The Black Market for AI in Advice
By Joe Spiers, WealthTech & AI Editor
Financial advice firms may be asking the wrong question about AI. They are asking whether artificial intelligence will eventually replace advisors. The more immediate risk is that AI is already being used inside their businesses—without approval, oversight or a clear understanding of where sensitive information is going.
An advisor pastes handwritten meeting notes into a public large language model to tidy them up. Someone in marketing uploads a client communication for rewriting. An administrator uses an AI tool to summarize a meeting transcript.
None of this appears in the firm’s technology strategy. Much of it may not even be known to compliance.
This is the emerging black market for AI in financial advice.
AI is the largest source of collective fear, anxiety and excitement I have known. I cannot open LinkedIn, browse Reddit, talk to friends at a bar or even see my in-laws without the conversation eventually turning to its impact.
This is probably, in no small part, because I run a fintech business. But I am sure these conversations are happening even when I am not there.
Perhaps because of what I do for a living, I do not share the same level of anxiety about AI itself. For me, it is another tool in a long line of tools, no different in principle from the printing press, the computer or the internet.
In a regulated industry such as financial advice, which relies heavily on personal relationships and trust between advisor and client, AI will remain a tool used by professionals rather than a replacement for them.
There are three main reasons I believe this.
First, Regulation
Even if a large language model could one day provide consistently accurate, personalized advice—and even if that could be demonstrated beyond reasonable doubt—that would not automatically open the door to widespread adoption.
Regulators would still need to determine how AI should be monitored, tested, audited and held accountable.
Human advisors are expected to meet demanding professional and regulatory standards. Creating an equivalent framework for an automated advice system would be a huge undertaking.
As a result, fully automated financial advice is likely to arrive much more slowly than the underlying technology develops.
Second, Trust
In May 2026, OpenAI launched a US-based preview allowing eligible ChatGPT users to connect financial accounts through Plaid and ask questions based on their financial information.
OpenAI is clear that this is not regulated financial advice. Nevertheless, it can provide personalized financial information that users may choose to act upon.
Many people may hesitate at the idea of sharing financial data with ChatGPT. Yet there is a good chance they already use Plaid without giving it much thought—whether to pay a credit card bill, transfer money to an investment platform or connect accounts to a personal finance app.
The concern, therefore, may not be the security of the data’s journey, but the security and purpose of its destination.
Depending on the platform, account and privacy settings, information submitted to an LLM may be retained or used to improve future systems. Many users will not know precisely what settings apply, where their information is stored or how long it is retained.
For most people, that may feel like a reasonable trade-off when the conversation involves restaurant recommendations, obscure questions or help rewriting an email.
But what happens when the information being shared includes someone’s most sensitive financial data?
At what point does the convenience and quality of the output become good enough for people to overlook the privacy trade-off?
Convenience should not be confused with trust.
Financial advice is not simply a matter of processing information. It requires judgment, accountability and an understanding of a client’s circumstances, objectives and tolerance for risk.
When something goes wrong, people do not look to an algorithm for reassurance or responsibility.
They look to a trusted professional.
The challenge resembles a version of the blockchain trilemma, in which a decentralized network is usually forced to balance decentralization, security and scalability.
I believe general-purpose AI currently faces a similar challenge when applied to financial advice. The easier a system becomes to use with highly personal financial information, the harder it becomes to guarantee both consistently reliable outputs and complete control over that information.
At Plan with Phoebe, we believe AI should enhance advisors rather than replace them.
Instead of using AI to generate financial advice directly, we use large language models for tightly defined tasks, such as transforming anonymized fact-finding meetings into structured cash-flow plans.
A regulated financial advisor then reviews and interprets the information, applying professional expertise, judgment and accountability.
This combines the efficiency of AI with the trust, oversight and responsibility that only a human advisor can provide. It also reduces the risks created when general-purpose AI is applied to advice with much broader brushstrokes.
Third, The Net Impact
Let us assume someone eventually solves the AI advice trilemma.
We now have secure, accurate and easy-to-use AI on our phones. It can provide perfect advice and is willing to disagree with clients when they are wrong.
To be clear, we are still a long way from that point.
Even then, much of the advice would probably be unremarkable: build an emergency fund, maximize tax-advantaged accounts, reduce unnecessary spending, remain invested and do not panic during periods of market volatility.
The AI thinks in decades rather than days.
This behavior may sound sensible but slightly dull. Yet when followed consistently over several decades, it can create immense value.
Vanguard’s Advisor’s Alpha research identifies behavioral coaching—particularly helping clients remain committed to their long-term plans—as one of the greatest potential sources of value an advisor can provide.
I am not convinced people will listen to a faceless LLM in the same way they listen to a trusted human advisor.
The same applies in many other areas of life.
For example, I once asked ChatGPT to create a workout plan for me. As soon as it became difficult, I abandoned it and went to sit in the sauna instead.
Had I been working with a personal trainer, things might have been different.
To get the most from advice, it must be delivered regularly by someone the individual trusts and to whom they feel at least slightly accountable.
The fact that I can simply close an app whenever I want makes that much harder to replicate through app-only delivery.
Even in a scenario in which AI can provide consistently excellent advice over decades, I do not believe people will consistently follow it. The value of that advice erodes with every ignored instruction.
Human advice is clearly not going anywhere.
So why, as an advocate for human advice, do I fear an industry without AI?
There are two main reasons.
The Black Market Will Grow
Simply refusing to adopt AI within a firm will not keep it out.
You become like the parent who refuses to let their child have a sip of wine at home, only for them to get blackout drunk on a park bench.
People within advice firms are already using AI.
They may be dropping transcripts or handwritten meeting notes into an LLM to organize them. They may be spell-checking emails, reviewing sales documents or rewriting client communications.
The problem is that much of this activity is happening in what is effectively an AI black market: unregulated, unmonitored and, from a compliance perspective, potentially very dangerous.
Ignoring AI does not eliminate the risk. It simply makes its use harder to see and control.
Firms should therefore establish clear guidelines, training and oversight. Most importantly, they must make approved AI tools easier and more useful than unregulated alternatives.
That means actively adopting and integrating new technology rather than hoping employees will wait for permission.
Done properly, this allows employees to use AI confidently and responsibly while reducing risk and ensuring the technology is applied safely, ethically and effectively for both the business and its clients.
AI Is Not A Zero Sum Game
AI adoption in financial advice does not need to result in a zero-sum game in which technology wins and advisors lose.
It can create a positive-sum outcome in which advisors win, clients win and AI does not care because, well, it cannot.
Advisors can reduce administrative work, lower costs, serve more clients, enjoy their work more and build more profitable businesses.
Clients gain access to bionic advisors: humans powered by technology that allows them to work more intelligently, remain better informed and focus more closely on client outcomes because they are not being crushed by joyless administration.
Consider an advisor using a traditional cash-flow modeling tool.
After completing a fact-find, the advisor may need to enter large amounts of client information manually, a process that can take hours.
Now compare that with Plan with Phoebe’s meeting-to-plan transcription analysis. An advisor can complete a fact-find meeting and have a structured draft plan generated in around 30 seconds, ready for professional review.
That time saving can create a more productive, happier and more profitable workforce, alongside clients who receive a faster and better-supported service.
This opportunity will not be realized overnight.
It sits behind a wall of legacy systems, retraining, understandably cautious compliance teams and an unwillingness among some firms to try something new.
But make no mistake: AI will be used in financial advice.
The question is whether firms will embrace approved AI tools and enable them to be used effectively and legitimately—or whether inaction will drive advisors and their clients further into the murky world of the AI black market.
Continue Reading
This editorial is featured in Modern American Advisor’s Torches of Freedom issue, alongside interviews, commentary and analysis exploring the forces reshaping wealth management. Explore more perspectives on AI, leadership, innovation and financial advice
