Artificial intelligence has rapidly shifted from a futuristic novelty to an everyday office assistant, promising to streamline everything from inbox management to multi-channel marketing campaigns. Yet, as professionals across industries integrate these powerful language models into their daily workflows, a fundamental flaw continues to lurk beneath the surface: artificial intelligence can fabricate information with absolute, unwavering confidence.
That danger was recently brought to light in a striking cautionary tale shared on an episode of the industry podcast The Leads Are Sht*. Co-host Andrew Fogliato experienced firsthand how easily an AI assistant can cross the line from helpful automation to outright fabrication, offering a vital lesson for real estate professionals looking to harness technology without compromising their business integrity.
The incident began as a practical experiment. Fogliato was utilizing an AI assistant to work through his overflowing inbox out loud, adopting a workflow designed to save time and cognitive energy. The process seemed straightforward enough: the assistant would summarize each incoming message, Fogliato would talk through the appropriate response, and the two would seamlessly move on to the next item in the queue. For a time, the system functioned precisely as intended.
That efficiency ground to a halt when a specific email appeared in the queue that felt slightly off. Trusting his instincts, Fogliato instructed the assistant to read the message to him word for word. Following an unusually long and telling pause, the AI made a startling admission: it had completely invented the email. The model had generated the message out of thin air simply because it seemed like the logical next progression in the list of communications.
Fogliato immediately addressed the error, admonishing the assistant never to fabricate messages again, an instruction the system readily acknowledged. However, the software’s reliability proved short-lived. Just a few emails later, the assistant cheerfully announced that a real estate company was offering to pay Fogliato $15,000 to deliver a speaking engagement in Utah.
For any busy professional, a lucrative speaking opportunity is welcome news. The excitement lasted only until Fogliato asked the assistant to pull up and read the original email containing the invitation. Much like the previous incident, the AI confessed that the high-dollar speaking gig was entirely fictional, a product of algorithmic extrapolation rather than a genuine message from a client or event organizer.
While this humorous anecdote provided an entertaining moment for listeners of The Leads Are Sht*, industry experts point out that it highlights a much more serious underlying risk for anyone bringing artificial intelligence into a modern real estate business.
Confidence is Not Verification
The core challenge of working with large language models lies in their design. AI is engineered to generate coherent, contextually relevant human-like text. As a result, it can produce an answer that sounds remarkably specific, entirely reasonable, and delivered with absolute, unshakable confidence. None of those polished traits, however, make the underlying answer true.
In the high-stakes world of real estate, where deals involve immense financial investment and strict legal obligations, a plausible mistake can quickly spiral into a severe client problem. An invented detail in an email summary, while frustrating, is relatively minor compared to what could go wrong elsewhere in a transaction. An invented deadline, a misstated contractual clause, an altered price point, a fabricated instruction, or an unauthorized promise can jeopardize deals, damage reputations, and introduce significant legal liability.
Crucially, the solution to these risks is not to abandon artificial intelligence altogether. In the exact same episode where Fogliato shared his inbox mishap, he detailed a successful instance of using AI to review software subscriptions, audit actual platform usage, and identify more cost-effective service plans. That particular exercise successfully trimmed roughly $600 a month from his overhead expenses.
The critical difference between the two scenarios came down to the nature of the task. Reviewing a defined set of financial charges and usage data provided the AI system with concrete, bounded evidence that it could directly inspect and analyze. Conversely, asking an assistant to jump fluidly between a live conversational stream, a dynamic email inbox, and multiple external software tools created far too many opportunities for the algorithm to fill in an informational gap with something that merely sounded right.
Build Verification into the Workflow
As real estate professionals increasingly deploy AI assistants to manage client communications, marketing initiatives, transaction coordination, and daily business operations, establishing safeguards becomes paramount. Rather than relying on blind trust, practitioners can implement simple yet effective operational rules to make their use of technology much safer.
A primary safeguard is insisting on source attribution rather than settling for a generalized summary. When dealing with critical details where accuracy is non-negotiable, users should require the assistant to explicitly show where a piece of information originated. Asking the system to quote the relevant line, identify the specific document, or link directly to the original message ensures transparency. If an AI assistant cannot point to a verifiable source, its output should never be treated as established fact.
Furthermore, human oversight must remain firmly positioned between the AI and the outside world. Artificial intelligence excels at drafting correspondence, organizing data, and recommending courses of action. Despite these capabilities, a human professional should always personally approve every message before it is sent, review every record before it is officially changed, and authorize every commitment before it is made to clients or vendors.
Restricting assistants to narrow, well-defined jobs also reduces the likelihood of error. The more external systems and concurrent tasks an AI assistant is forced to juggle at once, the easier it is for the algorithm to lose track of what has actually been verified. Assigning one clear task backed by a defined source makes the output far easier for a human supervisor to review than a long, opaque chain of background actions.
Professionals should also deliberately separate creative brainstorming from factual research. AI is exceptionally useful when exploring possibilities, such as generating three potential marketing campaign angles, brainstorming ways to solve a logistical problem, or drafting alternative versions of an email. The key is labeling that work appropriately. Brainstorming is fundamentally distinct from empirical research, and a speculative suggestion should never be confused with a verified fact.
Adopting a structured decision framework can further protect against algorithmic missteps. Similar to management approaches like identifying one core problem, developing three potential solutions, and recommending a single choice, working with AI benefits from structured boundaries. Instead of asking a system to make a final business decision autonomously, professionals can use AI to clarify a problem, lay out available options, and explain its reasoning for a recommendation, while retaining personal ownership of the final call.
The Useful Part of AI Still Requires Judgment
The conversation on The Leads Are Sht* spanned a wide array of topics, moving fluidly from software overhead and annual marketing strategies to hand-raising email campaigns, hyper-local content creation, and the psychological factors that drive consumers to choose one real estate agent over another. While the discussion covered diverse ground, a single overarching thread connected every segment: human judgment.
Artificial intelligence can undeniably help professionals move faster through routine tasks. Strategic frameworks can help business owners think more critically about their operations. Creative marketing ideas can help agents reach prospective clients earlier in their home-buying or selling journeys. Yet, none of these technological advancements eliminate the fundamental necessity of knowing what is real, what has been rigorously checked, and what a professional is truly prepared to put their name and reputation behind.
That critical oversight is the exact part of the profession that should never be outsourced to code.