AI Is Supercharging Scams and the Defenses Are Already Falling Behind

When ChatGPT launched in late 2022, it demonstrated something that researchers had long theorized but never quite seen at scale: a machine could produce human-like text so convincingly that distinguishing it from genuine human communication became genuinely difficult. Most people marveled at the technology. Fraudsters got to work.

We are now living through what security researchers are calling a new era of AI-driven scams, and the numbers are beginning to reflect just how serious the problem has become. The tools that make generative AI useful for drafting emails, summarizing documents, and writing code are the same tools that make it extraordinarily efficient for phishing, impersonation, romance fraud, and financial deception. The barrier to running a sophisticated scam operation has collapsed. What once required a team of fluent English speakers working overnight shifts in a foreign call center can now be handled by a single person with a laptop and a free API key.

The mechanics of this shift matter. Traditional scam emails were often riddled with grammatical errors and awkward phrasing, and that sloppiness was, paradoxically, a feature rather than a bug. Security researchers noted that poorly written scams filtered out skeptical targets, leaving only the most vulnerable. Generative AI eliminates that filter entirely. Scam messages can now be grammatically flawless, tonally appropriate, and personally tailored using data scraped from social media profiles. A fraudster targeting a grieving widow or a first-generation college student applying for financial aid no longer needs to guess at the right emotional register. The model handles it.

The same AI systems creating new vectors for fraud are simultaneously being studied as tools for improving healthcare delivery, and that tension sits at the heart of how societies are struggling to govern these technologies. Researchers and health systems are actively exploring whether large language models can assist with diagnostics, triage, patient communication, and clinical documentation. The potential benefits are real: AI could extend the reach of overstretched healthcare systems, flag drug interactions, and help patients in underserved areas access reliable medical information.

But the overlap between AI in healthcare and AI in scams is more than thematic. Medical fraud is already one of the most lucrative categories of financial crime in the United States, costing the system an estimated $100 billion annually according to the FBI. Generative AI gives bad actors the ability to fabricate convincing clinical documentation, impersonate healthcare providers in patient communications, and construct elaborate insurance fraud schemes with a level of polish that was previously out of reach for most criminal operations. The same chatbot interface that helps a patient understand their diagnosis could, in the wrong hands, be used to convince that same patient to hand over their Medicare number.

Regulators are aware of this, but awareness has not yet translated into adequate frameworks. The Federal Trade Commission has taken action against AI-enabled fraud in isolated cases, but the agency's resources are not scaled to the volume of the problem. The EU's AI Act, which began phasing in during 2024, creates risk classifications for AI systems but does not specifically address the use of general-purpose models in fraud operations. The gap between regulatory ambition and technical reality remains wide.

There is a second-order consequence here that deserves more attention than it typically receives. As AI-driven scams become more sophisticated and more common, public trust in digital communication erodes broadly. People become more suspicious of legitimate emails from their banks, their doctors, and their government agencies. This erosion of trust has real costs: people delay responding to genuine medical alerts, ignore authentic fraud warnings, and disengage from digital services that could genuinely help them. The scam epidemic, in other words, does not just harm its direct victims. It degrades the information environment for everyone.

This is a classic feedback loop. The more convincing AI-generated fraud becomes, the more people distrust all digital communication. The more they distrust it, the less effective legitimate institutions become at reaching them through digital channels. Institutions then struggle to communicate urgency when it matters most, whether that is a public health alert, a data breach notification, or a financial warning. The fraudsters do not need to win every interaction. They just need to poison the well.

The healthcare AI research now underway will eventually produce tools that are genuinely useful, and some already are. But their adoption will depend on a foundation of public trust that is being quietly undermined by the same technological moment that created them. How societies choose to rebuild that trust, through regulation, through technical countermeasures, or through new norms around verification, may end up being the more consequential design challenge of this decade.