AI safety discussions often focus on scenarios that are distant, superintelligent systems with misaligned goals, treacherous turns at capability thresholds not yet reached. AI persuasion risk is different. The basic form of the threat is present in systems available today, affecting elections and public discourse right now. The more dangerous long-term version is an extrapolation of technology that is advancing rapidly, not a purely hypothetical future.
The long-term version (AI that can model and exploit individual psychology with superhuman precision) is the direction the technology is moving.
The concern is not simply that AI makes it easier to write propaganda, previous technologies already did that. The concern is that AI enables something qualitatively different: highly personalized persuasive messaging at massive scale, tailored to individual psychological profiles, updated in real time based on responses. The combination of personalization, scale, and low marginal cost produces a capability for influence that no previous tool has matched.
What makes AI persuasion different
Traditional influence operations (broadcast advertising, political messaging, state propaganda) reach many people with the same message. The message may be carefully crafted and extensively tested, but it is necessarily a compromise between different audiences. A skilled human persuader can adapt their approach to the specific person they are talking to, but they can only talk to a limited number of people. These two capabilities (broadcast scale and individual personalization) have historically been in tension.
AI dissolves this tension. A system with access to individual data (browsing history, social media, prior responses) and the capability to generate varied persuasive content can simultaneously reach millions of people and tailor each interaction to the specific individual. The marginal cost of generating another personalized persuasive message approaches zero. The system can iterate based on which approaches are working and which are not, in real time, at a scale that no human influence operation can match.
The asymmetry problem
The power of AI persuasion is highly concentrated. Deploying a persuasion-optimized AI at scale requires substantial resources, data, compute, engineering capability, access to distribution channels. These requirements favor well-resourced actors: large corporations, nation-states, wealthy individuals. The people being targeted have no equivalent counter-capability. They may eventually have access to AI tools that help them evaluate the content they encounter, but the offensive and defensive asymmetry favors the attacker in the near term.
A campaign that once needed a warehouse of copywriters can now draft a thousand variants of a message before lunch, each tuned to a slice of voters. That is a product category already, not a far-future sketch. Persuasion at that resolution is an AI risk that shows up before anyone builds a god-model.
This is a structural problem, not a problem about any specific piece of disinformation. Even if every individual claim in an AI-enabled influence operation is technically true, the systematic targeting of psychological vulnerabilities at scale distorts the epistemic environment that democratic deliberation depends on. The goal of influence operations is to shape how people feel about issues, candidates, and institutions in ways that serve the operator's interests, not necessarily to deceive.
The near-term evidence
AI-generated influence content has been documented in national elections in multiple countries since 2023. Influence operations using AI to generate synthetic media, fabricate quotes from real public figures, and produce high-volume coordinated posting at a scale that would require large teams of human operators have been identified and reported by independent researchers and platform security teams. The operations have become harder to detect as AI-generated content has improved, and the detection tools are running behind the generation tools.
The risk is present in the current technology environment, affecting real political processes, and the trajectory is toward more capable and harder-to-detect persuasion tools, not hypothetical. The question is what governance responses are proportionate and effective, not whether to treat this as a real risk.
The long-term escalation
Current AI persuasion tools are effective but not optimized for persuasion at a fundamental level. Future systems with more sophisticated models of individual psychology, more detailed data about individuals, and more capacity for long-run relationship simulation could be substantially more effective. A system that can build a persistent model of an individual's beliefs, concerns, and psychological vulnerabilities, and interact with them over weeks or months, could produce effects far beyond what current tools achieve.
This is the form AI persuasion risk takes at the superintelligence end of the spectrum: not simple mass disinformation, but individually targeted long-run epistemic manipulation by systems with far better models of human psychology than any current tool. The connection to singleton scenarios and value lock-in is direct: an actor with highly effective AI persuasion tools could use them to shape public opinion about ASI governance itself, making it harder for democratic institutions to impose the constraints that would limit the actor's power.