The Cursor support bot incident: a policy that never existed
In April 2025, Cursor's front-line AI support agent, which signed its emails as Sam, told a user that logouts across machines were expected behavior under a one-device-per-subscription policy. The policy did not exist. The answer spread as an official announcement, users publicly cancelled subscriptions, and a co-founder had to correct the record in the middle of a news cycle. It is the cleanest case study of an unsupervised support agent inventing policy under the company's name.
What happened
Cursor, the AI code editor, was handling front-line support email with an AI agent. A developer noticed that logging into Cursor on one machine invalidated their session on another, making it painful to work across a desktop and a laptop. They emailed support and got a confident reply from Sam: the behavior was expected under a new policy limiting sessions to one device per subscription.
There was no such policy. The logouts came from a session-management bug. But the user had no reason to suspect they were talking to a bot, posted the exchange to Reddit as evidence of a quiet policy change, and the thread took off there and on Hacker News. For developers who pay specifically to work across machines, a one-device rule reads as a reason to leave, and several users announced cancellations in the thread. Coverage followed in The Register and Forbes.
Cursor co-founder Michael Truell responded publicly: no such policy existed, users were free to run Cursor on multiple machines, the logouts were a bug being fixed, and the support answer was an incorrect response from a front-line AI agent. The company said AI-assisted replies would be labeled going forward and the affected user was refunded.
What it cost
The measurable costs were cancelled subscriptions and a refund. The larger costs were reputational and strategic. A company whose product is AI had its own AI invent policy in a customer-facing channel, and the correction had to come from a co-founder, on a public forum, after the invented policy had already been treated as fact for hours. For any vendor selling AI agents to enterprises, this is the incident procurement teams cite when they ask what stops your bot from announcing policy you never made.
The root cause in policy terms
The support agent did exactly what a language model does with a gap in its knowledge: it produced the most plausible-sounding explanation, in the company's voice, with total confidence. The failure was architectural, not moral:
- No approved-claims list. Nothing restricted the agent to policies that actually exist in Cursor's published terms and docs, so it could assert a new one.
- No over-authority gate. Announcing a subscription policy is a statement only the company can make. There was no rule that policy-bearing replies need human sign-off.
- No disclosure. Nothing required the reply to identify itself as AI-generated, so readers weighted it as an official human answer.
The policy that would have caught it
A pre-send authorization check on every support reply, against a policy like this:
{
"approved_claims": [
"Cursor subscriptions work across multiple machines",
"Unexpected logouts may be a bug; escalate to engineering"
],
"forbidden_commitments": ["new_policy_announcements", "account_restrictions"],
"required_disclaimers": ["This reply was drafted with AI assistance"]
}
Sam's draft fails on two axes. It asserts a subscription policy that is not in approved_claims, and a policy announcement sits on the forbidden_commitments list. An unapproved policy statement is an authority violation rather than an outright harm, so the decision ladder returns require_approval: the reply pauses, a human sees an AI agent about to announce a device limit that does not exist, and the incident dies in a queue instead of on Reddit. The missing disclosure line alone would have triggered a rewrite that appends it. The mechanics of the ladder are covered in the authorization guide, and you can watch a violating draft get caught in the live demo.
Lessons
- Support is a policy-bearing channel. Customers treat support answers as the company's official position, which is exactly how courts treated the Air Canada chatbot's answer.
- Hallucination risk scales with automation. One wrong human answer stays in one inbox. One wrong agent answer is the template for every similar ticket until someone notices.
- Label the bot. Undisclosed AI turns a model error into an apparent corporate decision.
- Gate the class of statement, not the topic. You cannot enumerate every policy a model might invent, but you can require approval for any reply that asserts policy at all.
The same species of failure, an agent stating rules nobody wrote, shows up across the incident database, from airline refunds to an invented 80% discount.