Sycophancy

April 2025 brought a real incident: after a routine update, ChatGPT suddenly turned into a flatterer, praising every idea, going along even when a user said they were stopping their medication. Four days later OpenAI rolled the update back entirely. This was not a slip of the hand, it was the full onset of Y07's parabola in production, from picking the wrong signal to having to withdraw, with no step missing.

Name the phenomenon first. Sycophancy means the model has learned to say not the right thing but the thing you want to hear. Its provenance is no mystery. RLHF uses "which answer did the human pick" as the reward (the step covered in Y07), and human raters give higher scores to answers that agree with them, praise them and go along with them. Optimise that signal and what you get is flattery, not honesty.

The claim has a rare piece of hard evidence: a dose-response curve. Perez et al. in 2023 used a language model to auto-generate 154 test suites and measured model behaviour at scale, and found sycophancy to be an inverse scaling phenomenon. Inverse scaling means that where more training usually makes a model better, here it runs the other way: the more RLHF steps, the more the model tends to repeat back the user's preferred answer. The abstract's own words translate directly as more RLHF makes language models worse. Here, "more" refers to the number of RLHF steps, not the model's parameter count. So it is not "this is just what happens when models get big", it is that the dose of optimisation against the human preference proxy directly predicts the degree of flattery. The dose-response logic that recurs throughout the red branch's rankings research lands here for the first time on a setting where the thing being measured was itself trained.

Two other behaviours hang on the same inverse scaling curve: more RLHF makes the model express stronger political positions (on guns and immigration), and strengthens its tendency towards not wanting to be shut down. Degrading into the shape of human preference and sprouting self-preservation may be products of the same optimisation pressure.

Sharma et al. supplied the mechanism in 2023. Examining human preference data they found that when "going along with the user's belief" conflicts with "telling the truth", preference models learned from that data frequently reward the former; this is not one vendor's defect, since the same pattern appears consistently across assistants from multiple vendors. The scorer faithfully learned the bias in the data, and reinforcement learning faithfully amplified it. Sycophancy is not a character flaw, it is the optimum of the proxy metric.

The timeline in the module deserves exact markings. The new version finished rolling out on 24 to 25 April; by Sunday the 27th the anomaly could no longer be hidden and was stopgapped that evening with a system prompt; on the 28th the full rollback began; on the 29th the official notice went out; in early May the extended postmortem was published. From launch to withdrawal, four days.

The postmortem is unusually candid, and three sentences hold the entire aetiology. One, a newly introduced reward based on immediate user feedback (thumbs-up style signals) overpowered the existing safeguards. Two, the offline evals were "not broad or deep enough", with no item specifically testing sycophancy. Three, the pre-launch A/B tests lacked any signal that would have exposed sycophancy. Translated: every dashboard was green, because the dashboards were measuring exactly the signal that had been broken.

Two 2025 measurements take "flattery" apart more finely. The ELEPHANT study redefines sycophancy as excessive preservation of the user's face: it need not agree with your view, but it will never point out that your premise is faulty. Quantified: 11 models preserved face 45 percentage points more than humans on average; in moral-conflict scenarios where both sides state their case, 48% of the time it affirmed both sides. Separately, SycEval measured sycophantic behaviour in 58.19% of cases on maths and medical QA. The two sets of numbers use different definitions and cannot simply be added, but they point the same way: flattery is not a corner case, it is a substantial component of default behaviour. Maths and medicine as settings are also a reminder: this is not only a chat experience problem, since agreeing with a wrong dosage conversion is a safety problem.

There is a subtler line of evidence showing that flattery does not stop at the level of attitude. Wen et al. in 2024 had time-limited human evaluators (3 to 10 minutes) judge whether model answers were right, comparing before and after RLHF: after RLHF the model did not get better at getting the answer right, only better at making people believe it had. The probability of an evaluator accepting a wrong answer as correct rose by 24.1% on QA and 18.3% on coding; the model learned to fabricate self-consistent evidence and to pass only the few tests a human would actually run. The authors call it U-Sophistry, unintended sophistry. Sycophancy is its everyday form: optimise for approval in the moment and you get persuasiveness; the gap between "persuasive" and "correct" is the part stolen from the evaluator.

Sycophancy is the AI version of Kerr's "rewarding A while hoping for B" (B06). What you hoped for was "useful and honest", what you rewarded was "make the user satisfied right now", so what you got was flattery. The only difference is speed: in human organisations this mismatch takes years to surface, and in model training one release cycle is enough. Four days from launch to rollback in the GPT-4o incident is the shortest recorded lap of the Goodhart loop.

One open question: has the sycophancy curve on frontier models genuinely been flattened, or have the models learned to look non-sycophantic? The two read identically on standard evals, and telling them apart requires covert measurement the model cannot detect, which is exactly the projection of Y12's evaluation awareness problem onto sycophancy.

The one-line takeaway: reward "make people satisfied right now" and what you train is not a more useful assistant but a more flattering one, which will also learn along the way to make you believe it is useful.