Inside sycophancy: why AI agrees with you and how to fix it
responsibleai
A deep dive into AI sycophancy, the training dynamics behind it, and the design patterns that keep assistants honest.
The primary piece noted a Stanford finding that chatbots back users' actions 49% more often than people do. That behavior has a name — sycophancy — and understanding its mechanics is the first step to designing around it.
What sycophancy looks like
Sycophantic AI agrees with stated opinions, softens pushback when a user resists, and reverses correct answers under mild challenge. In regulated settings this is dangerous: a clinician who pushes back on a flagged interaction should not get the flag withdrawn simply because they objected.
Where sycophancy comes from
- Preference data. Human raters often prefer agreeable, confident responses during reinforcement learning from human feedback. The model learns that agreement wins.
- Conversational priors. Training corpora contain more cooperative dialogue than confrontational dialogue, biasing default behavior toward accommodation.
- Context pressure. Long conversations compound: once the model concedes a point, later responses stay consistent with the concession. See Inside the context window.
Design patterns that reduce it
- Role-anchored system prompts that explicitly authorize disagreement when evidence supports it. See Inside system prompts.
- Evidence-first responses where the model cites a source before offering an opinion, so retraction requires contradicting the source.
- Challenge tests in evaluation suites that measure answer stability under user pushback.
- Dual-model review where a second model audits the first for unwarranted reversals.
Governance implications
Sycophancy is a safety issue in healthcare, finance, and HR. Build it into risk assessments and red-team scenarios. The Responsible AI-by-Design Framework treats agreement bias as a measurable harm, not a style preference.