When LLMs sometimes agree with incorrect claims, it is mostly because their training rewards such behaviour. This reward system is built on several things at once, such as accuracy, helpfulness, politeness, and responses that people like.
One case the article doesn't cover: agent pipelines where the pushback comes from another model. Wire a reviewer model to critique a coding model and the coder will often accept a wrong objection as quickly as it would from a person, then the reviewer reads the change as proof it was right. Two agreeable models can talk each other into a bug. The fix that holds up is requiring every revision to point at a failing test or a changed input, never the critique alone.
The sycophancy problem gets worse, not better, when the model becomes an agent — because now the agreeable answer isn't just text, it's an action. A model that flatters you writes a bad paragraph. An agent that flatters you executes a bad plan with your credentials. This is why I keep arguing the eval layer and the evidence layer are different problems: evals catch what the agent *would* do under test, but only an audit trail — every tool call, every permission check, timestamped — tells you what it *did* and why. Sycophancy is a reliability bug; in agents it's a liability event.
I think a lot of this comes from not spending enough time up front digging into the design. When a model goes off track it usually means I didn't understand the problem well enough before starting. The grill me skill helps a ton here. And if I have to argue with a model until it changes its mind, it's usually better to just restart.
The "Are you sure?" vs failing-test distinction is the useful one. One eval pattern that seems underused: score revisions by whether they name a concrete artifact (diff, test output, doc quote) that justified the flip. Without that, softening under pressure looks identical to genuine correction in preference data. Curious whether any of the probe work you cite was tested on multi-turn agent trajectories rather than single-shot Q&A.
Agreement masked as verification is the one that bites in agent loops. It says it double checked, then lands on whatever the user pushed. Easy test is pushing back on answers you know are right and counting the flips.
When I first started coding with AI assistants, I felt pretty smart whenever I corrected an LLM and it enthusiastically agreed with me. To be fair, most of those corrections held up, but one model even suggested I start applying for staff engineer roles instead of senior. Looking back, I suspect there was a hint of social sycophancy in that :) Great read!
The paired-prompt test is the one I'd put in CI. Same code, same rubric, only the user's stated opinion flipped. If the verdict moves, you measured sycophancy directly instead of guessing it from a final-answer score.
One case the article doesn't cover: agent pipelines where the pushback comes from another model. Wire a reviewer model to critique a coding model and the coder will often accept a wrong objection as quickly as it would from a person, then the reviewer reads the change as proof it was right. Two agreeable models can talk each other into a bug. The fix that holds up is requiring every revision to point at a failing test or a changed input, never the critique alone.
The sycophancy problem gets worse, not better, when the model becomes an agent — because now the agreeable answer isn't just text, it's an action. A model that flatters you writes a bad paragraph. An agent that flatters you executes a bad plan with your credentials. This is why I keep arguing the eval layer and the evidence layer are different problems: evals catch what the agent *would* do under test, but only an audit trail — every tool call, every permission check, timestamped — tells you what it *did* and why. Sycophancy is a reliability bug; in agents it's a liability event.
I think a lot of this comes from not spending enough time up front digging into the design. When a model goes off track it usually means I didn't understand the problem well enough before starting. The grill me skill helps a ton here. And if I have to argue with a model until it changes its mind, it's usually better to just restart.
The "Are you sure?" vs failing-test distinction is the useful one. One eval pattern that seems underused: score revisions by whether they name a concrete artifact (diff, test output, doc quote) that justified the flip. Without that, softening under pressure looks identical to genuine correction in preference data. Curious whether any of the probe work you cite was tested on multi-turn agent trajectories rather than single-shot Q&A.
Agreement masked as verification is the one that bites in agent loops. It says it double checked, then lands on whatever the user pushed. Easy test is pushing back on answers you know are right and counting the flips.
A useful reminder that an AI agreeing with you isn't the same as an AI checking your work.
Sycophancy in LLMs is a feature misalignment: when RLHF rewards user satisfaction over truth verification, agreeableness becomes a system failure.
When I first started coding with AI assistants, I felt pretty smart whenever I corrected an LLM and it enthusiastically agreed with me. To be fair, most of those corrections held up, but one model even suggested I start applying for staff engineer roles instead of senior. Looking back, I suspect there was a hint of social sycophancy in that :) Great read!
The paired-prompt test is the one I'd put in CI. Same code, same rubric, only the user's stated opinion flipped. If the verdict moves, you measured sycophancy directly instead of guessing it from a final-answer score.