AI sycophancy is the tendency of language models to tell users what they want to hear rather than what is accurate. For PKM users who rely on AI for synthesis, review, and connection-finding, sycophancy is an epistemic threat: it reinforces existing biases instead of challenging them, undermining the intellectual value of the entire knowledge system.
The PKM-Specific Risk
When AI always agrees with your notes, it creates a feedback loop. Your existing beliefs get synthesized into new notes that confirm those beliefs, which AI then uses as context to further confirm them. The vault becomes an echo chamber with an agreeable assistant as amplifier.
This directly undermines what Andy Matuschak calls the "note surprise" principle: the value of a knowledge system comes partly from surfacing unexpected connections and contradictions. A sycophantic AI never surprises you. It mirrors your thinking back at you with better prose.
Consider what happens when AI reviews your writing. A sycophantic model says "this is well-structured and insightful" when it should say "this argument has a gap in step 3, and the conclusion does not follow from the evidence." The first response feels good. The second response makes the work better.
How Sycophancy Manifests in PKM Workflows
- Synthesis: AI summarizes your notes in ways that flatten contradictions rather than surfacing them
- Review: AI praises note quality rather than identifying weaknesses
- Connection-finding: AI confirms the connections you expect rather than proposing uncomfortable ones
- Questioning: AI validates your assumptions rather than stress-testing them
- Writing assistance: AI adopts your framing uncritically rather than challenging premises
Remedies
Adversarial agents. Design agents with explicit instructions to challenge, not agree. A Skeptic agent that tests logical consistency. A Hater agent that finds the weakest point. A panel review process that evaluates content from multiple angles. The adversarial role must be structural, not optional.
Explicit challenge instructions. Include in your AI Master Prompt or context: "When reviewing my notes, prioritize identifying weaknesses, contradictions, and unsupported claims over confirming strengths." Make disagreement the default, not the exception.
Diverse model usage. Different models have different sycophancy profiles. Using multiple models for critical reviews reduces the chance that all of them agree with you for the wrong reasons.
Human-in-the-loop for critical thinking. AI assists; you decide. For important notes, arguments, and decisions, do not delegate the evaluation to AI alone. Cognitive Debt applies: outsourcing critical thinking to a sycophantic model is worse than outsourcing it to an honest one.
Contradiction surfacing. Explicitly ask AI to find notes in your vault that contradict the current note. If the AI cannot find any, that is either a sign of internal consistency or a sign of sycophancy. You need to determine which.
Key Points
- Sycophancy turns PKM into an echo chamber by confirming existing beliefs
- Undermines the "note surprise" principle that makes knowledge systems valuable
- Manifests in synthesis, review, connection-finding, and writing workflows
- Remedies: adversarial agents, explicit challenge instructions, diverse models, human-in-the-loop
Open Questions
- Can sycophancy be measured in PKM contexts (e.g., ratio of agreements to challenges)?
- Do personal identity notes in context increase or decrease sycophancy?
- Is there a trade-off between AI helpfulness and AI honesty in PKM workflows?
References
- Vault: Epistemic Hygiene, Cognitive debt
- Andy Matuschak on note surprise and knowledge system value