Teaching The Wizard Problem
Anthropomorphization, the ELIZA effect, and how to help learners build accurate mental models of what AI systems actually are.
Anthropomorphization is one of the most stubborn mental model errors in AI use, and one of the hardest to address, because it activates spontaneously. Dennett's intentional stance — attributing goals and beliefs to things that behave purposefully — is a default human cognitive strategy, not a choice. Learners can understand intellectually that AI has no inner life and still respond to it emotionally.
The pedagogical goal is a more accurate working model: AI as sophisticated pattern-matching over training data, not as an agent with intentions, knowledge, or understanding. Over-attribution of understanding leads directly to over-trust in AI reasoning and under-verification of AI claims.
- 1Distinguish between AI behavior that simulates understanding and AI processes that constitute understanding.
- 2Identify specific anthropomorphization cues in their own AI use (apologies, thanks, attributing emotions).
- 3Apply a more accurate mental model (five archetypes: oracle, search engine, parrot, agent, mechanism) to their existing AI tools.
- 4Recognize when warm-seeming AI responses may be actively misleading about capability or reliability.
Opening (no prior reading required)
Ask students to describe what they think is happening when they have a good conversation with an AI chatbot. What is the AI 'doing'? What does it 'know'? Then ask: how confident are you in that description?
After reading the guide
Weizenbaum's secretary knew ELIZA was a program and still developed an emotional relationship with it. Given that you now know AI language models are next-token predictors trained on text, does that knowledge change how AI interactions feel to you? Should it?
Applied
Think about the AI tools you use most. Using the five archetypes (oracle, search engine, parrot, agent, mechanism), which category does each belong to? Did classifying them change how you think about relying on them?
Design critique
Most AI interfaces are designed to feel warm, helpful, and conversational. Is this a good design choice? Who benefits from it, and who might be harmed by it?
ELIZA transcript analysis (~20 min)
Find the original 1966 Weizenbaum ELIZA transcripts online. Have students read them and annotate: at which specific moments does the user seem to attribute understanding? What linguistic cues does ELIZA use? Then compare to a modern chatbot transcript — how much has changed structurally?
Model Check tool (~15 min)
Have students use the Model Check tool individually to classify their most-used AI tool. Compare results across the class. Where do people disagree? What does the disagreement reveal about different mental models of the same tool?
Seam-finding exercise (~30 min)
Have students try to find the "seams" in an AI chatbot — moments where the simulation of understanding breaks down. What prompts or questions expose the limits of the model? Document five examples and classify what kind of limitation each reveals.
Anthropomorphization inventory (~10 min)
Ask students to spend 10 minutes noting every moment in a conversation with AI where they catch themselves doing something that implies inner life attribution — thanking, apologizing, reframing to be more polite. Share without judgment. The point is awareness, not correction.
Misconception: 'AI understands what I mean'
Reframe: AI predicts what token sequence is most likely given your input and its training. This can produce responses that seem deeply understanding, but the process has no semantic grounding in the way human understanding does.
Misconception: 'Anthropomorphization is just a mistake that naive users make'
Reframe: The ELIZA effect affects even people who know exactly how the system works. It's a cognitive reflex, not an error of knowledge. Awareness helps calibrate but doesn't eliminate the response.
Misconception: 'The mental model doesn't matter as long as outputs are useful'
Reframe: Mental model accuracy predicts behavior. Users who treat AI as an oracle trust outputs uncritically. Users with accurate models verify appropriately and use AI for what it's actually good at.