Large Language Models (LLMs) are everywhere—drafting emails, writing code, even debating politics. But how well do they actually think?
In recent years, a new breed of models has emerged: Large Reasoning Models (LRMs). Designed to “think out loud” using Chain-of-Thought (CoT) prompting and self-reflection mechanisms, LRMs promise deeper, step-by-step reasoning. They’ve excelled on popular benchmarks like MATH500 and AIME. Yet a recent paper from researchers at Apple—“The Illusion of Thinking”—casts serious doubt on their reasoning abilities.
🚨 Key Insight: Good at Easy Tasks, Useless at Hard Ones
Shojaee et al. introduce a new framework that avoids the noise of contaminated benchmarks. Rather than math problems scraped from the internet, they use synthetic puzzles with tunable complexity: Tower of Hanoi, Checkers Jumping, River Crossing, and Blocks World. These puzzles allow precise control over problem difficulty while enabling direct inspection of reasoning steps.
The Big Findings:
- Three Regimes of Performance: (a) Low Complexity: Standard LLMs (without “thinking”) often outperform their reasoning-enhanced counterparts. (b) Medium Complexity: LRMs show their strength, offering better answers thanks to more elaborate reasoning paths. (c) High Complexity: All models fail. Accuracy drops to zero, and reasoning becomes shorter, not longer—despite having token budget left.
- The Collapse of “Thinking”: At high difficulty, models begin using fewer tokens to reason. They give up early, even though they could keep going. This suggests that models are not scaling their reasoning efforts appropriately. In short: they stop thinking when they need to think the most.
- Overthinking on Easy Problems: On simple tasks, models often arrive at the right answer early—but then waste effort exploring wrong paths. This behavior is inefficient and wastes compute. Worse, it signals that models don’t “know” when they’ve solved the problem.
- Failure to Execute Even When Taught: Perhaps the most damning evidence: when explicitly given an algorithm (e.g., for Tower of Hanoi), models still fail to execute it properly beyond a certain complexity. This isn't reasoning. It's pattern generation.
🤯 So, Are These Models Actually Reasoning?
The authors argue that what we’re seeing isn’t true reasoning at all—but sophisticated pattern completion. LRMs produce reasoning-like traces, but those traces fall apart under pressure. This explains why they perform well on public benchmarks, which may be part of their training data, but fail when tested in controlled environments.
Put differently: they look smart until they need to be.
💡 Why This Matters
As AI systems are increasingly entrusted with high-stakes decisions—medical advice, legal reasoning, scientific discovery—their reasoning integrity matters. Relying on models that “think” convincingly but collapse under complexity is dangerous.
This research reminds us that flashy outputs don’t equal robust understanding.
🔬 Toward Better Evaluation
The paper also advocates a shift in how we evaluate LLMs. Instead of just looking at final answers on benchmark tests, we should inspect their internal reasoning processes. Tools like puzzle simulators, logic checkers, and symbolic evaluators can help.
Controlled puzzle environments offer a replicable way to probe cognitive limits—something benchmark-based pass@k scores can’t capture.
🧩 Open Questions
- Why do models reduce effort as complexity increases?
- Can we design reward mechanisms that enforce adaptive reasoning?
- Will symbolic reasoning modules help models stay on track?
- How do these findings generalize to real-world multi-step reasoning, such as legal or financial analysis?
🧠 Conclusion
Large Reasoning Models may represent a step forward in structured output generation, but they don’t yet “understand” problems the way humans—or even basic symbolic systems—do. As Shojaee et al. put it, these models offer an illusion of thinking. And illusions can be dangerous when mistaken for competence.
It’s time we move beyond benchmark performance and start demanding models that not only speak well but reason reliably.
TL;DR: Despite their name, Large Reasoning Models don’t scale their reasoning as problem complexity increases. They “think” more for medium tasks but collapse at harder ones—and even fail to execute known algorithms. True machine reasoning still lies ahead.
source: https://ml-site.cdn-apple.com/papers/the-illusion-of-thinking.pdf
