For the last two years, the implicit scoreboard inside most companies measured one thing: how much AI you were using. Token leaderboards. Usage dashboards. A quiet pride in being the team that ran everything through the biggest model available. Satya Nadella has a name for the habit, "tokenmaxxing", and he has been candid that he is not immune to it. As he put it, the pull is real and a little addictive.
But the novelty is wearing off, and a more disciplined question is taking its place. Not how much intelligence are we consuming, but to what end? That shift, from volume to value, is the heart of what Nadella calls token efficiency.
What token efficiency actually means
Token efficiency is not about rationing AI or spending less for its own sake. It is about using tokens more effectively for the specific outcomes that matter, and recognising that reaching for a frontier model on every task is rarely the way to get there. The frontier model is extraordinary at genuinely frontier problems. The mistake is treating every problem as a frontier one.
Nadella's version of the rule is blunt: Don't use frontier models for non-frontier problems. Don't, in his phrasing, reach for a frontier model to schedule a meeting. The sledgehammer is impressive. Most nails don't need it.
Two ideas make this practical.
1. Match the model to the work
A large share of enterprise work is not frontier work. It's deterministic, repeatable, high-volume: grammar cleanup, summarisation, classification, structured extraction, routine drafting. For tasks like these, the largest model is not the best model, it is the most expensive way to get an answer that a smaller, optimised model could deliver as well or better.
The deeper point is that "smaller" does not mean "weaker." A compact model can be tuned to a specific job using reinforcement learning on an organisation's own data traces, the records of how that work actually gets done inside the enterprise. Optimised against the real task, a small model can outperform a general-purpose frontier system on that task, while running faster and at a fraction of the cost. The frontier model stays reserved for the problems that genuinely need its breadth.
This is why automatic model routing matters: a well-designed system classifies each request and dispatches it to the smallest model that can handle it. The intelligence isn't in always using the most powerful tool, it is in knowing which tool the moment calls for.
2. Define the outcome, then write the rubric
None of this works without clarity about what you're trying to create. Efficiency, in Nadella's framing, begins with a precise definition of the objective and the dimensions you'll use to evaluate success, the rubric for what "good" looks like on this specific task.
That clarity is what turns AI from a novelty into a system you can improve. Once you can name the outcome and score against it, you can hill climb: measure, adjust, measure again, and steadily get better within a system you control. And the rubric is where an enterprise's real advantage lives. Your domain expertise, how you judge a good contract review, a good support resolution, a good forecast, is the thing no general model has. Encode that judgment into your evaluation, and you're no longer renting generic intelligence. You're compounding your own.
Without that clarity, you get the opposite: tokens spent at scale on outputs no one defined and no one can evaluate. Activity without value. As Nadella has framed it, the goal can't be a race toward doing more things that simply don't add up to anything.
The shift worth making
The next phase of AI inside organisations will be won by whoever directs intelligence most deliberately, frontier models pointed at frontier problems, optimised models doing the deterministic work better and cheaper, and a clear rubric underneath it all that turns every task into something you can measure and improve.
That's the discipline behind token efficiency. Not less ambition. Just intelligence spent where it actually counts.
