Token usage is the wrong way to measure AI ROI
Token usage became the default way to show AI is working. It measures cost and effort, not return. What to measure instead.
This spring, a Meta employee built an internal leaderboard that ranked the company's more than 85,000 employees by how many AI tokens they used.
It was called Claudeonomics. Over 30 days, the company used 60 trillion tokens. The top user averaged 281 billion. Heavy users earned titles like "Token Legend".
It came down two days after the news broke.
Over the next few months, Microsoft and Uber backed away from the same idea. In an internal email, Microsoft's Jay Parikh wrote:
Tokenmaxxing is not what we are optimizing for. I want all of us focused on maximizing outcomes that move the needle for our customers and our business.
It's easy to laugh at a leaderboard. But it was built for a real reason. Boards told CEOs to transform the company with AI, CEOs passed that down, and the people who rolled out the tools now have to show it's working. Token usage is the number they have.
Tokens tell you what AI cost and how hard it worked. They can't tell you what it produced.
Why tokens became the metric
For most companies, token counts were the first AI number anyone could get. Every provider reports them. They go up when people use the tools. They fit on a chart.
Meanwhile, the pressure to show a return is real. In a global survey of 4,454 CEOs published in January, 56% said AI had brought them neither more revenue nor lower costs over the past 12 months. Only 12% had seen both.
When the return is hard to see and the usage is easy to count, usage becomes the stand-in. That's how you end up with a leaderboard.
"At least usage shows people are adopting it"
This is the fair version of the argument:
We spent a year getting people to try these tools. Usage going up means it's working. We'll worry about value once everyone's on board.
Early on, usage does tell you something. If nobody on a team has opened Claude Code in a month, that's worth knowing.
The trouble starts when usage becomes the goal. Reward people for tokens and you get more tokens. Agents left running. Long sessions that should have been short ones. The biggest model for every small task. Uber's CTO said the company moved past tokenmaxxing because it raised AI costs without clearly improving productivity or business results.
Engineering leaders have noticed. In a survey of engineering leaders this year, 19% said tokenmaxxing was effective, and 57% said it fails to gauge real value.
A high token count can mean a team found something AI is great at. It can also mean someone left an agent looping overnight. On a leaderboard, both look the same.
What to measure instead
None of this means you should stop tracking tokens. It means putting them in the right column.
Count tokens as cost
Return on investment needs both halves. Tokens are the investment. Track them as spend, by team, tool, model, and person, and make sure the total includes every tool, not just the one with the best dashboard.
This half is harder than it sounds. A survey of 2,000 finance leaders this fall found 83% had AI costs run past what they expected, while rating their confidence in AI's value at 8.2 out of 10. Confidence is high. The cost number underneath it is shaky.
If you can't say what AI cost last month, any ROI figure built on it is a guess.
Tie the spend to the work it went into
The return side starts with a simple question: what did this money produce?
For engineering, the answer is in work you already track. Pull requests, tickets, projects. If you can see that a feature took, say, $340 of AI sessions across 3 people and shipped in a week, you can have a real conversation about whether that was worth it. A token total can't start that conversation.
This is what Tokenize ROI does. It links each pull request to the AI sessions that wrote it, rolls AI cost up to every project in your tracker, and sends a weekly digest of the work to Slack.
Look at what kind of work AI is doing
Not every session maps to a ticket, and that's fine. Some of the most useful AI work is debugging, exploring an unfamiliar codebase, or building a throwaway prototype.
What matters is the mix. If most of a team's spend goes to fixing production errors and shipping features, that's a different story than if most of it goes to sessions that never turn into anything. Grouping sessions by what they were for shows you where AI is pulling its weight and where it's mostly exploration.
Find where people get stuck
Usage numbers that look healthy on the surface can hide a lot. People hit a broken connector and give up. They use the tool for one narrow task and never try anything else. They paste the same instructions into every session.
None of that shows up in a token count. It shows up in how people work: which tools fail, which setups differ between laptops, which habits waste time and money. That's where a little help changes the return more than another training session.
Tokenize shows how AI is catching on across teams and sends each person private tips based on how they use it. It doesn't rank people by output.
Give one person the whole picture
Engineering picks the tools. Finance pays the bills. IT manages the seats. When nobody owns the full number, nobody can answer the board's question.
The FinOps Foundation found that organizations with clear ownership of AI costs were 3.7 times more likely to show AI's value to their CFO.
What to bring to the board
A slide with token counts going up and to the right answers the wrong question. A better one shows what AI cost, what it went into, and how that's changing:
- Total AI spend, across every tool, by team
- What it produced: the projects, pull requests, and kinds of work behind the spend
- Where the mix is improving, and where people are stuck
- What you're changing next quarter, and why
It's less impressive than 60 trillion tokens. It's also something a CFO can check.
If you'd like to see what that looks like for your teams, book a demo and we'll show you on your own data.
Sources: Fortune, Meta killed the employee AI token dashboard, April 9, 2026. LeadDev, The tokenmaxxing hype didn't last long, August 17, 2026, including the Microsoft email first reported by 404 Media and LeadDev's AI Impact Report 2026. PwC, 29th Annual Global CEO Survey, 4,454 CEOs surveyed September to November 2025. Pigment, Q3 CFO Index, 2,000 finance leaders surveyed August to September 2026. FinOps Foundation, AI Spend Budget Busters, October 2026.