As businesses increasingly adopt artificial intelligence (AI), the costs associated with its usage are rapidly escalating. While investments in AI infrastructure, such as data centers, have surged in recent years, organizations are now facing rising expenses related to AI consumption.
KPMG estimates show that businesses anticipate nearly doubling their AI expenditures in the coming year, averaging over $200 million each.
“Just a few quarters ago, few paid attention to the costs associated with large language model usage,” remarked Swami Chandrasekaran, head of AI and data labs at KPMG North America.
However, in the current year, these costs are escalating to levels that may approach those of human labor.
A recent survey conducted by Goldman Sachs found that many large companies are significantly exceeding their AI budgets, predicting that AI spending could soon match engineers’ salaries.
AI usage is generally measured in tokens, which represent the output of a word or two.
“I like to think of it as the number of exchanges in your interaction with the AI,” explained Max Kan, a “tokenomics” analyst at SemiAnalysis.
For instance, a simple conversation with a chatbot typically involves a handful of exchanges: you ask a question, receive an answer, and may follow up with two or three more queries.
However, advanced AI agents are capable of conducting extensive tasks such as internet searches, coding, analyzing numerous documents, and interacting with other agents continuously.
“This could easily involve anywhere from 100 to 10,000 times more tokens,” Kan noted.
The average cost per token has been decreasing due to improvements in models and hardware efficiency, with predictions suggesting a 90% drop by 2030, as outlined in Gartner research.
Nevertheless, more sophisticated models often utilize pricier tokens, and the demand for advanced features is increasing more quickly than costs are dropping.
“The question of how much budgets need to grow is still uncertain,” stated Will Sommer, an economist at Gartner. “However, deploying unconstrained advanced AI tools is not a sustainable approach.”
He suggested that businesses will most likely need to restrict their use of the most advanced and costly models.
Interestingly, some companies, including Meta, have begun what is referred to as tokenmaxxing.
“Employees are being encouraged to use as many AI tokens as they can,” explained Daniel Newman, CEO at Futurum Group. “The belief is that there is a direct correlation between token usage and productivity.”
Newman reported that his own company has experienced triple-digit growth in the use of AI agents over the last few months.
“It’s an incredibly effective productivity tool,” he noted, “but the expenses associated with it are becoming increasingly burdensome.”
This rise in costs can partly be attributed to some AI firms scaling back on subsidizing the use of their services, according to tech critic Ed Zitron, who has discussed the “subprime AI crisis.”
“Users will now have to cover the actual costs of the AI tools they utilize,” he noted. “This development is causing concern among businesses.”
Companies like Anthropic and Microsoft have recently made several pricing adjustments, which could lead to increased expenses for heavy users.
“I doubt many businesses fully comprehend the actual costs involved,” Zitron remarked.
Additionally, most firms lack robust systems to measure the return on their AI investments, according to economist Brian Jabarian from the University of Chicago, who consults companies on AI transformations.
“How can you prove, with hard data, that AI has been beneficial for your organization?” he inquired. “The bill is coming, and it’s going to be substantial.”
As AI expenses grow, the justification for those costs will need to become equally significant.
