Understanding the Shift to Agentic AI Processing
The rise of Agentic AI introduces a distinct computing model that diverges from traditional genAI training on GPUs. As organizations navigate this new landscape, they must carefully assess the available hardware options and pricing structures offered by cloud service providers. “It’s more about model management than model building — and the CPU is vital for effective workflow management,” explained Jack Gold, a principal analyst at J. Gold Associates.
The Complexity of Pricing
Cost variations remain a significant challenge in this evolving environment. The pricing structure for CPU computations differs markedly from that of intensive GPU usage, which complicates expense forecasting. “GPUs used for training generally consume more electricity because they operate at near-total utilization levels,” Gold noted. In contrast, general-purpose computing sees CPU and server utilization rates fluctuate between 40% and 60%. “However, this can vary widely based on the specific tasks the agent is performing,” he added.
The Future of AI Workloads
Looking ahead, Gold anticipates that between 80% to 85% of AI workloads will transition to inference over the next two to three years, particularly as tools become increasingly agentic. “CPUs play a crucial role in facilitating all these processes, which is why major cloud providers are now focusing on increasing their CPU capacities, not just their GPU resources,” Gold stated.
Conclusion
As Agentic AI continues to evolve, understanding the dynamics of hardware and pricing models becomes essential for businesses. With a significant shift towards CPU utilization for workflow management, organizations must adapt to maintain efficiency and cost-effectiveness in their AI strategies.
