Report: AI’s Productivity Gains in Science Tempered by Time Spent Validating Outputs
Artificial intelligence (AI) has started reshaping scientific workflows, enabling significant time savings for researchers. However, findings from a collaborative study by Google, Google DeepMind, and MIT indicate that these gains are partially offset by the time scientists spend validating AI outputs.
Key Findings
Researchers documented immediate challenges rather than more sensational claims of “rogue AI” systems. Concerns about auditing AI outputs, the proliferation of low-quality research, and the pace of hypothesis generation outstripping human capacity for validation emerged as key issues.
The study highlights that AI-generated content often requires significant verification, raising questions about the peer review process. With AI sometimes producing “hallucinations” or “illusions of understanding,” the quality of research could be jeopardized.
Study Overview
The September paper, AI in Science: Early Insights, presents insights from:
- 15 million anonymized interactions with Gemini, an AI model.
- An inventory of 2,690 specialized scientific AI models.
- A survey of 637 active scientists in the U.S. and U.K.
Almost half of those surveyed use AI daily, reporting an average net time savings of nearly seven hours per week.
Google emphasized these findings in its Sept. 15 AI & Economy ATLAS update. Researchers face new bottlenecks due to the need for rigorous validation, meaning that quicker individual tasks do not necessarily lead to accelerated discoveries.
A Verification Tax on AI’s Time Savings
While the productivity gains reported are substantial—three-quarters of researchers indicated time savings, with the average saving nearly seven hours weekly—much of this time is consumed in verification. Among those who reported saving time, 89% noted that over 10% of it went into verifying or debugging AI outputs. About 46% spent more than a quarter of their saved time on these tasks. This discrepancy is referred to as a “verification tax,” emphasizing the high value placed on accuracy within scientific research.
