Understanding the Challenges of AI in Government Decision-Making
Cary Coglianese, Edward B. Shils Professor of Law and Professor of Political Science
Professor Cary Coglianese delves into the limitations of AI in the decision-making processes of federal agencies.
In a thought-provoking essay featured in the Yale Journal on Regulation’s online platform, Notice & Comment, Cary Coglianese, the Edward B. Shils Professor of Law, Professor of Political Science, and director of the Penn Program on Regulation, investigates the transformative impact of artificial intelligence (AI) on administrative governance.
In his piece, “AI, Taxi Drivers, and Administrative Law,” part of a significant symposium discussing AI’s implications for the Administrative Procedure Act (APA), Coglianese warns that although AI can produce policy suggestions with impressive clarity and confidence, these cannot replace the thoughtful discretion required from federal agency officials.
Using the metaphor of an overly self-assured taxi driver who confidently shares opinions on a variety of subjects, Coglianese emphasizes that the APA is a vital mechanism to prevent government officials from unduly depending on AI when making important regulatory choices.
While he acknowledges the potential benefits of AI tools in assisting government officials with decision-making and various tasks, Coglianese cautions against over-reliance on large language models for addressing essential policy matters. He asserts, “Officials should not place undue weight on the outputs of general-purpose AI tools.”
“Administrative decisions under the APA,” he concludes, “must be based on more than simply the digital equivalent of opinions from even the most knowledgeable taxi drivers.”
An edited version of Coglianese’s essay was later published under the title “Administrative Law and AI’s Overconfidence” in The Regulatory Review, the online publication of the Penn Program on Regulation.
In conclusion, Coglianese’s analysis serves as a critical reminder of the need for caution when integrating AI into federal decision-making processes. By emphasizing the continuing importance of human judgment under the APA, he highlights the balance required to navigate the complexities of modern governance in the era of artificial intelligence.
