Federal Regulation and Oversight of AI
While there are few federal standards governing the use of AI in the prior authorization and claims review process, existing coverage decision-making for both public and private sectors follows general standards that aim to ensure reviews are conducted fairly, substantively, and promptly. These standards are split across various federal agencies, each responsible for overseeing different health coverage markets.
For private employer-sponsored plans, the U.S. Department of Labor (DOL) regulates claims and appeals requirements under the Employee Retirement Income Security Act (ERISA). Generally, ERISA exempts self-insured plans established by private employers from a majority of state insurance laws, including those protecting claims review, and may supersede state laws regarding AI use in claims processes. Most employees with employer-sponsored insurance are part of self-funded plans, which means many consumers may lack state protections regarding AI use in claims review, where they exist.
The rules governing claims and appeals under ERISA served as the foundation for reforms incorporated into the Affordable Care Act. These reforms established a minimum level of protections for the internal claims and appeals processes for those covered under Marketplace and off-Marketplace private insurance, allowing all consumers with private coverage to appeal denied claims through an “external review” by an independent entity.
ERISA mandates that all employer plan sponsors ensure a “full and fair” review of health claims. The interpretation of what constitutes a “full and fair” review in the context of AI tools in claims processing is still pending clarification through guidance or updated regulations. Furthermore, ERISA contains “fiduciary” rules obligating employers and fiduciaries to act in the best interest of plan enrollees and to monitor vendors’ activities. Although these standards may provide some level of protection concerning AI use in employer plans, fiduciary standards have rarely been enforced against employer health plans, and enrollees have faced challenges in successfully pursuing litigation against employers for breaching fiduciary responsibilities.
A recent DOL case involving a large third-party administrator (TPA) alleged a violation of fiduciary duty and noncompliance with ERISA claims rules after the TPA automatically denied claims en masse without conducting individual medical necessity evaluations as required by the plan. While this instance did not explicitly involve AI, it was reported that the TPA used an automated system lacking human oversight in making denial decisions. The case was settled by establishing a fund aimed at compensating enrollees for incorrectly denied claims.
Federal guidance addressing AI use in prior authorization and claims review for Medicare and Medicaid remains sparse. Each program has distinct consumer protection measures concerning claims and appeals under federal requirements, alongside some state standards applicable to Medicaid.
Medicare. The 2023 Medicare Advantage regulations and additional guidance for 2024 stipulate that Medicare Advantage organizations are prohibited from making medical necessity determinations using algorithms or software that fail to account for individual circumstances. Denials based on medical necessity must be evaluated by a qualified health care professional. Proposed regulations introduced in 2024 that addressed potential bias and discrimination pertaining to AI usage by Medicare Advantage plans were not finalized during the Trump administration. Moreover, the federal government is piloting the application of AI for specific service prior authorization decisions in traditional Medicare through its Wasteful and Inappropriate Services Reduction (WISeR) Model, partnering with AI technology companies to implement this pilot program across six states.
Medicaid. Current Medicaid regulations do not specifically address the use of automation in prior authorization. Medicaid managed care regulations mandate that any services denied by a managed care organization (MCO) must be decided by “an individual” possessing relevant expertise, but do not expressly address the role of AI. States retain the authority to set requirements for plan performance and reporting through managed care contracts (which CMS reviews and approves), allowing for measures such as requiring plans to disclose their use of AI in prior authorization procedures. The Medicaid and CHIP Payment and Access Commission (MACPAC) recently put forth recommendations regarding the application of automation in Medicaid prior authorization.
State AI Consumer Protections in Prior Authorization and Claims Review
In recent years, several states have taken steps to implement laws and regulations designed to protect consumers from potential harms arising from algorithmic decision-making systems, such as privacy violations, inaccuracies, and biases. Ongoing discussions regarding AI-related legislation are taking place in nearly every state legislature, with a considerable number of these efforts receiving bipartisan support. Some states have opted to issue regulations and provide guidance based on existing laws instead of, or in addition to, introducing new legislation.
State laws define new and existing AI consumer protections. Certain state laws offer broad protections designed to span various sectors and apply to a range of entities, including developers and users of the technology for commercial purposes. Other legislation is tailored more specifically to industry sectors (e.g., health care), themes (e.g., employment, civil rights, education), or particular uses, such as utilization review within health insurance.
Broad consumer protection laws prohibit unfair or deceptive practices across all 50 states. These laws are enforced by state attorneys general and may allow consumers to pursue a private right of action for violations, meaning individuals can sue directly rather than solely relying on state enforcement. States like Colorado and Utah have amended their consumer protection statutes to include general AI protections.
Depending on the specific law, these broader consumer protections could be used to address harms related to AI in prior authorization and claims review. Moreover, a growing number of states have updated long-standing health insurance regulations regarding managed care and utilization review to specify how these rules are applicable to AI (Figure 1). Most of the newly established laws focus on the utilization review process, often defined as individualized decisions regarding whether a service is medically necessary based on a patient’s specific clinical circumstances. These regulations typically do not cover administrative claim review decisions that do not necessitate a medical necessity determination, such as whether a claim concerns excluded care.
Each state law governing AI in prior authorization and/or claims review comes with its unique requirements, but several common themes can be identified:
- Human review of claim denials required. Some state laws stipulate that adverse determinations (denials) must be made solely by a licensed health care provider; AI cannot serve as the only decision-maker. For instance, Illinois mandates that only a “clinical peer” can make adverse determinations based on medical necessity, prohibiting the exclusive use of an “algorithmic automated process” for these decisions.
- AI tools must consider individual clinical circumstances. Certain states require that any AI tools employed for utilization review consider an enrollee’s unique medical history. Alabama, for example, mandates insurers using AI for prior authorization to ensure that decisions reflect the enrollee’s clinical circumstances.
- Disclosure of AI use. Several states, including Utah, require organizations utilizing AI in utilization reviews to disclose its application to the public, their network health care providers, and each enrollee.
- Review of AI tool outcomes. Some state regulations require organizations conducting utilization reviews to periodically assess the performance and outcomes of their AI tools to verify their accuracy and reliability. California stipulates that an AI tool must undergo regular evaluations to ensure optimal accuracy and dependability.
- Limits on patient data usage for privacy protection. Several state laws incorporate provisions that forbid those conducting utilization reviews from utilizing patient data beyond its intended purpose in ways that conflict with HIPAA or state confidentiality protections. Maryland demonstrates this approach.
- AI tools must be open to inspection. Some regulations require that AI tools employed for utilization reviews be auditable by regulators. In Texas, the insurance commissioner has the authority to audit and inspect the utilization review agent’s use of an automated decision-making system at any time.
- AI protections against bias and discrimination. A few state laws, such as Washington’s, mandate that AI tools be applied “fairly and equitably” to prevent direct or indirect discrimination against enrollees.
New state guidance aims to exert authority over AI use. Some states have articulated guidance to clarify how existing legal protections apply to AI applications. For example, the Massachusetts Attorney General issued a public Advisory in 2024, elaborating on how the state’s consumer protection, civil rights, and data privacy laws affect developers, suppliers, and users of AI, and their implications for consumers in Massachusetts.
Insurance regulators in various states have similarly created guidance, clarifying how current laws pertain to AI and giving insurers specific information regarding their obligations regarding AI use. As of early April 2026, at least 25 states have released guidance stemming from a model bulletin adopted by the National Association of Insurance Commissioners (NAIC) in 2023. This model bulletin applies across all types of state-regulated insurance (not just health insurance) and addresses AI use in various stages of the insurance lifecycle, including claims administration, fraud detection, product development, and rating. It sets expectations that consumer-facing decisions made or supported by AI systems comply with existing regulations, including those against unfair trade practices and discrimination. Insurers are instructed to establish policies detailing their AI usage and to implement controls that mitigate risks of adverse outcomes. Additionally, insurance regulators hold oversight responsibilities, including the authority to inquire about the development, deployment, and outcomes of AI systems or predictive models utilized by insurers and their third-party vendors, as well as to request information about AI validation, testing, and ongoing audits.
