In a significant development regarding the use of artificial intelligence (AI) by the Home Office in asylum decision-making, a comprehensive legal opinion has revealed potential unlawful practices, particularly concerning a lack of transparency for applicants. This raises critical questions about fairness and accountability in the asylum process.
Image credit: WikipediaYou can download the 84-page legal opinion here.
This opinion was crafted by barristers Robin Allen KC and Dee Masters from Cloisters Chambers, alongside Joshua Jackson from Doughty Street Chambers. It was commissioned by the non-profit group, Open Rights Group, which focuses on digital rights.
According to the document, the Home Office’s deployment of generative AI tools within the asylum process may contravene legal standards, particularly those related to procedural fairness, data protection, and equality laws.
The findings disclosed in this opinion may empower asylum seekers to challenge decisions made while using AI tools in their evaluations. The Open Rights Group emphasizes that this could lead to legal actions from asylum applicants who suspect AI has influenced their eligibility for protection in the UK.
Specifically, the Home Office is utilizing two AI tools in the asylum application process. The Asylum Case Summarisation (ACS) tool compiles and summarizes information from applicants’ interviews, while the Asylum Policy Search (APS) tool helps caseworkers find relevant country-of-origin information.
The analysis highlights that these AI tools generate new text for decision-makers to review, rather than merely organizing pre-existing information. This process may inadvertently filter out essential facts that could impact legal obligations regarding refugee status assessment. Importantly, neither the asylum-seeker nor their legal representatives are informed that AI technology is being utilized in their application.
Concerns about the accuracy of these AI tools are also raised, noting that an AI pilot study revealed the ACS tool produced erroneous summaries 9% of the time, with 5% of APS users unsure of its reliability. Moreover, there’s insufficient publicly available information regarding the evaluation of these tools’ accuracy or the safeguards implemented to prevent erroneous outcomes in asylum decisions.
The legal opinion insists that the Home Office has an elevated responsibility to scrutinize the performance and implications of these AI tools before they are used in asylum assessments. Failure to adequately evaluate their accuracy, bias, and potential alternative solutions could lead to a breach of their Tameside duty of inquiry.
Additionally, there are concerns about how reliance on AI-generated summaries might impair decision-makers’ reasoning. Established public law principles dictate that caseworkers must consider all relevant evidence, including applicants’ testimonies and country-specific information. If decision-makers lean too heavily on AI summaries without fully reviewing the underlying evidence, crucial considerations may be overlooked, risking the integrity of asylum determinations.
Another significant concern is that inaccuracies in AI-generated summaries could result in decisions based on false information. This issue is exacerbated by the absence of mandatory procedures for caseworkers to verify AI outputs against original documents. As applicants are not given access to these summaries, they lack the opportunity to correct potential mistakes.
Given the serious implications surrounding asylum decisions, transparency is vital for ensuring procedural fairness. The opinion asserts that asylum seekers must be informed when AI tools are employed in their cases, and they should access the summaries generated by the AI.
The authors conclude: “[G]iven the gravity of the consequences for asylum-seekers if their claims are determined on the basis of inaccurate information and the nature of the interests at stake, we consider that – as a matter of procedural fairness – asylum-seekers have a common law right to be informed that AI is being used in the determination of their claims, how it is being used, and to be provided with the output of the AI-generated summaries.” They stress that this is especially pertinent in the case of the ACS tool that processes sensitive personal information.
The authors of the opinion also underline data protection and equality concerns stemming from the use of AI in the asylum decision-making process. The use of the ACS tool involves processing highly sensitive personal information that falls under UK GDPR regulations, necessitating transparency, accuracy, and access provisions.
Furthermore, the lack of a published Equality Impact Assessment means that the Home Office cannot demonstrate compliance with the Public Sector Equality Duty, nor can it prove that potential discriminatory effects of these tools have been addressed. Additionally, current oversight by civil society and regulators, such as the Independent Chief Inspector of Borders and Immigration, is limited, which undermines accountability and public scrutiny.
In light of these findings, Robin Allen KC and Dee Masters from Cloisters Chambers state: “If AI tools are influencing asylum decisions, there must be full transparency about how those systems operate and how their outputs are used. Without that transparency, it becomes extremely difficult to ensure that decisions affecting fundamental rights are lawful and fair.”
The implications of this legal opinion could reshape the landscape of asylum decision-making and ensure a more just and transparent process for individuals seeking protection in the UK.
