Want to Beat FOIA Backlogs? Embrace AI

Last year, federal government workers processed a record 1.1 million Freedom of Information Act (FOIA) requests. Despite this herculean effort, backlogs across agencies still grew to more than 200,000 cases, nearly tripling since 2012.

At the same time FOIA officers are swimming upstream against requests—increasing both in volume and complexity—the government is amassing ever-increasing amounts of data. According to some estimates, government data within the average agency is expected to grow more than 400% in the next five years.

If the status quo persists, the federal FOIA program will remain unsustainable, and backlogs will persist, damaging transparency and public trust. A recent report from the Government Accountability Office (GAO), a watchdog group, underscored the need for agencies to have more specific plans in place to beat backlogs.

While staffing and underfunding challenges are part of the problem, the reality is that no realistic number of new hires can complete the mounting task. The federal FOIA program can only function properly in the future if government agencies meaningfully embrace artificial intelligence, specifically machine learning programs overseen by humans, to sort through mass volumes of data and process hundreds of millions of records efficiently.

Some agencies have already begun to launch pilot programs, including a declassification effort at the State Department in which human-trained algorithms achieved accuracy 97-99% of the time and cut staff hours by 60%. Now is the time for agencies across the government to deploy automation projects, building proof points and trust in the technology that will encourage widespread adoption—and create the ability to beat the FOIA backlogs for good.

Why machine learning is the answer to FOIA’s troubles

Machine learning’s potential is enormous for FOIA, a program that relies on a series of time-consuming and often repetitive tasks that are ripe for automation. So why isn’t it being more widely adopted?

Within the public sector, integrating new technology can face multiple hurdles—from budget constraints to unease around security, integration with existing systems and team adoption. On top of that, many are weary of AI at large.

It often takes a dedicated government changemaker to champion innovative ideas, as was the case for the State Department project. The best chance these changemakers have in building confidence in AI is by working with trusted partners to set up specific, tailored and conservative use cases using machine learning, a technology that’s been shown to be effective and safe across industries, from healthcare to finance. For example, an agency may implement an algorithm that immediately identifies whether there are exact duplicates of request. This simple automation alone can save hours of staff time.

Hesitant agencies can also lean on guidance from the White House, including an Executive Order from 2023 that outlines how to deploy AI in a safe, secure and trustworthy manner. The White House also published a Blueprint for an AI Bill of Rights, which lays out considerations agencies should take regarding the effects of AI use on society and individuals, as well as notes their obligation to ensure models are free from discrimination, maintain data privacy, and promote accessibility.

Use cases for incorporating machine learning in FOIA

Equipped with guidance from the White House, certain agencies have launched pilot projects that incorporate machine learning into nearly every step of the FOIA process, including from the moment a requester hits submit.

For instance, in another project at the State Department, AI algorithms analyze language within FOIA requests and generate a tailored response directing requesters to already published information or suggesting narrowing the scope for faster response. At a broader level, FOIA.gov’s new FOIA Wizard tool lets users input the records they’re looking for, and AI directs them to the correct agency holding those documents.

Once a request is submitted, AI can analyze language within the requests to route records to the correct person quickly. It can also identify duplicate or vexatious requests, which are increasingly bogging down the system.

The biggest time savings occur during the actual reviewing and redacting records. AI can scan massive amounts of data quickly.  In the State Department example, AI algorithms learned from decisions made by human redactors to identify certain sensitive information and sort records into three buckets for human review.

AI can also help government agencies in their quest to more proactively publish data and records to promote transparency and reduce the number of incoming requests. AI is becoming essential to these open data projects, such as at the National Archives, where algorithms are consolidating classified presidential records into the National Declassification Center and improving searches of the 300 million documents within the National Archives Catalog.

AI can speed up mundane tasks, but human oversight will remain essential

As agencies begin to experiment with AI, it’s crucial they take careful steps to ensure technologies are adopted responsibly. AI is a tool that can make FOIA request fulfillment more efficient, but it is not an end-to-end solution or replacement for human judgment. Professional FOIA expertise and oversight remain indispensable at every stage of the process.

As requests continue to grow in complexity and volume, AI can automate repetitive and tedious components of the job so that FOIA officers can spend their energy acting as final decision-makers. As long as humans continue to provide expertise and careful oversight at every stage, AI is poised to revolutionize the FOIA process—and make backlogs a thing of the past.

[To share your insights with us, please write to psen@itechseries.com]