Addressing AI Bias in Online Identity Verification with 5 Critical Questions
When bias becomes embedded in machine learning models, it can have an adverse impact on our daily lives. It’s exhibited in the form of exclusion, such as certain groups being denied loans or not being able to use the technology. As AI continues to become more a part of our lives, the risks from bias only grow larger.
In the context of facial recognition, demographic traits such as race, age, gender, socioeconomic factors, and even the quality of the camera/device can impact software’s ability to compare one face to a database of faces. In these types of surveillance, the quality and robustness of the underlying database is what can fuel bias in the AI models. Modern facial recognition software uses biometrics to map facial features from a photograph or video. It then compares the information with a database of known faces to find a match (this is known as a 1:n match).
In 2018 the American Civil Liberties Union found that Amazon’s AI-based Rekognition facial recognition software falsely matched 28 U.S. Congress members with a database of criminal mugshots and that nearly 40% of the false matches were of people of color, even though they made up only 20% of Congress.
Demographic bias is also an issue for facial authentication which relies on an individual’s unique biological characteristics to verify that she is who she claims to be.
Facial recognition and facial authentication, however, are two very different kinds of animals.
Most leading identity verification solutions leverage AI and machine learning to assess the digital identity of remote users. Unfortunately, these algorithms are also susceptible to demographic bias. But, this type of bias has nothing to do with the underlying database because this type of authentication doesn’t perform 1:n-type searches against an established database of images.
It’s a whole different kind of AI that is used to solve a very different business problem — if the person is who they claim to be when creating new accounts online.
AI algorithms are used to compare the selfie of a customer with the photo in their identity document, and bias can creep into algorithms in several ways. AI systems learn to make decisions based on training data, which can include biased human decisions or reflect historical or social inequities, even if sensitive variables such as gender, race or sexual orientation are removed.
Here are five critical questions to ask solution providers to determine how well they are addressing demographic bias:
How Big and Representative Is Your Training Database?
Machine learning models use AI training datasets to learn how to recognize patterns and apply technologies such as neural networks so that the models can make accurate predictions when later presented with new data in real-world applications. When it comes to AI, size matters. The larger and more representative the training data set, the better its ability to withstand the introduction of demographic bias.
Where Did the Data Come From to Create the Training Data Sets?
When companies don’t have enough of their own data to build robust models, they often turn to third-party data sources to backfill this gap, and these purchased datasets can introduce unintentional bias. For example, a dataset of images of ID documents captured under perfect lighting conditions with high-resolution cameras is not representative of ID images that are captured in the real world. Not surprisingly, AI models built on unrealistic models will struggle with IDs that were captured in dim lighting. Algorithms that were built with real-world production data, on the other hand, will contain documents with real-world imperfections. As a result, these AI models are more robust and less susceptible to demographic bias.
How Were the Data Sets Labeled?
In most AI projects, classifying and labeling data sets takes a fair amount of time, especially with enough accuracy and granularity to meet the expectations of the market. In the context of identity verification, labeling is how the ID documents are tagged. If the photo of the ID has been manipulated, then the document will be tagged as fraudulent with photo manipulation. If the picture of the ID has an excessive glare, then the label should reflect this characteristic. If the wrong labels are used when tagging individual identity verification transactions, the AI models will bake that information into the algorithms, making the models less accurate and more subject to bias.
Some solution providers outsource or crowdsource the tagging exercise while others insource it to experienced agents who are instructed how to tag verification transactions to optimize the learning curve of the AI models. Naturally, the insourcing models generally result in more accurate models.
What Type of Quality Controls Are in Place to Govern the Tagging Process?
Unfortunately, a lot of this bias is unconscious because many solution providers do not necessarily know when they’re making the algorithm that it’s going to make incorrect outcomes. That’s why there needs to be some quality control injected into the process. In the identity verification space, there’s no substitute for having a trained crew of tagging specialists who know how to accurately tag individual ID transactions and auditing processes in place to check their work.
How Diverse Is the Team Developing the Algorithms?
Reducing bias is also about the people who are developing the AI algorithms and tagging the datasets. It’s not unfair to ask about the composition of the AI team. Ideally, the AI engineers and data scientists come from a variety of nationalities, genders, ethnicities, professional experiences and academic backgrounds. This diversity helps ensure that different perspectives are brought to bear on the models being created which can help reduce some demographic bias.
There is a growing concern that demographic bias in a vendor’s AI models could reflect negatively on a company’s brand and possibly raise possible legal issues, especially when economic decisions are dependent upon the accuracy and reliability of those algorithms. Believe it or not, these algorithms can result in some types of customers being unfairly rejected or discounted, which translates to lost business and downstream opportunities. That’s why it’s increasingly important to understand how vendors measure demographic bias and what measures they’re taking to address it.