Natural Language Processing: The Technology That’s Biased

What is Natural Language Processing?

Natural Language Processing (NLP) refers to building machines that can understand and respond to voice data with their own text and speech. Natural Language Processing falls under the umbrella of Artificial Intelligence (AI) and recent models like the Bidirectional Encoder Representations from Transformers (BERT), Generative Pre-Trained Transformer 3 (GPT-3) and Pathways AI Language Models (PaLM) have made accurate human-machine communication possible.

These Large language Models (LLMs) are trained on massive volumes of text with billions of parameters and are able to understand and answer reading comprehension questions as well as generating new text such as a summary.

Put simply, LLMs are trained to predict the next words in a sentence, such as by extending the autocomplete feature in messaging applications. But they can do much more, for example question answering, translation, image captioning, human-level dialogue agents, entity linking, or even data cleaning (for mixes of structured and unstructured data).

NLP is already being used to automate some human tasks (RPA – robotic process automation), however the breath-taking advances in the last 3 years, NLP open new potential for businesses to digitize company knowledge and disrupting incumbent business models.

Top AI ML Blog: How AI English Language Training Tools can be Leveraged Ahead of the Summer Travel Surge

NLP has multiple potential use cases, one of which is website chatbots and customer service.

NLP has become vital in some companies’ day to day activities and acts as a decision maker. Increasingly NLP is used as a strategic differentiator, where data driven decision making leveraging specialized knowledge is used to process claim, inspect, and ensure compliance, review contracts and risks, extraction and repudiation of events. However, issues like bias can arise with NLP and this must be acknowledged and mitigated in order to make it a more sophisticated system.

Companies are increasingly turning to this new technology to help them pick apart unstructured data whose potential would otherwise go unharnessed. It offers businesses a way to organize information, so it is easier to understand, more accessible and therefore more valuable.

What sets AI systems in NLP apart is that other than quantitative data, it doesn’t focus on the actual data, the words, but rather on the semantic interpretation of it. And that semantic interpretation is very context sensitive. And that sensitivity makes NLP especially prone to inherent bias.

What can biases look like in NLP?

Increasing Customer service efficiency, customer satisfaction, speed to resolution, churn prevention and up-sell are tremendous opportunities for the NLP technologies.

One of the keys for that is the intent analysis, whereby the AI attempts to understand the problem formulation or the intent of the client calling to (semi-) automate the request. The degree to which the AI system is able to understand the context of the request and create analysis which addresses this is the measure of good and bad AI.

NLP is used increasingly to automate decisions and provide valuable insights. Especially due to the context sensitive interpretive layer of the NLP systems, inherent or underlying bias can limit their power and even lead to wrong decisions.

Bias can be pervasive. AI, being built on statistical models has an inherent tendency towards bias.

Any quantitative data has bias in the way it’s collected (or not) and processed. Most commonly reporting bias happens, when the frequency of data doesn’t represent reality. Selection bias on the other hand appears, when training data is mainly available for larger groups, and the model then performs bad on the marginalized groups.

One limitation of NLP is that biases continue to exist due to institutionalized processes which can result in certain groups in society being excluded. This can lead to discriminatory, sexist, and prejudiced results. With Responsible AI, Boston Consulting Group is paving the way in AI ethics and addressing these biases and ethical challenges.

Bias can manifest in a variety of different ways. As an example, when the GPT-3 model tried to gender certain job roles, the system would determine that males were doctors and females were nurses. This bias is sexism and is usually embedded in the technological system from previous rules and patterns that it has followed.

Additionally, other biases that occur in machine learning surround racism and religion. Although this is dangerous, the fact we acknowledge it is happening is a step in the right direction – it’s not too late to eliminate these biases.

Avoiding and regulating bias in Natural Language Processing

How can a machine be biased?

It happens due to poor data quality or the incorrect use of data. The data must be representative, as those that aren’t frequently represented in these systems are then not included. These systematic errors then manifest into models, making them often discriminatory and not representative of the entire population.

Solving biases can be achieved with several vital components but avoiding it in the first place is crucial. Raising awareness among those involved in operations and development eliminates bias at the earliest stages.

Bias can happen as a result of methodological error or ignorance, but it can better be avoided by comprehensive versioning, documentation, access controls and logging of all AI input variable (features), AI model codes, AI result and strict data governance. These are all extremely important.

Some of the best practices we employ to control bias are:

• Understanding the context of data collection
• Training on representative data, testing models on broad representations
• Designing AI models with inclusion in mind
• Use and be wary of human bias in decisions and process
• When presenting results, leverage explainable AI concepts as much as possible to provide model insights

AI for AI

Another component in solving the issue is continually detecting bias and correcting it, which algorithms must be trained to do. It is essential to always obtain a weighting of the variables as in the data without bias. This gives a more accurate representation of data.

One way of achieving this is through the use of AI for AI – whereby an AI algorithm is developed to detect possible bias in its own models. While it can’t correct it, it can be trained to monitor drop-off levels of specific bias indicators and so pinpoint to the algorithmic pipeline possibly introducing the bias.

We have recently used such an NLP and AI for AI techniques to detect societal bias – for example by analyzing language differences from feedback managers give about their employees. While these biases are often reflective of underlying societal biases and therefore often unconscious, AI can help reduce such bias to make language more inclusive, actionable by (in real time) by bringing it to awareness.

An innovative piece of technology for Natural Language Processing

Natural Language Processing is an impressive piece of AI technology that will constantly evolve. It manages complex systems, provides us with insights and boosts customer relationships and their experiences. We should continue to use this system with caution as biases can sometimes be dangerous, and we should look to regulate them before they become embedded in a pattern that harms various social groups. It is also essential to raise awareness of the biases and ways in which we can educate those who work within the system.

The future of NLP is exciting and as AI technology continues to rapidly advance, so will Natural Language Processing; tasks will become simpler and it will become a widely used system for data interpretation and processing, aiding numerous organizations and companies.

Biases can be overcome and it’s never too late to start tackling them.

Recommended: DeepNash and the World of Model-free Multi-agent Reinforcement Learning (RL)

[To share your insights with us, please write to sghosh@martechseries.com]