Imagine if someone wrongfully convicted of a crime was asked to design the algorithm used by police to convict criminals. Imagine if a young person, newly immigrated to the US, was asked to design the algorithm for admissions at top US universities. Imagine if populations, historically marginalized from the use of your products, were asked to design your products. The chances that the outputs from these algorithms would replicate the same output that they do today are slim to none. That is in many ways what AI and machine learning offers — but rather than having systems that embrace diversity of perspective and opinion, if we aren’t vigilant, we can end up with systems that enforce existing biases at best and actively create brand new biases at worst.
Recommended: Deloitte Report: 83% of Creators Want Multiple Revenue Streams and Higher Share of Income from Brand Sponsorships
Society has managed to steadily progress despite the myriad issues around diversity, and some might argue that slow progress coupled with the benefits offered by AI is good enough. However, I would disagree. While there are plenty of moral reasons for diversity, DEI shouldn’t just mean Diversity, Equity, and Inclusion, it should also stand for Diversity Equals Income. Every time a company uses an algorithm that alienates a user, diminishes an outlier in order to fit a model, tamps down diversity when making a hiring decision, or works in a diversity-blind fashion, dollars are being left on the table — dollars that few businesses can afford to spare.
The first time the negative interaction of technology and race dawned on me was back in the late ‘90s when I – and many of my black friends – found ourselves unable to be properly identified by the face recognition software used by Facebook. As we soon learned, who was in the room doing the programing mattered. The programmers, the majority of whom did not look like us, trained the machines on faces that looked like theirs, leaving those of us with darker complexions as mysteries unable to be identified by computerized eyes. One would think that as the years have progressed, things would have gotten better, but a 2018 study by the National Institute of Standards and Technology (NIST) found that some facial recognition algorithms had error rates that were up to 100 times higher for African Americans than for Caucasians.
Read More: Smart Engines Launches AI Scanner for Handwritten Credit Card Numbers
Sadly, this bias isn’t just found in visual data. A 2019 study by the National Bureau of Economic Research found that algorithms used by credit scoring companies tended to underestimate the creditworthiness of African American and Hispanic borrowers. These algorithms routinely gave these borrowers lower credit scores and higher interest rates.
AI has also ushered us into a new age for HR. All across the world companies are using AI to screen resumes for potential hires, yet AI-powered hiring systems have been found to discriminate against women and minorities. A study by the University of Cambridge found that an AI-powered recruitment tool developed by Amazon downgraded resumes that contained words such as “women,” “female,” and “gender,” and as a result, candidates with female-sounding names were less likely to be selected for interviews.
There were two interconnected problems, which were difficult to solve but needed to be addressed. First, in all these situations, the training set was flawed. If a system is trained on biased information, it will generate and propagate a biased output. In the case of the recruitment tool, it had been trained on resumes submitted to the company over a 10-year period, most of which were from male applicants. Second, those in charge of these systems didn’t value or consider diversity enough to actually encode it in the system.
Like a child learning right from wrong, an AI needs to be taught. In order to deal with the myriad different situations it may encounter, organizations must expose the AI to past examples of right and wrong (or success and failure), which can be redolent with bias against women, immigrants, people with physical or neurodivergence, as well as race and ethnic groups. Currently, since the complexity of the AI’s computations is so high, the best way to check if a system is biased is through testing both the input and the output.
Testing for any sort of sampling bias in terms of a specific characteristic, geography, or demographic marker in what was fed into the system as well as unwanted correlations from what comes out of the algorithm, is critical. This extra step, while relatively simple, is time consuming and let’s face it, time is money.
Ask yourself however, is this enough?
As I said before, Diversity, Equity, and Inclusion is important but not just because it is morally right but because in business Diversity Equals Income.
For example, a 2009 study by the Center for American Progress found that companies with more racial and gender diversity were more likely to have higher profits.
While a 2016 study by the Peterson Institute for International Economics found that companies with more female executives were more profitable. What’s more, a 2018 study by the Boston Consulting Group found that companies with more racially and gender diverse boards of directors were more profitable.
Diverse teams bring a wider range of perspectives and experiences to the table, leading to more creative problem-solving and better decision-making. They can be more effective in understanding and serving a diverse customer base, and they can be more attractive to top talent, leading to higher productivity and innovation. AI provides businesses the opportunity not only to ensure their hiring practices aren’t biased but that their staff has the diversity needed produce the best goods and services for their evermore diverse consumers.
Organizations that rely on AI alone to screen resumes, sift through applications for schools, or make decisions about credit, are making a grave mis-judgement, mistaking the hammer for the carpenter and the car for the driver.
This is similar to a problem I occasionally run into while leading product ideation workshops. Clients get so invested in the rules and procedures of an exercise that they’ll get upset when I throw out the rules and start capturing the ideas that start pouring out. They often want to hold their tongue and risk losing their idea, rather than sacrifice the well laid out rules, that is until I remind them that the exercise is just a tool, and what really matters is the idea.
While it can be powerful, AI is still just a tool and one of many that businesspeople, and consumers have at their disposal. The goal must remain one of creating a more diverse, equitable, and inclusive business environment, so we can create better products, services, and experiences for consumers. Forget that and we leave money on the table, consumers unsatisfied, companies without the best talent, and ourselves exposed to the first competitor smart enough to capitalize on our blind spot.
The smartest companies I’ve worked with are the ones that define their goals first and find the tools to achieve those goals second. As Marshall McLuhan said, “We shape our tools, and thereafter our tools shape us.” We cannot, and must not, allow our tools to shape us if we hope to progress to a more diverse future, let alone a more profitable one for businesses.
Comments are closed.