The Challenges of Generative AI in Supply Chain and Procurement
This post is co-authored by Edmund Zagorin, Founder and CSO, Arkestro, and Alan Rice, VP of Global Procurement, Primo Water Corporation
Generative AI has captured the imagination of business leaders around the world. However, given that the technology may soon be performing tasks or heavily supporting human decision-making in operational areas like supply chain and procurement, there are significant risks to consider as we navigate into these new waters.
The truth is that without some degree of human intervention, AI could wreak havoc on supply chains. For example, if AI is able to detect or preempt a run on a specific commodity, food or medicine, then it could also trigger an autonomous buying cycle ahead of that run, thereby exacerbating it. In the worst-case scenario, this could drive shortages in life-saving supplies like cancer medicines which have proven to be vulnerable to supply chain fragilities.
Of course, AI can also be a force for good in supply chain and procurement functions, but there are several dangers that could arise if it’s left unchecked. For example:
Purchase Order Fraud and Supplier Selection Fraud –
Procurement and Accounts Payable teams have gotten used to a type of fraud committed by hackers where they impersonate a procurement organization and ask suppliers for sensitive financial information by sending them a fake Purchase Order. AI can be used to generate fake purchase orders that look authentic much more quickly, thus dramatically multiplying the costs of this kind of attack.
Additionally, suppliers might have to worry about the accuracy or fairness of an AI selecting the winning supplier as a result of a sourcing process. For example, the customer or maybe hackers could potentially manipulate the AI system to favor certain suppliers over others, meaning a potential loss of a lucrative contract.
Release of Sensitive Intellectual Property or Private Information via AI –
Unless handled properly, data entered into generative AI tools can be viewed by others using the service. While the spread of this information may happen without malicious intent, it is still very alarming that sensitive IP and trade secrets could potentially be stolen by AI. That’s why Samsung and other companies have recently banned or paused use of generative AI technology by their employees while investigating potential risks.
Algorithmic Bias, Lack of Transparency in Supplier Recommendations –
Many new AI systems provide algorithmic recommendations for suppliers. These recommendations incorporate training data that may show bias toward historically marginalized suppliers and run against the stated intentions of Supplier Diversity programs operated by many procurement and supply chain organizations. If biases present in the data are not properly addressed, the AI could perpetuate or even exacerbate these biases, leading to unfair outcomes and legal liability. It may also work against the efforts to develop new suppliers or suppliers from strategically relevant and historically underrepresented communities.
For example, one researcher on algorithmic bias found that algorithms used in making hiring decisions would down-rank candidates whose application contained the word “woman”.
Based on training data, the two features that most predicted the candidate would be hired were if their name was “Jared” and if they had played lacrosse. These algorithmic recommendations that can now appear inside of systems like SAP and Oracle, as well as powering rankings for all sorts of corporate functions, are also potential legal liabilities. In fact, New York has just adopted a law that will hold companies legally responsible for algorithmic bias in automated hiring tools that are deemed to run afoul of labor laws or create the basis for an Equal Opportunity Employment Office complaint.
Given the similarities of automated tools that are used in selecting the right person for a job and selecting the right supplier for a contract or purchase order, it is troubling to imagine how these bias scenarios could play out against the interests of procurement and supply chain groups. Making the AI’s decisions as transparent as possible will be critical in order to avoid these pitfalls.
Disruption in Critical Supplies Caused by AI-induced “Panic Buying” Demand Surges –
The classic model of a “run on the bank” is a panic induced by the fear caused by the need to access a needed resource. Earlier this year, we saw this dynamic play out dramatically in the Silicon Valley Bank collapse, but the fundamental structure of the disruption scenario is no different than many other common shortages.
It’s easy to imagine that as AI becomes more fused with autonomous systems that monitor markets and conduct buy cycles for repetitive transactions – similar to algorithmic trading in financial markets – that the risk of causing a shortage by trying to “beat the market” ahead of a price spike could unintentionally cause a disruption. For categories such as food, energy and medicine, the consequences of such disruptions could prove immensely painful to procurement and supply management teams, not to mention communities and citizens.
Understanding the consequences of these autonomous buying and selling agents in a supply chain will be challenging because of self-fulfilling prophecies.
For example, let’s imagine that a company that is worried about hurricanes sets up an AI agent to monitor satellite imagery of weather patterns in order to purchase bananas and other perishable fresh fruit before a hurricane makes land. Now let’s imagine that several companies have configured the exact same agent.
Top AI ML News: Zeta Launches Customer Growth Intelligence, a Snowflake Native App in the Data Cloud
Due to a satellite error, an alert is mistakenly triggered saying: “Hurricane detected! Lead times will soon increase. Recommendation: Increase the size of your banana order.” It is not hard to imagine such a trigger itself causing a temporary banana shortage by suddenly and unexpectedly increasing the aggregate demand for bananas, triggering what supply chain managers refer to as a “bullwhip” effect. This dynamic can be understood as Level 2 Chaos, which Yuval Noah Hari defines as chaos that “…reacts to predictions about it.”
There is certainly a great deal of AI hype floating around, and for many of us in procurement and supply chain, that is nothing new. In fact, there have previously been periods of AI hype and so-called “AI winters” when the rising expectations of total automation met the disappointing reality of the many limitations or drawbacks of current models. And yet, these models are only getting better, and the speed of improvement has never been faster.
We must have internal discussions and form clear boundaries and specific intentions ahead of this technology’s rapid acceleration. Commonsense guidelines and guardrails may help to ensure that AI is used responsibly and effectively in our procurement and supply chains. This approach will also help enhance the benefits that generative AI will provide to organizations, while minimizing potential risks and challenges.
Comments are closed.