Amazon Announced a New ‘Private Investigator’ AI Model

Amazon’s AI ‘Private Investigator’ flags damaged products, improving the customer experience and preventing waste. AI-enabled technology ensures customers receive products in the condition they expect, and is helping Amazon further its sustainability efforts

Inside Amazon fulfillment centers across North America, millions of products ranging from dog food and phone cases to T-shirts and books pass through imaging tunnels every day, where one artificial intelligence (AI) model named “Project P.I.,” which stands for “private investigator,” uses detective-like tools to scan items for defects. The goal? To ensure customers are delighted by every order they receive.

Using a combination of generative AI and computer vision technologies, Project P.I. is able to uncover defects, like damaged products or issues like wrong color or size, before products reach customers. In addition, Project P.I. can help identify the root cause of issues, enabling preventative measures upstream to prevent them from happening again. At the sites where the system is available, it has proven adept at sorting through the millions of items that pass through the tunnels each month and accurately identifying product issues.

Week’s Top Read Insight:10 AI ML In Supply Chain Management Trends To Look Out For In 2024

How Amazon’s ‘Private Investigator’ works

Before an item ships to a customer, it travels through an imaging tunnel, where Project P.I. uses computer vision to scan the product and evaluate the images to detect any defects, like a bent book cover. If a defect is found, Amazon isolates the product so it is not shipped to a customer, and investigates further to determine if there is a wider issue with similar items.

Amazon associates—who review the items Project P.I. flags—then decide whether the item is eligible to be resold at a discounted price as part of Amazon’s Second Chance site, donate it, or find another use for it. The model serves as an extra pair of eyes for Amazon associates and is already helping to enhance manual inspections at several fulfillment centers in North America. The technology is expected to expand to additional sites throughout 2024.

“We want to get the experience right for customers every time they shop in our store,” said Dharmesh Mehta, vice president of Worldwide Selling Partner Services at Amazon. “By leveraging AI and product imaging within our operations facilities, we are able to efficiently detect potentially damaged products and address more of those issues before they ever reach a customer, which is a win for the customer, our selling partners, and the environment.”

Read: 10 AI ML In Data Storage Trends To Look Out For In 2024

Ensuring a more sustainable customer experience

Project P.I.’s work isn’t just part of Amazon’s customer-obsessed culture. It’s also one of many ways the company is using AI innovation to help infuse its commitment to address climate change into the customer experience.

Preventing damaged or defective items from reaching customers is critical to a positive customer experience, and it’s also important for the planet. Accidentally shipping imperfect items can result in unwanted returns, which can lead to wasted packaging and unnecessary carbon emissions from additional transportation.

“Amazon is using AI to reach our sustainability commitments with the urgency that climate change demands, while also improving the customer experience,” said Kara Hurst, vice president of Worldwide Sustainability at Amazon. “AI is helping Amazon ensure that we’re not just delighting customers with high-quality items, but we’re extending that customer obsession to our sustainability work by preventing less-than-perfect items from leaving our facilities, and helping us avoid unnecessary carbon emissions due to transportation, packaging, and other steps in the returns process.”

What’s next: Preventing future errors

In parallel, Amazon teams are leveraging a generative AI system that uses a Multi-Modal LLM (MLLM) to investigate the root cause of negative customer experiences. When we learn of a defect from the customer that we missed to identify, we use that to understand the cause and continuously improve the system. The system first reviews customer feedback and then analyzes images taken from Project P.I. in fulfillment centers and other data sources to confirm what led to the problem.

For example, if a customer contacts Amazon because they ordered twin-size sheets but received king-size ones, the system cross-references that feedback with fulfillment center images and asks questions like, “Is the product label visible in the image?” and “Does the label read king or twin?”

This same technology is poised to help Amazon’s selling partners by making data on defects more easily accessible. For example, if a selling partner accidentally put stickers with the wrong size on a product, Amazon would communicate the issue to help prevent the error from happening again. More than 60% of sales in Amazon’s stores are from independent sellers— most of which are small and medium-sized businesses—who provide a vast selection of amazing products, competitive prices, and convenience for consumers. By reducing the number of defective products that are shipped to customers, we’re also reducing the overall number of returns. Project P.I. is a great example of our focus on improving the customer and selling partner experience.