New Monetate Research Proves the Power of AI in Testing and Segmentation

Findings Derived from Two Billion Personalized Experiences Confirm Value of AI over Manual A/B Testing

Monetate, the worldwide leader in personalization, released new research revealing the proven efficacy of AI-powered methods over traditional approaches to testing, segmentation and 1-to-1 personalization. Leveraging data from more than two billion personalized experiences, Monetate uncovered that less than 10 percent of traditional A/B tests – which measure the impact of changes to user experiences – reach statistical significance. Not only does this indicate that manual A/B tests do not deliver the conclusive results marketers require to generate better user experiences and increase engagements, but it also suggests the opportunity for marketers to explore AI as a better alternative.

“Our data findings underscore why marketers should be considering how AI can improve their testing practices and ultimately increase engagement”

Marketers have long relied on manual A/B testing to improve customer experiences and deliver strong business results. While this method for website optimization has created a strong foundation for personalization, the emergence of AI for testing and segmentation is prompting marketers to reevaluate their engagement toolkit.

Read More:5G NR: Inseego and Verizon Demonstrate Virtual Reality Telemedicine over 5G at Qualcomm Snapdragon Technology Summit in Maui

To help marketers understand whether AI-enabled tools are worth investing in, Monetate examined data from the Monetate Intelligent Personalization Engine™ to uncover how manual methods compare to AI solutions. The AI counterparts examined during the research were Majority Fit Experiences (MFEs) – which monitor the results of an A/B test and gradually direct new traffic to the more successful content – and Individual Fit Experiences (IFEs) – which use first and third party data to inform 1-to-1 individualized content decisions that provide each site visitor with a unique combination of content that is most likely to be relevant to them. The results confirmed that traditional testing often falls short, with the statistical significance of IFEs representing a more than 290 percent increase compared to traditional A/B tests.

Read More: Wind River Teams With Renesas to Advance Autonomous Driving Applications

Additional key findings include:

• 84 percent of MFEs outperformed their corresponding A/B test relative to a specific goal metric, indicating that AI personalization solutions offer increased potential to optimize for business KPIs
• The difference in KPIs was particularly noticeable in Revenue Per Session, in which the lift of the MFE over the A/B test exceeded nine percent, and New Customer Acquisition, which saw a lift of more than 15 percent
• In experiences that were designed to encourage user action—such as email signups or link clinks—the IFE outperformed a non-personalized control group 65 percent of the time, showing an average lift in click rates of more than 41 percent

“Our data findings underscore why marketers should be considering how AI can improve their testing practices and ultimately increase engagement,” said Brian O’Neill, CTO of Monetate. “While traditional A/B testing still deserves a place in web optimization programs, supplementing those practices with AI-powered personalization will enable marketers to more effectively meet their goals without draining resources, creating a better framework to focus on customer lifetime value.”

Read More: Virtual Assistant Technology – Empowering the Human Workforce of Tomorrow