AVCC and MLCommons Join Forces to Develop an Automotive Industry Standard Machine Learning Benchmark Suite

AVCC, a global autonomous vehicle (AV) consortium that specifies and benchmarks solutions for AV computing in the automotive industry and MLCommons, an open, global engineering consortium dedicated to making machine learning (ML) better for everyone, are announcing the industry’s first Automotive Benchmark to take specifications and benchmarks and move them through to open-source software and certification. Initial participation in the benchmark includes Arm (co-chair), Bosch, cTuning Foundation, KPIT, NVIDIA, Red Hat, Qualcomm, Inc. (co-chair), Samsung Electronics, and other industry leaders.

Recommended: AiThority Interview with Dorian Selz, Co-Founder & CEO at Squirro

The use of ML, especially for the perception system in autonomous vehicles, has increased dramatically over the last few years, unlocking new innovations such as automatic lane-keeping that make roads safer. As vehicles are becoming more intelligent, the industry needs a common set of ML benchmarks to enable fair and accurate comparisons between different technologies.

The goal of the partnership is to develop an industry-standard Automotive Benchmark Suite for use by OEMs and automotive suppliers using AI/ML Deep Neural Network (DNN) technology. The Suite will build upon the AVCC AI/ML Benchmark Technical Reports and the MLPerf™ benchmark suites developed by MLCommons. This common set of benchmarks will also help guide the industry’s collective engineering around future platforms, accelerating the development of new capabilities. Development of the automotive benchmark suite will occur in a multi-phase approach, with the goal of delivering open-sourced software solutions by the end of the year.

“We are excited to bring our expertise in machine learning to the automotive industry, working together with the AVCC,” said David Kanter, executive director of MLCommons. “We believe this opportunity will help spur innovation and encourage standards around increasingly intelligent and capable vehicles.”

AiThority: AiThority Interview with James Rubin, Product Manager at Google

“OEMs and automotive suppliers are currently challenged to understand a solution’s compute performance and system resource requirements,” said Armando Pereira, president of AVCC. “The work of our joint task force will finally give the industry an easy and certified source of information these players need to make significant decisions on their selection of suppliers and project investment.”

The technical work has begun, and broad participation from the industry is encouraged. The Automotive Benchmark initiative seeks input from all AVCC and MLCommons members, along with other industry players. If your organization is using (or looking to use) automotive AI technology, you are encouraged to participate in this joint initiative to support the development of the new Standard Automotive Benchmark Suite.

Read: AiThority Interview with Sumeet Arora, Chief Development Officer at ThoughtSpot

[To share your insights with us, please write to sghosh@martechseries.com]