OctoML Accelerates ML Innovation Across Broad Array of Arm Hardware and Embedded Environments

OctoML has announced a collaboration with Arm to deploy next-generation machine learning (ML) applications and models across its suite of hardware. The partnership enables Arm partners to upload machine learning models to the OctoML platform and receive optimized and packaged versions of the model fine-tuned to Arm® hardware.

Arm is also contributing engineering effort to Apache TVM which provides companies developing ML on Arm technology one unified software stack to deploy models seamlessly across microcontrollers, GPUs, NPUs and CPUs.

What’s OctoML?

OctoML is a machine learning acceleration platform based in Seattle, Washington. OctoML aims to accelerate model performance while enabling seamless deployment of models across any hardware platform, cloud provider, or edge device. The company’s investors include Addition, Madrona Venture Group, and Amplify Partners. OctoML was founded by creators of open-source Apache TVM, CEO Luis Ceze, CTO Tianqi Chen, CPO Jason Knight, Chief Architect Jared Roesch, and VP of Technology Partnerships Thierry Moreau.

Apache TVM is a software framework that provides a unified layer between the leading machine learning frameworks—like PyTorch, TensorFlow—and the vast array of hardware solutions available. This innovation means that ML models can be deployed anywhere from cloud to edge to mobile.

Arm’s open source efforts in providing Apache TVM support to Arm Cortex®-A processors, Cortex-M processors, Arm Mali™-GPUs and Arm Ethos™-NPUs, ensures that data scientists, software and embedded developers can now use a single software stack that works on the billions of chips based on Arm technology.

“Optimizing and deploying machine learning workloads across a diverse array of hardware is difficult, especially when working with embedded environments,” said Luis Ceze, CEO and co-founder, OctoML.

Luis added, “Arm has been an incredible partner in the open source Apache TVM community, especially in making ‘TinyML’ a reality on devices that lack a fully-fledged operating system. It’s great to see our collaboration with Arm now extend to our SaaS platform where their customers can both speed up deploying models and also enable new ML-based use cases that were not previously viable.”

Built on the Apache TVM open source framework, OctoML’s Platform provides an automation framework that optimizes trained models to achieve optimal performance across a breadth of hardware endpoints and cloud services—without compromising accuracy. The platform readily addresses the challenge of optimizing ML models to match the resources at the edge, which opens up opportunities for a new wave of intelligent apps.

“Arm has been committed to Apache TVM since its earliest days and we believe it is a key enabling technology for the ML ecosystem,” said Steve Roddy, Vice President of Product Marketing, Machine Learning at Arm. “OctoML and the TVM community have excelled at pushing the boundaries of where ML can run, and our continued collaboration with partners like OctoML will empower the industry to develop innovative new AI applications.”