Esperanto Technologies Announces RISC-V Industry Milestone of Generative AI Models Running on ET-SoC-1

Initial Work Includes Running a Range of Large Language Models, Including Meta’s Open Pre-Trained Transformer Generative AI Model, Running as Low as 25W for AI Inferencing on ET-SoC-1

Esperanto Technologies, the leading developer of high performance, energy-efficient artificial intelligence solutions based on the RISC-V instruction set, announced that it has ported and is running a range of Generative AI models on its low power RISC-V hardware. Esperanto plans to provide access to researchers in the RISC-V community as part of the company’s mission to help “democratize AI” for the broader industry and help accelerate development of Generative AI technology on RISC-V.

AiThority Interview Insights: AiThority Interview with Brad Anderson, President of Product and Engineering at Qualtrics

“RISC-V offers unparalleled opportunities for collaboration and customization, making it ideally suited for this next wave of AI innovation”

This development is part of Esperanto’s strategy to extend the benefits of RISC-V technology in AI and general-purpose applications from cloud to edge. Areas of focus for Esperanto’s Generative AI efforts include use cases for large language models (LLMs) where using low power hardware with improved total cost of ownership (TCO) compared to existing offerings are key.

In particular, several versions of Meta’s Open Pre-Trained Transformer (OPT) model are now running on Esperanto’s hardware at multiple precision levels and context sizes with power levels as low as 25W per chip for inferencing. The rapid porting and bring-up of the OPT models onto ET-SoC-1 silicon were enabled by Esperanto’s machine learning software development kit, which is currently in use by the company’s commercial customers. Researchers who are granted access to Esperanto’s solutions for R&D purposes will be required to comply with Meta’s open-source license as well as other program terms.

“Generative AI is one of the latest advancements in machine learning, and we are pleased to contribute elements of our efforts in the area of large language models to the RISC-V research community,” said Art Swift, president and CEO at Esperanto Technologies.

“Penguin Computing is proud to be a strategic partner of Esperanto, helping to accelerate the deployment of high performance, low power Generative AI solutions,” said Thierry Pellegrino, president of Penguin Solutions and senior vice president of Intelligent Platform Solutions at Smart Global Holdings. “We are excited to be working with Esperanto to extend the deployment of its RISC-V solutions to a broader set of customers that are searching for low power AI inference solutions and reduced total cost of ownership.”

“The growing interest in RISC-V signals an important inflection point for the semiconductor industry. This development in the area of Generative AI will further accelerate the evaluation of RISC-V by researchers looking for breakthroughs in this exciting field, and by developers seeking to increase their competitive differentiation beyond what existing platforms can provide,” said Ben Bajarin, CEO and principal analyst at Creative Strategies.

Read More about AiThority Interview: AiThority Interview with Ajay Sathyanath, Chief Technology Officer at Madison Logic

“RISC-V offers unparalleled opportunities for collaboration and customization, making it ideally suited for this next wave of AI innovation,” said Calista Redmond, CEO of RISC-V International. “Esperanto is one of the companies leading the charge in this space, pushing the limits of performance and power-efficiency to make Generative AI development more accessible.”

For commercial customers, Esperanto is currently shipping AI evaluation servers which deliver high performance combined with high energy efficiency and low TCO. Available in a standard 2U-high form factor, each Esperanto evaluation server includes dual Xeon host processors and either 8 or 16 ET-SoC-1 PCIe cards. Each Esperanto PCIe card has over 1,000 64-bit RISC-V CPUs with attached vector/tensor units, delivering up to 16,000 RISC-V CPUs per server. Esperanto’s evaluation servers enable customers to obtain performance and power data from running a variety of industry standard AI models, as well as the ability to bring their own models and data.

 Latest AiThority Interview Insights : AiThority Interview with at Brian Sathianathan, Co-Founder and CTO at Iterate.ai

 [To share your insights with us, please write to sghosh@martechseries.com]