Untether AI Ushers in the PetaOps Era With at-Memory Computation for AI Inference Workloads

• tsunAImi accelerator card packs 2 PetaOps of performance in a PCI-Express form factor
• Powered by the runAI200 chip, the industry’s first at-memory computation engine offering unrivaled 8 TOPs/W efficiency

Today at the fall Linley Processor Conference, Untether AI unveiled its tsunAImi accelerator cards powered by the runAI devices. Using at-memory computation, Untether AI breaks through the barriers of traditional von Neumann architectures, offering industry-leading compute density with power and price efficiency.

Recommended AI News: Study: 46% of Corporates Are Now Willing to Hire More Apprentices

The Need for Speed

Artificial Intelligence (AI) workloads for inference require increasing amounts of compute resources, far outstripping the gains available to traditional CPU and GPU architectures. The slowing of Moore’s law and the end of Dennard scaling further limits future gains in performance from traditional computing approaches. Solving this dilemma is important – as inference acceleration in the data center, using AI accelerators, is estimated to be a $10 billion market by 2025, according to McKinsey and Company. Untether AI was founded to radically rethink how computation for machine learning is accomplished. In current architectures, 90 percent of the energy for AI workloads is consumed by data movement, transferring the weights and activations between external memory, on-chip caches, and finally to the computing element itself. By focusing on the needs for inference acceleration and maximizing power efficiency, Untether AI is able to deliver two PetaOperations per second (POPs) in a standard PCI-Express card form factor.

“For AI inference in cloud and datacenters, compute density is king. Untether AI is ushering in the PetaOps era to accelerate AI inference workloads at scale with unprecedented efficiency,” said Arun Iyengar, CEO of Untether AI.

The Most Efficient AI Compute Engine Available – runAI200 Devices

Tailored for inference acceleration, runAI200 devices operate using integer data types and a batch mode of 1. At the heart of the unique at-memory compute architecture is a memory bank: 385KBs of SRAM with a 2D array of 512 processing elements. With 511 banks per chip, each device offers 200MB of memory and  operates up to 502 TeraOperations per second in its “sport” mode. It may also be configured for maximum efficiency, offering 8 TOPs per watt in “eco” mode. runAI200 devices are manufactured using a cost-effective, mainstream 16nm process.

“As AI compute requirements continue to explode, new architectures are needed to meet these demands,” said Linley Gwennap, principal analyst, The Linley Group. “Untether AI’s runAI200 devices, with their innovative at-memory compute architecture, break through traditional von Neumann architecture bottlenecks and represent a new breed of AI accelerators.”

Recommended AI News: ClearScore Selects Netacea for Bot Protection

2 PetaOps at the Lowest Price per TOP- tsunAImi Accelerator Cards

tsunAImi accelerator cards are powered by 4 runAI200 devices, providing 2 POPs of compute, more than two times any currently announced PCIe cards. This compute power translates into over 80,000 frames per second of ResNet-50 v 1.5 throughput at batch=1, three times the throughput of its nearest competitor. For natural language processing, tsunAImi accelerator cards can process over 12,000 queries per second (qps) of BERT-base, four times faster than any announced product.

“When we founded Untether AI, our laser focus was unlocking the potential of scalable AI, by delivering more efficient neural network compute,” said Martin Snelgrove, co-founder and CTO of Untether AI. “We are gratified to see our technology come to fruition.”

Simple, Automatic Tool Flow – the imAIgine Software Development Kit

Until now, making neural networks perform optimally has been a manual process. The Untether AI imAIgine Software Development Kit (SDK) provides an automated path to running networks at high performance, with push-button quantization, optimization, physical allocation, and multi-chip partitioning. The imAIgine SDK frees data scientists from having to perform low-level optimization tasks and instead spend time where it matters to them – crafting their models. The imAIgine SDK also provides an extensive visualization toolkit, cycle-accurate simulator, and an easily integrated runtime API.

Availability

The imAIgine SDK is currently in Early Access (EA) with select customers and partners. The tsunAImi accelerator card is sampling now and will be commercially available in 1Q2021.

Recommended AI News: Alegion Names David Mather President and CEO, Appointing Him to its Board of Directors