AiThority Interview with Fabrizio Del Maffeo, Co-Founder and CEO, Axelera AI

Fabrizio Del Maffeo, Co-Founder and CEO, Axelera AI chats about the Gap and launching the first M.2 accelerator with Intel. He emphasized his journey towards founding Axelera AI, focusing on developing advanced AI acceleration technology using in-memory computing and RISC-V architecture, which has significantly improved performance and efficiency in edge AI applications.

———————

Please share more on your tech and SaaS journey and the key milestones in the technology industry that have shaped your vision for Axelera AI.

I have a master degree in electronic engineering and I had worked in product development, product management and sales in computing and IoT markets for most of my career, covering different managerial roles in companies such as Advantech and Aaeon, the IoT subsidiary of Asus group.

During my experience in Asus group, in 2015 I have founded and launched via a Kickstarter campaign UP Bridge The Gap ( UP Bridge the Gap – Homepage – Edge Computing Devices (up-board.org)), a new product line which became the most popular Intel-based development platform in the B2B embedded and industrial computing space. Talking to our customers, I noticed the growing need of processing data on device – at the edge – and the lack of a good solution. Therefore in 2018 together with Intel we launched, leveraging UP product line, the first M.2 accelerator based on Intel Movidius Myriad 2: while it was initially a very good commercial success, the technology fell short on many aspects since it wasn’t tailored on pure on device AI acceleration. It appeared clear that there was a growing need of a specific solution to accelerate AI workload at the edge: CPU did not give enough parallelism of computations needed by neural networks, FPGA were versatile but too slow and costly, GPU were a good solution but still not optimized, costly and power hungry.

Also Read: Humanoid Robots And Their Potential Impact On the Future of Work

I went back “to school”, reading few scientific papers about new emerging technologies such in-memory computing and RISC-V which seem to fit the bill: in-memory computing to accelerate the vector-matrix multiplication which represents 70-90% of the calculations in any neural networks and RISC-V computing architecture to process all the rest.

I finally decided to team up with Bitfury Group and start the project and after almost 2 years of incubation, together with our co-founder Evangelos Eleftherious and other 16 people from imec, IBM, google, ETH Zurich, Intel and Qualcomm we started Axelera AI

Axelera AI is known for its AI-native hardware and software platform. Can you elaborate on the latest updates to this platform’s features?

At Axelera AI, we are revolutionizing the field of artificial intelligence by developing a cutting-edge hardware and software platform for accelerating computer vision and generative AI on device, at the edge. Our platform, built using proprietary digital in-memory computing technology and RISC-V dataflow architecture, delivers industry-leading performance and usability at a fraction of the cost and energy consumption of current solutions. Technically we are talking about 210+ Tera Operations Per seconds (TOPs) with core efficiency of 15 Tera Operations per second per watt (TOPs/W) in a chip  – our METIS AI Processing Unit – which cost way below 100 USD. We are at least 2-3x better than any other product in the same market segment.

A major advantage of our accelerator technology is that has been implemented in standard CMOS technology. Our SRAM-based D-IMC design uses proven, cost-effective and standard manufacturing processes, readily available in any silicon foundry. Memory technologies are also a key driver for lower lithography nodes. So, Axelera AI will be able to easily scale performance as the semiconductor industry brings advanced lithography nodes into volume production.

We are also going beyond just the chip development, building a complete product portfolio of AI acceleration cards and systems powered by a versatile and easy-to-use software stack: our Voyager SDK. One of the biggest challenges for Edge AI is optimizing neural networks to run efficiently when ported onto a mixed-precision accelerator solution. Our platform includes advanced quantization techniques and mapping tools that significantly reduce AI computational load and increase energy efficiency.

Our neural network zoo gives the possibility to customers to run out of the box state of arts neural networks which deliver high performance and high precision minimizing the power consumption.

Finally, our software tool chain allows customers to build up a complete application pipeline in a matter of minutes. Essentially, we do everything to simplify the deployment of artificial intelligence in any device.

Can you talk about some of the recent product launches from Axelera AI and their impact on the market?

The full production-ready Metis AI platform is now in mass production delivering high performance and preserving 99% of the original model’s precision, indistinguishable from GPU-based inference models, while offering 4-5 times throughput, energy efficiency and cost savings, opening up unprecedented opportunities for mass deployment of AI solutions.

We have a complete product portfolio starting from an M.2 card to PCI-E card which can deliver up to 214 TOPs, capable to handle the most demanding vision applications. We are going to release in coming month a new card equipped with 4 Metis AI Processing Units which can delivers up to  856 TOPs and early next year an AI single board computer which couples a powerful ARM based host CPU together with our Metis AI Processing Unit.

Metis AI is set to unleash a new wave of innovation putting in the hands of developers a very powerful and competitive AI acceleration technology to power AI applications in any computer vision application such as surveillance systems, retail systems, robots, drones, medical devices and more.

We have been also working on broadening our future product offerings from the edge to the enterprise servers to address the growing computing needs for generative AI, large language models and large multi-modal models. This market expansion includes high-performance computing by designing high efficiency, high-performance and price-competitive AI accelerators to power future exa-scale and peta-scale HPC centers.

What are the key areas within Enterprise AI that currently command your attention?

Our focus in on AI inference. Inference, the process of running a trained neural network, represents most of the AI workloads which we run today. Inference doesn’t require per se the same hardware used for training a network: a technology optimized around inference, like our digital in-memory computing technology coupled with RISC-V dataflow, can be orders of magnitude more efficient, faster and less expensive.

Power consumption is a critical factor, both on device and in data centres. So Axelera AI offers class leading compute density with exceptional core efficiency. The result means systems can easily crunch data without draining power or running hot, with a typical use case requiring just a few watts.

Another important factor is the computing performance: neural networks are getting bigger and they require more computations. Scaling performance using CPU and GPU is inefficient and extremely expensive. We are fully focused on tailor our technology around the new emerging needs, efficiently offloading completely the AI acceleration from the CPU inside our AI processing unit.

Also Read: Natural Language-Based Scientific Analytics

Our first focus was computer vision but we are now developing specific solutions to accelerate generative AI workloads, from large language models to large multi modal language models.

In your opinion, how do AI-native platforms offer a competitive advantage over traditional AI solutions?

Initially computing technology has been initially designed for simple tasks such writing and make calculations. Essentially the first computers were typewriters which can compute. Graphical interfaces required more sophisticated computing. Computer games required a different architecture such graphic processing units, optimized to parallelize high precision calculations.

AI is following the same path. Solving a neural network means to compute trillions of very simple calculations and therefore an architecture like the one we designed at Axelera AI,  tailored around this need, can deliver orders of magnitude higher throughput, efficiency at a lower cost.

As AI continues to evolve, where do you see the technology heading beyond 2025, particularly in the context of its integration with IoT and smart city applications?

Artificial intelligence is spreading from cloud computing to the edge, running on the devices. All new devices are getting smarter and today is possible to run tiny neural networks inside 32-bit microcontrollers. I expect analog sensor to become more intelligent in future, with a growing integration of analog computing to filter the first signal, identify the first patterns.

Most of the data will be processed real-time inside the edge computer which will have an heterogeneous computing architecture, with specific components to run AI and other to run standard applications. Not all of the data will be streamed to cloud, only the most relevant. There will be increasing numbers of computing layers between the sensor and the cloud and the architecture will be more decentralized with increasing collaborative computing at the edge. Inside the AI workload, inference will dominate the AI edge computing with very little amount of computing reserved for on device learning.

In a scenario like Smart City, you can imagine in a not too far future traffic light systems to be completely synchronized, sharing each other information to minimize the waiting time. Cars to communicate between each other to optimize the route and minimize the traffic, parking systems connected to cars to inform about parking space availability. The number of circulating cars will be dramatically reduced thanks to fully autonomous vehicles which will reduce the demand of new vehicle, maximizing the utilization. Large deployment of sensors and cameras will improve people safety and will allow to learn population behavior, flows, helping improving city infrastructure design and implementation.

Stores will be fully automatized giving a frictionless experience to customers which will enter the shops and grab the products they want without stopping at the cashier. It will be possible to know exactly in which shop there is the product we look for thanks to advanced product tagging and cloud information sharing. In the next decade robots will also be more present in our city, relieving people from heavy duty tasks or unpleasant jobs, becoming natural assistants in many jobs.

All of this “intelligence explosion” will be powered by advanced and sophisticated neural networks which are expected to improved dramatically once they will start learning from the physical world. Today the most important neural networks are transformer-based network which require a lot of bandwidth and computations due to the quadratic complexity of the attention mechanism, but I’m convinced that new topologies, more efficient, will emerge in coming 3-5 years accelerating the progress in artificial intelligence, and further democratizing the access to artificial intelligence to everyone.

Also Read: AiThority Interview with Eli Ben-Joseph, CEO at Regard

Given your experience in deep-tech startups and fundraising, what advice would you give to emerging AI startups looking to secure investment and scale their operations?

I don’t think there is a secret sauce to secure funds, and I’m convinced that early stage investors focus their attention on the usual suspects: the team, the market opportunity, the timing, the unique value proposition and the ability to scale. At later stage – Series A/B – investors tend naturally to focus on the market traction, the final validation of your solution. Many startups fall short in one or more of these elements.

My advise to founders is to be honest and very critical: we must constantly challenge our own assumption, admit our mistakes, learn from them, learn possibly from other people mistakes and don’t give it up in front of any failure. Running a startup means to eat failures for breakfast.

I advise to keep a constant focus on customer problem and adjust the solution around it. This doesn’t mean to follow the customers but to deeply understand the problem to be solved and finally provide a far better solution. Typically, if the solution doesn’t give 5-10x improvement in the key metrics, it is doomed to fail because the adoption will be extremely limited to few innovators and early adopters.

I also suggest you build up immediately a strong team, possible with few co-founders with complementary skills: they will be the pillars on which you can build and scale the company and they will give confidence to the investors that this is not a one-man show.

About securing funds, I strongly advise to keep 100% focus on finding at least one lead investor. Lead investors are the kings in the venture capital market, the leaders of the pack: when you find your king, all the other investors will just follow it. I suggest you run your own due diligence on each potential investor, understanding the stage of the fund (how much capital they still can deploy), the key decision makers within the fund and their strategy. It’s important to ask immediately as many questions as possible to see whether there is a real fit between their investment thesis, their appetite to invest and your company. Don’t try to force the fit, just move on to a new investor.

Could you tag one person in the AI/ML sector whose insights and answers to these questions you would love to read?

Jim Keller ( CEO of Tenstorrent)

Sid Sheth (CEO of D-Matrix)

[To share your insights with us as part of editorial or sponsored content, please write to psen@itechseries.com]