AiThority Interview with Cloudflare’s Rita Kozlov, Senior Director of Product, Developer Platform

Hi, Rita. Welcome to the Interview Series. Please tell us about your role at Cloudflare. How did you arrive at Cloudflare?

I joined Cloudflare seven years ago as a solutions engineer, helping our customers achieve success with our technical deployments for our performance and security services at the time. Before Cloudflare, I was working as a software engineer, so when Cloudflare Workers was released it seemed like it could present a massive opportunity to change the way applications were built, so I joined the product side to work on it, and have been helping grow our developer offering since (which now includes our AI products).

What has changed in the AI data engineering landscape since the launch of ChatGPT? How are generative tools transforming DevOps and programming techniques?

On the DevOps and programming side, we’ve started seeing generative AI transform the way developers were building before ChatGPT, with the release of Github’s Co-Pilot.

We’re already seeing massive productivity gains from developers using code-generation tools to accelerate development, especially when it comes to generating boilerplate code, tests, and tasks that have been solved many times but are time-consuming. The other big shift we’re already starting to see and I believe we’ll see even more of is generative AI opens up the door for more collaboration amongst developers, and designers, and a lower barrier to entry for beginners.

Could you highlight Cloudflare’s approach to innovating with generative AI tools? Please tell us more about Mistral 7B.

The challenges developers face with deploying AI today are very similar to the challenges we’ve been solving with our developer platform for the past 6 years — a lot of developer productivity is wasted on setup, provisioning, optimization of resources, scaling and performance. So we wanted to bring the same serverless approach we brought to generalized compute to GPUs and AI deployments and allow developers to start building applications with AI without having to think about infrastructure.

We launched Workers AI in September with support for several models in our model catalog out of the box to get started with an open-source LLM being as easy as writing a few lines of code. We’ve been expanding our model catalog since launch and recently announced support for Mistral 7B which is an exciting new open-source LLM that outperforms previous LLMs in many ways, despite being a much smaller model.

How do you think AI is going to reach critical mass for businesses?

There’s been a lot of excitement recently about really large language models, however, anyone running or deploying them in production will be faced with reality pretty quickly: it’s really expensive to run them, and hard to even find and provision infrastructure for them. The next wave of innovation we’re going to see will likely go the other way, in getting smaller models (single-digit-billion parameter models, as opposed to hundreds-billion parameter models) to perform better.

With fine-tuning, these models are more likely to provide value to businesses fast, and power better experiences without breaking the bank.

AiThority Interview with Snyk’s CMO Jonaki Egenolf

On a related note, there’s been so much great innovation going on with open-source models, and the tooling ecosystems that have sprung up around them. It will be interesting to see how open-source model adoption plays out compared to proprietary, but I would say open-source is always a good bet.

With models taking multiple seconds to execute, does performance really matter in AI?

AI is still a fairly nascent technology — so our expectations are in line with that. But if AI is to become an integral part of our daily lives, performance is going to improve significantly and become a standard.

The web started out very similarly, with users being accustomed to waiting multiple seconds if not minutes for a web page to load. Today, however, engineers are optimizing for every large millisecond of wait time because as it turns out, there’s a direct correlation between performance and conversion rates.

Especially once AI starts powering more real-time experiences that are meant to emulate humans (think ordering a coffee at a drive-through), we’re going to expect experiences that have processing and reaction times that mimic those of human interaction.

Your predictions for the AI engineering industry in 2024: 

I have two big predictions for AI development in 2024. 

The first is that surging AI bills will land on developers. As AI experimentation skyrockets, so do AI bills.

Developer teams will be required to answer for this AI spend as CFOs will not accept unbounded and unpredictable costs for much longer and there will be added pressure to prove the return on investment. Tools offering insights, guardrails, and monitoring, especially in experimentation, are going to be critical for every dev team’s AI arsenal. 

AiThority Interview with Abhishek Shrivastava, VP of Product at LinkedIn

The second is that smaller models are going to become the default.

While it’s been exciting to see what we can achieve through the hardware that’s allowed the training of extremely large hundred billion parameter models, the reality is that in production there are diminishing returns to running such large models, while the costs of running them are disproportionately higher. What we’re going to see is more fine-tuning of smaller models, and innovation around customizing them and making them smarter.

Lighter notes:

Burn the midnight candle or soak in the sun?

Neither! I’m a morning person, but I love a cozy day indoors.

Coffee, or Tea?

Coffee

Your favorite Cloudflare offering that you want everyone to know about?

I’m biased, but Cloudflare Workers! It takes just a few seconds to try and you can get an application up and running so quickly and it will continue to scale regardless of how much traffic you get.

First memorable experience in your career as a technology leader?

Shipping workers.dev and launching it at JSConf EU. It was surreal seeing apps pop up the next day built on top of the tools we had just launched.

One thing you remember about your employee (s):

There are so many different ways to be a successful PM. Some PMs are more analytical and live by dashboards, others are great speakers and presenters, some love getting down into the API design. You want a mix of these different skills on your team to both foster different perspectives and learning from each other. I feel so lucky about the team I get to work with, and learn from them myself every day.

Most useful app that you currently use:

Google Calendar – it dictates my entire day.

Thank you,  Rita! That was fun and we hope to see you back on AiThority.com soon.

[To share your insights with us, please write to sghosh@martechseries.com]