AiThority Interview with Pete Wurman, Director at Sony AI America

Tell us about your journey in the technology industry and how you reached Sony AI.

I got my undergraduate degree from MIT in mechanical engineering. I didn’t feel ready to be an engineer, so I went to the University of Michigan to get a Masters degree in mechanical engineering. Along the way, I got a job programming at the university, and eventually decided to go back to school and get a Ph.D. in computer science. From there, I became a professor in the Computer Science Department at North Carolina State. In 2004, as I went up for tenure, my roommate from my undergraduate days at MIT came up with an idea for a robotic warehouse system, and convinced me to help him start what became Kiva Systems. In 2012, Amazon bought Kiva and started deploying our robots to their warehouses. The division is now called Amazon Robotics, and has deployed approximately 500,000 robots. In 2015 I briefly retired, but soon joined Cogitai, a small AI startup focused on reinforcement learning. Sony acquired Cogitai in 2019 and founded Sony AI.

You recently unveiled Gran Turismo Sophy. Why do you call it “the first superhuman AI agent to outrace the world’s best drivers of Gran Turismo (GT) Sport”?

Gran Turismo Sophy (GT Sophy) is a revolutionary superhuman racing agent that has mastered the highly realistic game of Gran Turismo Sport at a world championship level. The goal of the GT Sophy is to compete with and enhance the gaming experience of the world’s top GT drivers.

GT Sophy was trained to master the following driving skills:

1. Race Car Control: Deep understanding of car dynamics, racing lines, and precision maneuvers to conquer challenging tracks.

1. Racing Tactics: Split-second decision-making skills in response to rapidly evolving racing situations. GT Sophy mastered tactics like slipstream passing, crossover passes and even some defensive maneuvers such as blocking.
2. Racing Etiquette: To play fair, GT Sophy had to conform to highly refined, but imprecisely specified, sportsmanship rules including avoiding at-fault collisions and respecting opponent driving lines.

Recommended: AiThority.com Interview with Nicolas Gaude, CTO at Prevision.io

Could you tell us what kind of AI capabilities are used in this project?

​​GT Sophy is an autonomous AI agent trained utilizing deep reinforcement learning. The agent learned to race through trial and error. By giving the agent rewards for doing the right things, and penalties for doing the wrong things, we encouraged GT Sophy to learn all of the skills needed to compete with the world’s best drivers. Sony AI and its partners built a novel deep reinforcement learning platform to allow us to experiment with many ideas at the same time until we found a combination that worked..

The innovative approach included a new training algorithm called Quantile-Regression Soft Actor-Critic (QR-SAC), agent-understandable encodings of the rules of racing, and a training regimen that promoted the acquisition of nuanced racing skills.

Can you explain why this is a breakthrough for gaming?

GT Sophy takes game AI to the next level, tackling the challenge of a hyper-realistic simulator by mastering real-time control of vehicles with complex dynamics, all while operating within inches of opponents.

How might AI in gaming transform experiences among players?

From the outset, GT Sophy was developed not just as a showcase of AI, but with a view to how AI agents in games (as opponents or collaborators) can enhance the players’ gaming experience. Building a superhuman agent was a way to find out how far we could push the technology. Now we’d like to focus on ways we can use the technology to make exciting races for players of all skill levels and help them become better racers.

The potential of AI also extends beyond game play into the realm of game creation. Here the opportunity is immense from automated testing to asset creation to even game control.

How is AI-based gaming different from conventional gaming experiences?

With regards to Gran Turismo, the existing in-game AI is competitive with casual players but is no match for the best drivers. Although there are physical limits to how fast any given car can get around the track, GT Sophy has managed to find a way to drive that is faster than the best human drivers. In addition, GT Sophy has learned to combine that speed with real racing tactics and racing etiquette, making it fun and challenging for the best drivers in the world.

Recommended: AiThority Interview with Will Keggin, Head of TV at LiveRamp

Is this something that we can expect to soon see being widely used among game developers?

Sony AI in partnership with Polyphony Digital and Sony Interactive Entertainment will continue to evolve GT Sophy’s capabilities as well as explore ways in which the agent can be integrated into the Gran Turismo series going forward.

In addition to Gran Turismo, Sony AI is also eager to explore new partnerships with other gaming studios to enhance the gaming experience for players through AI.

What are the “real world” applications of this AI achievement, including and beyond gaming?

In addition to making contributions to the gaming community, we believe this breakthrough presents new opportunities in areas such as autonomous racing, autonomous driving, robotics, and other automation applications.

For example, GT Sophy could be used to train race car drivers in both simulated and real racing, and the technologies could make their way into real autonomous race cars or into emergency maneuvers systems in real cars.

What other AI and Deep Learning projects is Sony AI currently working on?

In addition to Gaming, Sony AI’s flagship projects include Gastronomy, Imaging and Sensing, and AI Ethics.

For Gastronomy, Sony AI is looking to enhance chefs’ creativity by applying AI and robotics throughout the creation process. By using AI systems to put vast amounts of food data into the hands of chefs, supporting their knowledge and skills when it comes to ingredient selection and pairing.

With Imaging and Sensing – Sony AI is working on combining industry-leading sensor technologies developed at Sony Semiconductor Solutions with new machine learning methods and robotic actuators. By developing novel learning and control algorithms, while also evolving the computing hardware to handle the sensor data in optimal ways, we will discover and explore the full potential of Sony’s sensor technologies and sensor-based solutions.

Sony AI is leading the way towards more ethical AI to ensure AI applications are fair, transparent, and accountable. Sony AI is focused on conducting cutting-edge research on challenging real-world problems facing Sony Group’s businesses, which span imaging and sensing solutions, games, music, movies, and finance.

Which AI sub-fields and specializations are you most excited about?

I’ve been lucky to have had the chance to work in several AI fields in my career. I did my Ph.D. in computational auction theory, switched to robotics while at Kiva, and now, while at Sony, have had the chance to learn about deep reinforcement learning. In many ways, gaming is a perfect environment in which to study deep RL because the games are exceedingly complex, but have all of the advantages of being simulations. I’m quite excited about how these new technologies can impact gaming and significantly improve player experiences.

Recommended: AiThority Interview with Ross McNab, President, North America Advertising at Cardlytics

Your plans for 2022: What kind of future do you foresee for AI in gaming?

Regarding AI in gaming, our goal is to develop AIs that make games more fun for all levels of players. We are looking forward to working with Polyphony to figure out the best ways to incorporate this technology into Gran Turismo in the future.  And to work on similar challenges for other games.

Thank you, Pete! That was fun and we hope to see you back on AiThority.com soon.

[To share your insights with us, please write to sghosh@martechseries.com]