Robotics Ramp-Up: NVIDIA Sets Milestone in Delivering Unified Platform for Building Autonomous Machines

Updated Capabilities for Robotics AI Perception and Simulation Are Now Available with a New Version of the NVIDIA Isaac Software Development Kit.

NVIDIA has released updated capabilities for robotics AI perception and simulation, with a new version of the NVIDIA Isaac software development kit.

Announced by company founder and CEO Jensen Huang at NVIDIA’s latest GPU Technology Conference, the SDK achieves an important milestone in establishing a unified robotic development platform enabling AI, simulation and manipulation capabilities.

It includes the Isaac Robotics Engine (which provides the application framework), Isaac GEMs (pre-built deep neural networks models, algorithms, libraries, drivers and APIs), reference apps for indoor logistics, as well as the first release of Isaac Sim (offering navigation capabilities).

As a result, the new Isaac SDK can greatly accelerate developing and testing robots for researchers, developers, startups and manufacturers. It enables the AI-powered perception and training of robots in simulation allowing them to be tested and validated in a range of environments and situations.

Read More: Measuring CCPA Preparedness of Big Data Companies: Facts and Insights

And in doing so, it saves costs.

Start with AI-Based Perception

Every autonomous machine starts with perception.

To jumpstart AI robotic development, the new Isaac SDK includes a variety of camera-based perception deep neural networks. Among them:

• Object detection — recognizes objects for navigation, interaction or manipulation
• Free space segmentation — detects and segments the external world, such as determining where a sidewalk is and where a robot is allowed to travel
• 3D pose estimation — understands an object’s position and orientation, enabling such tasks as a robotic arm picking up an object
• 2D human pose estimation — applies pose estimation to humans, which is important for robots that interact with people, such as delivery bots, and for cobots, which are specifically designed to work together with humans

The SDK’s object detection has also been updated with the ResNet deep neural network, which can be trained using NVIDIA’s Transfer Learning Toolkit. This makes it easier to add new objects for detection and train new models that can get up and running with high accuracy levels.

Read More: World’s Largest Blockchain Application Competition ‘Klaytn Horizon’ Winners Announced

Introducing Isaac Sim

The new release introduces an important capability using Isaac Sim to train a robot and deploy the resulting software into a real robot that operates in the real world. This promises to greatly accelerate robotic development, enabling training with synthetic data.

Simulation enables testing in so-called corner cases that is, under difficult, unusual circumstances to further sharpen training. Feeding these results into the training pipeline enables neural networks to improve accuracy based on both real and simulated data.

Multi-Robot Sim Is Here

The new SDK provides multi-robot simulation, as well. This allows developers to put multiple robots into a simulation environment for testing, so they learn to work in relation to each other. Individual robots can run independent versions of Isaac’s navigation software stack while moving within a shared virtual environment.

Thus, manufacturers seeking to run multiple robots in large logistics operations can, for example, test their interactions and debug problems before deployment into the real world.

Isaac Integrates with DeepStream 

The new SDK also integrates support for NVIDIA DeepStream software, which is widely used for processing analytic capabilities. Video streams can be processed deploying DeepStream and NVIDIA GPUs for AI at the edge supporting robotic applications.

Developers can now build a wide variety of robots that require analysis of camera video feeds, provided for both onboard applications and remote locations.

Programming with Isaac SDK 

Finally, for robot developers who have developed their own code, the new SDK is designed to integrate that work, with the addition of a new API based on the C programming language.

This enables developers to connect their own software stacks to the Isaac SDK and minimize programming language conversions giving users the features of Isaac routed through the C API access.

The inclusion of C-API access also allows the use of the Isaac SDK in other programming languages.

Read More: AiThority Interview Series with Steve Auerbach, CEO at Alegeus