Banuba Increases the Performance of Virtual Backgrounds by 10X by Upgrading Neural Network Architecture

Banuba implemented a series of technical updates to Face AR SDK that drastically increase the maximum frame rate of live videos with background separation effect. Depending on the platform and hardware, this results in up to 10 times higher maximum FPS.

AiThority Interview Insights: AiThority Interview with Matthew Tillman, CEO and Co-founder at OpenEnvoy

Virtual backgrounds are a must-have feature for any modern video communication software and social media. They help alleviate camera shyness, protect users’ privacy and prevent potentially embarrassing situations like pets walking in during a serious business meeting. After the COVID-19 pandemic caused a massive increase in remote work, the demand for virtual backgrounds skyrocketed, so improving them has been one of Banuba’s top priorities. You can see them in action in the relevant video.

The results were achieved thanks to three main additions:

• New neural network architecture with improved utilization of CoreML Neural Engine (on Apple devices with Bionic processors);
• Algorithm optimization on Windows and Web, allowing the neural network to monitor every other frame instead of each of them;
• Improved anti-jitter algorithms that demand less resources from the device.

Read More about AiThority Interview: AiThority Interview with Rebecca Jones, General Manager at Mosaicx

This effect is available as part of Banuba Face AR SDK (for live streaming and video communication) and Video Editor SDK (for prerecorded videos).

Other updates include:

• Facial feature editing – a new functionality that allows changing size and shape of any part of the face;
• Optimized hand tracking and gesture recognition;
• Better lips segmentation. This is especially noticeable on the corners of the mouth and near the philtrum;
• Acne removal for photos and an option to change the size of the area affected by the effect.

Banuba is an augmented reality company with over 7 years on the market, pioneering face tracking, virtual try-on, and virtual background technologies. Besides online try-on solutions for makeup, headwear, jewelry, glasses, and more, it offers a Face filters SDK and Video Editor SDK – ready-made modules to apply effects and edit videos.

 Latest AiThority Interview Insights : AiThority Interview with Dan O’Connell, Chief AI & Strategy Officer at Dialpad

 [To share your insights with us, please write to sghosh@martechseries.com]