Lenovo’s AI Revolutionizes Sign Language in Brazil

Software engineer Gabriel greeted Lenovo chairman and CEO Yuanqing Yang enthusiastically in Libras, the official Brazilian Sign Language, as he crossed the stage at the Tech World event. Although YY, as he is called at Lenovo, is not familiar with Libras, he had a solid understanding of Gabriel because to AI. An innovative AI system translated Gabriel’s speech and text in real time as a camera tracked his every motion. Gabriel and YY were able to establish an immediate and genuine connection despite their language difference.

The innovative accessibility solution developed by Lenovo researchers was on display in this brief interaction. There are about 2.3 million people in Brazil who are hard of hearing or deaf, and this new technology has the potential to improve their lives in innumerable ways. Whether at a hospital or a retail establishment, the R&D team’s AI solution can help people communicate and build relationships even when a sign language interpreter isn’t present.

Read: How Lenovo Leverages AI and ML for an Unbreakable Supply Chain

Under the radar of Tech World, Lenovo edge servers powered the AI and deciphered the mountain of data collected as Gabriel greeted his audience. Edge servers offer superior speed and dependability right where AI is required, as opposed to cloud computation, which is an option. Although Gabriel’s AI voice—chosen by his family from thirteen different options—was largely a proof of concept during the demonstration, the technology behind it is already well-developed and ready for use. Already, hundreds of hours of anonymized video data have been donated to the training set and AI advancements by dozens of hard of hearing and deaf individuals using Libras.

How Generative AI is Revolutionizing Multilingual Translation

A software developer who is fluent in Libra brought up numerous accessibility difficulties during an internal team conversation at Lenovo in 2019. She encouraged Lenovo to do more to assist the deaf community’s independence and quality of life. A Brazilian Lenovo team started brainstorming potential solutions, including a chat application that could translate Libras into written or spoken Portuguese in real-time, allowing the deaf or hard of hearing to sign to a device’s camera. Generative AI and multilingual data sets have made translation into many more languages possible.

One of the biggest challenges with real-time video capture and translation between languages is the massive quantity of data that is involved. This data includes not only the grammar of each phrase but also the specific motions for each word. Individuals born under the sign of Libra can have their own unique movements and styles, much like different regions of the United States might have their own unique accents when speaking English. Lenovo teamed up with the Brazilian innovation hub CESAR to address the data dilemma by exchanging knowledge on video capture and categorization, which paved the way for AI. In order to train the main algorithm to identify and contextualize specific movements, Lenovo and CESAR have since produced a library of thousands of Libras films. Next, Lenovo took the lead in creating the innovative AI that is the solution’s backbone.

Read: Lenovo Selected for New Supercomputer at the Paderborn Center for Parallel Computing

The signer’s digital articulation points and hand locations are both detected by the AI. Once the AI has processed all of the motions and gestures, it can quickly translate the sign language into text by determining the sentence flow. The group worked in tandem with Lenovo’s Product Diversity Office (PDO), an organization whose stated goal is to make all Lenovo goods accessible to people with different types of disabilities. Experts in inclusive design from the PDO helped pinpoint possible trouble spots and ensured that product testing took them into consideration. These included things like hairstyle, skin tone, corrective lenses, and limb discrepancies.

 How AI is Revolutionizing Learning?

During a recent internal event in Brazil that focused on inclusion in Lenovo workspaces, an R&D team member heard the story of a deaf girl who had trouble communicating with her parents when she was growing up. She had a tough time of it as sign language interpreters weren’t always accessible, especially at home. The research and development team at Lenovo stressed that their approach fills communication gaps rather than encouraging more people to learn Libras or other sign languages. Moreover, AI has the potential to speed up the process of learning sign language by utilizing computer vision to monitor the precision of gestures and “instruct” users on how to improve their performance. With the use of AR or wearable tech, individuals could engage in fully immersive learning experiences guided by AI.

In order to discover a solution for edge computing, Lima’s research and development team collaborated with Lenovo’s Infrastructure Solutions Group. Depending solely on the cloud—and, by extension, extremely high Internet speed—may be sufficient in many cases, but it is far from foolproof. Potential customers in high-traffic areas, such airports or hospitals, would prefer not to depend on unreliable connections. Integrating computing at the edge with Lenovo’s pocket-to-cloud portfolio enables customers to access AI right from the data source.

Expanding the project’s scope beyond internal testing is the following stage. In order to provide a large-scale real-time sign language translation interface, additional data points are required. With the increasing number of users and data sets, the team is looking into self-learning algorithms and other technologies to speed up development. Since data sets may be fine-tuned and optimized to create an ideal user experience, Lenovo is also looking at methods to customize the translation solution to specific sector verticals like retail or banking. Artificial intelligence (AI) has the potential to greatly benefit the world’s 430 million deaf and hard of hearing people as the solution evolves and motivates the development of more inclusive technologies.