Unlocking Hyperautomation: What It Takes to Guard Quality with AI
The advances in neural machine translation (MT) and large language models (LLMs) in recent years have been nothing short of breathtaking. For many of the major language pairs and use cases, state-of-the-art MT is now both accurate and fluent.
However, despite such impressive strides, the use of MT alone still carries risks that are not tolerable for many if not most enterprise applications. To guard against these risks, human translators are tasked with performing MT post-editing and review, a method that is both time-consuming and costly. Alternatively, an organization may opt to employ MT without any human input at all. In doing so, they take a gamble on possible embarrassing errors and misinterpretations that could be damaging on a number of levels.
Read: AI in Content Creation: Top 25 AI Tools
Enterprises seeking to streamline translation and localization workflows face a conundrum: How can an organization achieve automation without compromising quality or risking flawed translations? The solution lies in harnessing cutting-edge technology to provide unparalleled visibility into translation quality at scale.
The Importance of Automating Quality Performance Scores
So, what does it take to enable organizations to automatically detect and address low-quality translations efficiently, minimizing the need for extensive human intervention? After all, isn’t human oversight the ultimate in quality control? Understandably, the bar is high when it comes to automating quality, and success begins with automated accurate detection of translation errors and assignment of quality performance scores.
From a translation workflow perspective, we need an automated translation quality scoring system that has the following properties:
- The system must be capable of assigning quality scores at the segment level, which can then be aggregated to the document and job level.
- Workflow “gating” decision-points should then be implemented supporting two major complementary workflow decisions:
- At the job-level: Is a translated job of sufficient quality to be completed without further human editing or review?
- At the segment-level, for jobs that are sent to human editing and review: Which segments are of sufficiently high quality and can be confidently “blocked” from further human editing and correction?
Crucially, the system described above must be designed to operate seamlessly across diverse translation scenarios. This includes not only machine-translated content but also human-edited MT and traditional human translation. This versatility ensures that enterprises can maintain rigorous quality standards across all their localization efforts, regardless of the translation method employed.
Of course, assigning quality performance scores so that content can be routed properly assumes quality can indeed be accurately and reliably assessed. This is yet another area where new technology capabilities are upending established protocols for enterprises.
Automating Linguistic Quality Assessments
The most reliable process for assessing the quality of translations has long been a human-expert-intensive process known as linguistic quality assessment (LQA). Human LQA has evolved, with increasing adherence in recent years to the multidimensional quality metrics (MQM) framework. The MQM framework is a complete model for assessing translation quality over several dimensions. It takes into account different requirements such as fluency, adequacy, and error types, offering a structured way of evaluating translated content. And when a structured system like this is available, it’s ripe for automation.
For years, human LQA has often been restricted to small samples due to the process’s cost and speed limitations. Still, there are many enterprises whose localization budgets have significant allocations for LQA. With the recent advent of LLMs, however, full automation of human LQA, at impressive levels of accuracy, has now become possible.
Read: Role of AI in Cybersecurity: Protecting Digital Assets From Cybercrime
Automated LQA is much faster and less costly than human LQA can ever be. It can be used for use cases where automated analysis is deemed sufficient, as well as an automated “pre-annotator” for human LQA (similar to the way MT is used in conjunction with human MT post-editing). Furthermore, since the MQM framework already includes a scoring algorithm, a fully-automated LQA based on MQM also provides a well-understood scoring function.
So, doesn’t automated LQA solve the automated assignment of quality performance scores problem that we described earlier? Eventually, this will probably be the case—but not today. While automated LQA with LLMs represents a significant leap forward, slow speeds and the high cost of LLMs currently hamper its scalability. To address that scalability challenge, a smaller, faster, and less costly AI model—specifically designed to predict the score that would be generated by either a human or automated MQM annotation—can be trained. This makes the process a useful complement to automated LQA.
The landscape of translation quality assessment is evolving rapidly, driven by advancements in technology and methodologies. These innovations signify a critical turning point in the market, revolutionizing automation and scalability within localization. It lays the groundwork for hyperautomation, where content is seamlessly processed through workflows employing various AI and machine learning models and techniques. This will transform both the efficiency and precision of multilingual content creation—a vital component for managing the growing volume of content in today’s globally connected world.
Comments are closed.