How Machine Learning and Visual Inspection Can Unlock Powerful Automation Techniques

Over the last several years, there have been notable advancements in the fields of computer vision, visual inspection, and machine learning that make it vastly easier to apply in more complex situations like manufacturing. In its earlier days, computer vision researchers were limited to initial capture and compare applications, the equivalent of a human being looking at an image and determining whether its contents were good or bad. Early automation of that process allowed software to achieve a deeper level of analysis: It could compare two images, and determine that if they’re not perfectly the same, or if key areas are not perfectly the same, then the product in the image likely contains a flaw.

This method posted serious limitations to industrial use. Namely, if the image the model is analyzing isn’t perfectly calibrated, the AI won’t work right. These types of systems used a complex series of rules, essentially a heuristic to model what was “right,” and what was “wrong.” Updating that rule set to work in a new context – even different lighting – requires significant effort, and of course changes to products, components, or the layout of a factory floor are ever-changing. In short, capture and compare is too unadaptable and too inflexible to be useful in modern industrial settings.

8 Ways AI Will Improve Healthcare Post-Pandemic – and Beyond

To apply visual inspection at scale and make it useful for people on the ground, visual inspection systems have to be able to generalize. They need to be able to reach the desired conclusions without pixel perfect images to work with. And they need to be able to adapt so that if the result the model is searching for changes, the model doesn’t need to be re-built from the ground up. By applying machine learning techniques to visual inspection, we can endow it with the nuance and the flexibility to be applicable and helpful in countless real-world settings, without having to write by hand a laundry list of nuances, exceptions, and rules.

Teaching computer vision to learn on its own 

 Computer vision models that are trainable using machine learning open up a number of new applications. Using machine learning, for example, we can train visual inspection systems to identify counts, apply rules for an inspection and identify when human intervention might be necessary. When it can count, for example, visual inspection can apply a system failure tag if a part has five screws instead of four. When this condition is met, the image and all the information contained can be automatically sent to an administrator to make sure the part is okay.

Critically, the administrator can then generate a feedback loop so that the model continues to improve the more determinations that it makes.

These applications would be impossible with the traditional capture and compare method for visual inspection. Take the previous example of counting screws on a part. In a piece of machinery like a car, there might be hundreds of different fixtures from a wide variety of different suppliers. Using capture and compare, a single change to the color of a single screw – even a slightly darker shade of silver – would be sufficient for overwhelming and breaking the model. Using machine learning, however, we can apply object recognition, and we can train the model to know what a screw looks like, without the screw needing to look identical each and every time. We can set increasingly more complicated logic rules, so that a single visual inspection model can be applied across many iterations of the same project.

COVID-19 and Artificial Intelligence: How Have Attitudes to AI Changed Through the Pandemic?

Object recognition makes it possible to automate even more elements of the manufacturing process. A mobile application can be trained to look for bar codes or vehicle identification numbers so that it can analyze whatever machines are coming off the assembly line that day. Optical character recognition can trigger whichever software system is relevant without having to interrupt the flow of the assembly line, using only a handheld camera, giving you enormous flexibility about what types of products or machines you produce and where.

Even for something as ubiquitous as a car, there is enormous complexity to the underlying machine. There are countless assemblies, sub-assemblies and parts. There are requirements regarding safety and function – does the car have a secure windshield – and there are requirements regarding aesthetics. Finally, to train software to ensure that your machine is meeting all those requirements, your software needs to adapt with the times. It needs to be resilient to change, changes in supply chains, equipment and trends. In short, it needs to be able to learn on the job. Thanks to advancements in machine learning, this flexibility is possible, making visual inspection a capable and effective colleague.