Industrial Image Processing Deep Learning—Gateway to New Machine Vision Applications

Related Vendors

MVTec Software GmbH

Wuxi InfiMotion Technology Co., Ltd.

In industrial manufacturing, the variety of produced parts and potential errors is virtually endless. The added challenge for quality assurance and identification tasks: deviations are often "OK," and not every deviation is a defect. Deep learning technologies can provide solutions here, enabling image processing applications that were previously unimaginable.

Industrial image processing (machine vision) has been considered a key technology in automation for many years. It has proven its worth in numerous industrial application scenarios by delivering faster, more reliable, and more robust results than the human eye. Traditional machine vision systems operate on a rule-based approach. This means specific algorithms must be programmed for all conceivable use cases, following fixed rules. These methods can solve a wide variety of tasks quickly and efficiently, enabling companies to achieve significant efficiency gains. Such is the state of the traditional machine vision world.

Developments ushered in by artificial intelligence (AI) mark the beginning of a new machine vision era. AI, and specifically its variant deep learning, break the limitations of rule-based methods.

Gallery

Gallery with 6 images

Clear the Way for New Applications

Perhaps the most crucial difference: the programming effort is reduced with deep learning-based methods. It is no longer necessary to manually program and teach every possible defect to the system; instead, an automated training process occurs, where the system independently learns through the input of numerous, suitable image data. Thanks to AI, machine vision applications are now feasible that once seemed unimaginable. An example of this is the food industry: natural products, unlike screws for instance, can exhibit significant variance or even change over time. And yet they remain consumable, or in technical terms: OK.

Deep learning-based solutions can distinguish between OK and NOK despite the high variances. Such distinctions would not be feasible with rule-based methods, as it is impossible to cover all possible variations, such as those of a baguette, with rule-based algorithms. A deep learning method like anomaly detection reliably identifies OK and NOK products without displaying false negatives.

Inspection of Weld Seams

Another example of new application possibilities is the inspection of weld seams. These also exhibit extremely high variance, which cannot be reliably inspected using traditional methods. With deep learning-based classification, automated and reliable quality assessment is now possible. The prerequisite for this is prior training of the underlying neural network, in this case with image data of OK weld seams.

The deep learning tool from MVTec supports data handling, labeling, and training of such deep learning applications. If defect classes (e.g., "scratch," "air inclusion," "too thin weld seam," etc.) are known, the aforementioned deep learning classification can also be performed here. In this process, the classes are specified in the deep learning tool and then trained to enable differentiation later, if desired.

Bin-Picking And Pick-And-Place Tasks

Even complex bin-picking and pick-and-place tasks are optimized through AI-based image processing. For gripping rigid objects based on CAD models, the "Deep 3D Matching" technology is an ideal solution. This makes bin picking in three-dimensional space possible solely based on 2D images.

With methods like "Object Detection" or "Gripping Point Detection," deep learning also enables robots to securely grasp differently shaped, translucent objects such as plastic bags containing assembly accessories. Here too, the software is first extensively trained with sample images, learning the numerous different shapes and positions the bags can take. This results in a very robust detection rate, even with an almost infinite variance of objects and positions. This makes it possible to grasp and pick stacked bags as well.

Application Field Optical Character Recognition (OCR)

Another application field enhanced by deep learning is optical character recognition (OCR). Rule-based OCR sometimes faces challenges when surface properties or lighting create diverse reflections and spots, significantly complicating the correct segmentation of characters. With AI-based Deep OCR, recognition rates remain robust even in such cases. These algorithms can locate characters regardless of their orientation, font, or polarity. Additionally, letters can be automatically grouped, enabling the identification of entire words. Misinterpretations of characters with similar appearances are completely avoided, significantly improving recognition performance. Thanks to extensively pre-trained deep learning networks, even hard-to-read texts can be recognized with high accuracy.

The Royal Road: Bringing Together Machine Vision Worlds And Enabling New Applications

Have AI-based developments made previous image processing technologies obsolete? Not at all! Rule-based methods will continue to have their place in the future. The reasons: they are simple and proven to work. Additionally, they require comparatively little computing power, making them very fast. The ideal approach, therefore, lies in combining both worlds. With powerful machine vision software products such as MVTec Halcon or MVTec Merlic, users can access and combine various methods—whether rule-based or deep-learning-based. This enables the automation of new applications with maximum speed and high robustness.

About the author

Felix Podhorksy has been with MVTec Software GmbH since 2019. In his current position as Business Development Manager, he is responsible for the strategic development of cross-product business development. This includes evaluating and exploring new business areas and application fields in the field of industrial image processing.

Previously, he worked as a Senior Application Engineer at MVTec. His professional career began in second-level support, where he assisted customers with technical questions about the software products Merlic and Halcon—with a focus on deep learning.

Felix Podhorsky studied Geodesy and Geoinformation at the Technical University of Munich. He wrote his master's thesis in collaboration with MVTec in the field of deep learning.

Smart-Factory

Access solutions for the Smart Factory - Industry 4.0