Autonomous Process Inspection of Welds with AI

Related Vendors

FAULHABER Drive Systems

Wuxi InfiMotion Technology Co., Ltd.

A technology with industrial image processing combined with AI automates the inspection of welds on car bodies and identifies anomalies. The application improves the consistency, speed, reliability, and accuracy of the entire inspection process—completely autonomously.

In automotive production, high-quality standards are mandatory. This naturally applies to welding processes on the car body shell (Body in White) as well. The importance of the body’s structural stability is self-evident. More intriguing is the question of how high-quality welds can be ensured—automated and seamlessly. The challenge lies in the fact that many different defects can occur, which impact the quality of the body. For example, cracks, incomplete welds, and irregular welding patterns must be precisely identified. DGH Group tackled exactly this challenge. The Spanish company, headquartered in Valladolid and recently integrated into Groupe ADF, supports various industrial sectors with innovative products. The result is an inspection system that automatically captures images of welds. These are then immediately analyzed by MV Tec Halcon's AI-based algorithms and DGH's image processing software. The software transmits the results—OK or NOK—to the PLC, which determines the next steps for the body accordingly. Halcon, the standard software for industrial image processing (Machine Vision) from MV Tec, plays a key role. The family-run company, headquartered in Munich and founded in 1996, has been developing hardware-independent image processing software for industrial applications and is among the technology leaders in this field—also because the company offers various powerful deep learning algorithms.

Deep Learning in Production: Optical Inspection of Welds

Deep Learning is a subset of Artificial Intelligence. In industrial image processing, Deep Learning enables the implementation of an increasing number of applications, including those that were previously impossible. Additionally, the performance of existing applications can be significantly enhanced. The DGH Group has also utilized these advancements. On behalf of a large French automotive manufacturer, DGH's expert team developed an automated system for the inspection of welds created by Metal Inert Gas welding (MIG welding). "Until now, inspections were always carried out by long-term employees. It's not always easy to determine whether the quality of welds from different processes is OK. When implementing the new system, we incorporated employees' experience. Specifically, we trained the underlying Deep Learning networks with their knowledge. The robust recognition rates required are only achievable through the use of Deep Learning," explains Guillermo Martín, Innovation & Technology Director at DGH. The primary goal of the implementation was to achieve a very high quality standard for all welds. In addition, the new autonomous quality inspection was designed to leverage the fundamental benefits of automation—namely higher speed, reliability, accuracy, and clear consistency in decision-making as opposed to the subjectivity of human decisions.

Seamless Process Integration of Industrial Image Processing

The implementation of such a system was accompanied by several challenges. "It was clear to us that we needed to implement the system based on Machine Vision. Sensors or conventional 2D vision systems fail due to the complexity of the welds. The first challenge was therefore to develop a viable solution and reliably detect the various types of defects. Furthermore, and this was the second challenge, it was necessary to transfer the knowledge of experienced employees into this system, a Deep Learning application. The third challenge was ultimately to conduct the inspection processes within a short timeframe due to the tightly scheduled cycle times," explains Martín. The system now implemented at the French automotive manufacturer operates as follows: When a car body arrives at the inspection station, the PLC triggers various inspection processes. Upon receiving a trigger, the mounted 2D cameras individually or sequentially capture photos of the welds and transfer them via the Gig-E-Vision protocol to the Machine Vision software, where they are processed. The system checks for anomalies around the welds. It is capable of reliably inspecting different types of weld seams, joints, and points produced by various welding processes.

The data is then sent to the PLC, and the corresponding results are visualized on a screen. The inspection application developed by the DGH Group was designed on an industrial PC, and the system continuously monitors communication with the production line's PLC as well as with multiple 2D cameras. The core of the setup is the Machine Vision software Halcon.

Deep Learning Methods

To reliably detect defects, the image processing software uses two Deep Learning methods. First, "Instance Segmentation" is applied to locate the relevant area, specifically the weld, on the captured images. This Deep Learning technology can assign objects pixel-precisely to different pre-trained classes. In the next step, "Anomaly Detection" is utilized. The Deep Learning-based anomaly detection enables automated surface inspection and accurately identifies deviations, i.e., defects of any kind. "Anomaly Detection had two decisive advantages for us: On the one hand, the detection rates are very high and robust. On the other hand, training the underlying neural networks was simple. This is because primarily 'good images,' meaning images of welds without defects, were needed for training the Deep Learning networks. The network for anomaly detection is trained only with good images. The presence of 'bad images' is not a requirement for anomaly detection; however, a few bad images can help determine the optimal threshold for distinguishing between good and defective welds. This threshold is applied to the anomaly score, which is the result of the anomaly detection network. Determining the threshold, however, is not part of the training. As a result, we required only a small number of good images. This is very practical since they are quick and easy to acquire. Defect images are significantly harder to organize, let alone that it is impossible to obtain images of all possible defects. Here, Deep Learning has a clear advantage," explains Guillermo Martín. In images of welds that differ from the trained images, anomalies or defects are reliably detected. The size of the delta between OK and NOK is determined by the threshold value. The threshold is a parameter within Deep Learning methods that defines how much the inspected image may deviate from the trained 'good image.' This parameter can be freely adjusted by the user, providing transparency in the "black-box" decision-making process of artificial intelligence.

Labeling the Images and Training the Neural Networks

The Deep Learning technology requires that neural networks are trained with images before operation. These images must first be labeled for training. For these preliminary tasks, DGH used MV Tec's Deep Learning Tool. With this free tool, image data can be easily labeled and then conveniently trained. To begin, DGH collected images of welds, incorporating the expertise of employees. These employees reviewed each image to ensure that primarily "good images" were used for training. A mistakenly used "bad image" would distort the training results. The "good images" are then uploaded into the Deep Learning Tool and specifically labeled for the Instance Segmentation technology. The "Smart Label Tool" is available for this purpose. The user simply clicks within the area of the weld using the computer mouse, and the tool automatically outlines the weld. This ensures that the Deep Learning Tool is then trained solely on the relevant parts of the image. Employees of the automotive manufacturer were also involved in this step. They knew which areas within the images contained important information about the weld and how large the corresponding frame around the weld should be. After labeling, a split is performed. The image dataset is typically divided into 50 percent for training, 25 percent for validation, and another 25 percent for testing. Training, validation, and testing are carried out simply and conveniently at the push of a button in the Deep Learning Tool. The trained model is then saved and, thanks to the seamless connection between the Deep Learning Tool and Halcon, loaded into the Machine Vision software. The software is now ready for operation.

Machine Vision Software as the Core Component of the Inspection System

"We at DGH have been working with MV Tec for over ten years and are therefore well aware of their powerful tools and algorithms. That’s why we decided to trust MV Tec Halcon for this project as well," reveals Guillermo Martín. The challenges regarding training and speed due to the tightly scheduled cycle times have already been mentioned. Additionally, there was another requirement for the Machine Vision software: the environment is challenging due to reflective metal surfaces and varying lighting conditions.

The DGH Group was able to overcome all challenges and deliver a system with the desired high quality. "In early 2024, the first system was put into operation at the automotive manufacturer’s plant. After it ran successfully, we received a new request in April 2024 from the same manufacturer to implement a second system for weld inspection," says Guillermo Martín with satisfaction. The goals, particularly during times of skilled labor shortages, to reduce reliance on specialists for quality inspection processes and consequently increase the level of automation, were achieved. Automation based on Machine Vision and Artificial Intelligence has demonstrably minimized errors and ensured consistent and reliable detection of welding defects. This is why Guillermo Martín also believes that despite the multitude of image processing systems already in use across various industries and production processes, there is still growth potential—such as for Deep Learning solutions in particularly demanding and complex applications.

*Freelance author