Development of Anomaly Detection Methods for a Collaborative Robot in Chemical Drum Assembly

Abstract

In this paper, anomaly detection methods for a collaborative robot during the chemical drum assembly process in the semiconductor industry are presented. The manual assembly of chemical drums has been automated using robots to prevent industrial accidents. However, the automation may increase downtime due to anomalies or failures in the robot manipulator tasks. To prevent this issue in advance, the methods to diagnose anomalous behaviors and conditions in the robotic automation workflow and subsequently resume tasks are proposed. To detect and diagnose anomalies in the tasks, the Random Forest classification method was utilized. Using this Random Forest classification, the collaborative robot anomaly detection model achieved an accuracy of 98.91%, successfully detecting all anomalies in the assembly process.

1. Introduction

In the semiconductor industry, over 100 types of chemicals are used for purposes such as cleaning, etching, stripping, and electroplating. The chemicals contained within clean drums are delivered to the Central Chemical Supply System (CCSS) through a dispensing system. The dispensing system is connected to the chemical drum by attaching a dispenser for suction. With the improvement in robot performance, robotic automation has been increasingly applied in the semiconductor industry to enhance productivity and efficiency and to address issues such as safety accidents during tasks [1]. Despite improvements in robot performance, robotic automation has experienced an increase in unexpected downtime [2]. When the system is suddenly halted in robotic automation due to failures, errors, or natural disasters, negative impacts on products, time, and injuries are caused. Therefore, additional strategies to detect and minimize problems in advance are required.

Fig. 1 shows the application of robot automation in the chemical drum assembly. In the chemical drum assembly, robotic automation is used to assemble the dispenser to the inlet of the chemical drum at the supply station using mobile manipulator [3,4].

Fig. 1 Process for chemical drum assembly

In this paper, a method for anomaly detection and classification in robotic automation during the chemical drum assembly is presented. The method is designed to minimize downtime and classify the causes of downtime.

2. Anomaly Detection Process in Chemical Drum Assembly

As shown in Fig. 2, anomaly detection is performed according to the automated assembly process of chemical drums and is represented in the following steps:

Fig. 2 Anomaly detection method for robotic automation in the chemical drum assembly process

1. Vision detection: The inlet where the dispenser will be assembled is recognized through vision, and the central coordinate of the inlet is returned. The robot end-effector is aligned according to the returned central coordinate.

2. Dispenser assembly: In the aligned state of the robot end-effector, the dispenser located at the end of the robot is assembled to the inlet. Force control is performed through the F/T sensor mounted at the end of the robot to minimize the reaction force generated during the assembly.

3. Assembly anomaly detection: The reaction force generated during the dispenser assembly process is monitored for anomalies, and a machine learning-trained prediction model is used to classify the success or failure of the assembly.

4. Anomaly detection of collaborative robot: Data on joint positions, speed, and current generated during the robot's operation are collected. Anomalies are detected using a machine learning-trained prediction model, and load and vibration are classified.

In industrial settings, vision systems can encounter various issues such as lens contamination from chemical gases of the chemical drum and installation position errors. To detect these anomalies, if the robot is aligned according to the coordinates returned during the detection phase, the Vision task returns 1. If the robot is not aligned, the Vision task returns 2.

If the vision task returns 2, the vision system recognizes that the inlet is not within the image frame and moves the robot upward along the z-axis to achieve a wider field of view (FOV), then returns vision task 3. After this, vision detection is performed again. If the inlet is detected, the robot aligns accordingly. If vision detection fails, the anomaly is reported to the operator, and the robot remains on standby.

Vision detection was performed using the Intel Realsense D455 [5], and the contours of the RGB image were detected using the Find Contours function defined in the OpenCV library. The circular shape of the contour was confirmed using the approxPolyDP function, and the minEnclosingCircle function was used to calculate the circumscribed circle and return the central coordinates of the circle.

This allowed the robot end-effector to be aligned with the assembly location. The measurement height for vision detection was defined as 500mm, and the pixel resolution obtained through experiments was 0.205mm/pixel. Fig. 3 shows the results of vision detection, indicating cases where the assembly location is outside the camera’s field of view and thus not detected, and cases where the assembly is successfully detected.

Fig. 3 Vision detection results(top: detection failure; bottom: detection success)

Fig. 4 is a demo configured for experimentation and data measurement for automated chemical drum assembly. The configured robot system (UR10e [6] equipped with a commercial F/T sensor, Onrobot HEX-E [7]) was used, and the dispenser {R} and assembly {H} were modeled to perform assembly tasks and classify detected anomalies.

Fig. 4 Demo setup for automated chemical drum assembly process

3. Anomaly Detection Methods for a Collaborative Robot

Anomaly detection for collaborative robots was performed using machine learning methods to classify the robot's state based on its data [8]. Among the machine learning methods, the Random Forest is characterized by using an ensemble learning method of a single tree structure. As shown in Fig. 5, Random Forest randomly extracts new sample sets from the original data set using the bootstrap resampling method and constructs decision trees with these samples. The classification result is determined by a majority voting strategy, where the class that receives the most votes becomes the final classification result [9].

Fig. 5 Configuration of the Random Forest model

This parallel structure of supervised learning is significantly faster in terms of training speed compared to other learning methods and was chosen to efficiently construct the collaborative robot anomaly detection prediction model when handling large-scale data sets, such as those in robot systems.

Fig. 6 provides an overview of the collaborative robot/assembly anomaly detection. Before training the model, the robot data(Joint Position(θ), Joint Velocity(\(\begin{align}\dot{\theta}\end{align}\)), Joint Current(I) and F/T sensor(f,m) data undergo a data preprocessing process to be transformed into various features. The purpose of transforming data into features is to obtain more accurate detection results more easily in machine learning. The differences between actual and target data generated from the raw data were critical, and the sliding window technique was used to divide the features into a one-second timeframe. It was observed that using data less than one second long for detection tended to decrease the model's prediction accuracy. The data used and the calculated features are shown in Table 1.

Fig. 6 Method overview for the robot/assembly anomaly detection

Table 1. Analyzed robot data

(Note) The subscripts ‘d’ and ‘a’ denote the desired (or target) and actual values in the 6-axis joints, respectively. All the vectors are 6×m ones.

Table 2 shows the classification according to the model type. Two different types of models are trained for collaborative robot/assembly anomaly detection. The anomaly detection model for the collaborative robot is trained to detect anomalies in the joint operations used by the robot and can classify multiple types of anomalies through multi-class classification. The assembly anomaly detection model is trained using data from the F/T sensor and can detect anomalies in the assembly operation.

Table 2. Classification by model type

Fig. 7 illustrates the setup for the chemical drum assembly demo experiment. The process of aligning the robot end-effector to the center of the assembly detected by the robot vision and performing the assembly was repeated to generate data for model training. The diameter of the dispenser and assembly model is Φ56mm, with a tolerance difference of h7 (0.03mm).

Fig. 7 Experiment on automated chemical drum assembly process

Fig. 8 shows the feature data used for training the collaborative robot/assembly anomaly detection models, with the following anomaly conditions created for each segment:

Fig. 8 Data according to anomalies for training

①: Robot operating under normal conditions

②: Robot operating under abnormal conditions with a 3kg load applied

③: Robot operating under abnormal conditions with a 6kg load applied

④: Robot operating under abnormal conditions with vibrations (300rpm)

ⓐ: Successful assembly operation

ⓑ: Failed assembly operation

Fig. 9 presents the validation feature data used to assess the prediction accuracy of the model. It distinguishes between normal state, anomaly state 1, anomaly state 2, and anomaly state 3 over the sequence of operation time, and is further divided into data from 3 successful assemblies and 3 failed assemblies.

Fig. 9 Data according to anomalies for validation

Fig. 10 and 11 compare the prediction results and actual labels of the feature data. To verify the accuracy of the model results, the accuracy of the multi-class classification can be determined by dividing the number of differences between the predicted labels and actual labels by the total number of samples in the test dataset.

Fig. 10 Prediction results of collaborative robot anomaly detection classification

Fig. 11 Prediction results of assembly anomaly detection classification

The classification accuracy of the collaborative robot anomaly detection model was 98.91%, and the assembly anomaly detection model detected all anomalies.

6. Conclusions

A method for detecting and classifying anomalies in the chemical drum assembly process used in the semiconductor industry was presented. By classifying work anomalies, the method aimed to enhance the productivity and efficiency of automated operations and address safety issues that could occur during the process.

Data for normal and abnormal states were generated through experiments, and models were developed to predict the success and failure of robot operation and assembly tasks. The accuracy of each model was evaluated by applying Random Forest. The classification accuracy of the anomaly detection model of the collaborative robot was found to be 98.91%, and the assembly anomaly detection model detected all anomalies, confirming the high predictive accuracy of the proposed method.

In this paper, the reliability of robotic automation was enhanced, and the applicability of anomaly detection method using machine learning methods in various manufacturing industries, including the semiconductor industry, was demonstrated. Consequently, the contributions were made to improving safety and production efficiency.