Block Sparse Low-rank Matrix Decomposition based Visual Defect Inspection of Rail Track Surfaces

Abstract

Low-rank matrix decomposition has shown its capability in many applications such as image in-painting, de-noising, background reconstruction and defect detection etc. In this paper, we consider the texture background of rail track images and the sparse foreground of the defects to construct a low-rank matrix decomposition model with block sparsity for defect inspection of rail tracks, which jointly minimizes the nuclear norm and the 2-1 norm. Similar to ADM, an alternative method is proposed in this study to solve the optimization problem. After image decomposition, the defect areas in the resulting low-rank image will form dark stripes that horizontally cross the entire image, indicating the preciselocations of the defects. Finally, a two-stage defect extraction method is proposed to locate the defect areas. The experimental results of the two datasets show that our algorithm achieved better performance compared with other methods.

1. Introduction

 With the rapid development of high-speed railways in China and around the world, the quality requirements for rails are constantly increasing. Because of the trains' high speeds, the friction between the wheels and the rails is greatly intensified, which will usually cause various defects, such as crack, rolling contact fatigue, spalling of rail-head and rail corrugation etc [1,2]. These defects always indicate potential dangers. They will usually cause traffic noise and can even lead to accidents. Rail-track quality is directly related to train safety, national economies, security, and other impacts [3]. Thus, rail-surface defect inspection is a crucial issue for the daily maintenance of rails and safe operation of trains.

 Conventional rail track defect inspection depends on trained people that walk around railroad lines at regular intervals and visually check the rail tracks. This is time-consuming, subjective, and dangerous. Therefore, automated and nondestructive inspection systems are strongly recommended for practical reasons [4]. For example, Fiber Bragg grating sensors [5,6], ultrasonic inspection [7], Eddy currents [8,9], MEMS gyroscope sensors [10], and visual inspection system (VIS) [11-13]. Ultrasonic inspection is very suited to internal crack detection, but the speed is very slow and it cannot detect surface defects in rail tracks. Magnetic fields geneterated by Eddy currents are used for defect inspection using the Eddy-current testing method. Although the inspection speed is relatively fast and it can be used for surface defect inspection, the defect detection results are very sensitive to the lift-off effect of the probes and the condition of the rail tracks tested.

 Visual inspection systems have been widely used in the surface defect inspection of industrial products in the recent years due to the major advances in computer vision techniques. For example, visual inspection for defects is used for TFT-LCD panels [14,15,16], capacitive touch panels [17], solar modules [18], touch screens [19], glass, tile and steel [20], and so on. In the studies cited, fractal transformation, low-rank matrix recovery, filtering methods, random forest analysis, ICA analysis, sparse representation, and more were used to explore the characteristics of the background textures. In recent years, deep learning is also used for anomaly detection[21,22] that included adversarial training of a generative model of normal appearance. The network is trained exclusively on fault-free data and completes the image patches with a fault-free version of the missing image region. The pixel-wise reconstruction error within the cut out region is an anomaly image that can be used for anomaly detection. However, to use deep-learning-based methods, a large amount of training data needs to be captured and labeled, which is not possible for some real-world applications. In [23], Charles et.al proposed using an RPCA method based on low-rank and block-sparse matrix decomposition to achieve foreground detection. In general, the defects and backgrounds appear differently. The defects usually have a random and sparse appearance, and the background is usually presented as a homogeneously textured image. This property allows for the defect areas to be segmented from the backgrounds.

 In recent years, low-rank matrix recovery [24-36] has attracted much attention in the areas of face recognition, image recovery, de-noising, and data reconstruction. The main purpose of low-rank matrix reconstruction is to recover the low-rank matrix from a large, but sparse error data set. In different applications, this is also called sparse and low-rank matrix decomposition, robust principle component analysis (RPCA), and rank-sparsity incoherence. The traditional low-rank matrix approximation methods only consider the low-rank approximation of a single matrix, such as RPCA [24], principal component analysis based on  -norm (PCAL1) [25,26], and the recently developed low-rank matrix theory [27,28]. These methods can be considered as generalized cases of the sparse representation (SR) [29,30] method. Generally, these methods decompose a noisy low-rank matrix into a low-rank matrix plus a sparse error matrix. In real video applications, the columns of each frame will be concatenated to form a vector, and all vectors obtained from a video clip are aligned to form a low-rank matrix. Low-rank analysis can be applied in this manner to this large matrix. This type of operation will immensely increase the dimensions of the resulting low-rank matrix and destroys the inherent 2-D correlations within an image. This produces very slow decomposition speeds and decreases the recovery performance.

 To address this issue, generalized low-rank approximations of matrices (GLRAM) [31] and robust generalized low-rank approximations of matrices (RGLRAM) [32] were developed with the ability to directly process a set of matrices. The GLRAM method uses the  -norm to measure the approximation error, but the  -norm is sensitive to outliers. Thus, RGLRAM methods based on the  -norm were proposed that are more robust to outliers and large sparse noise than existing  -norm-based methods. However, both of them ignore the rank of the approximation matrix. The rank of the approximation matrix is closely related to the amount of detail in the output image. A matrix with a higher rank often preserves more detail components in the output images. The noise points in an image often correspond to the detail image components. Thus, rank minimization is considered in the optimization framework presented in [33]. This study proposed to jointly minimize the nuclear norm of the approximation matrix and the  -norm of the sparse matrix. Furthermore, considering the smoothness of the approximation images, [34] proposed a smoothed and re-weighted low-rank matrix recovery (SRLRMR) method by using the re-weighted nuclear norm. Moreover, in [35], the TV norm and the pixel range constraint were incorporated into the existing reweighted low-rank matrix analysis to achieve structural smoothness significantly improve quality in the recovered image.

 

2. Related Work

 The rail track surface discrete defects (RSDDs) inspection system has three main parts[13]: the image-acquisition subsystem (IAS), image-processing subsystem (IPS), and automatic-control block. Using a CCD camera, high- quality rail-track images can be acquired under the LED light sources. Then, the images are processed by defect inspection software to execute an automated inspection. The control block will automatically control the LED light sources to adjust brightness and ensure image quality. [11] proposed a Michelson-like contrast (MLC) measure to enhance rail images, along with a proportion emphasized maximum entropy (PEME) thresholding algorithm to extract the defect areas. In [12], preprocessing including local normalization was applied to enhance the contrast of rail track images. Then, a projection profile was utilized to locate the defect positions. In [13], a hierarchical extractor scheme was proposed for rail-track defect inspection. The coarse extractor is responsible for finding the background modes. The fine extractor can reduce most of the impacts of noise interferences by vertically fusing the context information, and horizontally fusing the prior information. In [37], a new inverse Perona-Malik (P-M) diffusion model was proposed for image enhancement. Then, an adaptive threshold binarization method was proposed for defect inspection. By using the Gabor texture feature, Mandriota et al. [38] applied the k-nearest neighbor (KNN) method to identify potential defects in rail track images. In [39], local or small corrugations were detected by importing local frequency features. In [40],  by using the maximally stable extremal regions technique, geometrical features, such as the area, centroid, filled area and perimeter of the defect regions, were extracted for rail surface defect sensing. In [41, 42], wavelet transform was compared with Fourier analysis and showed its effectiveness for rail inspection. In [43], an adaptive multilevel thresholding method was applied to segment the ROI of the tramway to estimate the threshold parameters by using a local accumulated histogram. Then the region growing method was adopted to decrease the influence of environmental noise and to predict the trend of the tramway. Generally, global or local spatial patterns of intensity extraction is the key issue in the above mentioned approaches, which will obtain good results for uniform textured rail surface images. However, because different rail track appearances and dynamic backgrounds are caused by heavy random noise and varying camera quality, these methods may fail in complex situations[44]. Moreover, most techniques require case-by-case adjustment of parameters, which is complex and depends heavily on individual experience. This limits the applications of these techniques.  

 For the low-rank matrix decomposition-based defect inspection, in [17], combining with image tolerance models, a nonnegative matrix factorization (NMF)-based large-size image registration method was proposed to detect defects in capacitive touch panels. In [15], the textured background of an LCD image was modeled as a low-rank matrix, and the foreground image with defects is treated as a sparse matrix. Then, a joint minimization of  the nuclear norm and   norm is applied for defect image segmentation. Following the ideas of [15], an RPCA model was used for surface defect detection of aluminum foil [45]. In [46], the compressed sensing method was used in the rows and columns of defective images for defect inspection, then a hardware realization method was introduced for LCD mura detection. [47] proposed a low-rank representation method based on eigenvalue decomposition for the textile fabric defect detection, and the blocked-matrix method was proposed to improve the low-rank representation technique, which  improved the accuracy and efficiency of defect detection. [48] proposed a robust principal component analysis (RPCA) model with a noise term to handle fabric images with diverse patterns. The author decomposed the original image as a sum of a low-rank background, a noise matrix and a sparse defect matrix. The Gabor filter with multi-scales and multi-directions was applied in the fabric defect images for feature extraction in [49]. Then, the feature images were partitioned into blocks and the feature vectors in the blocks were re-constructed as a feature matrix. Finally, the RPCA model was used on the feature matrix to realize the segmentation of the sparse (defect) matrix.

 Based on the above, we can see that low-rank matrix decomposition plays an important role in defect inspection because it does not require training data, and the defect areas can be segmented well into the sparse matrix. Within the low-rank decomposition models, RPCA is a popular basic model. The model proposed in this paper for rail track defect detection is also based on RPCA, and is represented as

\(\min _{A, E} \operatorname{rank}(A)+\lambda\|E\|_{0}, \text { s.t. } \quad D=A+E\)       (1)

where D is the original image, A denotes the low-rank matrix (texture background) and E denotes the sparse matrix (the sparse foreground). However, the   norm is usually used as the relaxation of   norm (it counts the non-zero points of a vector). This model is better for defects with small areas or for noise-like defects. Thus, the optimization problem (1) can be re-written as:

\(\min _{A, E}\|A\|_{*}+\lambda\|E\|_{1}, \text { s.t. } D=A+E\)       (2)

 Here,   is the nuclear norm of A. This model is known as the Robust Principal Component Analysis (RPCA)  and can be solved by the accelerated proximal gradient (APG) algorithm, augmented Lagrange multipliers (ALM), and the inexact ALM (IALM) algorithm.

 

3. Motivations and Contributions

 The RPCA model shown in Eq. (2) can only decompose a noisy low-rank matrix into a low-rank matrix plus a sparse error matrix. The   norm in Eq. (1) is effective for the sparse noise, which corresponds to the small and sparse defects when the RPCA model is used in surface defect inspection. In rail surface defect inspection, different defect shapes will appear because of the long period of friction between the wheels and the rails, as shown in  Fig. 1. The small defects can be modeled as sparse noise in the images. However, for defects with large areas, the   norm will not be effective. This is because within the large defect areas (Fig. 1(c) and (d)), the images are also smooth. But from the perspective point of the entire image, the large defect areas are block-sparse. Thus the sparse term described by the   norm in Eq. (2) can not handle the block sparse defects well. The optimization models of Eq. (1) or (2) will treat the defects with large areas as the background (the defect areas will horizontally cross the whole track or be evenly merged into the background, which will be treated as the low-rank background), . such that the defect areas will remain in the low-rank background image rather than in the sparse foreground image. Thus, new methods needs to be considered for rail track defect inspection.

 Fig. 2 shows the defect decomposition results directly obtained by optimization problem (2). It can be seen that this model can be very effective for small defects (Fig. 2(a) and (b)).The defect areas are almost totally decomposed into the sparse matrix E. For the large defects, the defect areas fail to be decomposed into the sparse foreground image because they are also low-rank to some extent (Fig. 2(c) and (d)). However, large defect inspection is the crucial issue because they are dangerous for the train's safety. 

Fig. 1. Rail track images under different illuminations. The red rectangles denote the defect areas. (a) and (b) are the rail track images with small hole defects. (c) and (d) are the rail track images with large hole defects.

 To solve this problem, a new model needs to be established that enables not only small defects, but also large defects to be decomposed into the sparse matrix. In general, the defects (small and large defects only; tiny defects are not considered) can be considered as a kind of block-sparse areas in an image. Thus, we decided to use the 2,1-norm to describe the block sparsity of the defects in the rail track image. Using the optimization model with 2,1-norm minimization, all the defects can be decomposed well into the sparse matrix. In addition, another advantage by using this model is that the obtained low-rank images will contain some dark stripes cross the images, which just correspond to the defect areas one by one.  This provides us a easy way to find the row indexes of the defects and ultimately used to locates the defect positions. The contributions of this paper are as follows.

Fig. 2. Low-rank decomposition results of the four images in Fig. 1 using the RPCA model. For each sub-figure, the left image is the original image, the middle one is the resulting low-rank approximation matrix A, and the right image is the sparse matrix E of Eq. (2).

 (1) A block sparse low-rank matrix decomposition model is established that can decompose the original image into a low-rank approximation matrix and a block-sparse matrix.

 (2) A two-stage defect extraction method is presented in this paper. According to the low-rank decomposition model we proposed, the 2,1-norm will force the obtained low-rank matrix to be smooth, and the defect areas will become dark stripes that horizontally crossing the entire image. Thus the upper and lower vertical bounds of each defect area can be detected by the low-rank matrix. Then, a simple binarization method can be used in the corresponding small areas of the sparse matrix to locate the exact defect areas.

 (3) The experiment results show that our proposed model is effective and accurate for different defect shapes under varied illuminations.

 The rest of this paper is organized as follows. In Section III, the optimization model will be presented and the Alternating Direction Methods (ADM) will be used to solve the optimization problem. In Section IV, a two-stage defect extraction method is presented based on the decomposed low-rank matrix and the block sparse matrix. Section V presents the experiment results, and Section VI concludes the paper.

 

3. Low-Rank Matrix Decomposition with Block Sparsity

 For a matrix \(X \in R^{M \times N}\) , the 2,1-norm of  X is defined as

\(\|X\|_{2,1}=\sum_{i=1}^{M} \sqrt{\sum_{j=1}^{N} x_{i, j}^{2}}=\sum_{i=1}^{M}\left\|x^{(i)}\right\|_{2}\)       (3)

 where \(x_{i, j}\)  denotes the element in the ith row and jth column of  X and X⁽ⁱ⁾  denotes the ith row of matrix  X .

 According to Fig. 1, we can see that the defects can be considered as a kind of block sparsity in the image. Additionally, the background is obviously a low-rank matrix. Thus, we propose the following model for the decomposition of the original rail track image:

\(\min _{A, E} {rank}(A)+\lambda\|E\|_{2,1}, \text { s.t. } \quad D=A+E\)       (4)

 As shown in [50], the rank minimization problem is NP-hard, which is difficult to be solved directly. It has been recognized that the rank minimization problem can be relaxed by using the nuclear norm [51]:

\(\min _{A, E}\|A\|_{*}+\lambda\|E\|_{2,1}, \text { s.t. } \quad D=A+E\)       (5)

 In the rest of this paper, we will refer to the problem in (5) as the low-rank matrix decomposition with block sparsity (LDBS). The corresponding augmented Lagrangian function for this problem is

\(\begin{aligned} f(A, E, Y)=&\|A\|_{*}+\lambda\|E\|_{2,1}+<Y, D-A-E>+\frac{\mu}{2}\|D-A-E\|_{F}^{2} \\ &=\|A\|_{*}+\lambda\|E\|_{2,1}+\mu<\frac{Y}{\mu}, D-A-E>+\frac{\mu}{2}<D-A-E, D-A-E>\\ &=\|A\|_{*}+\lambda\|E\|_{2,1}+\frac{\mu}{2}\left\|D-A-E+\frac{Y}{\mu}\right\|_{F}^{2}-\frac{\mu}{2}<\frac{Y}{\mu}, \frac{Y}{\mu}> \end{aligned}\)      (6)

where Y is the Lagrange multiplier, \(\mu\)  is the regularization parameter, \(\|\cdot\|_{F_{F}}\)  denotes the Frobenius norm and   denotes the inner product. The LDBS optimization algorithm aims to find the optimal solution of (6). The pseudo code of our proposed LDBS algorithm is shown in Algorithm 1. Similar to ADM, we use the alternative method to solve this optimization problem. To simplify the solution, we can fix the other variables (or suppose that they are known) such that the original multi-variable optimization problem is transformed into a single variable optimization problem. Then, with multiple iterations, the original problem can be eventually solved. We now discuss how to update the optimization variables in each iteration.

 A. Updating A

 If A is unknown and the other variables are fixed, then A can be updated by minimizing   with respect to A, as follows.

\(\begin{aligned}\arg \min f(A, E, Y) &=\arg \min _{A}\|A\|_{*}+\frac{\mu}{2}\left\|D-A-E+\frac{Y}{\mu}\right\|_{F}^{2} \\&=\arg \min _{A}\|A\|_{*}+\frac{\mu}{2}\left\|A-\left(D-E+\frac{Y}{\mu}\right)\right\|_{F}^{2}=\Gamma_{1 / \mu}\left(D-E+\frac{Y}{\mu}\right)\end{aligned}\)       (7)

where \(\Gamma_{1 / \mu}(X)=U S_{1 / \mu}(\Sigma) V^{T}\)  and  \(U \Sigma V^{T}\) is the singular value decomposition of the matrix X.   is an absolute value shrinkage operator, defined by

\(\left(S_{1 / \mu}(X)\right)_{i j}=\left\{\begin{array}{ll}\left|X_{i j}\right|-\frac{1}{\mu} & \text { if }\left|X_{i j}\right|>\frac{1}{\mu} \\0 & \text { otherwise }\end{array}\right.\)       (8)

 for \(X=\left(x_{i j}\right)_{m \times n} \in R^{m \times n}\)  and  \(\frac{1}{\mu}>0\).

 B. Updating E

 If E is unknown and the other variables are fixed, then E can be updated by minimizing \(f(A, E, Y)\)  with respect to E, as follows.

\(\begin{aligned}\arg \min f(A, E, Y) &=\arg \min _{E}\lambda\|E\|_{2,1}+\frac{\mu}{2}\left\|D-A-E+\frac{Y}{\mu}\right\|_{F}^{2} \\&=\arg \min _{E}\lambda\|E\|_{2,1}+\frac{\mu}{2}\left\|E-\left(D-A+\frac{Y}{\mu}\right)\right\|_{F}^{2}=\arg \min _{E} \lambda\|E\|_{2,1}+\frac{\mu}{2}\|E-M\|_{F}^{2}\end{aligned}\)       (9)      

where \(M=D-A+\frac{Y}{\mu}\)  . Because of the non-smoothness of  \(l_{2,1}\) , solving Eq.(9) exactly is highly nontrivial. In [52], Xiao et.al decomposed Eq. (9) into N subproblems of dimension M, i.e.,

\(\hat{E}_{i,:}=\min _{E_{i, :}} \lambda\left\|E_{i,:}\right\|_{2}+\frac{\mu}{2}\left\|E_{i,:}-M_{i, :}\right\|_{2}^{2}, i=1, \ldots, N\)        (10)

 Additionally, by constructing the Lagrangian dual form, the closed-form solutions of Eq. (9) can be obtained as

\(\hat{E}_{i,:}=\left\{\begin{array}{lc}\left(1-\frac{\lambda}{\mu\left\|M_{i,:}\right\|_{2}}\right) \cdot M_{i,:}, \quad \text { if }\left\|M_{i,:}\right\|_{2}>\frac{\lambda}{\mu} \\0 \quad\quad\quad\quad\quad\quad\quad otherwise\end{array}\right.\)       (11)

where \(\hat{E}_{i, i}\) denotes the ith row of the optimal solution \(\hat{E}\) .

 C. Updating Y

 Y is the Lagrange multiplier of the original optimization problem. It should be updated after the other variables. When Y is unknown and other variables are fixed, Y can calculated by

\(Y=Y-\mu(D-A-E)\)       (12)

 The algorithm for solving (6) is described in Algorithm 1. The stopping criterion is set as

\(\|D-A-E\|_{F} /\|D\|_{F}<\varepsilon\)       (13)

where e  is a small positive number. In addition, similar to RPCA and RGLRAM, to obtain better performance, the value of the parameter  \(\mu\) will be tuned by \(\rho\)  in each iteration. The parameter   is set to be larger than 1 [31], [33]. In our experiments,   can be set as  \(1<\rho<5\).

Algorithm 1. Low-rank matrix decomposition with block sparsity (LDBS)

Require: \(D \in R^{M \times N}, \lambda\). Ensure: Initialize \(A=O^{M \times N}, E=O^{M \times N}, \rho, \mu_{\max }=10^{10}, \varepsilon=10^{-6}\) while \(\|D-A-E\|_{F} /\|D\|_{F}<\varepsilon\) do      Step 1: Update A according to Eq. (8).     Step 2: Update E according to Eq. (11).     Step 3: Update Y according to Eq. (12).     Step 4: Update \(\mu\)  as \(\mu=\min \left(\rho \mu, \mu_{\max }\right)\). end while

 

4. Two-Stage Defect Extraction Based on Low-Rank Matrix Decomposition

 According to Algorithm 1, a rail track image can be decomposed into a low-rank matrix and a block sparse matrix. The decomposition results of the four images in Fig. 1 are shown in Fig. 3.

Fig. 3. Decomposition results using the LDBS algorithm. For each sub-figure, the left image is the low-rank matrix A, and the right image is the sparse matrix E in the Eq. (6).

 According to Fig. 3, it can be seen that from the perspective of low-rank matrix recovery, the results of our algorithm are not good and are even worse than other low-rank matrix recovery method such as RPCA. We can see in Fig. 2(a) and (b) that the low-rank approximation images are much better than the results in Fig. 3(a) and (b). However, in Fig. 3, almost all the defect areas are clearly decomposed into the sparse matrix, especially the large defects, which is not possible in Fig. 2(c) and (d). The goal of our study is defect inspection, not low-rank image recovery, so the recovered low-rank image quality is not our key issue. Instead of focusing on the recovered low-rank image quality, we observed that because of the smoothness effect of the low-rank decomposition model, the defect areas will form dark stripes (indicated by the red circles in Fig. 3) that horizontally cross the entire recovered low-rank background image (i.e., the low-rank matrix A in the Eq. (6)). Even a small defect will produce a small dark stripe. These dark stripes reveal the row indexes of defect areas in the original image, and clearly distinguish them from other normal parts. According to this property, a two-stage defect extraction method is proposed in this section to locate the defect positions: In the first stage, the row indexes can be identified for each defect by the dark stripe in the low-rank image.  Because the rail track is narrow (the column number of the image is much less than the row number), once the row indexes of the defect are detected, the defect will be limited in a small image block. In the second stage, the column indexes of the defect area can be easily identified within the small image block by performing some simple operations.

 

4.1 First Stage: Finding the Row Indexes of Defect Areas

 To determine the upper and lower row bounds of each defect, we first draw a column (the values in this column are normalized into [0,1] by dividing the maximum absolute value in this column) of the obtained matrix A, as shown in Fig. 4.

 Fig. 4 shows that the pixel values within the defect areas will be much smaller than others, and the dark stripes form several peaks. Thus, we only need to find the indexes of the peaks to locate the positions of the defects.

 As seen in Fig. 3, the illumination of the rail tracks varies within an image. To enlarge the gaps between the peaks and other normal points in one column, we select the brightest column in a rail track image as the column to  find the start and end points of the peaks. The column selection criterion is as follows.

\(J=\arg \max _{j}\left\{\sum_{i=1}^{M} a_{i j}\right\}\)       (14)

where  a_ij is the pixel value in the (i,j)^th position of \(A \in R^{M \times N}\), J  is the column index used for peak detection.

Fig. 4. Plots of the first column corresponding to the four obtained low-rank matrices shown in Fig. 3.

 After the column is selected by Eq. (14), denoted as \(a_{i, j}\) , a threshold  \(\tau\) is set to this column:

\(a_{i, J}=\left\{\begin{array}{ll}0, & a_{i, J}<\tau \\1, & a_{i, J} \geq \tau\end{array}\right.\)       (15)

where  \(\tau=\beta \cdot \operatorname{mean}\left(a_{i, j}\right), \beta\)  is a constant and  \(\operatorname{mean}\left(a_{\pm},\right)\) is the average value of the column \(a_{i, j}\) . We call this operation column binarization. An example of column binarization is shown in Fig. 5(b). It can be seen that there are a total of four peaks in Fig. 5(b). However, the two peaks marked by the red rectangles only have one point at the top, which corresponds to two tiny defects. We do not consider tiny defects since they will not affect train safety. Thus, such peaks should be removed. To do this, we  simply measure the width of each peak and remove the ones with widths less than or equal to q points. The result after peak removal is shown in Fig. 5(c). For comparison, Fig. 5(d) shows the sparse matrix E obtained by Eq.(5). To align it with the other three sub-figures, below is the transpose of E.

Fig. 5. Column binarization and tiny peak removal. (a) The column of low-rank matrix A in Fig. 3(d) selected for peak detection; (b) Column binarization result; (c) The result after peak removal. (d) Sparse image of Fig. 3(d).

 Through the above column binarization and peak removal operations, we can identify the start and end points of each peak, which correspond to the vertical bounds of the defects. The red rectangles in Fig. 6 indicate the defect areas.

Fig. 6. Defect detection results.

4.2 Second Stage: Finding the Column Indexes of the Defect Areas  

 After the vertical index bounds of the defects are obtained, the image blocks containing the complete defect area can be cropped from sparse image E. In the cropped image blocks, the background is much smoother than in the original defect image, which is the influence of the original defect image background being almost completely removed by applying Algorithm 1. Additionally, because we use the block sparsity decomposition model, both the small and large defects can be decomposed into the sparse image. Therefore, we can simply binarize the image blocks (areas indicated by red rectangles in Fig. 6) to obtain the exact defect areas. In other words, the horizontal column index bounds of the defect areas can be determined by the maximal-connected  component in the binarized image block. Fig. 7 shows an example of the second stage of the image shown in Fig. 6(b).

Fig. 7. Process to find the column indexes of the defects. (a) Image blocks cropped from the sparse image E (image blocks in red rectangles, Fig. 6(b)). (b) Image block binarization results. (c) Results obtained by keeping the maximal-connected component.

 Finally, the defect areas can be marked in the original rail track images. The defect detection results of the 4 images in Fig. 1 are shown in Fig. 9. The complete process  proposed in this paper is summarized in the following flowchart.

Fig. 8. Flowchart of the proposed rail track defect detection process.

Fig. 9.  Defect inspection results of the 4 images in Fig. 1.

 

5. Experiment Results

 To evaluate our proposed algorithm, two datasets are used in our experiments in the same way they were used in [13]. The first is a Type-I RSDDs dataset of 67 images captured from high-speed rails. The second is a Type-II RSDDs dataset of 128 images captured from common/heavy-haul rails. Several image examples for both types are shown in Fig. 11 and Fig.  12. The experiments are conducted in Matlab  2016b with a Surface 3 laptop. Pixel-level and defect-level criteria are used to evaluate the performances of our algorithm. Both criteria are based on the common criteria of Precision, Recall and F-measure (abbreviated as Pre, Rec and F, respectively). They are defined as follows:

 A. Pixel-level criterion:

 In this situation, Pre, Rec and F are defined as:

\(\begin{array}{c} { Pre }=T P /(T P+F P) \\ { Rec }=T P /(T P+F N) \\F=2 * { Pre* Rec/(Pre+Rec) }\end{array}\)       (16)

where TP denotes the number of correctly detected pixel points, FP is the number of falsely detected pixel points, and FN is the number of undetected defect pixel points. In addition, a higher F-measure means that the detection result is consistent with human judgment. Note that the values we shown in this section are the average values for the whole dataset.

 B. Defect-level criterion:

 Pre', Rec' and F' are defined as:

\(\begin{array}{c} {Pre}^{\prime}=T P_{d} / P \\{Rec}^{\prime}=T P_{d}^{\prime} / N \\F^{\prime}=2 * {Pre}^{\prime} * {Rec}^{\prime} /\left(\text {Pre}^{\prime}+{Rec}^{\prime}\right)\end{array}\)       (17)

where  \(T P_{d}\) denotes the number of correctly detected defects,  \(T P_{d}'\) is the number of correctly detected defects mapping to ground truth, P refers to the total number of detected defects, and N refers to the total number of labeled defects in the ground truth. Suppose that the s^th  defect area is denoted as area(s)  and the defect area in the groundtruth image is denoted  as area(GT) , then TP_d  is defined as

\(T P_{d}=\bigcup_{s=1}^{P}\left(\frac{{area}(s) \wedge {area}(G T)}{{area}(s)}>0.8\right)\)       (18)

TP'_d  is defined as

\(T P_{d}^{\prime}=\bigcup_{s=1}^{P}\left(\frac{{area}(s) \wedge {area}(G T)}{{area}(G T)}>0.8\right)\)       (19)

 Here, \(\bigcup_{s}(\cdot)\) counts the number of s that satisfies the condition shown in the brackets.  denotes the intersection.

 In our algorithm, there are two parameters that need to be adjusted. One is q (decides the size of the defects that can be detected by our algorithm), the other is \(\beta \) (decides the threshold  ). Since tiny defects can be ignored here, we set q=3, which means that if the defect height is smaller than 3 pixels, the defect will not be detected. For \(\beta \)  , 20% of the images are randomly chosen to evaluate the proposed method. The relationship between the value of \(\beta \)  and the F-measure are shown in Fig. 10. It can be seen that for the Type-I RSSDs, the best performance is achieved when \(\beta \) , while the best performance is achieved when   for Type-II RSSDs. Thus, these two values are selected in the final testing.

Fig. 10. Relations between   and F-measure values. (a) Type-I RSSDs; (b) Type-II RSSDs.

 We compared our defect inspection results with several well-established methods, i.e., LN+DLBP[12], MLC+PEME[11], the method in [37], and CFE[13]. The performances of all  four methods are compared in [13], so we can just use the results shown in Tables 1 and 2 in [13]. It should be noted that LN+DLBP locates defects by bounding boxes, and the method in [37] detects contours to locate possible defects. Therefore, pixel-level criterion is not suitable for these two methods. In the Step 4 of our proposed Algorithm 2, a binarization operation is performed for the small areas obtained in Step 3, and the defect areas can be obtained in a pixel-wise manner. Thus, both the defect-level and pixel-level criteria are suitable for our algorithm. The results are compared and presented in Tables 1 and 2.

Table 1. Comparison of our method and other methods for the Type-I RSDDs dataset.

Table 2. Comparison of our method and other methods for the Type-II RSDDs dataset .

 

 The two tables above show that our algorithm achieves better performance compared to others. This is especially ture for the recall since a higher recall value means that more defects labeled in the ground truth are detected correctly. However, the precision of our algorithm is not always the best. This means that our algorithm has a high level of false detection. However, the F-measure of our algorithm is higher than others. Finally, some of the detection results are shown in Fig. 11 and 12 for Type-I and Type-II RSSDs, respectively. It should be noted that the subfigures (b)-(e) all come from [13], so we just added our result in subfigure (f) for comparison. These figures indicate the defects detected with red rectangles, which clearly show the accuracy of the detected defect bounds.

Fig. 11.  Several inspection results using different methods with the Type-I RSDDs dataset. (a) defect part (b) ground truth (c) LN+DLBP (d) MLC+PEME (e) Method in [37] (f) our method.

Fig. 12. Several inspection results using different methods with the Type-II RSDDs dataset. (a) defect part (b) ground truth (c) LN+DLBP (d) MLC+PEME (e) Method in [37] (f) our method.

 

6. Conclusions

 In this paper, we proposed a block sparse low-rank matrix decomposition based rail track visual inspection method to address the problem that traditional low-rank matrix decomposition algorithms cannot deal with large defects. Because the rail track is narrow, a large defect in the rail track will be treated as image background, so it cannot be decomposed well into the sparse foreground. Thus, the 2,1 norm is used in our decomposition model and the large defect areas are treated as block-sparse areas. After decomposition, the defect areas in the resulting low-rank background will form dark stripes that horizontally cross the entire image, indicating the precise locations of the defects. Finally, defect areas can be easily detected after simple thresholding and binarization processes. The experiment results showed the superiority of our algorithm compared to other methods, especially for the recall, as we achieved about a 10% improvement on the two test datasets.