Irregular Workpiece Template-Matching Algorithm Using Contour Phase

Abstract

The current template-matching algorithm can match the target workpiece but cannot give the position and orientation of the irregular workpiece. Aiming at this problem, this paper proposes a template-matching algorithm for irregular workpieces based on the contour phase difference. By this, one can firstly gain the profile curve of the irregular workpiece by measuring its radius in orderly fashion and then, calculate the similarity of the template workpiece profile and the target one by phase-shifting and finally, compute the rotation measure between the two according to the number of phase movements. The experimental results showed that in this way one could not only match the shaped workpiece in the template base accurately, but also accurately calculate the rotation angle of the irregular workpiece relative to the template with the maximum error controlled within 1%.

1. Introduction

At present, the world is going through the transformation of Industry 4.0. In order to successfully transition to Industry 4.0, companies must develop and implement certain key technologies of Industry 4.0 [1]. Machine vision technology is one of the key technologies of Industry 4.0, and the application of machine vision has gradually been added to many machines and production lines and is playing an increasingly important role. It can be used for product quality inspection and target positionings, such as some appearance defect detection and accurate positioning of precision workpieces. Machine vision can also be used for hazardous tasks in some hazardous environments that are not suitable for direct human operation. Additionally, in some workpiece inspection tasks, compared with the human eye’s inspection, machine vision systems can greatly improve the inspection accuracy and inspection efficiency, their own capabilities being incomparable to the human eye. Because machine vision plays an increasingly important role today, research on machine vision has also become popular. In machine vision, the technology of judging the object input to the computer by comparing it with the template in the sample database is called template matching [2,3]. Identifying and locating objects is an important research topic, and template matching plays an important role in it. With the continuous development of template-matching technology [4], this is widely used in industry and agriculture. In recent years, the shortage of labor and the increasing demands of workers on the working environment has made the spraying industry more and more automated. Regular workpieces usually use the robot’s teaching program to realize the automatic glue-spraying process. However, due to their irregular shape, it is difficult to locate the position and posture of irregular workpieces directly. In the field of image processing, template matching searches for the existence of a known template in another image based on a known object image (i.e., template). This method is best suited for the localization of irregular workpieces if there is a target workpiece whose position needs to be located in order to guide the robot for painting [5] Therefore, the theoretical significance and practical application of template matching have great research value [6].

Template-matching algorithms are a common feature acquisition method, which has been well applied in various fields of production and life. However, the accuracy of template matching is affected to some extent by light, background, and occlusion. When the size and rotation angle of the object change, the number of templates will increase, which will increase the time of template matching and affect the efficiency of matching. In recent years, research on the efficiency and robustness of template matching has made some progress [7,8,9,10,11,12,13].

Regarding the template-matching algorithm, its general features can be divided into two categories. The first category is template-matching algorithms based on grayscale information [14,15]. It completes the matching between the target image and the template image by calculating the similarity between the template and the target image. This template-matching method is relatively simple in principle and easy to implement, and can meet real-time requirements. In some well-lit environments, it can also obtain relatively satisfactory results. However, in the actual environment, the illumination is often uneven, which leads to poor matching results. Another category is template-matching methods based on geometric features [16,17]. The features applied to template matching can generally be divided into point features, surface features, and line features. Compared with the difficulty and indistinct features in the extraction of surface features and line features, point features are widely used and studied because of their simple, clear and robust extraction process.

But no matter what method is used, it only achieves the purpose of successful matching, and can further give the center position of the target workpiece, but it cannot measure the rotation angle of the workpiece well. In order to use the robot to grasp and spray the workpiece automatically, it is also necessary to know the rotation angle of the workpiece relative to the template. The classic template-matching algorithm needs to traverse all the location points to be searched in the reference image [18]. This exhaustive search algorithm is computationally intensive, slow, and has no good practical value [19]. Given that the classical template-matching algorithm has the disadvantages of being computationally intensive and time-consuming, the researchers proposed to reduce the computation by using a fast template-matching algorithm, simplifying the correlation metric function or using a non-orthogonal search strategy [20]. For regular workpieces, the literature [21,22,23] proposed a method to proxy the amount of workpiece rotation by the rotation of the external rectangle of the workpiece, which more accurately calculates the rotation of the regular set of workpieces with respect to the stencil workpiece. Reference [18] defines a saliency constraint similarity measurement method for the automatic recognition of planar objects with irregular shapes and low heights in complex backgrounds, so as to achieve accurate matching of planar object contours. Reference [24] used the Hausdorff distance for the registration of the template image, and used the classification algorithm of the support vector machine to detect and classify the defects of the PCB board. In [25], using five white circular features around the charging port (CP), the distance to the center of the ellipse is obtained according to the principle that the center is the furthest point from the contour. In addition, the pose information on the CP is obtained by the geometric solution method. Under 4 klux light intensity, the positioning error of this method is 1.4 mm, the angle error is 1.6°, and the insertion success rate is 98.9%. The cluster template-matching algorithm (CTMA) proposed in [26] uses contour matching and logarithmic evaluation metrics to obtain matching positions. Based on the image deformation rate and zoom rate, a matching template is established to achieve fast and accurate matching of textureless circular features and complex light fields. Reference [27] combines the pixel statistics method with the template-matching method and proposes a new fastener positioning method called local unidirectional template matching (LUTM). According to the experimental results of reference [27], the method described can effectively realize the precise positioning of ballastless track and track fasteners with ballast simultaneously. In order to solve the problem of slow speed and a large amount of calculation of traditional template-matching algorithms in power image recognition, reference [28] proposes a second template-matching algorithm for fast identification of target images. The algorithm can accurately identify and locate electric power equipment, detect equipment faults, and improve the matching speed. Reference [29] proposed an improved template-matching algorithm for tube-to-tube sheet-welding positioning, which can automatically create a template for each image, instead of using one template to match all tube images, and the matching result can accurately locate the target tube. Reference [30] proposed a template-matching method based on direction code, which roughly estimates the rotation angle, which struggles to play a role in practical applications.

Combining the above research, this paper proposes a template-matching algorithm based on contour phase difference to better obtain accurate information of irregular workpiece poses. We will briefly describe the stencil matching based on geometric features and Hu-invariant moments and its shortcomings in Section 2, which leads to the algorithm proposed in this paper. Then, we will expand the description of our proposed algorithm in Section 3, discuss the experimental validation in Section 4, and finally summarize the results in Section 5.

2. Template Matching Based on Geometric Features and Hu Invariant Moments

2.1. Construction of the Base Feature Vector

The most frequently employed picture-invariant moment today was first presented by Hu, M.K. [31] in 1961. It is a series of invariant moments with image characteristics obtained by using a nonlinear combination of some regular moments. No matter how these photos are resized, rotated, or otherwise altered, these invariant moments stay the same. For a discrete two-dimensional image $f\left(x,y\right)$, the $\left(p+q\right)$ order moments can be represented as follows in accordance with the definition of Hu invariant moments in the literature [31]:

$$m_{pq}={{\displaystyle \sum }}_x{{\displaystyle \sum }}_y{x}^p{y}^{q}f\left(x,y\right)$$

(1)

The corresponding $\left(p+q\right)$ order central moments are shown below.

$$m_{pq}={{\displaystyle \sum }}_x{{\displaystyle \sum }}_y{\left(x-\overline{x}\right)}^p{\left(y-\overline{y}\right)}^{q}f\left(x,y\right)$$

(2)

where $p,q=0, 1, 2, \dots ,+\infty$, $\overline{x}=\frac{m_{10}}{m_{00}},\overline{y}=\frac{m_{01}}{m_{00}},\overline{x},\overline{y}$ denote the gray center position of the image. According to Equation (1), the geometric moments are the projections of the image on the function $x^p{y}^q$. The standardized $()$ order central moments are shown as below [32].

$${\eta }_{pq}=\frac{m_{pq}}{m_{00}^r}$$

(3)

where $r=\left(p+q\right)/2$. Hu, M.K. generated seven image moments with invariance by expanding a nonlinear combination of second-order central moments and third-order central moments from the algebraic theory. These seven image moments are shown in Equation (4).

$$\begin{gathered} M_1={\eta }_{20}+{\eta }_{02} \\ {M}_2={\left({\eta }_{20}-{\eta }_{02}\right)}^2+4{\eta }_{11}{}^2 \\ {M}_3={\left({\eta }_{30}-3{\eta }_{12}\right)}^2+{\left(3{\eta }_{21}-{\eta }_{03}\right)}^2 \\ {M}_4={\left({\eta }_{30}+{\eta }_{12}\right)}^2+{\left({\eta }_{21}+{\eta }_{03}\right)}^2 \\ {M}_5=\left({\eta }_{30}-3{\eta }_{12}\right)\left({\eta }_{30}+{\eta }_{12}\right)\left[{\left({\eta }_{30}+{\eta }_{12}\right)}^2-3{\left({\eta }_{21}-{\eta }_{03}\right)}^2\right]+\left(3{\eta }_{21}-{\eta }_{03}\right)\left({\eta }_{21}+{\eta }_{03}\right)[3{\left(3{\eta }_{21}+{\eta }_{03}\right)}^2-{\left({\eta }_{21}+{\eta }_{03}\right)}^2] \\ {M}_6=\left({\eta }_{20}-{\eta }_{02}\right)\left[{\left({\eta }_{30}+{\eta }_{12}\right)}^2-{\left({\eta }_{21}+{\eta }_{03}\right)}^2\right]+4{\eta }_{11}\left({\eta }_{30}+{\eta }_{12}\right)\left({\eta }_{21}+{\eta }_{03}\right) \\ {M}_7=\left(3{\eta }_{21}-{\eta }_{03}\right)\left({\eta }_{21}+{\eta }_{03}\right)\left[3{\left({\eta }_{30}+{\eta }_{22}\right)}^2-{\left({\eta }_{21}+{\eta }_{03}\right)}^2\right] \end{gathered}$$

(4)

It has been shown that the invariant moments based on second-order moments (i.e., $M_1$, $M_2$) can better maintain the rotation, translation, and scaling of two-dimensional objects during the recognition of target objects in images. Other higher-order invariant moments bring larger errors and have little impact on object recognition. Therefore, in this paper, $M_1 \mathrm{and} M_2$ invariant moments are selected as components of the artifact base feature vector. To enhance the accuracy of the base feature vector, additional constraints need to be added to the base feature vector. In this paper, area, circumference (contour points), the ratio of long to short radius, mean radius and radius variance are selected as the remaining constrained feature values, which together with $M_1 \mathrm{and} M_2$ form the base feature vector of the artifact image.

2.2. Create Template Base Feature Vectors

The object of this paper is a shaped workpiece. Since the image acquisition camera has a field of view of 1280 × 1080 and a mounting height of 800 mm, the maximum size of the selected shaped workpiece is 800 mm × 800 mm and the maximum thickness is 100 mm in order to ensure that a complete product image is captured. New templates can be added to the template library for any product that meets the criteria. In this section, several artifacts from Figure 1 are selected for testing, where artifacts B and B-Mirror are mirror images of each other, and D and D-Mirror are mirror images of each other.

The basic feature vectors of the above artifacts are shown in

Table 1. Base feature vectors for template shaped workpieces.

Category	M1	M2	Area	Circumference	Ratio of Long to Short Radius	Radius Mean	Radius Variance
A	0.30242455	0.00825412	125,959	2292	4.429	232.49	87.62
B	0.17034508	0.00001640	362,931	2368	1.763	341.89	44.11
B-Mirror	0.17034508	0.00001640	362,931	2368	1.763	341.89	44.11
C	0.18838271	0.00157881	248,166	1983	2.684	290.29	62.25
D	0.25531456	0.03709374	212,668	1902	3.585	278.40	94.46
D-Mirror	0.25531456	0.03709374	212,668	1902	3.585	278.40	94.46

.

As can be seen from the above table, there is a clear distinction between the base feature vectors of different types of artifacts. However, the feature vectors of workpieces that are mirror images of each other are almost identical. These two types of workpieces are like left and right hands, which cannot be overlapped on the same surface and cannot be classified as the same type of workpiece. Therefore, this paper proposes a template-matching algorithm based on the contour phase difference for solving the above problem to obtain more accurate results.

3. Algorithm Principle

3.1. Scheme Description

The proposed algorithm’s work is as follows: First, calculate the center of mass of the formed workpiece using the Hu invariant distance. Then, select the starting point on the contour and expand the radius of the workpiece in turn by 360-degree rotation to obtain the contour curve of the workpiece. After that, calculate the similarity and the rotation angle between the contours of the template and the target workpiece. It is proved that the efficiency and accuracy of template-matching of irregular workpieces are greatly improved. Detailed instructions will be expanded in subsequent chapters.

3.2. Method of Calculating the Center of Mass Based on Hu Invariant Moments

Obtaining the center of mass of a workpiece image is a key step for template matching. If the gray value of the pixel on the image is regarded as the quality of the place, then the center of mass of the regular workpiece image is the geometric center of the image outline, and the center of mass of the irregular workpiece needs to be calculated from the geometric moment of the image. According to the definition of image invariant moments proposed by Hu, M.K. [31], the $\left(p+q\right)$ order moments for a discrete two-dimensional image $f\left(x,y\right)$ are as shown in Equation (1).

The 0th order moment ($p=0,q=0)$ represents the quality of the discrete two-dimensional image $f\left(x,y\right)$, and the calculation equation is:

$$m_{00}={{\displaystyle \sum }}_x{{\displaystyle \sum }}_{y}f\left(x,y\right)$$

(5)

The 1st order moment $\left(p=1,q=0 \mathrm{or} p=0,q=1\right)$ represent the centroid of discrete two-dimensional image $f(x,y$), and the calculation equation is

$$\begin{array}{c}\overline{x}=\frac{m_{10}}{m_{00}}\\ \overline{y}=\frac{m_{01}}{m_{00}}\end{array}$$

(6)

In Equation (6), $\overline{x}$ and $\overline{y}$ represent the gray center position of the image. After obtaining the center of mass of the workpiece, the next step of the workpiece contour unfolding and the contour phase difference matching calculation can be performed.

3.3. Contour Feature Vector Computing Based on Rotating Radius Cluster

The rotational radius cluster of a workpiece is the set of the ordered edge point–center-point distances on the workpiece contour. If the workpiece is circular, as shown in Figure 2a, $\mathrm{O}\left(\mathrm{x},\mathrm{y}\right)$ is the center of mass of the circular workpiece, $\mathrm{A}\left(\mathrm{x},\mathrm{y}\right)$ is a point on the contour of the workpiece, and $\left|\mathrm{OA}\right|$ denotes the radius of the workpiece. With point O as the center of rotation, starting from point A, the rotation is 360 degrees, and each radius is expanded in an orderly manner, and the workpiece profile obtained is a flat straight line. The result is shown in Figure 2b.

For an irregular workpiece, as shown in Figure 3a, $\mathrm{O}\left(\mathrm{x},\mathrm{y}\right)$ is the center of mass of the irregular workpiece, $\mathrm{A}\left(\mathrm{x},\mathrm{y}\right)$ is a point on the contour of the workpiece, and $\left|\mathrm{OA}\right|$ is one radius of the workpiece. With point O as the center of rotation, starting from point A, the rotation is 360 degrees, and each radius is expanded in an orderly manner, and the workpiece contour is a curved contour with interlaced peaks and valleys, as shown in Figure 3b.

It can be seen that when the workpiece is circular, the only basis for distinguishing different workpieces is the difference in radius. When reasoning to the irregular workpieces, with the same initial starting point $\mathrm{A}\left(\mathrm{x},\mathrm{y}\right)$, the cumulative difference of the unfolded profile curve at each rotation angle can be used as a criterion for determining whether the target workpiece and the template one can be matched. Since the contour curves of different irregular workpieces unfolded according to the radius must be different, the contour curves of shaped workpiece image unfolding can be used as the basis for stencil matching. Usually, the target workpiece has a rotation angle relative to the template one, so their initial point $\mathrm{A}\left(\mathrm{x},\mathrm{y}\right)$ is not consistent. In other words, the outline curve of the target workpiece and that of the template have a certain phase difference, as Figure 4 shows:

To sum up, the details about the algorithm of irregular workpiece template matching based on the contour phase difference are as follows:

• Phase difference matching calculation

Assume that there is a template workpiece (X), and the distance from all its contour points to the center of mass is recorded as an array $X_{\left[i\right]},i=0, 1, 2,\dots , n$. Under the same conditions, an image of the target workpiece Y is captured, and the distance from all its contour points to center of mass is recorded as an array $Y_{\left[j\right]},j=0, 1, 2,\dots ,m$. The number of pixels on the profile of the template workpiece and the target one represent n and m respectively, and if $m\ne n$, the arrays with more pixel points need to be isometric sparse processed. The relationship between $X_{\left[i\right]}$ and $Y_{\left[j\right]}$ is shown in Figure 5.

If $\epsilon$ is the coincident degree of the two curves, in the discrete state, it can be expressed as:

$$\epsilon ={\displaystyle {\sum }_0^{\mathrm{n}}\sqrt[2]{{(X_i-Y_i)}^2}}$$

(7)

Equation (7) is the Euclidean distance between two points, which is relatively complicated to calculate. It can be replaced here by Manhattan distance, that is:

$$\epsilon ={{\displaystyle \sum }}_0^n\left|X_i-Y_i\right|$$

(8)

There is only translation and rotation between work pieces, if they belong to the same product, the workpiece profile curve in the figure pans one contour point at a time, i.e.,:

$$Y_i=Y_{i-1}$$

(9)

where I = 1, 2, 3, … and the values that exceed the array index are added to the end of the new array in turn. Then we calculate the overlap degree ${\epsilon }_m$ at this time, m = 0, 1, 2, …, k, as shown in Figure 5, the two curves can basically overlap after panning k times.

$${\epsilon }_m=\{\begin{array}{l}{{\displaystyle {\sum }_0^n\sqrt[2]{{(X_i-Y_{k+i})}^2}}}_{}{}_{},k<m-i\\ {\displaystyle {\sum }_0^n{\sqrt[2]{{(X_i-Y_{k+i-m})}^2}}_{}{}_{}},k\ge m-i\end{array}$$

(10)

$${\mathsf{\epsilon }}_k<\mathsf{\Gamma }$$

(11)

where $\mathsf{\Gamma }$ represents the threshold value of the successful match of the workpiece, and a smaller value means a higher successful matching rate.

• Rotation angle calculation

As can be seen from the above, the workpiece profile image moves by one contour point at a time, i.e., the step angle $\mathsf{\lambda }$ of the workpiece image rotates one unit at a time.

$$\mathsf{\lambda }=\frac{2\pi }{N},N=\min \left(m,n\right)$$

(12)

So, after panning k times, the rotation angle $\mathsf{\theta }$ of the target workpiece with respect to the stencil workpiece that satisfies Equation (11) is calculated, which can be expressed as:

$$\mathsf{\theta }=\mathsf{\lambda }k$$

(13)

4. Experimental Analysis and Verification

The object of this experiment is the backrest and cushion of the child seat, and the irregular artifacts to which our proposed algorithm is mainly applied are based on these products. During the mixing spray adhesive process of the backrests and cushions, it is necessary to identify which product the incoming material is, that is, we need to compare it with the template database, which contains the backrests and cushions data of all models of the company. To verify the validity of this algorithm, several products need to will be taken from the template database to form a sub-template database. As Figure 6 shows, the KB-Left and the KB-Right are mirrored symmetrically and the contour curves of the four template workpieces are shown in Figure 7.

A target product is selected from each of the KB-Left, KB-Right, ZD631, and ZD632 models, and the target product relative to the template rotation angle is: −18°, 22°, 180°, 90°. Then the matching experiments are conducted using the algorithm proposed in this paper and the external rectangle algorithm proposed in the literature [21,22,23]. The matching results by the algorithm proposed in this paper are shown in Figure 8.

The specific matching results compared with the external matrix algorithm are shown in the following

Table 2. Target workpiece phase difference match.

Target Work Piece	Rotation Angle (Degree)	Template	The Proposed Algorithm (Degree)	Error	Calculation Angle of Exterior Rectangle [21,22,23] (Degree)	Error
KB-Left	−18	KB-Left	−17.9275	0.40%	−18.0098	0.05%
KB-Right	22	KB-Right	22.0169	0.07%	22.2304	1.04%
ZD631	180	ZD631	180	0	0	100%
ZD632	90	ZD632	90	0	0	100%

.

To further verify the effectiveness of the algorithm, some of the shaped workpieces mentioned in the previous sections are selected for verification comparison. The model’s names for the target workpiece are B and B-Mirror. The rotation angles of the target product with respect to the template are −70° and −180°. The matching results are shown in Figure 9.

The specific matching results compared with the external matrix algorithm are shown in the following

Table 3. B, B-Mirror phase difference match results.

Target Work Piece	Rotation Angle (Degree)	Template	The Proposed Algorithm (Degree)	Error	Calculation Angle of Exterior Rectangle [21,22,23] (Degree)	Error
B-R70	−70	B	−70.0068	0.0097%	−69.8874	0.05%
B-Mirror-R180	−180	B-Mirror	−179.8874	0.0623%	−0.1646	99.91%

.

From the above results, the phase difference algorithm proposed in this paper adapts well to the matching of various types of irregular workpieces, with the maximum error of the rotation angle of the target workpiece relative to the template calculated by the algorithm is at most 1%. When the long and short shafts of the workpiece differ greatly, the rotation angle calculated by the external rectangular algorithm proposed by the reference [21,22,23] is close to the actual angle, but the error is slightly larger than that calculated by the algorithm in this paper. With slightly difference, the external rectangular algorithm does not work and the algorithm proposed in this paper has high accuracy. As long as the target workpiece has a position or angle deviation from the template workpiece, the phase difference algorithm works. After being applied to several matching experiments for other workpieces, the phase difference algorithm reached the success rate of 100%, and the angle error was controlled within 1%, which has much higher accuracy than the methods proposed in the reference [5,14,15]. Therefore, it meets the requirements of product positioning.

5. Conclusions

In this paper, an algorithm of irregular workpiece recognition based on contour phase difference is proposed. It can not only help to match the template workpiece with slight differences accurately, but also conduces to calculate the rotation amount of the target workpiece relative to the template workpiece, and the error is controlled within 1%. Compared with the algorithm created in the reference [5,14,15,21,22,23], the algorithm in this paper has higher accuracy. It also helps solve the substantive problems such as the positioning, sorting, spraying and other processes of the irregular workpiece, which has a good application value. However, the algorithm does not apply to the scaling workpiece matching at present, which requires the camera to take an image at a fixed height.