Measuring the Positions of the Solder Pins of Electrical Connectors from the Side

Featured Application

The proposed method can be used in the electronics industry to inspect the positions of the solder pins of electrical connectors or electronic components from the side, which provides an alternative solution for pin position measurement.

Abstract

In the electronics industry, it is common to measure the positions of the solder pins of electrical connectors from the bottom using the automated optical inspection (AOI) technique. However, the pins’ surfaces facing the camera of the AOI system may generate image noises, leading to inaccurate measurement results. The objective of the paper is to develop a machine vision system with an algorithm to measure the pin position from the side. Two images must be captured in different view directions so that the algorithm can use the information on the images to calculate the positions of the pins. Comparing the results obtained using the proposed method and those obtained using a 2D vision measuring machine to measure the positions from the bottom, we found that the maximum difference in the results was 0.03 mm. The technique presented in the paper can provide an alternative solution for measuring the positions of the solder pins of electrical connectors or similar components if measurement from the bottom is not feasible.

1. Introduction

Electrical connectors are the components used in electronic equipment or devices, such as mobile phones, computers, network equipment, and medical devices, for signal or power transmission. Through-hole–type connectors often need to be soldered onto the printed circuit boards (PCBs). For properly placing the connectors on the PCBs, the positions of the connectors’ solder pins need to be made within the specification so that the solder pins can be fitted into the corresponding through holes on the PCBs without resistance.

To make sure that the positions of the connectors’ solder pins are within the specification, in addition to using visual inspection or gauges, the common practice in the electronics industry involves the use of the automated optical inspection (AOI) technique [1,2], which uses industrial cameras to capture the images of the object that is subject to inspection. Then, it applies image processing to obtain the measurement results. Using AOI can save inspection time and reduce human errors [2,3,4].

Several researchers have studied the solder pins of ICs or connectors. For instance, Furukawa and Sakuma [5] measured from the bottom of a connector. Because the bottom surfaces of the solder pins reflected light unevenly, the images from the bottom became partially white and partially black after binarization. Subsequently, they used the proportion, position, and shape of the black and white parts of the images and conducted a neural network analysis to determine whether the pin positions were normal. Further, they used 18 samples to train neural networks, and the detection success rate reached 100%. Luo et al. [6] used machine vision to detect the pins of the DIP IC from the side. First, edge detection was performed to obtain the pin positions. After wavelet transformation, the spacing between the pins was obtained to determine whether the spacing was qualified. The system can only detect one side of the pins of the DIP IC at a time. Kaitwanidvilai [7] also applied the wavelet transform and discrete Fourier transform to recognize various IC pin patterns. Huang et al. [8] proposed using a median filter to improve the Canny algorithm and remove the image noise along the edge of the IC pin. On the other hand, to increase detection efficiency, Qiao et al. [9] used mirror reflection to detect both sides of the pins simultaneously. Zhong et al. [10] also used a mirror set to detect the flatness of the SMT pins on both sides of the IC. The flatness-detection error of the system can be less than 4 µm. In addition, Zhong et al. [11] also used an object feature-matching algorithm to find the pin end faces and then took their center points, representing the pin’s position. Ruan et al. [12] placed the connector’s terminals on top of the backlight, allowing a camera to capture the terminals’ image from the top. Subsequently, they performed Gaussian filtering and Canny edge detection to determine the spacing between terminals.

Some researchers used two cameras to capture images in stereo vision. For instance, Tay et al. [13] used dual cameras with the centroid method to measure the pins, obtained the pins’ bright spots from the end of the pins, and rebuilt the 3D structure of the pins to obtain the height of each pin. The experimental results showed that of the 19 pins tested, the maximum difference between the results obtained and the contact measurement method was 3.7%. Stroppa and Cristalli [14] also adopted dual camera detection. They used high dynamic range (HDR) technology to deal with the uneven brightness problem. HDR uses different exposure times to capture images. Combining several images with low exposure and overexposure created a high dynamic range imaging, completely preserving the image’s details of high brightness and shadows, following which the pin’s profile was obtained. Then, they used the pyramid pattern matching algorithm to extract the top feature of the pin from the image. A standard connector was adopted as the reference. The in-plane accuracy of this system was 0.066 mm, and the depth accuracy was 0.084 mm. Zhao et al. [15] also adopted two cameras to inspect connectors. They developed an algorithm based on deep neural networks and pattern matching so that the vision system could be used on multiple types of connectors. They demonstrated that the algorithm could be used to inspect 33 different types of connectors successfully. Other methods that can be used to measure the positions of the pins include 3D laser triangulation [16] and a structured light pattern projector. A comparison of these methods can be found in [17]. Recently, augmented reality and deep learning [18,19,20] were introduced into the industry for inspecting connectors, but they are mainly for small pin detection, pin sequencing, and mismatched pin visualization.

If machine vision, or automated optical inspection, can be used to measure the positions of the connector pins and the measurement difficulties reduced, it will significantly help the connector industry in terms of inspection. Furthermore, the camera is usually placed on top of the pins when using machine vision to measure the pins’ positions. However, the area of the pin tip is small, and the surface is uneven. Thus, it is prone to irregular light source reflections that interfere with the image, causing measurement errors [5]. Nevertheless, this problem may be avoided if the image is not taken from the end face of the solder pins. Therefore, this research aimed to establish a machine vision inspection system, which measures from the side of the connector pins and calculates the positions of the pins by capturing images in two different view directions. The results of this research can be applied to the connector and the electronics industries to provide innovative methods for pin inspection.

2. Materials and Methods

2.1. Machine Vision Systems

The machine vision setup in this research is shown in Figure 1, including an industrial camera, a bi-telecentric lens, an LED ring light, a diffuser, and a PC (not shown). The software for image measurement was developed in the LabVIEW [21] environment provided by National Instruments (Austin, TX, USA). A Basler (Ahrensburg, Germany) industrial camera was used in this research. The model number was Aca2500-14gm [22], and its sensor type was CMOS with a size of 5.7(H) × 4.28(V) mm. The image’s length and width are 2592 × 1944 pixels, and the resolution is 5 million pixels. The unit pixel is converted to the actual length and width of 2.2 × 2.2 μm. The video communication interface is GigE. The bi-telecentric lens is from ES Technology (Taoyuan, Taiwan). Its model number is ESCA230-66 [23], with a working distance of 166 ± 3 mm. A bi-telecentric lens is used in this research because the size of the solder pins on the capture images is not affected by the distance from the pins to the camera.

A ring light source consisting of 96 LEDs was used as a light source. A diffuser is placed in front of the light source to create a back-diffused light to evenly distribute the light on the entire image. The image captured by the camera will be transmitted to a PC for subsequent image processing. The PC contained an Intel core 2 Duo E7500 dual-core central processing unit (CPU), 4096 MB dynamic memory (DDR SDRAM), and an NVIDIA VGA standard graphics card.

In this study, a 2D vision measuring machine (2D VMM) developed by Resson (Taiwan) was used to verify the measurement results of the proposed method. Its model number is EVM-2515 [24]. The resolution of the linear scale of the machine is 1 μm, and the accuracy is 4 + L/50 μm, where L is the measured length (mm). The 2D VMM will be mentioned several times in the following because the measurement results of this machine were used as a reference for the proposed machine vision system.

2.2. Measurement of Pins’ Positions from the Side

Before introducing the mechanism of measuring a connector from the side, some background information must be laid down. A typical through-hole connector, consisting of solder pins and plastic housing, is shown in Figure 2. This type of connector must be soldered on the printed circuit board (PCB), as shown in Figure 2a. One or two reference pegs are usually made to guide the connector to the proper position on the PCB. The PCB has holes for the pegs, allowing the multiple solder pins to easily slide into the corresponding plated through-holes when the reference pegs go into the holes. As shown in Figure 2a, the connector with the solder pins is at the bottom. In measuring the positions of the solder pins, the connector is usually placed on the table upside down so that the camera of the AOI or the 2D VMM can capture the positions of the solder pins from the bottom side of the connector, as illustrated in Figure 2b, where the x-axis is in the longitudinal direction of the connector, and the y-axis is in the transverse direction.

When viewing the pin of a connector from the bottom, the pin’s position can be seen in the x-direction and y-direction relative to the reference peg, or in-plane position, as illustrated in Figure 2b. The position’s information is two-dimensional. However, only one-dimensional information can be obtained when viewing the pins of a connector from the side. Therefore, the connector must be viewed from two different directions using two cameras oriented in different directions, as shown in Figure 3, to obtain the two-dimensional information.

The relationship between the two cameras and the connector in Figure 3 can be rearranged in Figure 4a,b separately. In this research, an equation to calculate the in-plane position from the two images was employed. The equation is derived as follows.

In Figure 4, ${\theta }_{b1}$ and ${\theta }_{b2}$ are two different connector orientations relative to the camera, ${\theta }_{ri}$ is the theoretical angular position for the i-th pin, ${\theta }_i$ is the difference between the actual angular position and the theoretical angular position of the i-th pin. Moreover, $r_i$ is the distance from the center of the reference peg to the center of the pin at its theoretical position, and $d_1$ and $d_2$ are the transverse distances from the center of the reference peg to the center of the pin at its actual position. Figure 4a shows a right-angled triangle consisting of a green line (the hypotenuse), a blue line (the leg), and a red line (another leg). The length of the hypotenuse, or the deviation from the theoretical position, is equal to the square root of the summation of the legs’ square, i.e.,

$$\mathrm{deviation}=\sqrt{{[r_i\times \cos ({\theta }_{b1}+{\theta }_{ri})-d_1]}^2+{[d_1\times \tan ({\theta }_{b1}+{\theta }_{ri}+{\theta }_i)-r_i\times \sin ({\theta }_{b1}+{\theta }_{ri})]}^2}$$

(1)

Similarly, in Figure 4b, the length of the hypotenuse, or

$$\mathrm{deviation}=\sqrt{{[r_i\times \cos ({\theta }_{b2}+{\theta }_{ri})-d_2]}^2+{[d_2\times \tan ({\theta }_{b2}+{\theta }_{ri}+{\theta }_i)-r_i\times \sin ({\theta }_{b2}+{\theta }_{ri})]}^2}$$

(2)

The hypotenuse, or the deviation, in Figure 4a and the one in Figure 4b are the same. In other words,

$$\begin{array}{l}\sqrt{{[r_i\times \cos ({\theta }_{b1}+{\theta }_{ri})-d_1]}^2+{[d_1\times \tan ({\theta }_{b1}+{\theta }_{ri}+{\theta }_i)-r_i\times \sin ({\theta }_{b1}+{\theta }_{ri})]}^2}\\ =\sqrt{{[r_i\times \cos ({\theta }_{b2}+{\theta }_{ri})-d_2]}^2+{[d_2\times \tan ({\theta }_{b2}+{\theta }_{ri}+{\theta }_i)-r_i\times \sin ({\theta }_{b2}+{\theta }_{ri})]}^2}\end{array}$$

(3)

or

$$\begin{array}{l}{[r_i\times \cos ({\theta }_{b1}+{\theta }_{ri})-d_1]}^2+{[d_1\times \tan ({\theta }_{b1}+{\theta }_{ri}+{\theta }_i)-r_i\times \sin ({\theta }_{b1}+{\theta }_{ri})]}^2=\\ {[r_i\times \cos ({\theta }_{b2}+{\theta }_{ri})-d_2]}^2-{[d_2\times \tan ({\theta }_{b2}+{\theta }_{ri}+{\theta }_i)-r_i\times \sin ({\theta }_{b2}+{\theta }_{ri})]}^2.\end{array}$$

(4)

In Equation (4), ${\theta }_{b1}$ and ${\theta }_{b2}$ are identified when the view angle of the camera is set up: ${\theta }_{ri}$ and $r_i$ are the theoretical values that can be obtained from the connector’s design, and $d_1$ and $d_2$ are the measurement results using the camera. The only unknown position in Equation (4) is ${\theta }_i$, which can be obtained by solving the nonlinear Equation (4). The Newton Raphson Zero Finder in LabVIEW provided a convenient way to solve the nonlinear equation to obtain ${\theta }_i$. Then, ${\theta }_i$ can be integrated into Equation (1) or Equation (2) to find the deviation of the pin from its theoretical position. The deviation is calculated using the proposed method, and the results are then compared with the deviation measured directly from the bottom of the connector using the 2D VMM.

3. Results and Discussion

The uncertainty of the proposed machine vision system and the 2D VMM were studied first. The type A uncertainty (standard deviation) of the proposed method was found to be 0.001 mm by repeated measurement. Then, the scale factor of the industrial camera was used in the type-B uncertainty calculation. The scale factor was 0.0148 mm/pixel. The type B uncertainty of the proposed method was calculated as 0.0148/2√3 = 0.004 mm. The combined uncertainty, 0.004 mm, was calculated by using the square root of the sums of the squares of the type A uncertainty and type B uncertainty.

Similarly, the type A uncertainty (standard deviation) of the 2D VMM was found to be 0.005 mm by repeated measurement, and the type B uncertainty of the 2D VMM, 0.002 mm, was calculated using the accuracy provided by the manufacturer. The combined uncertainty of the 2D VMM was calculated as 0.006 mm. Then two types of connectors were measured for validation in this paper. The details are presented in the following case studies.

3.1. Case Study One

The first case study involves a header with 18 solder pins made by TE Connectivity (part number 967618-2 [25]), as shown in Figure 5. There are nine pins in a row, two rows in total, and the pitch (spacing between pins) is 2.54 mm. The reference peg is the plastic locating one.

In measuring the positions of the pins from the side, the connector’s images were captured in two different orientations, as shown in Figure 6. This research only measured the first six pins of the three samples. Figure 6a shows the connector’s image when ${\theta }_{b1}=22°$. On the other hand, Figure 6b shows the connector’s image when ${\theta }_{b2}=28°$. By measuring d₁ and d₂ illustrated in Figure 4 and solving Equation (4), the deviation can be obtained, as listed in the rows marked by “side” for the three samples in

Table 1. The deviations of the first six pins of the three samples measured in case study one. “Side” means that the measurement was performed from the side of the samples using the proposed method, and “bottom” means that the measurement was performed from the bottom of the samples using the 2D VMM.

Deviation		Pin 1	Pin 2	Pin 3	Pin 4	Pin 5	Pin 6
Sample 1	side	0.008	0.096	0.076	0.080	0.150	0.137
	bottom	0.018	0.073	0.051	0.054	0.121	0.120
Sample 2	side	0.009	0.092	0.089	0.037	0.165	0.085
	bottom	0.018	0.063	0.061	0.036	0.139	0.114
Sample 3	side	0.120	0.202	0.059	0.118	0.233	0.121
	bottom	0.127	0.186	0.085	0.112	0.255	0.143

. Further, the 2D VMM was used to measure the deviation from the bottom. The results are also listed in the rows marked by “bottom” for the three samples in Table 1 for comparison. The results in Table 1 are also illustrated in Figure 7, in which the combined uncertainty of 0.004 mm for the proposed method and the combined uncertainty of 0.006 mm for the 2D VMM were used to generate the error bars for the data in Table 1. As shown in Figure 7, both measurement results have the same trend, although most of the results corresponding to the same pins do not overlap. The finding is not surprising, because each of the methods measured different parts of the pins. Subtracting the data in the “bottom” row from the data in the “side” row for the three samples in Table 1 gives differences ranging from −0.029 to +0.029 mm.

3.2. Case Study Two

The second type of connector was produced by AirBorn, a rectangular MIL specification connector (part number RM212-010-121-2900 [26]), as shown in Figure 8. The reference peg is the large metal pin on the left side of the solder pins shown in Figure 8. There are five pins in a row; it has two rows in total. Each row of pins is 1.905 mm apart, and the row-to-row spacing is also 1.905 mm.

In measuring the positions of the pins from the side, as shown in Figure 9, two images with different orientations for the connector were obtained. In this study, the first six pins are also measured. Figure 9a shows the image for the connector when ${\theta }_{b1}=7°$. On the other hand, Figure 9b shows the image of the connector when ${\theta }_{b2}=-7°$. The deviations, as listed in

Table 2. The deviations of the first six pins of the three samples measured in case study two. “Side” means that the measurement was performed from the side of the samples using the proposed method, and “bottom” means that the measurement was performed from the bottom of the samples using the 2D VMM.

Deviation		Pin 1	Pin 2	Pin 3	Pin 4	Pin 5	Pin 6
Sample 1	side	0.062	0.062	0.071	0.051	0.036	0.029
	bottom	0.086	0.064	0.064	0.040	0.034	0.030
Sample 2	side	0.055	0.095	0.070	0.298	0.019	0.270
	bottom	0.084	0.078	0.048	0.300	0.013	0.297
Sample 3	side	0.026	0.024	0.025	0.043	0.158	0.073
	bottom	0.007	0.037	0.051	0.067	0.177	0.086

, can be obtained using two measurement methods. The results in Table 2 are also illustrated in Figure 10, in which the combined uncertainty of 0.004 mm for the proposed method and the combined uncertainty of 0.006 mm for the 2D VMM were used to generate the error bars for the data in Table 2. As shown in Figure 10, some results overlap, and the others are close enough. Subtracting the data in the “bottom” row from the data in the “side” row for the three samples in Table 2 gives differences ranging from −0.029 to +0.022 mm.

3.3. Sources of Difference

There are several possible sources of difference for the measurement performed from the side of the connectors: the variation of the pin’s round corners, the view direction accuracy when capturing the image from the side, and the different accuracies between the two measuring systems used in the research. The cross-section of the rectangle solder pin measured in the first case study is not a rectangle with sharp corners, but a rectangle with round corners. The radii of the round corners are not the same, which caused differences between the measurement results from the bottom and the ones from the side. The results show that the radii of the round corners range from 0.03 to 0.08 mm. The difference between the deviation measured from the bottom and from the side for the two situations is calculated. In one situation, the left round corner’s radius is 0.03 mm, and the right round corner’s radius is 0.08 mm. In the other situation, all the round corners’ radii are 0.03 mm. The calculation results show that the difference between the deviations from the two situations is 0.008 mm, indicating that if the round corners are inconsistent, then the difference between the bottom and the side measurement can be expected to be as large as 0.008 mm.

The accuracy of the view direction is another possible issue; for instance, the 22° view direction in case study one. When measuring a distance of about 11.8 mm in the imaging system—as long as there is $0.1$ inaccuracy for the view direction—the difference between the bottom and the side measurement is also about 0.008 mm. The accuracies of the two measuring techniques may also cause some differences. There are also other sources of difference, such as the resolution of the industrial camera and the difference between the reference peg’s center lines determined from the side and the bottom.

4. Conclusions

When using machine vision to measure the positions of the solder pins from the bottom of an electrical connector or similar electronics components, it must be noted that the light reflection may affect the results. Nevertheless, measuring from the side may solve the problem. In particular, this research developed a machine vision system and the associated algorithm to address the issue. From the discussion in the paper, it can be understood that the proposed method can be applied to various electrical connectors or electronic components with solder pins and a reference peg. In validating the measurement results of this proposed method, a 2D VMM is used to measure the positions from the bottom of the connectors so that the results can be compared to the positions obtained using the proposed method. From the two examples presented in the paper, the difference between the measurement results using the proposed method and the 2D VMM was about ±0.03 mm. Considering all the possible causes of difference between measuring from the bottom and from the side of the connector, we can conclude that the proposed method in the paper can be used as an alternative for measuring the position of the solder pins of connectors, especially when the measurement from the bottom of the connectors is not feasible.