Multi-Camera Based Setup for Geometrical Measurement of Free-Falling Molten Glass Gob

Abstract

High temperatures complicate the direct measurements needed for continuous characterization of the properties of molten materials such as glass. However, the assumption that geometrical changes when the molten material is in free-fall can be correlated with material characteristics such as viscosity opens the door to a highly accurate contactless method characterizing small dynamic changes. This paper proposes multi-camera setup to achieve accuracy close to the segmentation error associated with the resolution of the images. The experimental setup presented shows that the geometrical parameters can be characterized dynamically through the whole free-fall process at a frame rate of 600 frames per second. The results achieved show the proposed multi-camera setup is suitable for estimating the length of free-falling molten objects.

1. Introduction

Characterizing molten material in industry without compromising its quality is a complex task. The high temperature environment is too harsh for direct measurements and so producers have to rely on indirect laboratory methods, with the associated risk of introducing contamination to the samples.

For molten materials such as glass, it is of the utmost importance that the characteristics remain consistent during the production of the final product. Characteristics such as viscosity, volume and weight [1] needed to be continuously monitored. The factors effecting the characteristics might be inside mechanical components of glass-forming machines, synchronization between different parts of the machine or any change in the raw material for example viscosity of glass melt [2].

Currently most of the monitoring is manual and is based on previous experiences, operator knowledge, and trial and error [3]. However, manual monitoring is slow, not continuous and not sufficiently accurate. The result is lost in production and too high rejection rate of the final product due to failure to meet quality requirements [2]. Therefore, continuous monitoring of the gob is required to ensure the quality of the glass products such as glass containers.

This paper proposes the use of a multi-camera setup to dynamically characterize small changes in the geometrical properties of free-falling molten material that are correlated to material characteristics [4]. The setup is demonstrated in a lab environment using a solid cylindrical object. Images of the free-falling object are captured by the high-speed multi-camera setup and processed with image-processing techniques. We evaluate the effectiveness of this technique as well as how the use of two cameras helps to improve the end-result, which is an accurate measurement of the length of the cylindrical object.

Shape analysis that includes the length, width and area of the solid cylindrical object in each falling position shows that the setup is able to accurately measure the length of free-falling molten materials such as glass gob.

2. Related Work

Different fields have adopted different approaches to measuring the size of objects of interest. In [5], information about the position of a camera, its height and pitch angle with respect to ground, and a vanishing point are used to determine the height of a person. However, the technique proposed in [5,6] are not suitable for the experiment presented in this paper, in which it is not possible to determine the vanishing point from the scene. Paper [7] proposes using a measurement method based on an edge detection algorithm to automatically segment images of oysters and thus determine the height and width of each oyster. The researchers use an object of known size to calculate the pixel size ratio that can be used to estimate the measurements, but the focus of the paper is mostly on the segmentation technique. Frantisek et al. [8] proposed a method to estimate human height from a single calibrated image. They estimated the angles from the scene to determine the height. The authors claim a 1% to 2% error with a reference height of 1700 mm in the real environment. However, when estimating human height, accuracy in millimetres is not a serious concern and this method might not be suitable for measuring smaller objects with high accuracy. The method used in paper [4] has similar approach but more simplified as the object of interest is spherical and possess no influence on measurements due to tilt or perspective errors. However, in this article the shape of the object and presented correction methods minimize the error induced due to tilt and perspective errors.

Another approach [9] uses triangulation and depth of field to estimate a person’s height. It is assumed that the camera is facing downwards and looking at the object from an angle. This approach is not suitable for our application as the end-point required to their equation will be inaccurate due to the cylindrical shape of our objects.

A system is needed that can measure free-falling cylindrical or oval objects with high accuracy and with no reference object in the scene. In this article, we present a suitable approach to achieve this goal.

3. Materials and Methods

The measurement setup consist of two high-speed cameras. The cameras are placed 90 degrees apart, covering the region of interest from different angles as shown in Figure 1. The geometrical parameters of the free-falling object are characterized continuously over the whole field of view (FOV) from the captured images.

The objective is to measure the length of the free-falling cylindrical object. Both cameras were first calibrated using a calibration pattern explained in [10]. Then the images of the cylindrical object were first converted into a binary image using the Otsu threshold method [11]. After detecting the boundaries, the images were cropped to remove background. The extracted object in the form of a bitmask was then used to determine the length of the object by first calculating the maximum Feret’s diameter [12] of the cylindrical object. Feret’s diameter is a caliper diameter that allows an object to be measured by placing it between the jaws of a Vernier caliper. Although simpler techniques to measure the length of the object could be used, the maximum Feret’s diameter was chosen for this experiment to determine whether this technique can accurately measure length in the intended application. A molten glass gob, which is a highly viscous liquid, may be subject to elongation during free-fall. We believe that the technique presented in this paper would represent the true length of the glass gob.

In this setup, camera 1 measures the length, and camera 2 provides a reference image to remove any tilt error in the measurements.

3.1. Proposed Approach to Measure the Length from One Camera

Step 1: Capture grayscale images of a free-falling cylinder as input from camera 1 and camera 2 simultaneously.

Step 2: Convert the grayscale images to binary images using the Otsu threshold method [11].

Step 3: Find the centroid $\overline{x},\overline{y}$, orientation $\theta$, and major axis length l of the object in the binary image using moment-based geometrical properties [13].

Step 3(a): The centroid $\overline{x},\overline{y}$ of the image is found using (1) and (2) [13].

$$\overline{x}=\frac{1}{\left|R\right|}.\sum _{(u,v)ϵR}u$$

(1)

$$\overline{y}=\frac{1}{\left|R\right|}.\sum _{(u,v)ϵR}v$$

(2)

where R is the binary region of the image, and u, v are the pixel co-ordinates.

We can define the binary image (p,q) moment of a contour m as:

$$m_{p,q}=\frac{1}{\left|R\right|}.\sum _{(u,v)ϵR}I(u,v)u^p{u}^q$$

(3)

where p is the x-order and q is the y-order for $I(u,v)ϵR$.

For the binary image $(u,v)ϵ\{0,1\}$, only the foreground pixels with $I(u,v)=1$ in the region R are considered, giving a simplified form of (3):

$$m_{p,q}=\frac{1}{\left|R\right|}.\sum _{(u,v)ϵR}{u}^p{u}^q$$

(4)

Now the area of the region A(R) can be expressed as:

$$A\left(R\right)=\left|R\right|=\sum _{(u,v)ϵR}1=\sum _{(u,v)ϵR}^n{u}^0{u}^0=m_{00}\left(R\right)$$

(5)

$$Centroid\{\overline{x},\overline{y}\}=\left\{\frac{m_{10}\left(R\right)}{m_{00}\left(R\right)},\frac{m_{01}\left(R\right)}{m_{00}\left(R\right)}\right\}$$

(6)

Step 3(b): The major axis length l shown in Figure 1 has the same normalized second central moment of the region, that is, the central moment $\mu$ is the same as (4). The only difference is that now the values of u and v in the (7) are subtracted or displaced by the calculated centroid $\overline{x},\overline{y}$ in (1) and (2), which gives.

$${\mu }_{p,q}\left(R\right)=\sum _{(u,v)ϵR}I(u,v){(u-\overline{x})}^p{(v-\overline{y})}^q$$

(7)

For the binary image with $I(u,v)=1$ within region R, (7) becomes:

$${\mu }_{p,q}\left(R\right)=\sum _{(u,v)ϵR}{(u-\overline{x})}^p{(v-\overline{y})}^q$$

(8)

The normalized central moment is obtained by scaling the reciprocal of the area $m_{00}\left(R\right)$ as in (9) [14]:

$${\overline{\mu }}_{p,q}\left(R\right)=\frac{{\mu }_{p,q}\left(R\right)}{{\mu }_{00}^{\frac{p+q}{2}+1}\left(R\right)}$$

(9)

where ${\mu }_{00}=m_{00}\left(R\right)$ and the central moments of order up to two are [15]

$$\begin{array}{c}{\mu }_{00}=0,\hfill \\ {\mu }_{10}=0,\hfill \\ {\mu }_{11}=m_{11}\left(R\right)-\overline{x}\ast m_{01}\left(R\right)=m_{11}\left(R\right)-\overline{y}\ast m_{10}\left(R\right),\hfill \\ {\mu }_{20}=m_{20}\left(R\right)-\overline{x}\ast m_{10}\left(R\right),\hfill \\ {\mu }_{02}=m_{02}\left(R\right)-\overline{y}\ast m_{01}\left(R\right),\hfill \\ {\mu }_{21}=m_{21}\left(R\right)-2\overline{x}\ast m_{11}\left(R\right)-\overline{y}\ast m_{20}\left(R\right)+2x^{-2}\ast m_{01}\left(R\right),\hfill \\ {\mu }_{12}=m_{12}\left(R\right)-2\overline{y}\ast m_{11}\left(R\right)-\overline{y}\ast m_{02}\left(R\right)+2y^{-2}\ast m_{10}\left(R\right).\hfill \end{array}$$

(10)

The covariance matrix in (14) is constructed using the information from (10):

$${\overline{\mu }}_{20}=\frac{{\mu }_{20}}{{\mu }_{00}}=\frac{m_{20}\left(R\right)}{m_{00}\left(R\right)-{\overline{x}}^2}$$

(11)

$${\overline{\mu }}_{02}=\frac{{\mu }_{02}}{{\mu }_{00}}=\frac{m_{02}\left(R\right)}{m_{00}\left(R\right)-{\overline{y}}^2}$$

(12)

$${\overline{\mu }}_{11}=\frac{{\mu }_{11}}{{\mu }_{00}}=\frac{m_{11}\left(R\right)}{m_{00}\left(R\right)-\overline{x}\overline{y}}$$

(13)

$$\left[\begin{array}{cc}{\overline{\mu }}_{20}& {\overline{\mu }}_{11}\\ {\overline{\mu }}_{11}& {\overline{\mu }}_{02}\end{array}\right]$$

(14)

Equation (15) represents the major axis length l is approximated by constructing a covariance matrix of the normalized second central moment of the region R in (14). The eigenvectors of this matrix corresponds to the major ${\lambda }_1$ and minor axes ${\lambda }_2$ [16], calculated as:

$${\lambda }_{1,2}=\frac{{\overline{\mu }}_{20}+{\overline{\mu }}_{02}\pm \sqrt{{({\overline{\mu }}_{20}-{\overline{\mu }}_{02})}^2-4{\overline{\mu }}_{11}}}{2{\overline{\mu }}_{00}}$$

(15)

where ${\lambda }_1$ takes the + sign and ${\lambda }_2$ takes the – sign, so that ${\lambda }_1\ge {\lambda }_2$.

$$l=2\sqrt{{\lambda }_1}$$

(16)

Step 3(c): Orientation $\theta$ is calculated by fitting an ellipse [13] over the whole object after extracting the major axis lengthl.

Using Pythagoras we can obtain (17) and (18):

$$sin\left(2{\theta }_R\right)=\frac{A}{\sqrt{A^2+B^2}}$$

(17)

$$cos\left(2{\theta }_R\right)=\frac{A}{\sqrt{A^2+B^2}}$$

(18)

where $A=2{\mu }_{11}\left(R\right)$ and $B={\mu }_{20}\left(R\right)-{\mu }_{02}\left(R\right)$.

Now for region R, the orientation vector $xd={(x_d,y_d)}^T$ is computed as in [12] using $co{s}^2\alpha =\frac{1}{2}[1+cos2\alpha ]$ and $si{n}^2\alpha =\frac{1}{2}[1-cos2\alpha ]$.

$$x_d=cos\left({\theta }_R\right)=\left\{\begin{array}{ll}0& \mathrm{for}\mathrm{A}=\mathrm{B}=0\\ {\left[\frac{1}{2}\left(1+\frac{B}{\sqrt{A^2+B^2}}\right)\right]}^{\frac{1}{2}}& \mathrm{otherwise}\end{array}\right.$$

(19)

$$x_d=sin\left({\theta }_R\right)=\left\{\begin{array}{ll}0& \mathrm{for}\mathrm{A}=\mathrm{B}=0\\ {\left[\frac{1}{2}\left(1-\frac{B}{\sqrt{A^2+B^2}}\right)\right]}^{\frac{1}{2}}& \mathrm{for}A\ge 0\\ {\left[\frac{1}{2}\left(1-\frac{b}{\sqrt{A^2+B^2}}\right)\right]}^{\frac{1}{2}}& \mathrm{for}A<0\end{array}\right.$$

(20)

The orientation ${\theta }_R$ for the region R is calculated as:

$${\theta }_R=\frac{arctan(A,B)}{2}$$

(21)

Step 4: To calculate the length of the cylindrical object, we use the major axis length l from (16) and orientation ${\theta }_R$ from (21) of the cylinder. We first extract the x and y co-ordinates of a line segment with high intensity values that starts from one end of the boundary to the other end of the region’s boundary and passes through the centroid $(\overline{x},\overline{y})$ of the cylinder as shown in the Figure 2.

$$k=\frac{1}{2}$$

(22)

$$x_{co}=\overline{x}+k\ast \left[\begin{array}{cc}cos{\theta }_R& -cos{\theta }_R\end{array}\right]$$

(23)

$$y_{co}=\overline{x}+k\ast \left[\begin{array}{cc}-sin{\theta }_R& sin{\theta }_R\end{array}\right]$$

(24)

Step 5: From Figure 2 first find the last index of the points with high intensity values for both $x_{c0}=(x_1,y_1)$ and $y_{c0}=(x_2,y_2)$ co-ordinates.

Step 6: Calculate the Euclidean distance for the length in pixels $L_{px}$ as shown in (25).

$$L_{px}=\sqrt[2]{{(x_1-x_2)}^2+{(y_1-y_2)}^2}$$

(25)

Step 7: Divide the length in pixels by the camera’s pixel size ratio ${\displaystyle \left(\frac{px}{mm}\right)}$ to get the length in millimetres, L, as in (26).

$$L=\frac{L_{px}}{\left(\frac{px}{mm}\right)}$$

(26)

Perspective errors can occur at the top or bottom elliptical parts of the cylindrical object (the ends) due to camera angle, as shown in Figure 3. The minor axis of this elliptical part is added to the binary image and becomes part of the measured length, adding perspective error to the measurements. To eliminate this error, the centre of this elliptical part is calculated. Minor and major axes are then extracted to fit an ellipse as shown in Figure 2. The minor axis can then be subtracted from the measured length. The length of the cylindrical object before and after correction is shown in the results section of this paper.

Step 8: The diameter of the minor axis of the elliptical part $P_{px}$ shown in Figure 4 can be calculated using (27):

$$P_{px}=\left(\sqrt[2]{{(x_2-x_{ec})}^2+{(y_2-y_{ec})}^2}\right)\ast 2$$

(27)

Step 9: The perspective correction in length L shown in Figure 4 of the cylindrical object can now be achieved by subtracting $P_{px}$ from $L_{px}$. Equation (28) shows the correct length in pixels $L_{px}^{\prime }$.

$$L_{px}^{\prime }=L_{px}-P_{px}$$

(28)

Equation (28), which is in pixels, is divided by the camera’s pixel size ratio ${\displaystyle \left(\frac{px}{mm}\right)}$ to obtain the length in millimetres.

$$L^{\prime }=\frac{L_{px}^{\prime }}{\left(\frac{px}{mm}\right)}$$

(29)

3.2. Proposed Method to Remove Tilt Error Using Two Cameras

Using the images from the two cameras pointing at the same object from two different viewpoints, 90 degrees apart as shown in Figure 1, any tilt in a falling object can be identified and rectified in post-processing. Figure 4 shows the method of calculating the tilt error e and then eliminating that error from the length $L_{px}^{\prime }$.

Step 10: The tilt error is calculated with the orientation angle $\beta$ as shown in Figure 4. The orientation angle $\beta$ is taken from the reference camera (i.e., camera 2).

Equation (30) gives the tilt error e:

$$e=rtan\beta$$

(30)

Step 11: To measure the width of the object, the minimum Feret’s diameter of the object is determined from the binary image.

Step 12: Repeat step 2 to step 3 on the images taken from camera 2.

Step 13: From a binary image with labelled matrix, the minimum Feret’s diameter r is measured between two boundary points on the antipodal vertices of a convex hull that encloses the cylindrical object. $r_b$ in Figure 5a shows the distance from the centroid to one of the boundary points on the antipodal vertices and can be calculated as:

$$r_b=2\sqrt{{\lambda }_2}$$

(31)

where ${\lambda }_2$ is taken from (15).

The width of the cylinder r is calculated from $r_b$ from (31):

$$r=2\ast r_b$$

(32)

Step 14: After calculating the tilt error e from the images taken from camera 2, the tilt error e is added into the length in pixels $L_{px}^{\prime }$ from (28).

$$L_{px}^{\prime \prime }=L_{px}^{\prime }-e$$

(33)

Step 15: Convert the length $L^{\prime \prime }$ to millimetres by dividing the length in pixels $L_{px}^{\prime \prime }$ by the camera’s pixel size ratio ${\displaystyle \left(\frac{px}{mm}\right)}$ as shown in (34):

$$L^{\prime \prime }=\frac{L_{px}^{\prime \prime }}{\left(\frac{px}{mm}\right)}$$

(34)

4. Results

An experiment was designed to evaluate the proposed multi-camera setup and measurement methods described above. The experimental setup for this paper consists of two high-speed uEye UI-3060CP-M-GL-R2 monochrome cameras with 25 mm fixed focal length lenses. The selected frame rate for this experiment was 600 fps, with a cropped region of interest of 1936 × 300 pixels. This gives a combination of relatively large number of pixels in the free-fall direction for geometrical characterization with low quantization error. This enables high resolution of the free-fall as the expected velocity of the falling cylinder is around 4 m/s; for this frame rate, the cylinder will only move 5.7 mm in each frame [17]. The exposure time was selected on the basis of the expected motion blur. For this experiment the calculated exposure time was 0.132 ms. The motion blur due to the chosen exposure time was calculated as stated in [17]. The motion blur will be 0.528 mm, which is adequate considering the size of the cylindrical object used in this experiment. Both cameras were placed 90 degrees apart with a working distance of 200 cm, facing towards the solid cylindrical object with true length of 186 mm (Figure 1).

The object was dropped from 100 cm above the ground. Both cameras covered the vertical field of view (VFOV), that is, 90 cm in such a way that the pixel size ratio would be 2.15 pixels/mm for camera 1 and 2.14 pixels/mm for camera 2. The pixel size ratio was calculated by dividing the resolution of the camera 1936 px by the VFOV. The object was dropped 20 times. In Figure 6a–c, different colors represent each drop with length measurement of the falling cylinder. Figure 7 shows a sequence of images for the free-falling solid cylindrical object captured in the experimental setup.

Figure 8 shows the processing steps from captured image to binary image to measured length in pixels $L_{px}$.

The different methods for measuring the object length described in the method section were applied to the captured images. Measurement of raw object length based on (25) is shown in Figure 6a. The measured object length with perspective compensation, (29) is presented in Figure 6b. The two camera-based tilt compensation (34) is shown in Figure 6c.

Length Estimation of a Falling Glass Gob

Measurement setup was installed on a real glass plant between the Feeder mechanism and Delivery system. The addition to the measurement setup a high speed thermal camera was placed to get the temperature of the glass gob. The recorded temperature range was between 1090 °C to 1125 °C. The VFOV was limited to 75.6 cm due to the presence of Gob accelerator below the Feeder mechanism. The working distance was 170 cm. Figure 9 shows the two falling gobs (after being cut by shear mechanism located inside Feeder) between the Feeder and Delivery system. Length of the falling gob was measured and corrected with proposed method. Figure 10 shows the length (in pixels) of gob at different falling positions.

5. Discussion

This paper has proposed and evaluated a multi-camera setup for geometrical characterization of free-falling objects. The setup was designed to evaluate its ability to measure the length of an object accurately and dynamically with high temporal resolution during free-fall as illustrated in Figure 7. The intended application is to measure a free-falling molten glass gob. The drop height in this setup was selected based on the average spacing between the fore-hearth (feeder) and the delivery system of the actual glass container machine [18].

The captured image is processed and analysed to measure the length of the object. The processed image is also used to estimate the perspective error caused by the elliptical part of the object. Figure 11 shows the effectiveness of this technique. The perspective error is reduced to 0.43 for L’ with standard deviation of 0.69 mm for full FOV and relative accuracy of 99.8 %. Considerable reduction in perspective error can be seen near the top and the centre part of the FOV where the error is reduced from 4.32 to 0.62 with a standard deviation of 0.74, and 1.07 to 0.28 mm with a standard deviation of 0.53 mm, respectively (see

Table 1. Root mean square error (RMSE) and standard deviation (SDE) for length estimation methods and selected field of view (FOV).

FOV	L		L’		L”
	RMSE	SDE	RMSE	SDE	RMSE	SDE
Full	2.57	0.59	0.43	0.69	0.41	0.60
Top	4.32	0.61	0.62	0.74	0.64	0.70
Centre	1.07	0.53	0.28	0.53	0.25	0.53
Bottom	0.33	0.60	0.31	0.60	0.18	0.36

and

Table 2. Relative accuracy of the length measurements for the different length estimation methods and selected field of view (FOV) in percent (%).

FOV	L	L’	L”
Full	99	99.8	99.8
Top	97.8	99.7	99.7
Centre	99.6	99.9	99.9
Bottom	99.8	99.8	99.9

).

Camera 2, at an angle of 90 degrees apart from camera 1 and also focused on the free-falling object, is used to identify the error caused by tilt in the falling object. Tilt towards or away from the camera makes an object appear smaller than its actual length. If the object is tilted that is towards or away from camera 1, the object will appear to be tilted sideways to camera 2. Figure 6c shows the correction in length due to this tilt error. The results in Table 1 and Figure 11 show that the total error due to both perspective and tilt is reduced to 0.41 with a standard deviation of 0.6 mm and relative accuracy of 99.8 % for full FOV. The largest reduction in error is in the centre and the bottom part of the FOV: the error is reduced to 0.25 with standard deviation of 0.53 mm, and 0.18 with standard deviation of 0.36 mm, respectively (see Table 1 and Table 2).

In Figure 6a–c slight deviations and variability in measurement are due to segmentation error, which is about 0.16%. That means the standard deviation in L” (Table 1) is basically due to segmentation error.

The segmentation error may increase where there is insufficient illumination. The error can be reduced by using better illumination and by setting the binary threshold manually. However, this is not an issue in the intended application as a molten glass gob glow due to its very high temperature.

Figure 9 shows the actual free fall of a glass gob between feeder and delivery system. The proposed method to measure length is applied and presented in Figure 10 and Figure 12. The gob deforms and started to shrink during the free fall.

Results shows that proposed simple method gives good accuracy in measurement, which gives small opportunities for improved accuracy with more advanced setups for example stereo vision. However, due to harsh environment careful configuration of the setup is still required when the setup is placed in real plant. An addition of a stereo vision to automatically calculate all the setup parameter can remove the need of careful configuration of the setup.

In glass container production, different sizes of containers produced which eventually require various size of gobs. Molten gobs can be cut in small chunks or big depending on the size of the required glass container. The curvature of the gob on top and bottom part may change with the size of glass gob. Further investigation and comparison between the shapes of gobs is required.

6. Conclusions

This paper shows that the size of a cylindrical object can be characterized in free-fall with 99.8% accuracy with either one camera or two cameras. Using two cameras makes the setup more robust against object tilt. The relative accuracy with one camera might decrease and the RMSE might increase as the number of tilted cylinders increases. The results show that a higher accuracy can be achieved if the observation area is limited to the centre/bottom part of the camera FOV. The results presented show that the proposed camera-based setup can characterize dynamic geometrical changes in a free-falling object such as a molten glass gobs, enabling contactless analysis of the characteristics of the glass. Further, the results show that if the falling object has limited tilt, one camera will give as good results as two will, but two cameras will remove errors caused by tilted objects. The proposed method of length measurement is applied on the free falling glass gob and results shows that there are geometrical changes on the free falling glass gobs that require further investigation to correlate the results with glass characteristics. During the measurements on glass plants, it was observed that careful configuration of the setup is required, and an addition of stereo vision could overcome this issue.