Computer Vision-Based Approach for Reading Analog Multimeter

Multimeters are useful instruments for measuring electronic parameters. Even though the digital multimeter is commonly used in our daily life under the considerations of precision and cost, the analog multimeter is still preferable in many applications due to its easy use to monitor promptly varying values. However, the reading of analog multimeters (or A-meter) usually relies on human eyes with two obvious drawbacks of inefficiency and easy fatigue, while visual inspection onto an A-meter is needed for a long period of time. From the viewpoint of optical sensor application, computer vision, like human eyes, can also be used to sense stimuli from the real world. Therefore, in this paper, an approach of reading an A-meter based on a computer vision technique is proposed. Reading an A-meter relies on information from the arrow on the function selector and the pointer on the instrument meter; the presented method is thus mainly composed of horizontal alignment of the A-meter, detection of the instrument meter region, angle detection of the selector arrow, and angle detection of the pointer. In addition, the schemes of edge-based geometric matching (EGM) and pyramidal gradient matching (PGM) are adopted to detect the regions of interest. The mapping relationship between the function selector and the selector arrow as well as that between the instrument meter and the pointer are built and formulated to finally read the A-meter. The often used scenarios for reading AC voltage, DC voltage, and DC current as well as resistance are used for experiments and evaluations. The experimental results show that the accuracy of detecting the function selected is 100%, the mean accuracy of reading a value from the A-meter is 95% or above, except for some cases of reading resistance that are affected by the so-called little-change-large-multiplier effect. The proposed method can perform very well as long as the mean intensity is ≥7.5. Based on a suitable modification of the proposed method, an application of monitoring a storage level meter and pressure meter installed on a 15 m3 liquid nitrogen (LN2) tank is demonstrated. Our experiments and demonstrations confirm the feasibility of the proposed approach.


Introduction
Multimetesr are significant instruments for sensing or measuring electronic parameters (e.g., voltage, current, resistance) and are indispensable to the fields of science and technology. Moreover, they usually appear in our daily life, e.g., factory, school laboratory, home tools, and so on. Although the digital multimeter has been widely used for several decades due to the considerations of precision and cost, the analog multimeter is still preferable in many applications, in particular, for monitoring promptly varying values and easy understanding. The disadvantage of the traditional analog multimeter (abbreviated as A-meter for short) is the lack of data communication interface within it to allow further data processing. Therefore, reading an A-meter usually relies on human eyes with two obvious problems-inefficiency and easy fatigue-while visual inspection onto an A-meter is needed for a long period of time. Therefore, as the goal of this study, from the viewpoint of optical sensor application, we develop a computer vision-based method to automatically read the meter value so that the mentioned drawbacks of A-meter can be overcome . Figure 1a illustrates a scenario where a 1 KΩ resistor is being measured with the A-meter model YF-370A [1], which is used in this study. The 1 KΩ resistor apparently shown in Figure 1b can be identified by the four-band of colors (brown-black-red-gold) which represents 10 × 10 2 ± 5% (Ω) as studied in reference [2]. According to human operation and visual inspection, in this case, the selector arrow of the function selector is switched to the location of ×10 (see also Figure 1c) and the pointer in instrument meter indicates around 100 with the Ω-scale (see also Figure 1d). The resistance about 1000 Ω is thus obtained by 100 × 10 (Ω). In fact, the result obtained by human visual inspection is an approximate value due to the viewing angle and the nonlinear Ω-scale, as demonstrated in Figure 1d. Despite such an approximate measurement obtained from an A-meter, from the viewpoint of practical application, it is convenient to let an engineer quickly observe an electronic component characteristic or understand a circuit system's behavior for instance. According to this philosophy of people using the A-meter, the proposed approach based on computer vision can be simply introduced as follows. The measured resistor can be identified by the four-band of colors (brown-black-red-gold) which represents 10 × 10 2 ± 5% (Ω) [2]. Note that (c) the selector arrow of the function selector is switched to the location of ×10 and (d) the pointer in the instrument meter indicates around 100 by observing the nonlinear Ω-scale. Figure 2 shows the flowchart of our computer vision-based approach. In order to facilitate the effective angle detections in our method, the A-meter image (say original image or I org ) should be adjusted first in the horizontal orientation using the horizontal alignment stage, and the adjusted image can be obtained as I adj . With the edge-based geometric matching and pyramidal gradient matching techniques, the region of the instrument meter (R IM ) can then be located by the instrument meter region detection stage. Because the pointer appearing in the R IM is needle-shaped (refer to Figure 1d), the angle (θ p ) of pointer can be detected at the angle detection of pointer stage. Similarly, the angle (θ s ) of the selector arrow appearing in the adjusted image I adj can be detected at the angle detection of selector arrow stage. Since reading the A-meter is dependent upon the indications of θ s and θ p , we need two mapping functions, mapping function for selector arrow, F(θ s ), and mapping function for pointer, M(θ p ) F(θ s ) , to determine which function of the A-meter and which scale in R IM should be used, respectively. Once the F(θ s ) is known, the measurement result can then be obtained by M(θ p ) F(θ s ) . The related works are simply reviewed in the following text. Several methods have been developed to determine the angle of pointer, where the binary image obtained by image differencing and binarization has often been used for study. For example, in a previous study, two images with different pointer angles were required for image differencing operation to make an instrument meter background image. The background image was used to operate image differencing again with the new position of pointer. Then, binarization and the Hough transform method were used to calculate the pointer angle [3,4]. Guan et al. collected the line intersection points to form a fitting curve which reflected the relationship between the abscissa values and the resistance values [5]. The method proposed by Lima et al. [6] needed the meter to be perfectly horizontally lined up with the camera which would be troublesome for users. Kim et al. described about the automatic recognition of analog and digital meters installed in a nuclear power plant [7]. They applied simple and elementary techniques, such as binarization, region labeling, and projection, to extract needles in analog meters and numeric codes in digital meters. Thresholding was performed to obtain the binary image. Region labeling was applied to extract the image elements. They used a simple run length technique to detect the needle in a meter area and calculated the run length from the center to the edge of the meter image as sweeping circularly. The position with the maximum run length value was selected as the needle position [7]. Li et al. used red light and green light to help detect the pointer for automobile meter calibration and an instrument qualification test before leaving the factory [8]. Zhang et al. studied an improved method of the Hough transform method to detect the pointer angle [9]. Ocampo-Vega et al. used the centroid and tip coordinates of the pointer of a water meter to form a straight line and calculated the pointer angle between the x-axis and the line that was created by joining the tip and the centroid of the pointer [10]. Zhang and Li developed a preprocessing method to automatically read a water meter, where Fourier transform was used to correct a tilted dial image of a water meter [11]. Recently, to develop a Flight Guardian (FG) system which is not physically connected to an aircraft, Khan et al. [12,13] developed a simple method to automatically read the airspeed and revolutions per minute (RPM) of instruments where the video streams are acquired with two fixed-position camera devices. Based on the features of circular shape and needle locations in both dials, the focus of the camera devices is fixed at the center of this shape, and a circular-range sub-image composed of black pixels, except the white pixels occupied by the needle, are adopted for processing. By converting the sub-image pixels into a one-dimensional vector, the needle's position is identified by finding the maximum convolution output.
In practical applications of automatically reading analog meters, rotated/tilted/shifted situations under variant illumination need to be considered. Based on the research mentioned above, the effectiveness of applying image differencing and binarization methods onto the pointer detection were investigated in the early stages of developing our automatic approach for reading an A-meter, where we found that the useful information lost, as well as the noise yielded, under camera/meter movement or light variation significantly affects the performance of a reading meter. Therefore, in this study, the grayscale images acquired from the A-meter (refer to the I org , shown in Figure 2), are adopted for algorithm development. Moreover, some techniques are involved in our approach to cope with the situations just mentioned. In order to track a target under variant illumination or a moved situation, a technique to update the template could be helpful [14][15][16][17]. Considering the localization capability under variant illumination and dusky environments, edge-based geometric matching and pyramidal gradient matching techniques are embedded in our approach instead of template updating to localize the wanted regions of interest (ROI) of the A-meter (refer to the main stages of instrument meter region detection, angle detection of pointer as well as angle detection of selector arrow in the flowchart of Figure 2). As a short summary, as depicted in Figure 3, a computer vision system for automatically reading an A-meter is presented in this paper, where the feasibility of our approach is demonstrated by reading the AC voltage, DC voltage, and DC current as well as the resistance value. Here, AC and DC represent "alternating current" and "direct current", respectively. Inspection by the human eye regarded as a ground truth for comparison and discussion. The rest of this paper is organized as follows. Section 2 describes the framework and system setup of our study, where the mapping relationship of the function selector and instrument meter are constructed. Moreover, two methods for locating ROIs by template matching as well as the schemes for horizontal alignment of A-meter images and locating the useful ROIs are developed. Section 3 presents the procedure for detecting the pointer's angle (or position) including the calculation of the pointer's spin center and a scanning approach to detect the pointer. An illustration is provided in Section 4 to demonstrate the functional behavior of the whole system and confirm the feasibility of our approach. The experimental results and discussion are given in Section 5. Based on a suitable modification of the proposed method, an application to monitor the storage level meter and pressure meter installed on a 15 m 3 liquid nitrogen (LN2) tank is demonstrated in Section 6. The conclusions of this paper and further work related to this study are finally drawn in Section 7. Figure 4 shows our experimental environment including a YF-370A A-meter (Tenmars, Taipei, Taiwan), a C-920 USB camera (Logitech Far East Ltd., Hsinchu, Taiwan), a controllable light source, and a laptop computer. The A-meter image is captured by the USB camera with a size of 2304 × 1536 pixels. The LED light source, used for testing the capability of reading the A-meter under different illuminations, is controlled by an Arduino Mega 2560 microcontroller [18] with its PWM (pulse width modulation) channel. In order to facilitate the system's integration and build the user interface the proposed algorithms were implemented in a LabVIEW environment [19] combined with the C codes. The capability of reading the A-meter was tested with random orientation under variant illumination and a dusky environment inside a room.   [20]) was used to output resistance to the probe of the A-meter to build lookup data and measure resistance. (b) Measuring the DC current of a 1.5Vdc battery.

Mapping Relationship of Function Selector
According to the framework depicted in Figure 3, the sources measured by the A-meter may be the AC voltage, DC voltage, DC current, and resistance. Parts of the system's setup should be made before the algorithms are developed and experiments conducted, such as the mapping functions used in mapping function for selector arrow and mapping function for pointer, as indicated in Figure 2, as follows. We assumed that the A-meter image was aligned in the horizontal direction. By observing the function selector sub-image illustrated in Figure 1c, the mapping function for the selector arrow was built with the relationship between θ s and F(θ s ). Since there were N functions that were selectable in this A-meter, each function could be identified within a delta range, ∆ = 360 N . Ideally, the i-th function, say F i , would be indicated with the orientation i × ∆ ∈ [0 • ∼360 • ), i = 0, 1, . . . , N − 1. However, according to the arrangement of functions in the A-meter, each orientation corresponding a function could be modified within a range, i.e., . Therefore, the index of selected function to be used in mapping function for selector arrow was expressed as Because there were 20 selectable functions in the illustration presented in Figure 1c, N = 20 and ∆ = 18 • . Thus, based on Equation (1), the wanted mapping table, F(θ s ), could be expressed as listed in Table 1. Note here that the selected functions for F(θ s ) = 0, 1, . . . , 15 correspond to the linear scales on the instrument meter region (R IM ), whereas those for F(θ s ) = 16,17,18,19 correspond to the nonlinear ones. In order to facilitate the manipulation of reading value from the pointer, an auxiliary value, A(θ s ), is also provided in Table 1, where A(θ s ) represents the maximum value used in the functions, F(θ s ) = 0, 1, . . . , 15 (linear cases) and the multiplier used in the functions, F(θ s ) = 16,17,18,19 (nonlinear cases).  (1). Here, N = 20 and ∆ = 18 • . The auxiliary value, A(θ s ), represents the maximum value used in linear cases (i.e., F(θ s ) = 0, 1, . . . , 15) and the multiplier used in nonlinear cases (i.e., F(θ s ) = 16,17,18,19), respectively. Here "NA" denotes "not available."  16 1 17 10

Index of Function
1000 10000

Mapping Relationship of Instrument Meter
Several scales can be found in the instrument meter sub-image (or R IM ), as illustrated in Figure 1d. In order to correctly read the measured value, we focused on the value indicated by the pointer, ∈R IM , at the scale relative to the selected function in Table 1. The scales in R IM may be arranged in linear (e.g., the scales of ACV, DCV, and DCmA) or nonlinear (e.g., the scale of Ω) form. In addition, they can be designated in sector form, as depicted in Figure 5, where the pointer (θ p ) with a spin center rotates from right end (θ r ) to left end (θ l ). It was assumed that the A-meter had been adjusted in the horizontal direction. For the linear case, we found that the maximum value, say v max , was set at the right end of the scale (see Figure 1d for reference). Therefore, the observing or measuring value (M(θ p )) from the pointer (θ p ) could be simply expressed as If θ s is detected and its F(θ s ) in Equation (1) belonged to a linear case, then we had the corresponding maximum value, A(θ s ), in Table 1. Therefore, the above Equation (2) was rewritten as where F(θ s ) ∈ {0, 1, . . . , 15} in Table 1. For the nonlinear case, in this study, only the Ω-scale was used, where the maximum value (∞) was located at the left end of the scale, whereas the minimum value (0) was located at the right end. According to the known Ω-scale ∈ R IM , as given in Figure 1d, a nonlinear mapping table can be built as listed in Table 2 to solve this situation, where the ground-truth angle, θ gt (k), k = 0, 1, 2, . . . , K − 1 (K = 58 in this study) corresponding to the resistance value, R(k), on the Ω-scale ∈ R IM can be obtained by means of a protractor. If the pointer was located within two consecutive ground-truth angles, e.g., θ gt (k) ≤ θ p ≤ θ gt (k + 1), the θ p was estimated by interpolation as below where i = {k|θ gt (k) ≤ θ p ≤ θ gt (k + 1)}. If θ s was detected and its F(θ s ) in Equation (1) belonged to a nonlinear case, then we had the corresponding multiplier A(θ s ) in Table 1. Therefore, the above Equation (4) was rewritten as where F(θ s ) ∈ {16, 17, 18, 19} in Table 1. Table 2. Nonlinear mapping table built for measuring the resistance from the pointer (θ p ) indication on the Ω-scale ∈ R IM where the ground-truth angle θ gt (k) corresponded to the resistance value (R(k)) which was obtained by means of a protractor. The values listed here were used in the current study.

Two Methods for Locating ROIs by Template Matching
According to the Equations (1), (3) and (5), it is obvious that the identification of θ s , θ l , θ r , and θ p are based on a horizontally aligned A-meter and can be achieved with the template matching scheme. Therefore, several fundamental templates were built in prior and used in our approach. To perform the template matching, the edge-based geometric matching and pyramidal gradient matching techniques were adopted in our system. Edge-based geometric matching uses the curves obtained from the edge information of a template image as the basis for the features that are used for matching. Pyramidal gradient matching is able to reduce the size of the image and the template to improve the accuracy and the execution time of pattern matching as well as to increase the performance of matching the blur or low contrast image. The edge information was used in both of the two algorithms. Since the algorithms for edge-based geometric matching (EGM) and pyramidal gradient matching (PGM) have been widely used and embedded in the NI (National Instruments) vision software for machine vision applications [21] and for LabVIEW [19], we do not review their principles here, but we applied them into our approach with the following expressions. Let I be an input image and T denote the template, which will be introduced later, for matching process. Both I and T are grayscale images. The return result of each matching includes three elements, i.e., flag, ROI, and rotation, and can be formulated as where flag denotes the matching successes (1) or not (0). ROI represents the four coordinates of the located region if flag is 1. In addition, since the functions of EGM and PGM can deal with the rotation cases, the parameter rotation reflects the rotation angle of the located ROI if the flag is 1.

Horizontal Alignment of an A-Meter Image
For the flowchart designated in Figure 2 to read an A-meter (I org ), the horizontal alignment is the first step to adjust I org into I adj for further processing. In this study, the EGM method expressed in Equation (6) was used for horizontal alignment where the input image was the original A-meter image (I org ) and one useful middle part of the A-meter shown in Figure 6 is used as the template T A-meter for matching. Thus, the Equation (6) may be expressed by Once the ROI A-meter and rotation A-meter were detected, the horizontally adjusted image I adj was obtained by rotating the I org about the center of the located ROI A-meter minus the rotation A-meter .

Locating the Useful ROIs
After obtaining the I adj , some ROIs were located in order to compute the pointer angle, θ p ∈ R IM , as indicated in Figure 2, where the R IM was located first. Because the R IM , including the pointer and scale details, is of great importance in this study, both the EGM and PGM methods were applied for template matching to increase the correctness of locating R IM from the image, I adj . In addition, in order to compute the right end (θ r ) and the left end (θ l ) of the scale, ∈ R IM , as indicated in Figure 5, the right end ROI R right-end and left end ROI R left-end also needed to be located carefully. Therefore, both the EGM and PGM methods were also applied to locate the R right-end and R left-end . That is, to locate the ROI's R IM , R right-end and R left-end , the designated approach was the same and was unified as follows. For each case, two kinds of templates, namely T EGM and T PGM , were adopted for template matching and used in the EGM and PGM methods, respectively. In order to let our method cope with a reasonable matching range from a bright to dark situation, two templates, T PGM bright and T PGM dark , were built for the PGM method. T PGM bright or T PGM dark was chosen and used in the PGM based on the mean intensity (Ī) of the given input image I and may be expressed by In this study, the threshold (TH = 52) was heuristically selected and used in our experiments. Accordingly, in order to locate the ROI R with both EGM and PGM methods, the Equations (6) and (7) were rewritten respectively as below and f lag PGM , ROI PGM , Rotation PGM = PGM(I, T PGM ).
If both methods successfully located an ROI for R, the one with a smaller rotation angle was the desired ROI, since its position would be closer to the horizontal base (0 • ). Thus, the located R was finally determined by the following expression,  (10) and (11), to find R IM , the input image was I adj , whereas to find R right-end and R left-end , the input image was R IM . The notations are summarized in Table 3 to locate the R IM , R right-end , and R left-end .     By following the method presented in Section 2.4, the information to locate the selector arrow (R selector-arrow ) was also found with the PGM method using the given template (T selector-arrow ), as shown in Figure 8, which may be expressed as below. In this situation, the image I adj was used for matching.
[ f lag selector-arrow , ROI selector-arrow , Rotation selector-arrow ] = PGM(I adj , T selector-arrow ). (13) Note here that the located ROI is ignored but the rotation is used and is just the wanted θ s as noted in Section 2.1. That is, θ s = Rotation selector-arrow .
As a short summary, the found information of R right-end and R left-end helped us to identify the parameters θ r and θ l . Furthermore, they were used to determinate the spin center of pointer ∈ R IM , as indicated in Figure 5 and therefore, to detect the angle of pointer θ p . All these algorithms are detailed in Section 3.

Spin Center Calculation of Pointer
Based on the located R right-end and R left-end in R IM presented in Section 2.5, the relationship between the pointer indicated by θ p and the scale in R IM (refer to Figure 5) were replotted by adding the coordinates {(x r1 , y r1 ), (x r2 , y r2 ), (x r3 , y r3 ), (x r4 , y r4 )} of R right-end and {(x l1 , y l1 ), (x l2 , y l2 ), (x l3 , y l3 ), (x l4 , y l4 )} of R left-end , as depicted in Figure 9. In this study, the templates for the right-end scale, i.e., T EGM right-end , T PGM bright-right-end and T PGM dark-right-end , given respectively in Figure 7d-f, were selected with a W r × H r rectangular ROI so that one pair of diagonal corner points were just passed over by a dummy straight line of the right-end scale (from the spin center of pointer) as redrawn in Figure 10a-c. Therefore, the angle of the right-end of the scale (θ r in degrees) was expressed by Similarly, the templates for the left-end scale, i.e., T EGM left-end , T PGM bright-left-end and T PGM dark-left-end , given respectively in Figure 7g-i, were selected with a W l × H l rectangular ROI so that one pair of diagonal corner points were just passed over by a dummy straight line of the left-end scale (comes from the spin center of pointer), as redrawn in Figure 10d-f. Therefore, the angle of the left-end of the scale (θ l in degrees) was expressed by In the current study on the YF-370A A-meter, because of W r H r and W l H l , we thus had θ r 45 • and θ l 135 • . These matched with the ground-truth angles, i.e., θ gt (0)| R(0)=0 = 45 • and θ gt (57)| R(57)=∞ = 135 • , as listed in Table 2, which were obtained by means of a protractor.
By reconsidering the illustration given in Figure 9, once the R right-end and R left-end were located, the Equations (14) and (15) could be rewritten as and In addition, by observing the Figure 9, the straight line of right end of scale was shown to pass approximately over (x r2 , y r2 ) and (x r3 , y r3 ) ∈ R right-end and the left-end of the scale passed approximately over (x l1 , y l1 ) and (x l4 , y l4 ) ∈ R left-end . By solving the two line equations joining one point, the spin center of pointer (x c , y c ) was thus obtained by the following expressions Note here that the origin (0, 0) is the top-left coordinate of the image. Figure 9. Relationship between the pointer indicated by θ p and scale in the R IM of the A-meter by adding the coordinates of R right-end and those of R left-end , where the scale is confined within the range from θ r to θ l . Note here that the straight line of the right-end of scale passes approximately over (x r2 , y r2 ) and (x r3 , y r3 ) ∈ R right-end and that of left end of scale passes approximately over (x l1 , y l1 ) and (x l4 , y l4 ) ∈ R left-end , which helps with the calculation of the spin center of pointer (x c , y c ). The origin (0, 0) is the top-left coordinate of the image. Figure 10. (a-c) illustrate that the templates, T EGM right-end , T PGM bright-right-end and T PGM dark-right-end , given respectively in Figure 7d-f ,were selected with a W r × H r rectangular ROI so that one pair of diagonal corner points were just passed over by a dummy straight line of the right-end scale; (d-f) illustrate that the templates, T EGM left-end , T PGM bright-left-end and T PGM dark-left-end , given respectively in Figure 7g-i, were selected with a W l × H l rectangular ROI so that one pair of diagonal corner points were just passed over by a dummy straight line of the left-end scale.

Scanning Approach to Detect the Pointer
The traditional projection scheme was borrowed here to detect the pointer. Our approach was to scan the image (R IM ) with a scanline (which has a length, L) about the spin center (x c , y c ) from θ r to θ l , as illustrated in Figure 11 so that all pixels within the sector region were visited once at least by the scan. Thus, the scan step (∆θ) was selected carefully and is presented later in this section. Here, L can be determined by the distance between (x c , y c ) and (x r2 , y r2 ) (or (x l1 , y l1 )), i.e., L = (x c − x r2 ) 2 + (y c − y r2 ) 2 + δ, as illustrated in Figure 9. In this study, the extent (δ) was set heuristically to 15 for our experiments. The method was as follows: Let S(i) be the set of all pixels belonging to the i-th scanline, and θ(i) represent the corresponding θ-value of the i-th scanline. In addition, let g(p) be the image intensity (or grayscale value) of the pixel (p). Therefore, the accumulation of all pixels' intensities within the i-th scanline, A(i), otherwise known as the A-quantity, was expressed as Ideally, since the appearance of the pointer tends to be a dark color, the pointer indicated by the k-th scanline should possess the lowest accumulation value, A(k) = min ∀i A(i), and the pointer's angle (θ p ) is intuitively the corresponding θ(k). However, this detection rule can fail due to the very thin pointer as well as the interferences of characters and the pointer's shadow on the meter surface. In order to enhance the detection correctness, the gradient of A(i), ∀i was further derived as GA(i), ∀i or called the profile of GA. Since along the scanning direction, the A-quantity of the thin pointer and its one neighbor may be changed from a large amount to a small amount or from a small amount to a large amount, the pointer's position indicated by the k-th scanline should possess the largest change with zero-crossing property, where the sign of GA(k) and that of GA(k + 1) are mutually opposite. The pointer's angle θ p was thus detected and expressed by Referring to Figure 11 in order to perform effectively the scanning process, the incremental amount of scanning degree ∆θ for each step needed to be determined so that the total number of scanning steps with ∆θ in the image domain was satisfied but not redundant. The following method was used: Let the changing amount in the horizontal and vertical directionswith the scanning range (θ R ) be a full range of FR x and FR y , respectively. Theoretically, the larger one between FR x and FR y will dominate the ∆θ so that there are more scanlines to cover all the pixels within the sector region with the θ R during the scanning process. This was expressed by As shown in the illustration in Figure 11, FR x and FR y can be replaced by double half range HR x and HR y , respectively. That is, FR x = 2HR x and FR y = 2HR y . Using trigonometric expressions, HR x and HR y may be expressed by and Note here that since the pointer may be located at the right end (θ r ) or the left end (θ l ) of the scale, to detect the pointer, the scanning range needs to be wider than θ R in real applications. Therefore, we heuristically set θ r = θ r − 5 • to be the starting position and θ l = θ l + 5 • to be the ending position for our scanning process, where the pointer's angle θ p was found with Equation (20). Figure 11. Illustration of our scanning approach to detect the position of pointer θ p . The scanning direction is from θ r to θ l , i.e., the range (θ R ), to find θ p where the corresponding scanline indicates the largest gradient change with zero-crossing property in the profile of GA.

Illustration of Reading the A-Meter
After presenting the methodology of our approach with the framework and system setup in Section 2 and the angle detection of pointer in Section 3, a series of results of measuring a 1KΩ resistor by the A-meter YF-370A are shown here to demonstrate the feasibility of the proposed approach. Figure 12a shows the original image (I org ), where one 1KΩ resistor is being measured by an A-meter YF-370A. The gray channel of Figure 12a was used for further processing, where its mean intensity wasĪ = 131.1. By template matching using Equation (8) with the template (T A-meter ) given in Figure 6, rotation A-meter = 16.914 • was detected, where the center of ROI A-meter was (1225.9, 1420.6), as depicted in Figure 12b. Then, the horizontal alignment image (I adj ) was obtained by rotating the I org in Figure 12a about the center (1225.9, 1420.6) with −16.914 • , as shown in Figure 12c.
Next, by performing template matching using Equation (12) with the templates given in Figure 7a-c onto the I adj , the R IM was located as the top region marked with the red rectangle in Figure 12d, where the selector arrow was also found using Equation (13), with the template given in Figure 8. The R IM was used later for the detection of the right-end R right-end and left-end R left-end of the scale. In this step, the detected selector arrow reported the rotation angle as θ s = 314.194 • . According to Equation (1) and the mapping table in Table 1, this θ s ∈ [297 • , 315 • ) indicates that the current selection represents the function name "Ω × 10"; F(θ s ) = 17 a nonlinear case; and A(θ s ) = 10 is a multiplier.
Within the found R IM , Figure 12e shows that the useful right-end R right-end and left-end R left-end of the scale were obtained by means of Equation (12)  According to the results obtained above, we the scanline length L = 499.318 (pixels) and ∆θ = 0.128 • , based on Equation (21), were used in our scanning process to detect the θ p . As noted in Section 3.2, to detect the pointer, the scanning range needed to be wider than θ R in real applications. Thus, the process was scanned from θ r = θ r − 5 • to θ l = θ l + 5 • to produce the accumulation profile of A and the gradient profile of GA, as shown in Figure 13a,b, respectively. In this example, itwo valleys obviously appeared (Figure 13a), which indicated the positions of θ r and θ l , respectively. From the GA-profile in Figure 13b and according to Equation (20), the detected index was k = 704 with the corresponding θ(704) = 118.405 • , which was just the detected pointer's position (θ p ) in the current illustration.  Based on θ p = 118.405 • and the selector function "Ω × 10" with the multiplier A(θ s ) = 10, Table 2 and Equation (5) were used to determine the M(θ p ) F(θ s ) , i.e., the resistance of the measured 1KΩ resistor. From Table 2, since θ gt (47) < θ p < θ gt (48), we had θ gt (47) = 118.4, θ gt (48) = 119.6, R(47) = 100, and R(48) = 120. Using the computation of Equation (5), a resistance of 1000.83Ω was obtained which is very close to 1 KΩ. Thus, the feasibility of the proposed approach is confirmed.

Results and Discussions
In order to further demonstrate the effectiveness of the proposed approach to read an A-meter, the often used functions-reading the AC voltage, DC voltage, and DC current as well as resistance values-were adopted for experiments and evaluations. Since there is no true value for reading a measurement from the A-meter, in this study, inspection by the human eye was regarded as the ground truth for evaluating the accuracy of the proposed method as the framework given in Figure 3. V h is the value from the A-meter read by human eyes, and V p is that read by the proposed method. The accuracy (%) was defined as and was used for the performance evaluation. With the A-meter YF-370A, there were 15 experimental scenarios that were designated as follows.
(2) function select "ACV 250" to measure the 110 Vac line power. Scenarios (1)-(10) belong to the linear cases and their results are presented in Section 5.1. Scenarios (11)-(15) belong to the nonlinear cases and their results are presented in Section 5.2. Section 5.3 further discusses the influence of illumination to the proposed approach and uses a digital meter instead of human eyes to reconfirm the accuracy of the proposed method. In addition, the influences of translation and/or rotation as well as the effective range of using our approach are also investigated.

Experimental Results on Linear Cases
In order to test the feasibility of our method under different rotations (rotation A-meter ) and illuminations (mean intensityĪ) of the A-meter YF-370A, five measurements for each scenario were performed by human eyes and the proposed approach and were analysed and compared. We checked the function select (FS h ) detected and the read value (V h ) obtained by human eyes and those (FS p and V p ) by the proposed method, respectively and then evaluated the accuracy with Equation (24). Note here that if the signs of rotation A-meter were positive, it meant that the A-meter was rotated to the left; otherwise it was rotated to the right. In addition, the lower theĪ was, the darker the illumination was, and vice versa.
The first scenario (1) was to let function select be "OFF", where the measurement resulted in 100% accuracy, as reported in Table 4. This means that the rotated A-meter was horizontally adjusted, and the function select and the meter pointer were detected correctly for each case by the proposed method. Note here that since both V h and V p were 0, they were identical and Equation (24) was not needed. For scenario (2), the A-meter was set to function "ACV 250" to measure the 110Vac line power. Table 5 reports the mean accuracy as 97.28%. Note here that the second row is a slightly tilted case with low illumination, as shown in Figure 14. In this tilted example, θ s = 40.827 • ∈ [27 • , 45 • ) represents that the function "ACV 250" was selected and A(θ s ) = 250 was used according to Equation (1) and Table 1, where F(θ s ) = 2 implies that this example is a linear case. In addition, θ r = 45 • , θ l = 134.393 • were obtained respectively by Equations (16) and (17) using the detected R right-end and R left-end , as detected in Figure 14c. Through our scanning process (using ∆θ = 0.12) on the GA-profile as shown in Figure 14b, θ p = 95.974 was thus detected. Since the current example is a linear case, Equation (3) was used and the read result V p = 107.444 Vac was finally obtained. Even the A-meter was rotated 15.947 • and the illumination was low:Ī = 49.8. In this case, the result V p = 107.444 Vac obtained by the proposed method is very close to the V h = 108 Vac observed by human eyes. Table 6 reports the results for scenario (3), where the function "ACV 250" was also selected to measure the 220Vac line power. The mean accuracy was 98.86% except for the fifth data point, since it was difficult to read by the human eye (but our system read the value as 236.450 Vac) under very low illumination (Ī = 7.5).  The results of the other linear cases (scenarios (4)-(10)) using function selects of "DCV 2.5", "DCV 10", "DCV 50" as well as "DCmA 25" are reported in Table 7. From these experimental results, it is shown that all of the function selects were detected with 100% accuracy. The pointer was detected very accurately and the mean accuracy of the reading value for most of cases was 95% or above.

Experimental Results on Nonlinear Cases
In this study, the nonlinear scale merely considered the resistance measurement with the A-meter YF-370A. The DC resistance controller ZX99A [20], as shown in Figure 15, was used to provide the reference resistance for our experiments. The results of the nonlinear cases (scenarios (11)-(15)) of using function selects of "Ω × 10", "Ω× 1 K", and "Ω× 10 K" are reported respectively in Table 8. From these experimental results, all the function selects were detected with 100% accuracy. The pointer was detected very well and the mean accuracy of the reading value for most of cases was above 97% except for the case using "Ω × 1 K" to measure the 1 KΩ resistor reported in Table 8d, where the mean accuracy was 87.27%. By observing the nonlinear scale on the meter surface, as shown in Figure 1d, it appears that it this accuracy decay for the function select "Ω× 1 K" to measure the 1 KΩ resistor was reasonable since a little difference around "1" may result in a large change due to the large multiplier "×1000" (we call it the little-change-large-multiplier effect). Contrarily, the phenomenon was reduced using the function select "Ω × 10" to measure the 1 KΩ resistor (refer to Table 8c) since the value around "100" indicated by the pointer is greater than that around "1", and the multiplier of "×10" is less than that of "×1000". Figure 15. The DC resistance controller ZX99A [20] was used to provide the reference resistance to validate the performance of reading resistance from the A-meter by the proposed method. Table 7. Results from using the A-meter YF-370A with function select (a) "DCV 2.5" to measure the 1.5 Vdc battery, (b) "DCV 10" to measure the 5 Vdc adaptor, (c) "DCV 10" to measure the 7.5 Vdc adaptor, (d) "DCV 10" to measure the 9 Vdc battery, (e) "DCV 50" to measure the 12 Vdc adaptor, (f) "DCV 50" to measure the 24 Vdc adaptor, and (g) "DCmA 25" to measure the DC current of the 1.5 V battery.   Table 8. Results of using the A-meter YF-370A with function select (a) "Ω × 10" to measure the resistance of the 50 Ω resistor, (b) "Ω × 10" to measure the resistance of the 220 Ω resistor, (c) "Ω × 10" to measure the resistance of the 1 KΩ resistor, (d) "Ω × 1K" to measure the resistance of the 1 KΩ resistor, and (e) "Ω × 10 K" to measure the resistance of the 70 KΩ resistor.

Further Results and Discussion
So far the proposed approach has been confirmed by experiments using both linear cases and nonlinear cases, as presented in Sections 5.1 and 5.2, respectively. In these experiments, in addition to obviously rotating the A-meter, we also randomly set the A-meter to be slightly tilted and adjusted the illumination from dark to bright during the experimental period where the illumination controlled by the LED of our system was reflected by the mean intensity ofĪ. An experiment was further conducted as follows to verify the usable range ofĪ. The A-meter YF-370A was rotated by 15.425 • and the 110 Vac line power was measured with function select "ACV 250". Table 9 reports that the proposed method can perform very well as long as the mean intensity isĪ ≥ 7.5.
The performance evaluations of those experiments presented previously were based on inspection by the human eye (V h ), where looking at the A-meter (or viewing angle) will result in an error due to reading the value indicated by the pointer. Similarly, such a situation (like the "tilted surface" noted in the previous experiments) will also appear in our computer vision-based approach. Hence, it can be concluded that the errors resulting from the A-meter itself and the observation from human eyes or a vision sensor are inevitable under the scenario of using an A-meter. In order to further confirm the feasibility of the proposed method, a digital multimeter (model DMM-85) instead of human eyes was used to read the value of measurement as ground truth (V dm ). By replacing V h by V dm , Equation (24) was also used for the accuracy evaluation. Table 10 reports that a mean accuracy of 95.61% can be achieved. Note here that the little-change-large-multiplier effect appears in the experiment of the last row of Table 10, where the accuracy decayed to 88.77%. To investigate the influences of translation and/or rotation using our approach, we experimented with shifting and/or rotating the camera of our system to read the A-meter. The tester calibrator (model AOIP CMU 310) was used to provide a stable 24 Vdc as the ground truth and the function of the A-meter was switched to "DC 50V" for the experiments. Six scenarios were designated as the A-meter in the captured image: (1) shifted right, (2) rotated right, (3) shifted right and rotated left, (4) shifted left, (5) rotated left, and (6) shifted left and rotated right, where scenarios (2) and (5) were obtained by the arrangements shown in Figure 16a,b, and their illustrative results are presented in Figures 17 and 18, respectively. In this experiment, there were 100 consecutive runs for each scenario in our approach. The measured DC voltages for the six scenarios are plotted in Figure 19, and their statistical results are reported in Table 11. The plots shown in Figure 19, demonstrate that a slight fluctuation existed when reading the A-meter using our computer vision-based system. In addition, as reported in Table 11, when the camera was located at the left-side relative to the A-meter (e.g., scenarios (1)-(3)), the reading value was higher than 24 Vdc, and vice versa (e.g., scenarios (4)- (6)). This matches with the relationship among the observer, needle, and scale along a line of sight. Note here that the lowest value (0) of the "DC 50V" scale on the A-meter was located at the left-end, whereas the highest value (50) was located at the right-end (see Figures 17 and 18 for reference).Even though the recognition value produced by our system was not always as stable as the digital value displayed in the tester calibrator, a mean accuracy of around 98% was still able to be achieved. As a short conclusion, an "uncertainty problem" exists when obtaining measurement values from an analog meter. Two obvious uncertainties can be observed during the measuring process. First, since the measurement value is indicated by rotating a needle-type pointer onto a marked scale, an error will result from the mechanical rotation, the quality of the pointer, and the scale. Secondly, viewing directly onto the meter by not only the human eye or a camera will also lead to a reading error as the experimental results showed.
Because the location of the ROIs in our approach was based on template matching, as presented in Section 2.3, the fundamental templates needed to be trained and built in prior. Even though the adopted templates are fixed and limited in the current system, based on the previous experiment, it was determined that it was worth conducting another experiment to examine the rough effective range. Again, the tester calibrator (model AOIP CMU 310) was used to provide a stable 24 Vdc as ground truth and the function of A-meter was switched to "DCV 50" for experiments. We moved the camera to read the A-meter and examine the effective result repeatedly. Finally, an effective range, as drawn in Figure 20, was obtained, namely, it was determined that the camera should be placed at a location about 13∼15 cm distance from the A-meter and the viewing orientation should be within [−18 • , +18 • ] about the center line. Note here that F has a distance from the A-meter of 16 cm; in this case, our system still can function well. The statistical results reported in Table 12 confirm that our computer vision-based approach can allow the A-meter to be read within the effective range drawn in Figure 20. If the camera is placed outside the effective region, reading the A-meter will fail. For example, consider the H and I in Figure 20. H and I have distances of 12 cm and 17 cm from the A-meter, respectively, and represent the camera being placed outside the effective range. Their experimental results displayed in Figures 21 and 22, respectively show that they fail to read the A-meter. As a comment, the scalability of the current system will depend on the A-meter included in the processed image, the resolution of the image, as well as the set of templates to be trained. Table 11. Statistical results of the measured DC voltages plotted in Figure 19.       Currently, the proposed algorithms are implemented in the LabVIEW environment [19] by combining with the C codes and plotting the visualized results for functional verification only. The execution time still bounds in the "second" level with an image size of 2304 × 1536 pixels for processing. Most of the time is spent on template matching and horizontal alignment. It takes about 90% of the whole execution time in the current implementation. The time taken to detect the pointer and read the value takes around 1 sec, as the videos demonstrate in reference [22,23]. The first video [22] shows the stability of locating the R right-end and the R left-end ∈ R IM and reading the value. The second video [23] demonstrates the scanning process to detect the pointer and read the value. Based on the effectiveness of reading the A-meter with the presented method, an application to read the dials attached on a liquid nitrogen tank is further demonstrated in the next section.

Scenario
As a result, the computer vision-based approach has been proposed to effectively read the value on not only the linear scale but also on the nonlinear scale indicated by the pointer from an analog multimeter. Through a comparison of the results obtained by the proposed method to those obtained by both human eyes and the digital multimeter, our experiments confirmed that the accuracy of locating the function select is 100%, and the mean accuracy of reading the value from the A-meter is 95% or above ,except for some cases affected by the little-change-large-multiplier effect. Our method was also shown to perform well under situations where the A-meter is rotated, slightly tilted, and/or illumination varied and thus, its feasibility and effectiveness are confirmed.

Application of Monitoring Liquid Nitrogen Meters
By suitably modifying our algorithms, the proposed approach can be applied for reading the similar kinds of A-meters. In this section, we demonstrate a successful application of monitoring a storage level meter and pressure meter installed on a 15 m 3 liquid nitrogen (LN2) tank which is located at outside our laboratory, as shown in Figure 23a. These two analog needle-type meters are very significant for our laboratory and need to be checked frequently to guarantee that the storage level and pressure of tank are maintained within a safe range. For example, excessive pressure will damage the solenoid valve of testing equipment and cause other dangerous situations, and once the LN2 storage level is below the low limit, the technician should call for LN2 filling to avoid sudden interruption of testing in laboratory. Reading such an analog meter based on human eyes is rather inefficient and will easily cause fatigue problems during long-term monitoring. Therefore, our computer vision approach is a valuable way to deal with such a scenario. In this application, an Internet Protocol (IP) camera (model AXIS P3364-LVE 12mm [24]) was installed to acquire the video stream (with a frame resolution of 1280 × 960 pixels and frame rate 30 frames/sec) including both the level and pressure meters, as shown in Figure 23b. At night, the LN2 tank meters can be illuminated by a road lamp and a LED flood light as shown in Figure 23c. For each frame, the subimage of the level meter and pressure meter is extracted first according to the predefined ROIs, and then processed individually by our method to read the value indicated by the meter pointer. Note here that only the pointer's angle detection (presented in Section 3) was implemented in this application. Figure 24 illustrates one frame containing the level and pressure meters from a video stream at rainy night. Using our monitoring system, the results shown in Figure 25 demonstrate the usefulness of our method in this application. Even during worst-case scenarios, such as rainy nights, our system can work well, as shown by the results in Figure 26. Since the storage level of LN2 gradually decreases during consumption in the laboratory and increases during filling, the pointers shown on the two meters normally vary very slowly, as the real video (recorded in rainy day) demonstrated in [25], and the pressure should be maintained within a range between 3 to 5 bar, and the storage of LN2 should be maintained above 80, as indicated on the scale used by our laboratory. In order to show the capability of reading the LN2 meters by our method, we intentionally set the pointer of meter moving fast by manual as the real video demonstrated in [26]. The two videos given in [25,26] demonstrate that our system can read the two dials on the LN2 tank in a real application with an execution time of about 0.2∼0.6 second for each reading. This can be regarded as real-time processing, compared to the very slow change of the meter pointer. As a result, such a non-invasive computer vision system provides us a convenient alternative to monitor the LN2 tank information without installing additional expensive digital sensors and truly helps us to guarantee the safety of using LN2.

Conclusions and Future Work
The analog multimeter (A-meter) is a significant instrument for sensing or measuring electronic signals, e.g., voltage, current, resistance, and so on. Even though the digital multimeter is commonly used in our daily life under the considerations of precision and cost, the analog multimeter is still a preferable device for many applications due to its ability to monitor promptly varying values. In order to retain the merits of A-meter and to reduce the tiredness of human eyes observing the A-meter over long durations, it is of great importance to develop a computer vision-based method to automatically read the measured value indicated by the pointer of the A-meter. To effectively read an A-meter, the presented algorithms include horizontal alignment of the A-meter, detection of the instrument meter region, angle detection of the selector arrow, and angle detection of the pointer. The desired ROIs used in our algorithms are extracted by two template matching methods-edge-based geometric matching (EGM) and pyramidal gradient matching (PGM)-which have been widely used and embedded in the NI vision software for machine vision applications [21] and for LabVIEW [19]. Moreover, the mapping relationship between the function selector and the selector arrow, as well as that between the instrument meter and the pointer are built and formulated to read the A-meter, where the scale on the meter may be linear or nonlinear. The often used scenarios of reading AC voltage, DC voltage, and DC current as well as resistance were adopted for the experiments and evaluations in this study. Our experimental results showed that using our method, the accuracy of detecting the function select was 100%, and the mean accuracy of reading values from the A-meter was 95% or above, except for some cases of reading resistance affected by the so-called little-change-large-multiplier effect. In addition, the proposed method performed very well as long as the mean intensity, i.e., illumination was ≥ 7.5. The "uncertainty problem" in reading an analog meter was explored using translation and/or rotation of the camera to read the A-meter by the proposed computer vision-based approach. The effective range to use our method within was also investigated and validated. Based on suitable modification of the proposed method, an application involving the monitoring of a storage level meter and pressure meter installed on a 15 m 3 liquid nitrogen (LN2) tank was demonstrated successfully. In conclusion, even if the A-meter is rotated, slightly tilted and/or illumination varied, the proposed computer vision-based approach can effectively read the analog multimeter. A general purpose method for reading the A-meter, its real-time implementation and a time complexity analysis, as well as a more deep investigation on the uncertainty issue are worthy of further study and could be regarded as our future works.