1. Introduction
Multimetesr are significant instruments for sensing or measuring electronic parameters (e.g., voltage, current, resistance) and are indispensable to the fields of science and technology. Moreover, they usually appear in our daily life, e.g., factory, school laboratory, home tools, and so on. Although the digital multimeter has been widely used for several decades due to the considerations of precision and cost, the analog multimeter is still preferable in many applications, in particular, for monitoring promptly varying values and easy understanding. The disadvantage of the traditional analog multimeter (abbreviated as A-meter for short) is the lack of data communication interface within it to allow further data processing. Therefore, reading an A-meter usually relies on human eyes with two obvious problems—inefficiency and easy fatigue—while visual inspection onto an A-meter is needed for a long period of time. Therefore, as the goal of this study, from the viewpoint of optical sensor application, we develop a computer vision-based method to automatically read the meter value so that the mentioned drawbacks of A-meter can be overcome.
Figure 1a illustrates a scenario where a 1 KΩ resistor is being measured with the A-meter model YF-370A [
1], which is used in this study. The 1 KΩ resistor apparently shown in
Figure 1b can be identified by the four-band of colors (brown-black-red-gold) which represents
(Ω) as studied in reference [
2]. According to human operation and visual inspection, in this case, the selector arrow of the function selector is switched to the location of
(see also
Figure 1c) and the pointer in instrument meter indicates around 100 with the Ω-scale (see also
Figure 1d). The resistance about 1000 Ω is thus obtained by
(Ω). In fact, the result obtained by human visual inspection is an approximate value due to the viewing angle and the nonlinear Ω-scale, as demonstrated in
Figure 1d. Despite such an approximate measurement obtained from an A-meter, from the viewpoint of practical application, it is convenient to let an engineer quickly observe an electronic component characteristic or understand a circuit system’s behavior for instance. According to this philosophy of people using the A-meter, the proposed approach based on computer vision can be simply introduced as follows.
Figure 2 shows the flowchart of our computer vision-based approach. In order to facilitate the effective angle detections in our method, the A-meter image (say original image or
) should be adjusted first in the horizontal orientation using the
horizontal alignment stage, and the adjusted image can be obtained as
. With the edge-based geometric matching and pyramidal gradient matching techniques, the region of the instrument meter (
) can then be located by the
instrument meter region detection stage. Because the pointer appearing in the
is needle-shaped (refer to
Figure 1d), the angle (
) of pointer can be detected at the
angle detection of pointer stage. Similarly, the angle (
) of the selector arrow appearing in the adjusted image
can be detected at the
angle detection of selector arrow stage. Since reading the A-meter is dependent upon the indications of
and
, we need two mapping functions,
mapping function for selector arrow,
, and
mapping function for pointer,
, to determine which function of the A-meter and which scale in
should be used, respectively. Once the
is known, the measurement result can then be obtained by
.
The related works are simply reviewed in the following text. Several methods have been developed to determine the angle of pointer, where the binary image obtained by image differencing and binarization has often been used for study. For example, in a previous study, two images with different pointer angles were required for image differencing operation to make an
instrument meter background image. The background image was used to operate image differencing again with the new position of pointer. Then, binarization and the Hough transform method were used to calculate the pointer angle [
3,
4]. Guan et al. collected the line intersection points to form a fitting curve which reflected the relationship between the abscissa values and the resistance values [
5]. The method proposed by Lima et al. [
6] needed the meter to be perfectly horizontally lined up with the camera which would be troublesome for users. Kim et al. described about the automatic recognition of analog and digital meters installed in a nuclear power plant [
7]. They applied simple and elementary techniques, such as binarization, region labeling, and projection, to extract needles in analog meters and numeric codes in digital meters. Thresholding was performed to obtain the binary image. Region labeling was applied to extract the image elements. They used a simple run length technique to detect the needle in a meter area and calculated the run length from the center to the edge of the meter image as sweeping circularly. The position with the maximum run length value was selected as the needle position [
7]. Li et al. used red light and green light to help detect the pointer for automobile meter calibration and an instrument qualification test before leaving the factory [
8]. Zhang et al. studied an improved method of the Hough transform method to detect the pointer angle [
9]. Ocampo-Vega et al. used the centroid and tip coordinates of the pointer of a water meter to form a straight line and calculated the pointer angle between the
x-axis and the line that was created by joining the tip and the centroid of the pointer [
10]. Zhang and Li developed a preprocessing method to automatically read a water meter, where Fourier transform was used to correct a tilted dial image of a water meter [
11]. Recently, to develop a Flight Guardian (FG) system which is not physically connected to an aircraft, Khan et al. [
12,
13] developed a simple method to automatically read the airspeed and revolutions per minute (RPM) of instruments where the video streams are acquired with two fixed-position camera devices. Based on the features of circular shape and needle locations in both dials, the focus of the camera devices is fixed at the center of this shape, and a circular-range sub-image composed of black pixels, except the white pixels occupied by the needle, are adopted for processing. By converting the sub-image pixels into a one-dimensional vector, the needle’s position is identified by finding the maximum convolution output.
In practical applications of automatically reading analog meters, rotated/tilted/shifted situations under variant illumination need to be considered. Based on the research mentioned above, the effectiveness of applying image differencing and binarization methods onto the pointer detection were investigated in the early stages of developing our automatic approach for reading an A-meter, where we found that the useful information lost, as well as the noise yielded, under camera/meter movement or light variation significantly affects the performance of a reading meter. Therefore, in this study, the grayscale images acquired from the A-meter (refer to the
, shown in
Figure 2), are adopted for algorithm development. Moreover, some techniques are involved in our approach to cope with the situations just mentioned. In order to track a target under variant illumination or a moved situation, a technique to update the template could be helpful [
14,
15,
16,
17]. Considering the localization capability under variant illumination and dusky environments, edge-based geometric matching and pyramidal gradient matching techniques are embedded in our approach instead of template updating to localize the wanted regions of interest (ROI) of the A-meter (refer to the main stages of
instrument meter region detection,
angle detection of pointer as well as
angle detection of selector arrow in the flowchart of
Figure 2). As a short summary, as depicted in
Figure 3, a computer vision system for automatically reading an A-meter is presented in this paper, where the feasibility of our approach is demonstrated by reading the AC voltage, DC voltage, and DC current as well as the resistance value. Here, AC and DC represent “alternating current” and “direct current”, respectively. Inspection by the human eye regarded as a ground truth for comparison and discussion.
The rest of this paper is organized as follows.
Section 2 describes the framework and system setup of our study, where the mapping relationship of the function selector and instrument meter are constructed. Moreover, two methods for locating ROIs by template matching as well as the schemes for horizontal alignment of A-meter images and locating the useful ROIs are developed.
Section 3 presents the procedure for detecting the pointer’s angle (or position) including the calculation of the pointer’s spin center and a scanning approach to detect the pointer. An illustration is provided in
Section 4 to demonstrate the functional behavior of the whole system and confirm the feasibility of our approach. The experimental results and discussion are given in
Section 5. Based on a suitable modification of the proposed method, an application to monitor the storage level meter and pressure meter installed on a 15 m
liquid nitrogen (LN2) tank is demonstrated in
Section 6. The conclusions of this paper and further work related to this study are finally drawn in
Section 7.
4. Illustration of Reading the A-Meter
After presenting the methodology of our approach with the framework and system setup in
Section 2 and the angle detection of pointer in
Section 3, a series of results of measuring a 1KΩ resistor by the A-meter YF-370A are shown here to demonstrate the feasibility of the proposed approach.
Figure 12a shows the original image (
), where one 1KΩ resistor is being measured by an A-meter YF-370A. The gray channel of
Figure 12a was used for further processing, where its mean intensity was
. By template matching using Equation (
8) with the template (
) given in
Figure 6,
was detected, where the center of
was (1225.9, 1420.6), as depicted in
Figure 12b. Then, the horizontal alignment image (
) was obtained by rotating the
in
Figure 12a about the center (1225.9, 1420.6) with
, as shown in
Figure 12c.
Next, by performing template matching using Equation (
12) with the templates given in
Figure 7a–c onto the
, the
was located as the top region marked with the red rectangle in
Figure 12d, where the selector arrow was also found using Equation (
13), with the template given in
Figure 8. The
was used later for the detection of the right-end
and left-end
of the scale. In this step, the detected selector arrow reported the rotation angle as
. According to Equation (
1) and the mapping table in
Table 1, this
indicates that the current selection represents the function name “
”;
a nonlinear case; and
is a multiplier.
Within the found
,
Figure 12e shows that the useful right-end
and left-end
of the scale were obtained by means of Equation (
12) with the templates given in
Figure 7d–f and
Figure 7g–i, respectively. In this step, the coordinates of
were found by
,
,
, and
, and those of
were found by
,
,
, and
. Therefore,
according to Equation (
16) and
according to Equation (
17), respectively. In addition, by means of Equation (
18), the spin center was obtained as
.
According to the results obtained above, we the scanline length
(pixels) and
, based on Equation (
21), were used in our scanning process to detect the
. As noted in
Section 3.2, to detect the pointer, the scanning range needed to be wider than
in real applications. Thus, the process was scanned from
to
to produce the accumulation profile of
A and the gradient profile of
, as shown in
Figure 13a,b, respectively. In this example, itwo valleys obviously appeared (
Figure 13a), which indicated the positions of
and
, respectively. From the
GA-profile in
Figure 13b and according to Equation (
20), the detected index was
with the corresponding
, which was just the detected pointer’s position (
) in the current illustration.
Based on
and the selector function “
” with the multiplier
,
Table 2 and Equation (
5) were used to determine the
, i.e., the resistance of the measured 1KΩ resistor. From
Table 2, since
, we had
,
,
, and
. Using the computation of Equation (
5), a resistance of
was obtained which is very close to 1 KΩ. Thus, the feasibility of the proposed approach is confirmed.
6. Application of Monitoring Liquid Nitrogen Meters
By suitably modifying our algorithms, the proposed approach can be applied for reading the similar kinds of A-meters. In this section, we demonstrate a successful application of monitoring a storage level meter and pressure meter installed on a 15 m
liquid nitrogen (LN2) tank which is located at outside our laboratory, as shown in
Figure 23a. These two analog needle-type meters are very significant for our laboratory and need to be checked frequently to guarantee that the storage level and pressure of tank are maintained within a safe range. For example, excessive pressure will damage the solenoid valve of testing equipment and cause other dangerous situations, and once the LN2 storage level is below the low limit, the technician should call for LN2 filling to avoid sudden interruption of testing in laboratory. Reading such an analog meter based on human eyes is rather inefficient and will easily cause fatigue problems during long-term monitoring. Therefore, our computer vision approach is a valuable way to deal with such a scenario.
In this application, an Internet Protocol (IP) camera (model AXIS P3364-LVE 12mm [
24]) was installed to acquire the video stream (with a frame resolution of
pixels and frame rate 30 frames/sec) including both the level and pressure meters, as shown in
Figure 23b. At night, the LN2 tank meters can be illuminated by a road lamp and a LED flood light as shown in
Figure 23c. For each frame, the subimage of the level meter and pressure meter is extracted first according to the predefined ROIs, and then processed individually by our method to read the value indicated by the meter pointer. Note here that only the pointer’s angle detection (presented in
Section 3) was implemented in this application.
Figure 24 illustrates one frame containing the level and pressure meters from a video stream at rainy night. Using our monitoring system, the results shown in
Figure 25 demonstrate the usefulness of our method in this application. Even during worst-case scenarios, such as rainy nights, our system can work well, as shown by the results in
Figure 26.
Since the storage level of LN2 gradually decreases during consumption in the laboratory and increases during filling, the pointers shown on the two meters normally vary very slowly, as the real video (recorded in rainy day) demonstrated in [
25], and the pressure should be maintained within a range between 3 to 5 bar, and the storage of LN2 should be maintained above 80, as indicated on the scale used by our laboratory. In order to show the capability of reading the LN2 meters by our method, we intentionally set the pointer of meter moving fast by manual as the real video demonstrated in [
26]. The two videos given in [
25,
26] demonstrate that our system can read the two dials on the LN2 tank in a real application with an execution time of about 0.2∼0.6 s for each reading. This can be regarded as real-time processing, compared to the very slow change of the meter pointer. As a result, such a non-invasive computer vision system provides us a convenient alternative to monitor the LN2 tank information without installing additional expensive digital sensors and truly helps us to guarantee the safety of using LN2.
7. Conclusions and Future Work
The analog multimeter (A-meter) is a significant instrument for sensing or measuring electronic signals, e.g., voltage, current, resistance, and so on. Even though the digital multimeter is commonly used in our daily life under the considerations of precision and cost, the analog multimeter is still a preferable device for many applications due to its ability to monitor promptly varying values. In order to retain the merits of A-meter and to reduce the tiredness of human eyes observing the A-meter over long durations, it is of great importance to develop a computer vision-based method to automatically read the measured value indicated by the pointer of the A-meter. To effectively read an A-meter, the presented algorithms include horizontal alignment of the A-meter, detection of the instrument meter region, angle detection of the selector arrow, and angle detection of the pointer. The desired ROIs used in our algorithms are extracted by two template matching methods—edge-based geometric matching (
EGM) and pyramidal gradient matching (
PGM)—which have been widely used and embedded in the NI vision software for machine vision applications [
21] and for LabVIEW [
19]. Moreover, the mapping relationship between the function selector and the selector arrow, as well as that between the instrument meter and the pointer are built and formulated to read the A-meter, where the scale on the meter may be linear or nonlinear. The often used scenarios of reading AC voltage, DC voltage, and DC current as well as resistance were adopted for the experiments and evaluations in this study. Our experimental results showed that using our method, the accuracy of detecting the function select was 100%, and the mean accuracy of reading values from the A-meter was 95% or above, except for some cases of reading resistance affected by the so-called
little-change-large-multiplier effect. In addition, the proposed method performed very well as long as the mean intensity, i.e., illumination was
. The “uncertainty problem” in reading an analog meter was explored using translation and/or rotation of the camera to read the A-meter by the proposed computer vision-based approach. The effective range to use our method within was also investigated and validated. Based on suitable modification of the proposed method, an application involving the monitoring of a storage level meter and pressure meter installed on a 15 m
liquid nitrogen (LN2) tank was demonstrated successfully. In conclusion, even if the A-meter is rotated, slightly tilted and/or illumination varied, the proposed computer vision-based approach can effectively read the analog multimeter. A general purpose method for reading the A-meter, its real-time implementation and a time complexity analysis, as well as a more deep investigation on the uncertainty issue are worthy of further study and could be regarded as our future works.