Smart OCR Application for Meter Reading ^†

Abstract

The typical current meter reading method used by power distribution companies is outdated and complex since it relies on a manual process of capturing images of meter values by a human and then manually updating a server with the new readings. This process is not only complex, but also time consuming, and has a huge margin for human error. The rise in the human population has caused an increase in the total number of electricity meters, thus increasing the number of readings to be manually recorded, which has increased the chances of introducing human error into data. This research presents a prototype of an alternative method to reduce the manual process by replacing the existing method with a smart android application. We are suggesting a smart web application with a web-based server designed to collect data from electricity meters and then transferring them to a database/server for calculation and analysis using optical character recognition. After an image of an electricity meter is taken, the calculated bill will be sent to the customer via e-mail. This process is much easier, and it reduces the workloads of both employees and companies.

1. Introduction

Reading electricity meters is an area where different types of research are being conducted as electricity companies want new solutions to make their meter reading process easier and more cost effective, reliable, and accurate, in addition to reducing customer complaints. It is necessary that the calculation of bills should be an efficient and accurate process, since clients want to pay their utility bills without any hassles and moreover without any need of correction. Companies are conducting research to make this process more and more efficient and one notable solution is optical character recognition (OCR).

Optical character recognition (OCR) is a process that translates scanned or printed image documents into a text format [1]. Once the image is scanned, it can be converted into an ASCII or Unicode format so it can be used as text. A typical OCR program performs several techniques to extract text with some pre-processing and post-processing techniques, along with few optimizations to increase the accuracy.

Our main focus in this research is to deploy an OCR program combined with an application which can extract readings from electricity meters and convert them to machine-readable code.

2. Literature Review

In the modern era, there is a push towards switching to automatic and smart systems for almost anything and everything, as these systems are efficient, reliable, and economical. Most companies want to change their their methodology from traditional outdated methods to smart ones as the world progresses. Everyone wants to accomplish their task as quickly and efficiently as possible. Each company is moving towards automation to achieve their goals easily with minimum costs.

Utility companies are inclined to bring the same changes to their meter reading methods. In recent years, metering devices have gone through much improvement and are expected to become even more sophisticated, offering more and more services [2]. A much better solution to GSM-based energy meters has already been introduced to market, which can transmit meter readings to a server automatically using telecommunications, but it is not economical. Traditional meters are still dominant in most markets because GSM-based meters have significant running cost, which means a loss in revenue, and the utility companies cannot afford to bear such losses. So, despite the resourcefulness of GSM-based meters, companies have not found it feasible to switch to them. Utility companies require a solution which is not only economical, but also reliable.

3. Problem Formulation

3.1. Prototype Details

The whole reading process consists of an android application and a server. After an image is captured, the OCR program will use its techniques to convert the image into digital numbers, which is interpreted by our application and the data are sent to the server, which is where calculations will be performed as shown in Figure 1.

3.2. Preprocessing Techniques

De-skewing: If an image is tilted at some angle, it is said to be skewed. De-skewing is the process of detecting a tilted image and fixing the angle to make the image correctly aligned. This helps the OCR program to detect the text easily.

De-speckling: Sometimes, documents contain unwanted noise in an image in the form of spots, or the overall image is blurred. This can cause the OCR program to malfunction. We can categorize different types of noise by identifying their features and can search for similar patterns in an image to choose appropriate methods for their removal [3]. Generally, linear filters are used for noise reduction in this context.

Binarization: Binarization is the process of converting a colored image into a black and white image. This process is done to aid the OCR program. A threshold value from 0 to 255 is given (where 0 is white and 255 is black) and all the values below the threshold value will be converted to white, while the values greater than the threshold will be convert to black. This is known as Otsu method of binarization.

Morphological Operation: To remove imperfections left after the process of binarization morphological operations are used. In morphological operation, we test the location of each pixel from input image; if test is successful then new binary image is created with non-zero values of pixel.

3.3. Post Processing Techniques

Single Shot & Region Based Detectors: There are different single-shot and region-based detector techniques. In this case, the system classifies a region and tests if the selected region has text or not. The algorithm will check the whole region and highlight the axis where text is present.

Efficient Accurate Scene Text Detector (EAST): In artificial intelligence, neural network is a series of algorithms is created using different sets of data through a process which replicates a human brain. The EAST detector uses neural networks to predict text or numbers. The overall performance of the neural network is determined by the interplay of multiple stages and pipeline components. The pipeline directly predicts words or text lines of arbitrary orientations and quadrilateral shapes in full images, eliminating unnecessary intermediate steps (e.g., candidate aggregation and word partitioning) with a single neural network [4].

Convolutional Recurrent Neural Network (CRNN): A convolutional neural network (CNN) is deep neural network that contains different convolutional and pooling functions to introduce non-linearity. Recurrent neural networks (RNN) are a kind of artificial neural network that can take a series of inputs and perform some deep learning techniques while retaining their original state to process the next sequence of inputs. A CRNN is a combination of both a CNN and RNN. This neural network architecture integrates feature extraction, sequence modeling, and transcription into a unified framework. This neural network extracts text directly from an input image, i.e., it does not need to perform the process of binarization, although it can be added for more accuracy.

Tesseract: Tesseract is an open-source OCR engine that was developed at HP between 1984 and 1994 [5]. After the release of Tesseract 4, deep learning capabilities were added for better recognition. We can combine the capabilities of Tesseract with all the above-mentioned techniques to enhance the system’s capabilities.

3.4. Training of Data

For the deep learning model, we will import InceptionResNetV2 from the tensorflow.keras.applications (developed by Keras) module, which is a convolutional neural network that is trained by the ImageNet Database, consisting of more than a million images. The network of this model is 164 layers deep and it can recognize a wide range of images.

Our main focus is to create our own custom trained model using the training and testing method of InceptionResNetV2. Our custom model has a region box that is bound to the readings of the wattage meter and we need to create a model that can easily recognize a region of a wattage meter from any given image. The reason for selecting InceptionResNetV2 is that both the Inception-v4 and ResNetV2 components result in state-of the art single-frame performance with the ImageNet validation dataset [6].

In InceptionResNetV2, multiple sizes of convolutional filters are combined with residual connections, which helps to avoid degradation problem caused by deep structures and it also boosts the training speed significantly.

3.5. Development of Application

After completing all the processing techniques, code will be written in the Python programming language and a web application will be developed using the Flask framework. All the techniques mentioned previously will act as the backend and the application will be designed using front-end technologies.

4. Results

The development and implementation of this application will provide a solution to utility companies for a problem that has been burdening them for many years. This application features almost no development cost and requires less manpower than the current process. On the other hand, it reduces human error and the time of operation as well. The development of this application demonstrates the concept and implementation of a new process that has a low infrastructure cost, low operating costs, greater data security, and lower manpower requirements. It not only solves the problem of conventional meter reading, but also provides an additional feature, namely, bill generation on mobile devices. The inputted image is preprocessed with the help of OCR, and the result is updated in a database. With the help of OCR techniques, a bill may generate and emailed to customers instantly.

Figure 2 show examples of extracted numbers from captured images where process is completely automated. Such an application also enhances the customer’s experience, as they do not have to wait for their bills, but instead they will be emailed to them within seconds of meter reading. Furthermore, since the application is doing all the work, it eliminates all the complaints of erroneous meter reading.

5. Conclusions

Despite ubiquitous claims that optical character recognition (OCR) is “a solved problem”, many categories of documents continue to break modern OCR software, such as documents with moderate degradation or unusual fonts [7]. There is no single perfect algorithm for all OCR problems, so modern systems try to adapt themselves to the actual features of the image or document to be recognized [8]. OCR can fail in certain circumstances. Even with the best machine learning techniques and best engines, OCR can still fail, and this is because the human eye and cameras work differently. The human factor has been widely considered for the evaluation of digital displays. However, it is rarely used as a reference point in the design and evaluation of digital cameras [9]. Still, digital camera/OCR methods have less accuracy as compared to the human eye. This problem can be overcome by improving OCR. As a solution proposed by Gor [10], this has been improved by the introduction of InceptionResNetV2, a convolutional neural network that has been trained on more than one million images and by integrating that neural network in our system. The epoch loss with different datasets can be seen in Figure 3, with a trend over time close to zero, which denotes fewer differences between the training data and test data.