Problems occur when banknotes used in such devices as automated teller machines (ATMs) and vending machines cannot be recognized due to higher-than-normal levels of soilage. This can cause these devices to malfunction, force users to re-insert the banknotes, or create other inconveniences during the insertion of the banknotes. Banknote fitness classification calculates the quality of a banknote to determine whether it is usable. Incorrect fitness classification can lead to unusable banknotes being circulated again or usable banknotes being discarded, which is also problematic because of the cost of discarding bad banknotes and creating new ones. Previous researches have thus been conducted on fitness classification because of growing demand for technology that can accurately perform such classification.
In research by the De Nederlandsche Bank on efficient banknote usage, using banknote data acquired via color imaging [1
], soiling was found to be one of the main reasons for the decline in the fitness of banknotes with use over time [1
]. Other research has also involved the use of color images of banknotes to extract soil characteristics [3
]. Geusebroek et al.
] and Balke et al.
] proposed a machine learning technique for fitness classification on Euro banknotes using the mean and the standard deviation of brightness values extracted from square-shaped areas in banknote images. Intensity, red-green-blue (RGB) color, and a mix of the yellow-blue and red-green channel colors were used as fitness classification features. The classifiers used in [1
] were a mixture of simple linear weak classifiers using an adaptive boosting (AdaBoost) algorithm. Instead of proposing an automatic fitness classification method, Balke et al.
described the classes of, and various contributing factors in, fit banknotes and unfit banknotes [3
]. The research in [4
] did not propose an automatic fitness classification method, but instead presented test values to classify the levels of soiling of Euro banknotes into five categories using various sensors [4
]. Aoba et al.
proposed a technique for recognizing Euro banknote denomination and fitness classification using visible light reflection images and near-infrared light reflection images as input data [6
]. This research involved a classification part, which used a three-layered perceptron, and a verification part, which used a radial-based function (RBF) network to reject partial unfit data. He et al.
performed research that classified the fitness of Chinese (Renminbi (RMB)) banknotes using a neural network [7
]. In this technique, the gray level histogram of the image of a banknote was used as an input feature, and the classifier was designed based on a neural network that used a sine basis function [7
Pham et al.
captured Indian banknote images with a visible light camera, performed a discrete wavelet transform (DWT) on pre-selected regions of interest (ROI), and used a support vector machine (SVM) to classify the banknotes’ fitness using the mean and the standard deviation of image brightness selected from the transformed banknotes based on a correlation with densitometer data [8
]. Rather than finding fitness levels through banknote images, Kang et al.
] and Teranishi et al.
] performed fitness classification by using acoustic energy, which is emitted when a banknote is inserted into a counting machine and passes over the roller.
Most of the methods proposed in relevant research require the training procedures for fitness classification. A fairly large amount of data and a lot of time are required for such training. Moreover, if there is a change in the national currency being used for classification, difficulties arise because the training procedure would need to be performed again, and new data relating to the currency would need to be gathered.
To solve problems in prevalent technologies for banknote fitness classification, this study proposes a method that uses a fuzzy system to achieve accurate fitness classification results from visible light and infrared images using rules determined. There exist various factors such as crumpling and tear, etc, which affect the banknote fitness, and our research is focused on the factor of soiling level on banknote surface for the classification of banknote fitness.
In comparison with previous researches, the results presented herein are novel in the following four ways:
First, this is the first study to carry out fitness classification of all denominations of the United States Dollar (USD) and the Korean Won (KRW).
Second, the regions of interest (ROIs) for extracting features from the visible light reflection (VR) images and near-infrared light transmission (NIRT) images were set differently according to banknote denomination and direction; from these ROI, the average values of the VR and NIRT images were extracted as features.
Third, a fuzzy system was applied to the features acquired from input images to determine the final fitness values for the input images and determine whether the relevant banknotes were usable based on two classes (Fit, Unfit).
Fourth, by using a fuzzy system, high performance of fitness classification can be achieved regardless of the currency of the banknote used, such as USD, KRW, and INR.
The remainder of this paper is structured as follows: Section 2
contains a description of the proposed algorithm for banknote fitness classification. Section 3
is devoted to a presentation of the results of experimental performance analyses of the algorithm, and Section 4
summarizes the conclusions of this study as well as plans for future research in the area.
3. Experimental Results
For the first experiment, we used the US dollar database to evaluate the performance of the proposed fitness classification method. This database contained images of denomination of $1, $2, $5, $10, $20, $50, and $100 banknotes, each of which was captured from four directions as shown in Figure 2
. The number of banknote images used in this research is shown in Table 3
Based on previous research that presented actual densitometer measurement values [18
] and those based on soiling levels, human experts classified the banknotes in the database as fit banknotes (capable of being used) or unfit banknotes (incapable of use) as shown in Table 4
In this study, Min and Max rules as well as six defuzzification methods (FOM, LOM, MOM, MeOM, COG, and RWA) were used to test performance. We defined two types of errors: Type I and Type II errors. A Type I error occurred when a fit banknote was incorrectly classified as unfit, and a Type II error occurred when an unfit banknote was incorrectly classified as fit. As explained at the end of Section 2.3
, depending on the threshold of the output of the fuzzy system, Type I and Type II errors were switched. Based on the threshold, which existed when the difference between Type I and Type II errors was minimized, the equal error rate (EER) was found, which is the mean error of Type I and Type II errors. The experimental results showed that when the Max rule and the RAW method were used for fitness classification, the EER was at its lowest at 0.04%. Figure 8
shows examples of receiver operational characteristic (ROC) curves for errors according to each defuzzification method when the Max rule was used in direction A data for each denomination. As shown in Figure 8
, when the RWA and the Max rule were used, the error rate was lowest.
The RWA method recorded a higher degree of accuracy than the COG method in fitness classification. This is because in case of COG, the center of gravity was calculated for the area under the maximum value of the L and H output membership functions among the IVs. However, for the output membership function used in this research, the shape exhibited left-right symmetry as shown in Figure 5
b; thus, the area’s center of gravity tended to cluster around 0.5.
However, in case of RWA, all IVs were used, and each IV was multiplied by a corresponding weight before the mean was found. Thus, the values were more effectively distributed than in case of COG. For example, in the case of fit banknote data where the input values (VR mean, NIRT mean) were (0.65746, 0.53974), and in the case of unfit banknote data where the values were (0.00666, 0.06438), if the IVs were selected using the Max rule, and COG or the RWA method was used to find values, the COG produced an output of 0.5 for the first input and 0.495795 for the second input, as shown in Figure 9
. On the contrary, RWA produced an output of 0.550639 for the first input and 0.409881 for the second input.
As we can see, in case of COG, the output values for fit and unfit banknote data were 0.5 and 0.495795, respectively, which are similar, whereas in the case of RWA, the outputs for fit and unfit banknote data were 0.550639 and 0.409881, which has a greater difference; hence, the latter was accurate in classifying banknote data.
shows a comparison of the fitness classification results of the proposed method and a method proposed in previous research [9
]. To perform banknote fitness classification, the method proposed in [9
] used acoustic energy patterns that occur when banknotes pass over rollers. In the relevant paper, 40 one-dollar banknotes were used to perform an experiment to test performance; thus, in this study, the degree of accuracy was compared by using one-dollar banknotes as well. As can be seen in Table 5
, the performance of our proposed method was superior to that of the previous method.
A comparison experiment was also performed with another previously proposed method that used a neural network [7
]. Since the neural network requires training, a two-fold cross-validation method was employed, where the database used in this research was divided into two halves (T1 and T2). The EER was measured using T1 as the training set and T2 as the test set, and the EER was measured using T2 as the training set and T1 as the test set. The average EER was calculated to reflect the final performance. As shown in Table 6
, the method proposed in this research (0.04%) yielded better performance than the other fitness classification method that used a neural network (3.32%). As shown in Table 6
, the proposed method, unlike previous research [7
], employs a fuzzy system that uses a suitable fuzzy rule in order to perform accurate banknote fitness classification.
In the next experiment, in order to check the performance of the proposed method on different types of banknotes, we used it on a database of images of banknotes of two other countries. The currencies used in the experiment were KRW and INR, and the number of banknote images and number of classes used in the experiment are shown in Table 7
. Figure 10
shows examples of banknote images used for each national currency.
In the results of the experiments, a fitness classification mean error rate of 0.92% was achieved for KRW. In previous research [8
], a discrete wavelet transform and support vector machine method had been used to perform fitness classification on INR. Thus, this method’s INR fitness classification accuracy was compared with that of the proposed method, as shown in Table 8
. As the table shows, the proposed method yielded an EER of 0.06%, which was better than the results of the relevant research [8
The previously mentioned performance by proposed method for the USD database (EER 0.04%), the KRW (EER 0.92%), and the INR (EER 0.06%) shows that the method has high fitness classification accuracy regardless of the type of database.
In Figure 11
, we show the examples of correctly recognized cases by our method on USD, KRW, and INR, respectively. As shown in Figure 11
, we can confirm that our method can correctly discriminate the fit and unfit banknotes irrespective of various qualities of banknote images.
In Figure 12
, we show the examples of incorrectly recognized cases (Type I error) by our method on USD, KRW, and INR, respectively. As shown in Figure 12
, although they were manually classified as fit data based on densitometer measurement values [18
], the average pixel values of ROI of VR and NIRT images are similar to those of unfit images, which causes the Type I error.
In Figure 13
, we show the examples of incorrectly recognized cases (Type II error) by our method on USD, KRW, and INR, respectively. As shown in Figure 13
, although they were manually classified as unfit data based on densitometer measurement values [18
], the average pixel values of ROI of VR and NIRT images are similar to those of fit images, which causes the Type II error.
The processing time of proposed method was also measured. Measurement of processing time was performed on the desktop computer composed of an Intel(R) Core(TM) i7 Extreme 975 CPU of 3.33 GHz [19
] and 8 GB RAM. In addition, the processing time was measured on the embedded system using a Texas Instruments (TI) digital media system-on-chip of 729 MHz (model name is TMS320DM6467 [20
]) in an actual banknote counting machine. The processing time on the desktop computer was measured at an average of 0.1 ms per each image, and that the banknote counting machine was measured at an average of 1 ms per each image. All the time taken by all the procedures of our method (including the segmentation of banknote area and ROI detection, etc
.) are included in the measured processing time. Because of this, we knew that the proposed method could perform fitness classification in a banknote counting machine at a rate of 1000 banknotes per second.
Finally, in order to use the proposed algorithm in banknote counting machines with limited resources, the total memory usage by our program was measured in the banknote counting machine. The measurement results, which are shown in Table 9
, confirmed that the total memory usage was 931,231 bytes, and our program could run on the actual banknote counting machine without memory problems.
In addition, we make all our datasets of USD, KRW, and INR open through the website [21
] so that other researchers can easily get access to our datasets. It is easy to obtain large banknote datasets for classifying the kinds of banknote such as $1, $10, and $100, etc.
However, it is much difficult to obtain large banknote datasets for classifying the fitness and unfitness of banknote. That is because based on the actual densitometer measurement values [18
] about soiling levels, human experts classified the banknotes in the database as fit banknotes (capable of being used) or unfit banknotes (incapable of use) for ground-truth data. Therefore, we performed the experiments with three datasets of USD (3856 images from 28 classes), KRW (3956 images from 16 classes), and INR (2300 images from 24 classes) as shown in Table 3
, Table 4
and Table 7
. Our experimental datasets are much larger than those of previous researches [7
In our research, fuzzy-based classifier (based on heuristic design of fuzzy rules and membership functions) is used. Therefore, the possibility of overfitting with our experimental datasets can be less in our method than that by the methods requiring much amount of training procedure such as neural network and SVM, etc.
As shown in the comparative results of Table 5
, Table 6
and Table 8
, our method outperforms previous researches based on neural network [7
], SVM [8
], and dynamic time warping (DTW) [9
], which shows that the possibility of overfitting by our method is less than that by previous researches.