A New Enhancement Edge Detection of Finger-Vein Identification for Carputer System

Abstract

Developments in multimedia and mobile communication technologies and in mobilized, personalized information security has benefitted various sectors of society, as traditional identification technologies are often complicated. In response to the sharing economy and the intellectualization of automotive electronics, major automobile companies are using biometric recognition to enhance the safety, uniqueness, and convenience of their vehicles. This study uses a deep learning-based finger-vein identification system for carputer systems. The proposed enhancement edge detection adapts to the detected fingers’ rotational and translational movements and to interference from external light and other environmental factors. This study also determines the effect of preprocessing methods on the system’s effectiveness. The experimental results demonstrate that the proposed system allows more accurate identification of 99.1% and 98.1% in various environments, using the FV-USM and SDUMLA-HMT public datasets. As results, the contribution of system is high accuracy and stability for more sanitary, contactless applications makes it eminently suited for use during the COVID-19 pandemic.

1. Introduction

Artificial intelligence (AI), the Internet of things (IoT), bigdata, mobile networks (MN) and cloud computing (CC) have been advancing rapidly. The growth of autonomous vehicles (AVs) and the Internet of Vehicles (IoV) means that these technologies are increasingly integrated into people’s lives. However, consumers who do not often drive find automobiles to be too costly. Costs include a high purchasing price, depreciation, insurance, maintenance, taxes and various other fees during and after the purchase of an automobile. A sharing economy would address this issue. The sharing economy business model promotes the redistribution of unused automobiles by carsharing, changing the traditional business model by providing reciprocal benefits for both the sharer and the user. This economic structure allows the sharing of automobiles as an alternative to the greater economic burden of individual purchasing.

Most carsharing platforms identify users using independently developed applications (App), which involves a registration process and the submission of an identity card, a driver’s license, or other documents to verify identity and driver qualifications. These documents must be verified before users rent a vehicle. Vehicles can be unlocked using virtual keys but logging into the app requires a traditional account and password verification. This identification process for carsharing platforms has numerous flaws: if an account were to be compromised, it is impossible to determine whether the driver is the registered user. Automobiles can be unlocked using a key or a keyless entry system. Physical keys can be replicated using pictures or molding techniques, but keyless entry systems are more secure. However, keyless entry systems unlock automobiles using near-field communication (NFC) [1] or Bluetooth [2] technology but a Bluetooth connection is unstable and media access addresses (MAC) are also susceptible to replication, so security breaches are possible. NFC technology is more stable and safer than a Bluetooth connection, but it is also easily replicable.

To address the security flaws of traditional identification documents, passwords, key and keyless entry systems, high-end automobiles use biometric authentication to increase security. As the demand for privacy, security and convenience increases, biometrics have become increasingly popular as a fast and simple identification process that offers security, reliability, and accuracy. Its convenience also allows for safe monitoring and system management and integration [3,4]. Biometric characteristics are universal, permanent, distinctive and collectable [5,6,7]. Biometrics are more secure than traditional identification methods because keys or ID cards are not required. The physiological characteristics that biometrics detect comprise external characteristics and internal characteristics. External characteristics include traits such as a fingerprint, a face, and a palm-print and internal characteristics include traits such as finger-veins and palm-vein.

Each of these characteristics are used for biometric analysis but internal characteristics allow more accurate and secure identification. Internal characteristics that involve vessels such as finger-veins beneath the skin are not prone to wear or interference from external staining, water, light or other environmental variables that undermine identification [3,4]. External characteristics are more easily counterfeited and affected by an external environment, so they are less suited to biometrics. Internal vein textures do not change significantly over time. Even the veins of twins have different texture to allow for identification [5]. Vein imaging uses myoglobin’s absorbance of infrared light to capture images using infrared (NIR) and visible light (RGB) detectors. Blood does not flow in deceased humans so finger-vein textures can only be acquired from living bodies [5,6].

Finger-vein identification systems use 1 to 1 verification and 1 to N recognition systems. A 1 to 1 verification system begins the identification process by requiring the user to verify identity using a card or password. The system then captures images of the user’s finger-vein, calculates the image’s similarity to samples in the registered dataset and confirms the identity and allows entry if the similarity index exceeds a specific threshold value. A 1 to N recognition system compares captured vein textures with samples in the registered dataset. When the system identifies data, it verifies the user’s identity. A 1 to N recognition system is more suited to the identification of numerous personnel. This study uses a 1 to N recognition system for the renting of shared automobiles to improve customer convenience.

2. Review of Literature

Finger-vein biometrics use statistical computations such as local binary histograms (LBH) [8], local derivative patterns (LDP) [9] and local line binary patterns (LLBP) [10] to analyze images and compare similarities between vein textures. However, these computational methods are easily affected by interference from external light sources or the rotational and translational displacement of the user’s fingers, which decreases identification accuracy.

A previous study [5] and the study by Miurea et al. [11] propose algorithms that use vein textures. These algorithms identify the darker shades of veins on the grayscale of digital images and track them to identify the texture. In [6], mathematically converted identified features to allow for correcting the errors due to rotational and translational finger displacements, but interference from external light sources cannot be countered. External light sources affect an image’s brightness and prevent the capture of vein features, so the identification rate decreases. Liu et al. [12] used bicubic interpolation to decrease the spatial resolution and create a more lightweight system but this approach inhibits the acquisition of vein texture data, so identification accuracy is decreased. Previous studies proposed finger-vein recognition using tri-branch feature points [5,13], but unstable light sources decrease the number of identifiable tri-branch feature points, so there are errors. However, these algorithms require significant computational power, so response times are slow. As CPU and GPU computing power increases and costs decrease, identification algorithms use deep learning (DL) to improve accuracy by processing and learning from large data sets of images, so identification using biometrics is more accurate. Other studies use a convolutional neural network (CNN) to extract and categorize texture features for finger-vein biometrics. CNNs are highly effective but a previous study [3,4] proposed an identification process that is slower because the system must process more samples for DL.

These recognition technologies all possess flaws. Traditional image algorithms have lower error rates for specific equipment and illumination conditions but when these settings are altered, the system’s recognition effectiveness varies. Image algorithms only use a small portion of all data during experimentation and if the dataset is expanded, the error rate changes. Traditional image algorithms retain the users’ vein features they are insufficiently secure to ensure total information safety for financial businesses and government agencies. DL-based algorithms also require large data sets, so they are susceptible to overfitting of finger-vein biometrics.

Contemporary shared automobiles verify user identities by verifying traditional ID documents. The registration processes require manual checking of driver licenses and ID documents. After registration, users rent cars by logging in with their account and password. Studies show that traditional identification features numerous risks and informational security weaknesses. Access to a registered carsharing service is possible using an account and password so anyone, including those underaged or those without a driver’s license, can use shared automobiles if they have the password. The traditional method is also vulnerable to invalid registrations using fabricated documents. Personal data breaches are possible, so the system does not protect information. To increase information security for carsharing systems, this study uses finger-vein biometric algorithms in shared automobiles’ carputers and an enhanced edge detection method to correct for rotational and translational movements of fingers and interference from external light sources and other factors in the environment. The system allows stable, recognition if there is interference from external light sources and complex environments. In terms of the COVID-19 pandemic, the proposed system verifies users’ identities by contactless interactions, which is a growing trend in the development of human-computer interaction (HCI) systems.

3. Proposed Finger-Vein Identification System

Figure 1 shows the conceptual flowchart for the proposed method. The identification process includes the extraction of regions of interest (ROI) and CNN training. This study uses a ROI algorithm to locate the area to be identified. Image brightness varies depending on the device and the environment, so the captured image of the ROI is adjusted using parametric-oriented histogram equalization (POHE) and contrast limited adaptive histogram equalization (CLAHE) to enhance and stabilize the vein characters. The system uses DL to match the users’ (drivers’) vein images against images for the registered user in the dataset and outputs the identification results.

3.1. Enhancement Edge Detection Method

When the user places a finger in the biometric scanner, any rotation or translation of the finger affects the identification process. Using a ROI preserves the relevant information for identification. The market uses contact-based and contactless biometrics but the hardware design for contact-based devices restricts the displacement of fingers, so interference from rotation and translation has little effect on the identification process.

Contactless options have become popular during the COVID-19 pandemic, so these devices capture more noise signals, which complicates image processing. Depending on the time of image capture, contactless biometrics are susceptible to rotation and translation of fingers, so a series of algorithms [5,6] are required to ensure that images that extracted using biometrics are sufficiently clear for analysis. The components of the algorithm for the proposed enhancement edge detection method are detailed in the next section.

Bilateral Filter

In Figure 2a, many denoising methods compromise image edge details so high frequency information is lost. A bilateral filter is a nonlinear filter that measures the spatial domain and gray levels to preserve edges, as demonstrated in Equation (1). A bilateral filter has two components, one of which relates to the pixel distance, for which the weight is product of the spatial nearest factor and similarity factor. The spatial nearness factor is a Gaussian filter coefficient and increased pixel distance decreases the weighting. An increase in the δ_r value increases image smoothing, as described by Equation (2). Equation (3) describes the relationship between the luminescence similarity factor and the spatial pixel difference. The greater the spatial pixel difference, the lower is the weighting for the luminesce similarity factor. An increase in the δ_d value increases the smoothing effect for pixels with the same gray level so less edge data is preserved. A bilateral filter is used with a Sobel operator to improve edge detection, as shown Figure 2b.

$$f_{BF}\left(x,y\right)=\frac{{\sum }_i{\sum }_{j}f\left(i,j\right)w\left(w,y,i,j\right)}{{\sum }_i{\sum }_{j}w\left(x,y,i,j\right)}$$

(1)

$$r\left(x,y,i,j\right)=exp\left(-\frac{{\left|f\left(x,y\right)-f\left(i,j\right)\right|}^2}{2{\delta }_r^2}\right)$$

(2)

$$d\left(x,y,i,j\right)=exp\left(-\frac{{\left(x-i\right)}^2+{\left(y-j\right)}^2}{2{\delta }_d^2}\right)$$

(3)

$$w\left(x,y,i,j\right)=exp\left(-\frac{{\left|f\left(x,y\right)-f\left(i,j\right)\right|}^2}{2{\delta }_r^2}\right)-\frac{{\left|f\left(x,y\right)-f\left(i,j\right)\right|}^2}{2{\delta }_d^2}$$

(4)

3.2. Multi-Layer Edge Detection

The Sobel operator is a common edge detection system that is used in image processing. It uses the algorithm for surrounding pixels’ gray level weighting and performs edge detection using the most extreme edge pixel values [14]. The operator uses two 3 × 3 masks, a horizontal operator S_x and a vertical operator S_y to allow convolutional computation by Equation (5).

$$G_x=\left[\begin{array}{ccc}1& 0& -1\\ 2& 0& -2\\ 1& 0& -1\end{array}\right], G_y=\left[\begin{array}{ccc}1& 2& 1\\ 0& 0& 0\\ -1& -2& -1\end{array}\right]$$

(5)

The calculated values for G_x and G_y are combined using Equation (6) to complete the Sobel edge detection process.

$$G=\sqrt{G_x^2+G_y^2}$$

(6)

Figure 3a shows the bilateral image after processing using a Sobel operator. The pixel and the weighting result between the Sobel operator and the bilateral filter’s processed images are shown in Figure 3b. This method contrasts and enhances the finger outline and increases the effect of the Canny edge detector. It is difficult to achieve denoising and accurate edge detection so smoothly edged images are less accurate. However, increasing the sensitivity of edge detection increases the perceived noise signals. To address this issue, a Canny edge detector [15] is used to balance denoising and detection accuracy. This algorithm gives precise detection, a low error rate and produces detailed edge outlines, so the Sobel operator’s results are processed using Canny edge detection. The multi-layer edge detection method gives stable finger outline information, as shown in Figure 3c. An image that is processed using Canny edge detection still contains small cracks and unevenness so a closing operator from morphology studies is used to complete the finger edge outline while maintaining the finger position and shape, as shown in Figure 4.

3.3. Region of Interest Detection

A user’s finger is displaced when it is inserted into the biometric device so this study proposes a region of interest (ROI) detection method to acquire stable and applicable vein character information. Canny edge computation firstly identifies X₁ and Y₁ on the left half and X₂ and Y₂ on the right half of the subject finger, as shown in Figure 5a. The center of the left and right halves of the finger is located through these points. The center points are then used to calculate the finger’s center baseline, as shown in Figure 5b. The calculated baseline is compared to the finger’s inherent center line, as shown in Figure 5c. Adjustments for skewed and shifted angles use calculated rotations. This normalization process decreases the chance of incorrect identification. The green bounding box (ROI) is derived from the normalized image, as shown in Figure 5d. Then, the normalized image, as shown in Figure 5e.

3.4. Image Enhancement and AlexNet

Current finger-vein biometrics are easily affected by external light sources, so image contrast levels differ. CLAHE [16] denoises and enhances image contrast, but the noise filtering process removes some vein character information, which decreases image quality. Therefore, this study uses POHE [17] to contrast and enhance the image. POHE uses Gaussian parameters to construct the functional model for a Gaussian kernel and regional histogram equalization (HE) to enhance the image. An image that is processed using POHE features more significant vein enhancement than the original, as shown in Figure 6. This method is used to extract a CNN’s vein character.

This study uses the AlexNet CNN because it categorizes very effectively. Unlike earlier neural networks (NN), AlexNet uses the activation function ReLu to accelerate convergence and dropout, in order to prevent overfitting. This NN has 60 million parameters and 65 million neurons and the AlexNet’s network architecture contains 12 layers: 1 input layer, 5 convolutional layers, 3 pool layers and 3 fully connected layers, as shown in Figure 7.

4. Experimental Results and Analysis

4.1. Experimental Environment and Datasets

The hardware environment for this study uses an Intel Core i7-8700K 3.7GHz as the central processing unit (CPU) and a Nvidia Geforce GTX 1080 Ti with DDR4 32GB computing memory as the graphics processor unit (GPU). The software is developed using Python and C++ and TensorFlow 1.15.0, Keras 2.3.1 and OpenCV 2.4.9 program libraries. To measure the effectiveness of the proposed edge enhancement detection method, this study uses public two finger-vein libraries: contact-based FV-USM [18] and contactless SUMLA-HMT [19].

The FV-USM dataset is created by Universiti Sains Malaysia. It contains vein images of 123 subjects’ left and right index and middle fingers. The subjects include 83 males and 40 females, with ages between 20 and 52 years. The vein imaging process requires subjects’ fingertips to contact the device base, as shown in Figure 8a. Each subject provides 6 images of each finger on each hand. The testing environment has unstable light sources, but the fingers are stably positioned so the resolution is 640 × 480, as shown in Figure 8b.

Anyway, the SDU-MLA dataset is created by Shandong University. It contains 636 vein images from 106 subjects. Six images each of every subject’s left and right index, middle, and ring fingers are contained in the dataset. The light sources in the testing environment are unstable and finger positioning is unstable, so the resolution is 320 × 240, as shown in Figure 9.

4.2. Setting and Evaluation

Unsuitable parameters hinder the model’s performance on systems that use DL so minor details affect the final result. This study separates the FV-USM and SDUMLA-HMT datasets into training and testing data sets. The training dataset is used to train and test the model and the testing dataset is used for verification. The AlexNet optimization component uses an Adam optimizer with a learning rate of 0.001, a batch size of 64 and 600 epochs.

In recent years, the security of biometric systems has been the subject of much interest. The definition of analysis of variances (ANOVA) test and T-test refers to the degree of difference between samples [20,21]. This is not suitable for finger-vein identification that each biometric is corresponding to the person individual. However, a critical index for the safety of 1:N identification systems is the error equal rate (EER). Identification systems incur two types of errors: the false rejection rate (FRR), whereby the system identifies a registered user as unregistered, as shown in Equation (7), and the false acceptance rate (FAR), whereby the system mistakes an unregistered user as registered, as shown in Equation (8). The security of the device is adjusted from the most to least secure by fine-tuning the system threshold. Changing the threshold increases the FAR value, until the probability approaches 1. The FRR value decreases to 0 and increases with the FAR. The EER is the intersection of the FRR and FAR curves. The threshold value is adjusted for different applications and scenarios but setting it at the EER value provides the most balanced performance. The EER is the index that denotes the system’s level of security.

$$FRR=\frac{FN}{TP+FN}$$

(7)

$$FAR=\frac{FP}{FP+TN}$$

(8)

This study proposes DL training for testing with edge detection and various image enhancement methods. Three different experiments give a range of analysis for future applications. Three finger-vein images were randomly selected from the dataset for training, and three were selected for testing. This is a 3T/3T setup. 4T/2T and 5T/1T setups were also used with different combinations of testing and training images. The HE using a wavelet transform (WT) [22] was used for these testing sets, as shown in

Table 1. Stability of identification for three vein image analysis scenarios using the FV-USM dataset.

3T/3T	EER	Accuracy Rate	Memory Size
WT + DL	0.007%	96.7%	5349 Mib
This work	0.003%	97.9%	5514 Mib
4T/2T	EER	Accuracy rate	Memory size
WT + DL	0.005%	98.4%	5814 Mib
This work	0.002%	99.0%	4467 Mib
5T/1T	EER	Accuracy rate	Memory size
WT + DL	0.004%	98.0%	5908 Mib
This work	0.002%	99.1%	5560 Mib

.

For the contact-based FV-USM dataset, 3T/3T vein image analysis using HE and DL gives the lowest EER value of 0.003 and highest accuracy of 97.9% because HE gives a more stable luminance contrast than WT-based image processing. For 4T/2T analysis, the lowest EER value is 0.002 and the highest accuracy is 99.0%. For 5T/1T analysis, the lowest EER value is 0.002 and the highest accuracy is 99.1%. Experiments show that the device is stable for 4T/2T and 5T/1T image analysis trials, as shown in Table 1.

To measure the effectiveness of the proposed enhancement edge detection method in processing images for contactless biometrics, this study uses the contactless-based SDUMLA-HMT public dataset for testing. For 3T/3T testing using HE and DL training, the EER is 0.010 and the accuracy is 96.0% because WT increases the brightness of excessively dark regions through the low-low band. However, 5T/1T is the most common training and testing setup for practical applications. The 5T/1T setup also gives a stable accuracy rate for this study, so performance with a weighting trade-off is acceptable. For 4T/2T analysis, the lowest EER value is 0.005 and the highest accuracy is 98.1%. Experiments show that the device is stable in 4T/2T and 5T/1T image analysis trials, as shown in

Table 2. Identification stability for three vein image analysis setups using the SDUMLA-HMT dataset.

3T/3T	EER	Accuracy Rate	Memory Size
WT + DL	0.008%	95.9 %	10,404 Mib
This work	0.010%	96.0%	9820 Mib
4T/2T	EER	Accuracy rate	Memory size
WT + DL	0.005%	97.6 %	6399 Mib
This work	0.005%	98.7%	6193 Mib
5T/1T	EER	Accuracy rate	Memory size
WT + DL	0.006 %	95.0 %	7015 Mib
This work	0.003%	98.1%	6989 Mib

. The EER values for the FV-USM and SDUMLA-HMT datasets are compared with those for other recent studies. The SDUMLA-HMT is newer and contains more finger displacement, but

Table 3. Comparison with the results for other recent studies.

Method	Year	Dataset	ERR
Hsia et al. [5]	2018	FV-USM	0.250%
Hsia et al. [6]	2018	FV-USM	0.130%
Kono [23]	2002	FV-USM	0.100%
Yu et al. [24]	2009	FV-USM	0.760%
Chen et al. [25]	2009	FV-USM	2.360%
Wang et al. [26]	2010	FV-USM	0.020%
Mulyono et al. [27]	2010	FV-USM	0.280%
Qin et al. [28]	2011	FV-USM	0.040%
Peng et al. [29]	2012	SDUMLA-HMT	4.420%
Hong et al. [30]	2017	SDUMLA-HMT	3.453%
Tao et al. [31]	2021	SDUMLA-HMT	2.551%
Hsia et al. [32]	2022	FV-USM	0.470%
Shen et al. [33]	2022	SDUMLA-HMT	1.130%
This study	2022	FV-USM	0.002%
		SDUMLA-HMT	0.003%

shows that EER values are less than those for other recent studies.

5. Conclusions

This study uses a 1 to N finger-vein biometric for carputer systems and proposes a method to enhance edge detection to correct for translation and rotation of fingers, interference from external light sources and other environmental factors. The effect of the preprocessing method on the developed finger-vein system is also studied using the public FV-USM and SDUMLA-HMT datasets. This gives the best values for EER of 0.002 and 0.003 for various datasets. The indices for the EER clearly show that the proposed method outperforms state-of-the-art systems. A stable biometric system for carputer systems is constructed, which operates through contactless interactions to provide superior convenience and sanitation for the user.