1. Introduction
Unimodal biometric verification based on physiological characteristics such as the iris, retina, face, fingerprint, palmprint, hand geometry, and vein patterns are increasingly demanded for security systems. However, they have still many challenges, such as limited information, limited feature representations and weak antispoofing capabilities. Thus, achieving a high accuracy rate in unimodal biometric verification remains a challenge. As a result, bimodal biometric verification was developed. The greatest advantage of bimodal biometric verification is that multiple information points can be acquired from different modal characteristics.
This paper proposes a biometric verification method based on two physiological characteristics: the palmprint and palm veins. A palmprint image is rich in line features such as wrinkles, ridges, and principal lines. A palm vein image contains rich texture features shown in its vein patterns. The proposed method fuses these two modal images and hence results in richer and complementary information of one or more biometric characteristics. Many techniques can be used to perform the fusion, including the well-known discrete wavelet transform (DWT) and inverse discrete wavelet transform (IDWT). The proposed method uses DWT and IDWT to fuse palmprint and palm vein images.
Biometric verification methods using palmprint features have been developed over the past decades. Jain
et al. [
1] identified 14 different biometric features that could be used to verify hand shapes by using deformable matching technology. Huang
et al. [
2] proposed a palmprint verification technique based on principal lines. These principal lines were extracted using a modified finite Radon transform, and a binary edge map was used for representation. Han
et al. [
3] extracted features such as finger length, finger width, and palmprints to be used as inputs for principal component analysis. Zhang
et al. [
4] applied the line-matching idea to print matching. They transferred palmprints to line sections and used these features to identify people. Lu
et al. [
5] applied a Karhunen–Loeve transformation to transform an original palmprint into an eigenpalm, which could represent the principal components of the palmprint. Then, the weighted Euclidean distance classifier was applied for palmprint recognition. In another study, texture-based codes such as the competitive code [
6] and the orthogonal line ordinal feature [
7] were used to extract the orientation of lines which exhibit state-of-the-art performance in palmprint recognition. Kong
et al. [
8] applied a two-dimensional Gabor filter to obtain texture information from palmprint images. Two palmprint images were compared in terms of their Hamming distance of texture information. Zhang
et al. [
9] obtained a palmprint feature by using a locality-preserving projection based on a wavelet transform. Lin
et al. [
10] presented a palmprint verification method that involved using a bifeature, palmprint feature-point number and a histogram of oriented gradient. Lu
et al. [
11] proposed a system of capturing palm images in peg-free scenarios by using a low-cost and low-resolution digital scanner. Lin
et al. [
12] applied a hierarchical decomposition mechanism to extract principal palmprint features inside the region of interest (ROI), which included directional and multiresolution decompositions. They used a normalized correlation function to evaluate similarities. Han
et al. [
13] used four Sobel operators and complex morphological operators to extract the features of a palmprint, and applied the backpropagation neural network and template matching with a normalized correlation function to verify persons.
Compared with the palmprint, the use of palm veins is a relatively new hand-based biometric trend. MacGregor
et al. [
14] were the first to present a system for personal identification using palm veins. Im
et al. [
15] employed a charge coupled device (CCD) camera to capture vein pattern images. Their research focused on implementing fixed-point operations to improve verification speeds and reduce hardware costs. Mirmohamadsadeghi
et al. [
16] investigated two new feature extraction approaches based on a variety of multiscale, local binary patterns and high-order local derivative patterns to identify the optimal descriptors for palm veins. Lin
et al. [
17] obtained multiresolution representations of images with feature points of the vein patterns (FPVPs) by using multiple multiresolution filters that extracted the dominant points by filtering the miscellaneous features for each FPVP. Shahin
et al. [
18] proposed biometric authentication using a fast spatial correlation of hand vein patterns, and designed a system with a near infrared cold source to provide back-of-hand illumination. Wang
et al. [
19] combined support vector machines (SVMs) with a k-nearest neighbors algorithm and a minimum distance classifier for palmprint and palm-vein feature matching. Recently, the effectiveness of finger vein recognition was proved by Liu
et al. [
20] using a novel point manifold distance metric.
Bimodal biometrics have been deployed with particular fusion schemes, including sensor-level, feature-level, and match-score level fusions. Wang
et al. [
21] fused palmprint and palm-vein images and proposed a Laplacian palm representation, which attempts to preserve local characteristics. Kisku
et al. [
22] used a few selected wavelet fusion rules to fuse biometric face and palmprint images at the sensor level. The technique proposed in this paper efficiently minimizes irrelevant distinct variability in the different biometric modalities and their characteristics by performing the fusion of biometric images at the sensor level.
The proposed method adopts two biometric modals: the palmprint and palm-dorsum vein. A function block diagram is shown in
Figure 1. The method is composed of five stages: image acquisition, preprocessing, image fusion, feature extraction, and multiresolution analysis and verification. In the image acquisition stage, a digital scanner and infrared (IR) camera were applied to capture palm and palm-dorsum images. The resolution of the digital scanner used in this study was 100 dpi and that of the IR camera was 320 × 240 pixels. One hundred volunteers were used to capture 3000 palmprint and 3000 vein-pattern images. The preprocessing stage included palm region segmentation, the locating of finger webs, and ROI localization. In the image fusion stage, DWT and IDWT were applied to fuse the two ROI images—the palmprint and vein pattern—into one new fused image. Iterative histogram thresholding was employed to extract line-like features (LLFs) from the fused images. The extracted LLFs were analyzed using a multiresolution filter to obtain the multiresolution representations. Finally, an SVM is adopted to perform the verification between reference templates and testing images.
Figure 1.
Block diagram of the proposed method.
Figure 1.
Block diagram of the proposed method.
The rest of this paper is organized as follows:
Section 2 describes the preprocessing procedures, which include palm region segmentation, the locating of finger webs, and ROI localization and alignment.
Section 3 describes the process of image fusion based on DWT and IDWT. In
Section 4, LLF extraction, iterative histogram thresholding, and multiresolution analysis with the multiresolution filter are described. The mechanism of verification based on SVM is demonstrated in
Section 5.
Section 6 presents the results to verify the validity of the proposed method. Finally, concluding remarks are presented in
Section 7.
3. Image Fusion Based on DWT and IDWT
Image fusion has been employed in diverse fields such as computer vision, remote sensing, medical imaging, and satellite imaging. Irrespective of the field, the aim of image fusion is the same that is to create more useful information from two single images.
DWT is a useful technique for numerical and functional analysis. It has long been used as a method of image fusion [
25], and its practical applications can be found in digital communication, data compression, and image fusion. A key advantage of DWT over Fourier transforms is that it captures both frequency and location information. The proposed method fuses images by using DWT and IDWT with a novel hybrid fusion rule at different decomposition levels in wavelet-based.
For the described DWT, some necessary signals and filters must first be defined. Signal
xi is the input signal. Signal
yo, which includes
ylow and
yhigh, is the output signal. Filter
l, as expressed in Equation (5), is a low pass filter that filters out the high frequency of the input signal and outputs the low frequency signal called approximation coefficients. Filter
h, as expressed in Equation (6), is a high pass filter that outputs the high frequency signal called detail coefficients. Variables
k and
n are the
k-th and
n-th data of the signal, respectively. The filter outputs are downsampled by two with the downsampling operator ↓:
The DWT of signal
xi is calculated by passing
xi through a series of filters. The signal is decomposed using a low pass filter and a high pass filter simultaneously. The decomposition is repeated to further increase the frequency resolution, and the approximation coefficients are decomposed with the high and low pass filters and the downsampling.
Figure 7a shows the one-stage structure of the two-dimensional DWT where
l(−
x) and
l(−
y) are the low pass decomposition filters,
h(−
x) and
h(−
y) are the high pass decomposition filters, and
CLL,
CLH,
CHL, and
CHH are the decomposition coefficient matrices.
Figure 7b shows the relative locations of decomposition coefficient matrices in the two-dimensional DWT.
Figure 7.
(a) One-stage structure of the two-dimensional DWT; (b) Relative locations of decomposition coefficient matrices in the two-dimensional DWT; (c) One-stage structure of the two-dimensional IDWT; (d) Relative locations of fusing coefficient matrices in the two-dimensional IDWT.
Figure 7.
(a) One-stage structure of the two-dimensional DWT; (b) Relative locations of decomposition coefficient matrices in the two-dimensional DWT; (c) One-stage structure of the two-dimensional IDWT; (d) Relative locations of fusing coefficient matrices in the two-dimensional IDWT.
Once the coefficients are merged, the final fused image is achieved using IDWT.
Figure 7c shows the one-stage structure of the two-dimensional IDWT. The fused image is denoted by
If (
x, y).
Figure 7d shows the relative locations of the fusing coefficient matrices in the two-dimensional IDWT.
ILL,
ILH,
IHL, and
IHH are the fused coefficient matrices.
Wavelet-Based Image Fusion of Palmprints and Vein Patterns
Palmprints have low gray levels in palm images, whereas vein patterns have high gray levels in palm-dorsum images. To make the gray-level properties of palmprints consistent with those of vein patterns, the gray levels of the palm images are reversed. Thus, inverted palm images have high gray-level palmprints with low gray-level backgrounds.
In addition, the ROI sizes for palmprints and vein patterns are different. To address this problem, palmprint ROIs are decomposed using two-dimensional DWTs three times to obtain first-, second-, and third-level coefficients with the sizes of 128 × 128, 64 × 64, and 32 × 32 pixels, respectively (see
Figure 8a). Vein pattern ROIs are decomposed using a two-dimensional DWT to obtain the first-level coefficient with a size of 32 × 32 pixels (see
Figure 8b). The size of the third-level coefficient for the palmprint ROI is the same as the size of the first-level coefficient for the vein pattern ROI. The two-dimensional DWT used in the proposed method is the Haar filter, which includes lowpass filter
l and highpass filter
h.
By analyzing the three-dimensional (3D) profiles of the palmprint and vein pattern ROIs, it is revealed that palmprints and vein patterns possess different characteristics. The 3D profile of the palmprint ROI shows a sudden change in the gray levels of adjacent pixels near the principal palmprint, which possesses a high frequency as shown in
Figure 9a [
12]. In contrast, the 3D profile of the vein pattern ROI demonstrates that the gray levels of the vein pattern varies smoothly. The vein pattern has a low frequency as shown in
Figure 9b [
17].
Coefficient fusion is the key step in image fusion based on wavelets. Many coefficient fusion rules have been presented including maximum, minimum, average, weighted, down-up, and up-down [
25]. According to an analysis of the 3D profiles for palmprint and vein pattern ROIs, the proposed method introduces a hybrid fusion rule consisting of average and maximum fusion rules. The hybrid fusion rule applies the average rule to combine the approximation coefficients and the maximum rule to combine the detail coefficients. The hybrid fusion rule is named the
Avg-Max fusion rule and is expressed as follows:
where
If (
x, y) is the coefficient value of the fused image at pixel (x, y),
Ip(
x, y) is the coefficient value of the palm image, and
Iv(
x, y) is the coefficient value of the palm-dorsum vein image.
Figure 8.
(a) Three levels of DWT in palmprint ROI decomposition; (b) One level of DWT in vein pattern ROI decomposition.
Figure 8.
(a) Three levels of DWT in palmprint ROI decomposition; (b) One level of DWT in vein pattern ROI decomposition.
Figure 9.
3D profile of palmprint and vein pattern ROIs. The vertical axis represents the gray level of pixels. The other two axes represent the coordinates of pixels. (a) The profile of the palmprint ROI shows a sudden change in the gray levels of adjacent pixels near the principal palmprint, which possesses high frequency; (b) The vein pattern profile demonstrates that the gray levels of the vein pattern vary smoothly. The vein pattern has a low frequency.
Figure 9.
3D profile of palmprint and vein pattern ROIs. The vertical axis represents the gray level of pixels. The other two axes represent the coordinates of pixels. (a) The profile of the palmprint ROI shows a sudden change in the gray levels of adjacent pixels near the principal palmprint, which possesses high frequency; (b) The vein pattern profile demonstrates that the gray levels of the vein pattern vary smoothly. The vein pattern has a low frequency.
Since the sizes of the palmprint and vein pattern ROIs are different, the proposed method adopts the different resolution fusion scheme [
26] to fuse palmprint and vein pattern ROIs at the third and first levels, respectively. The size of the third-level coefficient of the palmprint ROI is the same as the size of the first-level coefficient of the vein pattern ROI which is 32 × 32 pixels. The proposed method applies the novel hybrid fusion rule to combine the 32 × 32 coefficients and performs IDWT to fuse the 64 × 64 image. IDWT is then performed again with the remaining palmprint coefficients. The final fused 256 × 256 image is shown in
Figure 10. We can observe that the fused image retains the high frequency palmprint and low frequency vein pattern information.
Figure 10.
Three stages of IDWT fused image composition.
Figure 10.
Three stages of IDWT fused image composition.
7. Conclusions
In this paper, fused images of palmprints and palm-dorsum vein patterns are used to verify the identity of individuals. The experimental results show that the proposed biometric verification system is robust and reliable. The findings of this research can help extend bimodal biometric verification technology to security access control systems and bio-cryptosystems [
33,
34].
There are five advantages in our proposed method. First, no docking devices or fixed pegs are needed while acquiring palm images, which makes the personal verification device easier and more convenient for users. Second, the low-resolution images are used to verify and result in a smaller database. Third, the threshold values to binarize the original image to the background and palm region are automatically set. Hence, the palm region is segmented adaptively using the proposed thresholding technique. Fourth, according to the palmprint and vein pattern characteristics, this paper proposes a novel hybrid fusion rule, Avg-Max, to fuse the different coefficients decomposed by DWT. In addition, the palmprint and vein pattern images are of different sizes, yet the proposed method combines the different coefficients at different decomposition levels with coefficients of the same size. Finally, the fused image creates richer and more useful information than each individual image and the dimensions of the feature vectors are the same as in each individual image.
As with most biometric verification methods, the proposed method has some operational limitations. Because the IR camera used in this study has a low resolution and sensibility, this limits the accuracy of biometric verification. A high performance IR camera should be used to capture high-quality and more discriminative images. Furthermore, there may exist other effective feature extraction methods that could obtain more information from palm and palm-dorsum images. In addition, a biometric verification method combining additional biometric features such as palm geometry, fingerprints, or palm creases could increase verification accuracy. Finally, most biometric features vary with the age of the person, an improved biometric verification method would be capable of predicting feature variations to maintain accuracy.