Boosting Iris Recognition by Margin-Based Loss Functions

Abstract

In recent years, the topic of contactless biometric identification has gained considerable traction due to the COVID-19 pandemic. One of the most well-known identification technologies is iris recognition. Determining the classification threshold for large datasets of iris images remains challenging. To solve this issue, it is essential to extract more discriminatory features from iris images. Choosing the appropriate loss function to enhance discrimination power is one of the most significant factors in deep learning networks. This paper proposes a novel iris identification framework that integrates the light-weight MobileNet architecture with customized ArcFace and Triplet loss functions. By combining two loss functions, it is possible to improve the compactness within a class and the discrepancies between classes. To reduce the amount of preprocessing, the normalization step is omitted and segmented iris images are used directly. In contrast to the original SoftMax loss, the EER for the combined loss from ArcFace and Triplet is decreased from 1.11% to 0.45%, and the TPR is increased from 99.77% to 100%. In CASIA-Iris-Thousand, EER decreased from 4.8% to 1.87%, while TPR improved from 97.42% to 99.66%. Experiments have demonstrated that the proposed approach with customized loss using ArcFace and Triplet can significantly improve state-of-the-art and achieve outstanding results.

1. Introduction

During the pandemic of COVID-19, non-contact or contactless biometric-based identification methods led the industry in 2020. There is a growing interest in contactless biometric recognition, whether it is using the face [1] and ears [2] or using the iris [3] and voice [4] as the most functional biometric techniques. Contact-based biometrics, such as fingerprints [5], where sensors require that the finger be tightly held, are secure for a person’s data, but they are not as secure for their health. Wearing a facial mask is the new norm among people in this pandemic, and applications of face recognition technology [6,7] have encountered severe problems [8]. A recent work [9] focused on the system for recognizing faces with and without masks. Due to these challenges, iris recognition technology may become more widely adopted since it is contactless and not impacted by masks.

Based on Daugman [10], the four fundamental stages of the iris recognition system are segmentation, normalization, feature extraction, and feature matching. Although accurate iris segmentation is necessary, extracting discriminative features of iris images is the most challenging aspect of these systems. Convolutional neural networks (CNNs) can learn and extract features from highly complex datasets, such as iris images. A variety of solutions can improve CNNs, such as batch normalization, data augmentation, regularization, dropout, and different activation functions. Considering reducing the computational parameters, researchers designed improved structures of the pre-trained network.

Researchers have paid great attention to optimizing the loss functions to obtain distinctive feature distributions. The SoftMax loss function is one popular loss function for recognition tasks. In terms of inter-class separation, SoftMax loss performs well but is ineffective in minimizing intra-class compactness. For iris recognition systems, where only a few samples are available for each class, the generation of discriminating features is crucial to achieving high classification accuracy. There are differences in the way that SoftMax-based loss functions apply margin penalties. The Angular Softmax Loss (SphereFace) [11] applies a multiplicative angular margin penalty. The Large Margin Cosine Loss (CosFace) [12] adds a cosine margin penalty to the logit directly. Additive Angular Margin Loss (ArcFace) [13] places the additive cosine margin within a geometric angular space.

Using the MobileNet architecture with the combination of the ArcFace and Triplet loss functions in this paper, we train an iris recognition network taking advantage of both loss functions. We also compare the performance of this improved loss function to three others, consisting of original SoftMax loss, CosFace, and ArcFace. Notably, we try to optimize the network with segmented iris images without normalization. The testing datasets, especially CASIA-Iris-Thousand, have many classes with reflection or eyeglasses that will aid in clarifying the effect of challenging irises on loss functions by analyzing these two iris datasets. This work follows the main contributions:

• To extract discriminative iris features, our study focuses on implementing state-of-the-art loss functions, especially a combination of ArcFace and Triplet loss with a light-weight MobileNet architecture for the training approach;
• The performance of the proposed system is tested on two public datasets that interfere with the various noises: CASIA-Iris-Thousand [14] and IITD [15] datasets.

The remaining sections in this paper are arranged as follows: In Section 2, we review a couple of related works with more focus on deep learning-based methods. Section 3 presents an overview of the proposed deep learning approach with described combined loss function. Section 4 demonstrates the proposed system’s experimental results. The last section includes conclusions and future directions.

2. Related Work

Recent studies by Adjabi et al. [1] reviewed 2D and 3D face recognition from traditional to deep-learning models. For the recognition of single samples of faces, Adjabi et al. [16] used multiblock color-binarized statistical images. A machine learning technique called Deep Unsupervised Active Learning (DUAL) was proposed by Khaldi et al. [2], to determine the feasibility of active learning that could be used in ear recognition and biometrics in general. Researchers have exploited neural networks or deep learning approaches not only to identification, but also with biometric trait classification. Classification is often used in large-scale datasets to reduce the search space, busting the performance. According to Militello et al. [17], the first study has compared the most common CNN architectures by dividing fingerprints into four, five, and eight categories. The purpose of this work is to demonstrate the potential of CNN architectures for fingerprint classification, by reducing the amount of data needed to be processed during identification and search.

Most recent iris recognition work focuses on segmenting the iris and extracting iris features. Noisy iris images, such as those occluded by eyelids or eyelashes, specular reflections, and off-angles, are behind the main problem in segmenting and localizing them [18]. Denoising images using first and second generation wavelets [19,20] or fast matching algorithms [21] are some ways of preprocessing images.

Processing speed and recognition time are two critical parameters of any real-time biometric system. The normalization process consumes a significant amount of time. In work [22], authors claimed that by skipping the classical normalization step, they could improve iris recognition by saving time comparable to that spent on it. Because of the normalization errors, the normalized samples showed inconsistencies, leading to unstable recognition. Considering the capability of deep learning to model highly complex distributions and nonlinear data changes, Lozej et al. in [23] tried to investigate how crucial the use of traditional segmentation techniques is in a deep learning setting. Chen et al. [24] presented a novel non-normalized preprocessing method based on dynamic path search for iris segmentation. Using two public iris datasets, which are the same as in this work, the authors proved that iris segmentation images without normalization are a better choice for evaluating iris recognition techniques based on deep learning techniques.

For feature extraction, two categories of previous approaches are handcrafted approaches and learning-based approaches. Daugman [10] proposed the first study in handcrafted approaches in 1993, which used an integral-differential operator to segment the iris, followed by a Gabor filter to extract features. Wildes used the Laplacian pyramid and Fisher’s linear analysis [25]. Krichen et al. [26] used Gabor wavelet packets. Miyazawa et al. [27] used 2D Discrete Fourier Transforms (DFTs).

In the field of learning-based methods, CNNs are effective techniques. Liu et al. [28] provided the first use of CNN for iris recognition in 2016. They used a simple CNN with a multiphase filter bank. According to Minaee et al. [29], using more layers is ineffective for recognizing more distinctive features. Their method used the VGG network architecture. Gangwar and Joshi [30] proposed DeepIrisNet in two categories. The first case used standard convolution layers, and the second case incorporated inception layers. The model can recognize images from cross-sensors with high accuracy. Nguyen et al. [31] extracted features using some popular pre-trained networks and classified them using Support Vector Machine (SVM). DensNet architecture achieved the best result in their work. Al-Waisy et al. [32] developed a deep learning system called IrisCOnvNet, which trained a CNN with a SoftMax classifier. Using the ranking-level fusion method, they integrated the classification results. Zhang et al. [33] fused iris and periocular features with a weighted concatenation. They claimed that maxout units do better than ReLUs in their study. Using capsule networks, Zhao et al. [34] developed a deep learning algorithm for iris recognition. These networks can learn part-whole deep features relationships and lead to high classification accuracy.

The SoftMax loss is used to supervise the training of the reviewed iris recognition systems. Zhao and Kumar [35] proposed a network called UniNet. They designed an Extended Triplet Loss (ETL) function inspired by work in [36] and fed the samples to the model in triplet sets. Ahmad and Fuller [37] trained a network using ResNet with SoftMax classifier, thus optimizing the model using the triple loss function. Their vital target was to evaluate the network performance by eliminating the normalization stage. On constrained datasets, their network performed well. The primary demand of the triplet loss function is triplet mining or selecting semi-hard samples. In addition, comparisons among triplet samples within mini-batch are local. Inspired from the approach proposed in [38], Chen et al. in [39] introduced a novel loss function called Tight Center (T- Center). The optimized center loss function solved the problem of insufficient discrimination caused by SoftMax loss, but there is an incompatibility between Softmax loss and center-based loss functions. The SoftMax loss has an intrinsic angular distribution, while the center loss applies the Euclidean margin to penalize the distance between the deep features and their corresponding.

To angular discriminative deep feature learning, some loss functions such as SphereFace [11], CosFace [12], and Arcface [13] have been proposed recently. Hsu et al. [40] analyzed the performance of these loss functions on several face datasets containing challenge factors. Our paper uses a similar algorithm as in [40] for iris datasets.

3. Proposed Approach

Figure 1 provides an overview of the proposed framework. First, a preprocessing method removes the specular reflection from the input images. Then, a deep learning algorithm is applied to detect iris location and segment it into iris/noniris. The segmented irises are presented without being normalized to a feature extraction framework. In Section 3.1, we provide more details on image processing. During the training phase, a MobileNet architecture is implemented using a combined loss function with Triplet and ArcFace. Through the testing phase, embedding is the model output before the classifier header, which is regarded as the image feature map. The embedding vector is used for evaluation, by calculating cosine similarities between each feature. First, we pick an image most similar to others in a single class as registered base embedding, which is the yellow dot in Figure 1. Then, we test all of the left-over images if classified correctly, which in Figure 1 are the red and blue dots representing the positive and negative samples. The task we present in this paper focuses on both angular measurement between two features and distance metric, so we prefer cosine similarities to compute extracted embedding values.

3.1. Preprocessing and Segmentation

With only a slight modification, we use a deep learning segmentation method that has been proposed in [41]. It trained a Mask R-CNN framework with segmentation ground truth datasets prepared by Hofbauer et al. [42]. For the images with reflections, the proposed method does not segment accurately. We solve this issue by adding a preprocessing stage, which uses an expansion operator and a fast-matching algorithm [21] to remove the existing reflections. Figure 2 shows examples of the iris bounding box detection and mask segmentation implementing this preprocessing. It can be observed that the removal of reflection points leads to accurate segmentation. In this work, we eliminate the normalization step common in most recent works after segmentation. With data augmentation, it is possible to increase the variation in the training datasets and help overcome over-fitting. In our augmentation, we randomly randomly half of all iris images horizontally and vertically. A vertical flip is equivalent to rotating an image by 180 degrees and then performing a horizontal flip.

3.2. Combined Loss of ArcFace and Triplet

The selected loss function to combine, ArcFace, is based on changes in the SoftMax with a triplet loss function. Softmax loss can be expressed as follows:

$$L_{Softmax}=-\frac{1}{N}\sum _{i=1}^{N}log\frac{e^{{W_{y_i}}^T{x}_i+b_{y_i}}}{{\sum }_{j=1}^n{e}^{{W_{y_i}}^T{x}_i+b_{y_i}}},$$

(1)

where $x_i\in {\mathbb{R}}^d$ represents the deep feature vector or output from the last CNN layer to the fully connected layer of the i-th sample belonging to the $y_i$-th class. $W_j\in {\mathbb{R}}^d$ is the j-th column of the weight matrix $W_j\in {\mathbb{R}}^{d\times n}$ and $b_j\in {\mathbb{R}}^n$ is the bias term. n is the class number, and N is the batch size. The SoftMax loss function cannot optimize deep feature vectors to achieve better intra-class similarity and the scattered distribution of samples can cause problems for clustering.

Large Margin Cosine Loss [12] followed [11] initially set the bias value to zero. In $W_j^T{x}_i=\Vert W_j\Vert \Vert x_i\Vert cos{\theta }_j$, where ${\theta }_j$ is the angle between feature vectors $x_i$ and the ground truth weight $W_j$, the authors fixed $\Vert W_j\Vert =1$ and $\Vert x_i\Vert$ by $l_2$ normalization and re-scaled $\Vert x_i\Vert$ to s. Normalizing the weights and features yields predictions based on only the angle between them. Each embedding has a radius s and is distributed on the hypersphere. They named it “CosFace”.

$$L_{co}=-\frac{1}{N}\sum _{i=1}^{N}log\frac{e^{s(cos\left({\theta }_{y_i}\right)-m)}}{e^{s(cos\left({\theta }_{y_i}\right)-m)}+{\sum }_{j=1,j\ne y_i}^n{e}^{scos{\theta }_j}}$$

(2)

By taking advantage of the fact that the embedding features are distributed around each center on the hypersphere, the work in [13] added an additive angular margin penalty between $x_i$ and $W_{y_i}$ to enhance the intra-class compactness and inter-class dispersion simultaneously. The authors named their method “ArcFace”.

$$L_{aa}=-\frac{1}{N}\sum _{i=1}^{N}log\frac{e^{s(cos({\theta }_{y_i}+m))}}{e^{s(cos({\theta }_{y_i}+m))}+{\sum }_{j=1,j\ne y_i}^n{e}^{scos{\theta }_j}}.$$

(3)

Compared with CosFace and similar loss functions, ArcFace has a better geometric property because of the exact alignment of angular margin with geodesic distance.

Table 1. Comparing binary classification decision boundaries.

Loss Functions	Decision Boundaries
Softmax	$\left(W_1-W_2\right)x+b_1-b_2=0$
CosFace	$s\left(cos{\theta }_1-m-cos{\theta }_2\right)=0$
ArcFace	$s\left(cos\left({\theta }_1+m\right)-cos{\theta }_2\right)=0$

provides us with geometric comparisons differences of different decision margins. In binary classification, ArcFace has a fixed linear margin across the decision boundaries, whereas CosFase creates only a nonlinear margin. ArcFace loss functions can classify more accurately, since $\theta \in \left[0,\pi -m\right]$, $cos\left(\theta +m\right)<cos\left(\theta \right)$. In addition, because of $sin\left(\theta \right)$ in $cos\left(\theta +m\right)=cos\theta cosm-sin\theta sinm$, the new margin is more dynamic.

FaceNet [36] first uses triplet loss to learn deep representations of faces. In the presence of triplet loss, the positive pairs tend to have smaller distances than the negative pairs. In terms of mathematics, the loss can be described as follows using the Euclidean distance function:

$$\begin{gathered} L=\frac{1}{N}\sum _i^{N}max\left(\Vert f\left(x_i^a\right)-f\left(x_i^p\right){\Vert }_2^2-{\Vert f\left(x_i^a\right)-f\left(x_i^n\right)\Vert }_2^2+\alpha ,0\right) \\ \forall \left(f\left(x_i^a\right),f\left(x_i^p\right),f\left(x_i^n\right)\right)\in \tau \end{gathered}$$

(4)

where $(f\left(x_i^a\right),f\left(x_i^p\right))$ and $f\left(x_i^n\right)$ represent the feature vectors of the positive pairs $(x_i^a,x_i^p)$ and the negative pairs $(x_i^a,x_i^n)$ and $\alpha$ is the distance margin. Training samples with too few samples limit the effectiveness of feature embedding. One approach [43] proposes hard sample mining to help diminish this problem by utilizing the effective information in each batch, which is necessary for maximizing the performance of feature embedding.

ArcFace enlarges the margin between different classes (the so-called inter-class margin), while Triplet Loss also narrows the margin for the same class (the so-called intra-class margin). Moreover, the semi-hard triplet focuses only on the more complex samples. Combining losses allow deep for feature learning with high optimization flexibility and more accurate convergence targeting.

3.3. Network Architecture

Choosing a network architecture is still a challenge and depends on its specific application. One of the most significant light-weight networks is MobileNet [44]. Google designed MobileNet and optimized it to implement on mobile devices. The key feature of this structure is the low number of learning parameters with high classification accuracy. Blocks that build MobileNet take advantage of depthwise convolution. By applying only one convolutional filter for each input channel, depthwise convolution performs light-weight filtering. Also, MobileNet blocks have a $1\times 1$ convolution layer called a pointwise convolution. This component builds new features through linear combinations of input channels. These layers form a depthwise separable convolutional block. This block is illustrated in Figure 3. Each convolution layer is followed by batch normalization, which leads to faster convergence for deep learning models. MobileNet also benefits from ReLU6 activation function. ReLU6 activation is identical to the original ReLU, except that it is limited to six activations. In this paper, we use MobileNetV1. It begins with a regular $3\times 3$ convolution and follows by 13 times the above building block.

4. Experiments and Results

In this section, we evaluate the performance of the proposed method on two selected datasets (CASIA-Iris-Thousand, IITD) and plot the Receiver Operating Characteristic (ROC) curves to calculate Area Under the ROC Curve (AUC) and Equal Error Rate (EER).

4.1. Datasets

• CASIA-Iris-Thousand. The CASIA-Iris-Thousand [14] includes 20,000 images collected from 1000 subjects using the IKEMB-100 camera. Eyeglasses and specular reflections cause the main challenges in this dataset. CASIA-Iris-Thousand is the first available iris dataset of 1000 people. This large number of classes causes a challenge for analyzing unique iris characteristics and requires new classification methods. The images in this collection were recorded in $640\times 480$ resolution in jpg format. The resolution of the segmented image is $200\times 200$. we used the 10 left iris images from all subjects for the training set. We also select five random right iris images from each category to test the network performance;
• IITD. The IITD [15] contains 2240 images of the iris collected from 224 students and staff in Delhi, India.The database consists of 176 males and 48 females in the age range 14–55 years. There are different (iris) sizes and distributions of colors in this dataset. The images were recorded at $320\times 240$ resolution and BMP format. The segmented images have a size of $200\times 200$. We use the five images from the left iris images for network training and the five right iris images for testing.

4.2. Experimental Settings

We implement the deep CNN model in Python, using the TensorFlow back-end, with Keras library, a stochastic gradient descent (SGD) optimizer, which is used to optimize the model’s hyper-parameters. Momentum and weight decay are set at 0.9 and $5\times e-4$ for this optimizer. The initial learning rate is 0.009, divided by 10 on the 18th epoch of the 35 total epochs. The batch size is set to 64. According to [12], we would set the angular margin m of ArcFace at 0.5 and the feature scale s to 64.

4.3. Evaluation

To generate a stable evaluation result and reduce the calculating complexity, we first extract the embedding value from all images, and then sum up and normalize all embedding vectors in the same class as the registered base embedding vector. We use this vector as an embedding vector for this class and measure distance by calculating cosine distances between all embedding vectors and all registered base embedding vectors. Evaluation results consist of two parts: distances among all images and between all vectors registered, a label that shows whether each score is positively or negatively classified. The final evaluating results contain two parts score for distances between all images and all registered vectors, and a label showing whether each score should be classified positive or negative.

We compare the performance of selected loss functions using ROC curves and the EER to quantify the verification process. In addition to ROC curves for each dataset, we also select TPR at FPR = 0.1% and calculate the AUC to provide other feedback. Figure 4 and Figure 5 show the ROC curves obtained on IITD and CASIA-Iris-Thousand datasets, respectively.

Table 2. Testing results of different implementation on IIT Delhi.

Implementation	EER (%)	TPR (%)	AUC (%)
MobileNet + Softmax	1.11	99.77	99.92
MobileNet + Cosface	0.78	100	99.98
MobileNet + ArcFace	0.45	100	99.98
MobileNet + (ArcFace + Triplet)	0.45	100	99.99

and

Table 3. Testing results of different implementation on CASIA-Iris-Thousand.

Implementation	EER (%)	TPR (%)	AUC (%)
MobileNet + Softmax	4.80	97.42	98.97
MobileNet + Cosface	2.64	99.22	99.67
MobileNet +ArcFace	1.89	99.58	99.80
MobileNet + (ArcFace + Triplet)	1.87	99.66	99.84

show the results based on TPR, EER, and AUC. The proposed CNN model that uses ArcFace + Triplet loss is more effective for recognizing iris patterns. To make our conclusion clearer, we plot the ROC curve for combined loss separately in Figure 6 and Figure 7. Implementing ArcFace + Triplet loss achieves the lowest EER compared with the Arcface, CosFace, and SoftMax loss. The performance drops to the lower amount of AUC and TPR when evaluating the algorithm with Arcface and CosFace. Additionally, we plot the ROC of the SoftMax loss function to compare two margin-based loss functions with the SoftMax loss function. CASIA-Iris-Thousand is a more complex dataset than IITD because it contains images with eyeglasses and specular reflections. Therefore, the EER for CASIA-Iris-Thousand is higher than EER for IITD.

On the evaluated results, we also plot the histograms of positive and negative distributions. This histogram visually compares how the proposed method with the margin-based loss functions can distinguish iris feature vectors. We can see a clear decision margin distinguished in Figure 8, where the positive similarities for ArcFace are more gathered to the right of the figure, and the overlapping area is smaller. Though Figure 9 is not as clear as Figure 8, we also see a more narrow distribution for CosFace and ArcFace positive similarities. As illustrated in Figure 8 and Figure 9, ArcFace + Triplet is the best among all of them because there is a smaller overlap area between the red and blue distributions, and its positive distribution is more tended to the right than the other one. SoftMax loss is worst at separation. The margin between red and blue is smaller, and the overlapped area is broader. However, its negative red distribution is narrower and more concentrated, which means a smaller variance value. CosFace works similar to ArcFace, only adding a different margin between negative and positive. Thus, their results are identical, as only ArcFace + Triplet has a smaller overlapped area, making it a better separator.

We compare our model with three deep learning models, including DeepIrisNet [30], UniNet [35], CapsuleNet [34] and T-Center [39].

Table 4. Comparison of the performance on two public datasets.

	IITD			CASIA-Iris-Thousand
	TPR (%)	EER (%)	AUC (%)	TPR (%)	EER (%)	AUC (%)
DeepIrisNet [30]	95.03	1.17	99.02	86.86	4.01	97.93
UniNet [35]	97.35	0.89	99.20	90.61	2.79	98.67
CapsuleNet [34]	97.39	0.76	99.27	91.04	3.05	98.60
T-Center [39]	97.43	0.74	99.30	92.54	2.36	98.75
Proposed method (MobileNet + (ArcFace + Triplet))	100	0.45	99.99	99.66	1.87	99.84

shows that the proposed method with ArcFace + Triplet achieves the lowest value of EER for both datasets compared with prior state-of-the-art works. It has greatly improved about 2.57% and 7% for IITD and CASIA-Iris-Thousand compared with T-Center An important point to note is that the experiments we compared used normalized images, while our model does not involve normalization.

5. Conclusions and Future Work

Through the present article, the researchers proposed a deep learning framework for iris recognition based on MobileNet architecture by using a combined loss function with ArcFace and Triplet. ArcFace has a larger margin on semi-hard instances and keeps a small margin on easy ones; and increases the margins between classes, Triplet Loss narrows margins within a single class This somehow makes ArcFace work better on iris recognition tasks. As expected, the robust discriminative proposed CNN architecture using the combined loss function achieved significant performance with the EER = 0.45% for IITD and EER = 1.87% for CASIA-Iris-Thousand without normalization step. The obtained results, especially for CASIA-Iris-Thousand, have proved that the combined loss function can effectively be applied to large-scale iris identities. According to the above finding, it can be realized that the results of the Arcface studied in the present investigation lead to better use of existing state-of-the-art loss functions for iris recognition systems.

Various limitations, such as noisy data, differences and similarities between classes, have led to the development of multimodal biometrics, which utilizes multiple sources of information to authenticate the person. Using a fusion of matching and decision score level method, the proposed approach could be applied in a multimodal context. As shown in Conti et al. [45], fusing iris and retina or periocular modalities represents a robust combination. For future developments of the proposed approach, we will focus on multimodal biometrics. In future research, we will apply the proposed method to various datasets collected in an unconstrained environment with more recent deep learning architecture. In addition, we will use other loss functions independent of determining the margins and scale.