Additive Orthant Loss for Deep Face Recognition

Abstract

In this paper, we propose a novel loss function for deep face recognition, called the additive orthant loss (Orthant loss), which can be combined for softmax-based loss functions to improve the feature-discriminative capability. The Orthant loss makes features away from the origin using the rescaled softplus function and an additive margin. Additionally, the Orthant loss compresses feature spaces by mapping features to an orthant of each class using element-wise operation and 1-bit quantization. As a consequence, the Orthant loss improves the inter-class separabilty and the intra-class compactness. We empirically show that the ArcFace combined with the Orthant loss further compresses and moves the feature spaces farther away from the origin compared to the original ArcFace. Experimental results show that the new combined loss has the most improved accuracy on CFP-FP, AgeDB-30, and MegaFace testing datasets, among some of the latest loss functions.

1. Introduction

Face recognition (FR) [1,2] based on deep convolutional neural networks (DCNN) [3,4,5,6,7] is one of the fields in computer vision that found great success in the 2010s, and it has been actively studied as a biometric authentication [8,9,10] technique for face verification and identification. In particular, deep FR has better recognition accuracy for images with large variations such as pose, illumination, age, and expression compared to FR without using DCCNs. The good recognition accuracy of the deep FR comes from the fact that each DCNN layer is trained on a face image from various perspectives. It is important to design an efficient loss function that can distinguish identities well using a training dataset with a limited number of images and identities. To improve the recognition accuracy, various loss functions [11,12,13,14,15,16,17] as well as network architectures [18,19,20,21] have been studied.

Many loss functions for FR have been studied based on the softmax function [22] for image classification problems. In [23,24], the authors first trained a deep face verification model using the softmax function. Wen et al. [25] proposed the Center loss that uses both the softmax function and an additive term to improve the intra-class compactness. Liu et al. [26] proposed a multiplicative angular margin to increase the cosine similarity of each class. In [27], Wang et al. showed that the cosine similarity can be optimized using a new scaling constant replacing the magnitudes of the inner product between weight and feature vectors. Liu et al. [28] proposed the angular softmax function that moves to the hypersphere manifold through weight normalization. In [29], Gao et al. proposed the Margin loss that combines an additive term with the Center loss to enlarge the distances between centers of each class. Wu et al. [30] proposed the Center invariant loss combined with the center loss to solve a bias caused by imbalanced image data between identities. In [31,32], the authors proposed a loss function by introducing an additive angular margin to increase the cosine similarity of each class. Deng et al. [33] proposed the ArcFace function that performs feature and weight normalizations in addition to the use of an additive angular margin. In [34], Ou et al. proposed the LinCos-Softmax that replaces the cosine similarity between feature and weight vectors with an approximated linear angle to prevent the angle saturation caused by the nonlinearity of cosine similarity. In [35], Meng et al. proposed an adaptive loss function that efficiently learns the distributions of features using the magnitudes of features. Tao et al. [36] proposed the FCGFace loss that combines the ArcFace with two additive terms to compactly guide profile face features to each center of the frontal face features. In [37], Wang et al. proposed the RVFace that drops the noisy feature vectors and adaptively learn the distributions of features using semi-hard feature vectors to improve the feature discriminative capability. For surveys on more network structures and loss functions for deep FR, readers are referred to [38,39].

Many recent loss functions [31,32,33,34] only use the cosine similarity between weight and feature vectors in the hypersphere manifold without feature magnitudes. Then, even if feature vectors have high cosine similarities with weight vectors of their corresponding class, the feature magnitudes may be small. In this case, the feature vectors have a problem that the cosine similarity can be changed significantly when the signs of features generated from images with large variation are changed due to their small magnitudes. On the other hand, in the property of monotonicity of [35], it has been proven that feature magnitudes increase with the cosine similarity. It has also been shown that an optimal feature magnitude exists for a feature by the property of convergence of [35]. Based on the above properties, we can see that it is necessary to increase the feature magnitude to find an optimal one as well as to improve cosine similarity.

In this paper, we propose an additive orthant loss function, termed the Orthant loss, to improve the inter-class separability and the intra-class compactness. The Orthant loss improves the inter-class separability by increasing the distance between features and the origin using the rescaled softplus function and an additive margin. Additionally, the 1-bit quantization and element-wise operation of the Orthant loss simultaneously improve the intra-class compactness of each feature space by attempting to make the features of the same identity have the same signs. In this paper, the proposed loss function is termed as the Orthant loss, since we try to map features of the same identity to an orthant where the center of the corresponding class is located. The Orthant loss has two major differences from the Center loss [25]. First, the Center loss requires an additional fully connected layer to learn the center location of each class, whereas the Orthant loss does not require the layer. In addition, the Center loss learns the distributions of all features to gather at the center of each class without considering the feature magnitudes, while the Orthant loss learns the distributions to efficiently utilize the feature spaces by considering the feature magnitudes. Since our new loss function has no network modification, it is compatible with many of the latest softmax-based loss functions. Thus, we construct a final loss function by combining one of the lastest softmax-based loss functions [33,35] with the Orthant loss.

The experimental results show that a new loss function combining the ArcFace and the Orthant loss together, termed the ArcOrthFace, has the better accuracy compared to softmax-based loss functions, e.g., Center loss [25], SphereFace [28], ArcFace [33] and MagFace [35], respectively, on  CFP-FP [40], AgeDB-30 [41] and MegaFace [42] testing datasets. Additionally, we confirm that the Orthant loss combined with the softmax-based functions [22,33,35] improves the recognition accuracy in most benchmarks. In conclusion, the Orthant loss, combined with the latest softmax-based functions, can improve the accuracy of deep face recognition by enhancing the inter-class separability and the intra-class compactness of feature spaces.

2. Background

2.1. Deep Face Recognition System

As shown in Figure 1, a deep FR system consists of training and testing phases. In the training phase, DCNNs such as AlexNet [3], VGGNet [4] and ResNet [6] are trained to obtain the discriminative features $\mathbf{x}$ of preprocessed face images. The goal of training is to learn a set of parameters $\mathcal{W}$ of the DCNN and the weight matrix $\mathbf{W}=({\mathbf{w}}_1,\cdots ,{\mathbf{w}}_M)\in {\mathbb{R}}^{l\times M}$ of the last fully connected layer, where l is the dimension of features and M is the number of identities of a training dataset. The optimization problem of the deep FR is formulated by:

$$\left(\widehat{\mathcal{W}},\widehat{\mathbf{W}}\right)=\underset{\mathcal{W},\mathbf{W}}{arg\; min}\mathcal{L}(\mathbf{x};\mathcal{W},\mathbf{W}),$$

where $\mathcal{L}$ is a loss function, and $\widehat{\mathcal{W}}$ and $\widehat{\mathbf{W}}$ are learned parameters of $\mathcal{W}$ and $\mathbf{W}$, respectively. Once the model is trained, verification or identification is performed in the testing phase using a distance measure, e.g., cosine similarity or Euclidean distance, to determine whether the feature obtained from a test image is from a registered user of the system.

2.2. Some Known Loss Functions

The softmax function [22] has been mainly used as a loss function, which is given by:

$$\begin{array}{cc}\hfill {\mathcal{L}}_{\mathrm{Softmax}}& =-\frac{1}{N}\sum _{i=1}^{N}log\frac{e^{{\mathbf{w}}_{y_i}^T{\mathbf{x}}_i}}{e^{{\mathbf{w}}_{y_i}^T{\mathbf{x}}_i}+{\sum }_{j\ne y_i}{e}^{{\mathbf{w}}_j^T{\mathbf{x}}_i}}\hfill \\ \hfill & =-\frac{1}{N}\sum _{i=1}^{N}log\frac{e^{s_{y_i}cos\left({\theta }_{y_i}\right)}}{e^{s_{y_i}cos\left({\theta }_{y_i}\right)}+{\sum }_{j\ne y_i}{e}^{s_{j}cos\left({\theta }_j\right)}},\hfill \end{array}$$

(1)

where ${\mathbf{w}}_{y_i}$ and ${\mathbf{w}}_{{}_j}$ are the $y_i$th and the jth column of $\mathbf{W}$, respectively, ${\mathbf{x}}_i$ is a feature obtained by inputting the ith image into DCNN, and N is the batch size. In (1), ${\mathbf{w}}_{y_i}^T{\mathbf{x}}_i=s_{y_i}cos\left({\theta }_{y_i}\right)$ and ${\mathbf{w}}_j^T{\mathbf{x}}_i=s_{j}cos\left({\theta }_j\right)$. While ${\mathcal{L}}_{\mathrm{Softmax}}$ is known in [23] to have good capability to improve the separability between classes (inter-class separability), it has insufficient capability to compress the feature space of each class (intra-class compactness).

In order to improve the intra-class compactness, many softmax-based loss functions have been studied. The Center loss [25] is given by

$$\begin{array}{c}\hfill {\mathcal{L}}_{\mathrm{Center}}={\mathcal{L}}_{\mathrm{Softmax}}+\frac{\lambda }{2}\sum _{i=1}^N{\parallel {\mathbf{x}}_i-{\mathbf{c}}_{y_i}\parallel }^2,\end{array}$$

(2)

where ${\mathbf{c}}_{y_i}$ is the center position of the $y_i$th class, $\lambda$ is a hyperparameter for balancing the two terms, and $\parallel ·\parallel$ is the $l_2$-norm of a vector. ${\mathcal{L}}_{\mathrm{Center}}$ improves the intra-class compactness by penalizing the distances between features and their corresponding class centers.

The SphereFace [28] is formulated as:

$$\begin{array}{c}\hfill {\mathcal{L}}_{\mathrm{Sphere}}=-\frac{1}{N}\sum _{i=1}^{N}log\frac{e^{\parallel {\mathbf{x}}_i\parallel \psi \left({\theta }_{y_i}\right)}}{e^{\parallel {\mathbf{x}}_i\parallel \psi \left({\theta }_{y_i}\right)}+{\sum }_{j\ne y_i}{e}^{\parallel {\mathbf{x}}_i\parallel cos\left({\theta }_j\right)}},\end{array}$$

where $\psi \left({\theta }_{y_i}\right)={(-1)}^{k}cos\left(m_p{\theta }_{y_i}\right)-2k$, ${\theta }_{y_i}\in \left[\frac{k\pi }{m_p},\frac{(k+1)\pi }{m_p}\right]$ and $k\in [0,m_p-1]$, in which $m_p$ is a multiplicative angular margin. ${\mathcal{L}}_{\mathrm{Sphere}}$ uses an multiplicative angular margin to improve the intra-class compactness on a hypersphere manifold.

The ArcFace [33] has been proposed by:

$$\begin{array}{c}\hfill {\mathcal{L}}_{\mathrm{ArcFace}}=-\frac{1}{N}\sum _{i=1}^{N}log\frac{e^{scos({\theta }_{y_i}+m_{\theta })}}{e^{scos({\theta }_{y_i}+m_{\theta })}+{\sum }_{j\ne y_i}{e}^{scos\left({\theta }_j\right)}},\end{array}$$

where $m_{\theta }$ is an additive angular margin, and s is a scaling constant. ${\mathcal{L}}_{\mathrm{ArcFace}}$ improves the intra-class compactness using an additive angular margin on a hypersphere manifold.

The MagFace [35] is given by:

$$\begin{array}{c}\hfill {\mathcal{L}}_{\mathrm{MagFace}}=\frac{1}{N}\sum _{i=1}^N\left({\mathcal{L}}_{\mathrm{Mag}}+{\lambda }_{g}g\left(\parallel {\mathbf{x}}_i\parallel \right)\right),\end{array}$$

where ${\mathcal{L}}_{\mathrm{Mag}}=-log\frac{e^{scos({\theta }_{y_i}+m_g\left(\parallel {\mathbf{x}}_i\parallel \right))}}{e^{scos({\theta }_{y_i}+m_g\left(\parallel {\mathbf{x}}_i\parallel \right))}+{\sum }_{j\ne y_i}{e}^{scos\left({\theta }_j\right)}}$, in which $m_g\left(\parallel {\mathbf{x}}_i\parallel \right)$ is a strictly increasing function, $g\left(\parallel {\mathbf{x}}_i\parallel \right)$ is a strictly decreasing function and ${\lambda }_g$ is a hyperparameter that controls the trade-off between ${\mathcal{L}}_{\mathrm{Mag}}$ and $g\left(\parallel {\mathbf{x}}_i\parallel \right)$. ${\mathcal{L}}_{\mathrm{MagFace}}$ uses an adaptive margin $m_g\left(\parallel {\mathbf{x}}_i\parallel \right)$ and regularization $g\left(\parallel {\mathbf{x}}_i\parallel \right)$ to improve the intra-class compactness.

3. Orthant Loss

In this paper, we propose an additive orthant loss, termed the Orthant loss, to improve the inter-class separability and the intra-class compactness. The Orthant loss learns the distributions of features away from the origin and compresses the feature spaces by attempting to make the features of the same identity have the same signs. The Orthant loss is proposed by:

$${\mathcal{L}}_{\mathrm{Orth}}=\frac{1}{N}\sum _{i=1}^N\sum _{k=1}^{l}a{\left[\frac{1}{r}\log \right(1+e^{-r\left({\overline{x}}_{i,k}{Q}_1\right(w_{y_{i,k}}-m_c)}\left)\right]}^2$$

(3)

where ${\overline{x}}_{i,k}$ is the kth element of a normalized feature ${\overline{\mathbf{x}}}_i=\frac{{\mathbf{x}}_i}{\parallel {\mathbf{x}}_i\parallel }$, $w_{y_{i,k}}$ is the kth element of ${\mathbf{w}}_{y_i}$ and $Q_1\left(w\right)$ is the 1-bit quantization function, i.e., $Q_1\left(w\right)=+1$ if $w>0$ and $-1$ otherwise. In addition, $m_c$ is an additive margin, a is a hyperparamter to adjust the size of the loss, r is a hyperparameter to control the slope of the rescaled softplus function, l is the feature dimension and N is the batch size. The proposed algorithm of ${\mathcal{L}}_{\mathrm{Orth}}$ is given in Algorithm 1. The reasoning behind the design of Orthant loss is to make each feature element have the same sign as the weight vector of each class by making ${\overline{x}}_{i,k}{Q}_1\left(w_{y_{i,k}}\right)>0$ using 1-bit quantization and element-wise operation, which improves the intra-class compactness. In addition, the Orthant loss makes the magnitude of each feature element larger than $m_c$ to improve the inter-class separability by using the additive margin and the rescaled softplus function. Since ${\mathcal{L}}_{\mathrm{Orth}}$ only considers the feature and weight vectors of target identity $y_i$ in (3), we combine ${\mathcal{L}}_{\mathrm{Orth}}$ with a softmax-based loss ${\mathcal{L}}_{\mathrm{Soft}}$, taking into account non-target identities $j\ne y_i$, which gives a final loss:

$${\mathcal{L}}_{\mathrm{Final}}={\mathcal{L}}_{\mathrm{Soft}}+{\mathcal{L}}_{\mathrm{Orth}}.$$

(4)

Algorithm 1 The pseudo-code of Orthant loss ${\mathcal{L}}_{\mathrm{Orth}}$

Input: Feature vectors ${\mathbf{x}}_i$, Weight matrix $\mathbf{W}=({\mathbf{w}}_1,\cdots ,{\mathbf{w}}_M)$, Ground-Truth ID $y_i$, Hyperparameters $a,r$ and $m_c$. 1: Normalize the feature vectors by ${\overline{\mathbf{x}}}_i=\frac{{\mathbf{x}}_i}{\parallel {\mathbf{x}}_i\parallel }$ 2: Clone and detach the weight matrix $\mathbf{W}$ to prevent gradient computation by $\mathbf{W}=\mathbf{W}.\mathrm{clone}\left(\right).\mathrm{detach}\left(\right)$ 3: Quantize the chosen weight vectors corresponding to $y_i$ by ${\tilde{\mathbf{w}}}_{y_i}=\left[Q_1\left(w_{y_i,1}\right),Q_1\left(w_{y_i,2}\right),\cdots ,Q_1\left(w_{y_i,l}\right)\right]$ 4: Calculate the element-wise magnitudes including margin $m_c$ by ${\mathbf{u}}_i={\tilde{\mathbf{w}}}_{y_i}⨀{\overline{\mathbf{x}}}_i-m_c$, where ⨀ is the element-wise product of two vectors. 5: Calculate the proposed loss in (3) using the rescaled softplus function for a batch by ${\mathcal{L}}_{\mathrm{Orth}}=\frac{1}{N}{\sum }_{i=1}^N{\sum }_{k=1}^{l}a{\left[\frac{1}{r}log\left(1+e^{-r{u}_{i,k}}\right)\right]}^2$ Output: Loss score ${\mathcal{L}}_{\mathrm{Orth}}$

In what follows, we describe the features of the Orthant loss.

Rescaled softplus function. To learn the distributions of features to be well separated, we use the rescaled softplus function [43] $f_s\left(x\right)=\frac{1}{r}log\left(1+e^{-rx}\right)$, where r controls the slope of $f_s\left(x\right)$. As r goes to infinity, $f_s\left(x\right)$ converges approximately into the hinge function [44] $f_h\left(x\right)=max(0,-x)$. In $f_h\left(x\right)$, if $x\ge 0$, learning no longer proceeds as the gradient becomes 0. On the other hand, in $f_s\left(x\right)$, even if $x\ge 0$, learning is possible so that x is farther away from zero since the gradient of $f_s\left(x\right)$ is not zero. Therefore, we use $f_s\left(x\right)$ to improve the inter-class separability by distinguishing features around the origin.

Additive center separating margin.Figure 2a shows that a softmax-based loss function [28,29,30,31,32,33,34,35] learns the distribution of features centered on ${\mathbf{w}}_1$ and ${\mathbf{w}}_2$ to improve the intra-class compactness using an angular margin $m_{\theta }$, respectively. However, since features with small magnitudes can easily move to the feature spaces of other classes, the softmax-based loss function using an angular margin has difficulty in distinguishing these features correctly. To overcome this problem, this paper uses an additive margin $m_c$ to improve the inter-class separability by pushing the features away from the origin, as illustrated in Figure 2b. In the Center loss [25], which also uses an additive term, both ambiguous and discriminative features can be gathered at the center of each class without considering the feature magnitudes. On the other hand, both ambiguous and discriminative features move away from the origin in the Orthant loss, which improves the inter-class separability by increasing their magnitudes.

One-bit quantization and element-wise operation. To improve the intra-class compactness, we quantize each weight vector of the last fully connected layer and perform element-wise product on the normalized feature vectors. When the weight vectors are not quantized, the signs of some elements of normalized feature vectors can be easily changed if the magnitudes of the corresponding elements of the weight vectors are small. Therefore, we train the DCNN model so that the signs of feature and weight vectors are identical by multiplying the normalized feature by the 1-bit quantized weight vector. Then, we can improve the intra-class compactness through the element-wise sign consistency between feature and quantized weight vectors.

Despite the low computational cost, the 1-bit quantization may reduce the recognition accuracy due to the inaccurate gradient update [45]. Since 1-bit quantization has zero gradients on positive and negative sides, weight vectors of the last fully connected layer are updated in a wrong direction using a surrogate gradient function such as hard hyperbolic tangent (tanh) function. To maintain the accuracy while reducing the computational cost, we exclude the 1-bit quantized weight vectors from the gradient calculation. In other words, the weight vectors are updated using the softmax-based loss ${\mathcal{L}}_{\mathrm{Soft}}$ without the Orthant loss ${\mathcal{L}}_{\mathrm{Orth}}$ in the back-propagation process. Note that the Orthant loss ${\mathcal{L}}_{\mathrm{Orth}}$ is combined to a softmax-based loss when updating the set of parameters $\mathcal{W}$ of the DCNN.

Squared loss function. As r increases, the gradient for a misclassified feature in the rescaled softplus function $f_s\left(x\right)$ is approximately $-1$. Since $f_s\left(x\right)$ has the gradient of $-1$ for both easy and hard features to distinguish, features do not quickly converge. To improve the convergence speed, we introduce a squared loss to give a larger weight to a hard feature. In many studies [44,46], the squared hinge function has been used to give a larger weight to misclassified features. Inspired by this idea, we introduce a squared softplus function to expect fast convergence of misclassified features.

In the learning process, 1-bit quantization and element-wise operation of the Orthant loss make feature vectors have the same sign as the weight vectors of the corresponding class. However, feature vectors may converge to an orthant of the corresponding weight vector before the distances between weight vectors are sufficiently large, which fixes the sign of the weight vectors prematurely. In other words, since the speed at which feature vectors converge to the corresponding weight vector is faster than the speed at which the weight vectors move away from each other, the sign of the weight vectors may be fixed before the distance between the weight vectors increases. To avoid a premature decision of weight vectors, we apply the Orthant loss after the learning progresses sufficiently. In this paper, we apply the Orthant loss from 20,000th iteration.

Figure 3 displays 2D and 3D plots of features for the ArcFace and the ArcOrthFace, respectively. We use the LResNet18E-IR [33], where $a=2$, $r=30$, and $N=128$. In addition, we use $s_{y_i}=s_j=4$, $m_{\theta }=0.1$ and $m_c=1/\sqrt{2}$ for 2D features of Figure 3a,b and $s_{y_i}=s_j=8$, $m_{\theta }=0.2$ and $m_c=1/\sqrt{3}$ for 3D features of Figure 3c,d, respectively. For training and testing, we use a total of 2907 images of top four classes and a total of 4186 images of the top six classes sorted by the number of images in the CASIA-Webface dataset [47] for 2D and 3D features, respectively. We assume that there are $d=4$ and $d=6$ identities for 2D and 3D features, respectively. Here, we choose the number of identities by $d\le 2^l$, which is valid in practice. In Figure 3, we project features outside the unit sphere onto the sphere, where the features do not cause inter-class ambiguity due to the sufficiently high magnitude. In this example, Figure 3a,b have 2907 features, respectively, while Figure 3c,d have 4186 features, respectively.

In Figure 3b,d, we can see that ArcOrthFace has fewer features in the unit sphere by attempting to push features away from the origin, compared to ArcFace. It suggests that ArcOrthFace has improved inter-class separability compared to ArcFace, since the fewer features cause less inter-class ambiguity around the origin. In specific, for 2D and 3D features, we can observe that ArcOrthFace reduces the number of features inside the unit sphere to 278 and 100, while 604 and 344 features remain by ArcFace, respectively. As a result, ArcOrthFace, which learns the distributions of features away from the origin, shows the improved inter-class separability, compared to ArcFace.

Figure 3b,d also show that ArcOrthFace has the better intra-class compactness than ArcFace by compressing feature spaces further at the center of each class through 1-bit quantization and element-wise operation. The feature spaces learned by ArcOrthFace are distributed around 1-bit quantized vectors, i.e., ${\{-1,+1\}}^l$, which demonstrates the improved intra-class compactness, compared to ArcFace.

4. Experimental Results

Preprocessing. We use the datasets given in

Table 1. Datasets for Training and Testing.

Datasets	Number of Identities	Number of Images	Types
CASIA-Webface [47]	10 K	$0.5$ M	Training
LFW [49]	5749	13,233	Validation
CFP-FP [40]	500	7000	Validation
AgeDB-30 [41]	568	16,488	Validation
MegaFace(pro.) [42]	530	3530	Testing
MegaFace(dis.) [42]	690 K	1 M	Testing

for training and testing several loss functions. All the preprocessed images used for training and testing are obtained by [33] for fair comparisons. For preprocessing of each face image, five landmarks are acquired using the multi-task cascaded convolutional networks (MTCNN) [48], alignment is performed using similarity transformation, and then cropped to the image with a size of $112\times 112$. Each RGB image having a range of $[0,255]$ is normalized to have a range of $[-1,1]$ and exceptionally to have a range of $[0,1]$ on the MagFace [35].

Training.

Table 2. Training Setting and Hyperparamters.

Network model	LResNet50E-IR [33]
Batch size N	512
Feature length l	512
Optimizer	SGD Optimizer
Momentum	$0.9$
Weight decay	$0.0005$
Total iteration	$32,000$
Scaling constant s	64
Margin size $m_c$	$1/\sqrt{l}$
Slope control factor r	30
Margin control factor a	2

presents common training setting and hyperparameters. In order to fairly compare verification and identification accuracies that depend only on loss functions, we fix the network structure and training dataset. The recognition accuracies for the methods of comparison may be different from those in the original papers, which used their own training datasets and network structures. The learning rate starts at $0.1$ and is changed to $0.01$ at 20,000th iteration and $0.001$ at 28,000th iteration, respectively. We use a horizontal random flip for data augmentation. For the hyperparameters of Center loss, SphereFace, ArcFace and MagFace not shown in Table 2, we use the recommended values of [25,28,33,35], respectively. In

Table 3. Verification Accuracy (%) on Several Benchmarks with Various Loss Functions.

Loss Function	LFW	CFP-FP	AgeDB-30
Softmax	$99.27$	$94.67$	$92.90$
SoftOrthFace	$99.30$	$94.94$	$92.60$
N-Softmax	$98.47$	$89.81$	$89.12$
N-SoftOrthFace	$98.75$	$92.96$	$90.62$
ArcFace [33]	$99.43$	$95.57$	$94.95$
ArcCentFace	99.45	$95.41$	$95.13$
ArcOrthFace	$99.42$	95.73	95.18
MagFace [35]	$99.18$	$94.97$	$94.62$
MagCentFace	$99.32$	$95.33$	$94.88$
MagOrthFace	$99.42$	$94.80$	$94.67$
Center loss [25]	$99.28$	$94.84$	$92.05$
SphereFace [28]	$99.12$	$94.49$	$92.68$

Note: The accuracies in bold represent the highest ones among the listed loss functions.

and

Table 4. Identification and Verification Results (%) on MegaFace with Various Loss Functions.

Loss Function	Rank1 Accuracy	Verification Accuracy
ArcFace	$91.70$	$93.95$
ArcCentFace	$91.39$	$93.85$
ArcOrthFace	91.77	94.32
MagFace	$89.54$	$92.57$
MagCentFace	$91.22$	$93.96$
MagOrthFace	$89.79$	$92.83$
Center loss	$79.96$	$83.45$
SphereFace	$81.95$	$88.06$

Note: The accuracies in bold represent the highest ones among the listed loss functions.

, the loss function replacing the suffix ‘max’ or ‘Face’ with ‘OrthFace’, respectively, indicates a combined one with the Orthant loss. In addition, the loss function replacing the suffix ‘Face’ in ArcFace and MagFace with ‘CentFace’ indicates a combined one with the Center loss.

Testing. In the testing phase, features are extracted using the trained DCNN model, as shown in Figure 1. To make a feature from a testing image, we add the features obtained by original and flipped images and then normalize the combined feature. To evaluate the accuracy for LFW [49], CFP-FP [40] and AgeDB-30 [41], we measure the mean accuracy of 10 subsets of each dataset using the leave-one-out cross validation [50], following the unrestricted protocol with labeled outside data [49]. For the MegaFace dataset, we investigate the true positive rate (TPR) at ${10}^{-6}$ false positive rate (FPR) for face verification and rank-1 accuracy [3] for face identification with ${10}^6$ distractors, respectively.

Figure 4 shows the training loss of the new loss functions combined with the proposed Orthant loss function. We can observe that the new combined loss functions reduce the training loss to make a good-fitting model as training progresses. In particular, at the 20,000th and 28,000th iterations, we use a learning rate decay method to improve the training speed and model accuracy. In addition, it can be confirmed that the training loss does not diverge or converge to zero to avoid underfitting and overfitting, respectively, since we use dropout and weight decay techniques.

Table 3 shows the verification accuracy of various loss functions for LFW, CFP-FP and AgeDB-30, respectively. Table 3 demonstrates that the loss functions using Orthant loss improve the accuracy over the original ones in most benchmarks. This means that Orthant loss can improve the inter-class separability and the intra-class compactness by combining it with softmax-based loss functions. Additionally, ArcOrthFace, which combines ArcFace with Orthant loss, shows the highest accuracy on CFP-FP and AgeDB-30. In addition, it shows that SoftOrthFace has higher accuracy than Center loss, which demonstrates that Orthant loss has more discriminative capability than the additive term of Center loss. Although the accuracies of ArcCentFace and MagCentFace have increased compared to original ones, it can be seen that ArcOrthFace has the best accuracy in most benchmarks.

Table 4 shows identification and verification results of various loss functions for MegaFace. Similar to the results in Table 3, it can be seen that the loss functions using Orthant loss improve identification and verification accuracies. It demonstrates that Orthant loss makes feature spaces distinguishable by improving the inter-class separability and the intra-class compactness. In addition, it shows that MagOrthFace has higher accuracy than MagFace, which demonstrates that Orthant loss can improve the distinguishability of adaptively learned feature spaces of MagFace. In Table 4, it is shown that MagCentFace further improves the distinguishability of the adaptively learned feature spaces compared to MagOrthFace. However, ArcOrthFace shows the highest identification and verification accuracies compared to all the other loss functions.

5. Conclusions

In this paper, we proposed an additive Orthant loss to improve the inter-class separability and the intra-class compactness. The Orthant loss uses the rescaled softplus function and an additive margin to move features away from the origin. In addition, the Orthant loss uses 1-bit quantization and element-wise operation to map features to an orthant of each class. Experimental results demonstrated that loss functions combined with Orthant loss improve the recognition accuracy in most benchmarks compared to original loss functions. In particular, we showed that the loss function combining ArcFace with Orthant loss has the best accuracy on MegaFace, a challenging test benchmark containing more than 1 million distractors. In conclusion, Orthant loss, combined with a softmax-based loss function, can improve the accuracy of deep face recognition by improving the intra-class compactness and the inter-class separability. We believe that the proposed Orthant loss can be used as a loss function in recognition systems using various human characteristics, e.g., fingerprint, iris, and palmprint, which will require a future study for each recognition system.