# Optimal Face-Iris Multimodal Fusion Scheme

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Unimodal Biometric Systems

#### 2.2. Fusion Techniques on Face and Iris Biometrics

#### 2.2.1. Feature Level Fusion

#### 2.2.2. Match Score Level Fusion

#### 2.2.3. Decision Level Fusion

#### 2.2.4. Architecture of the Proposed Scheme

#### 2.2.5. BSA Feature Selection Algorithm

Algorithm 1 General Structure of Backtracking Search Algorithm [25]. |

1. Initialization |

Repeat |

2. Selection-I |

Generation of Trial Population |

3. Mutation |

4. Crossover |

End |

5. Selection-II |

Until stopping conditions are met |

_{best}and the global optimum value is updated to be fitness

_{Pbest}.

## 3. Results and Discussion

## 4. Conclusions

## Author Contributions

## Conflicts of Interest

## References

- Liau, H.F.; Isa, D. Feature selection for support vector machine-based face-iris multimodal biometric system. Expert Syst. Appl.
**2011**, 38, 11105–11111. [Google Scholar] [CrossRef] - Proenca, H.P. Towards Non-Cooperative Biometric Iris Recognition. Ph.D. Thesis, University of Beira Interior Department of Computer Science, Covilhã, Portugal, 2006. [Google Scholar]
- Nandakumar, K.; Chen, Y.; Dass, S.C.; Jain, A.K. Likelihood ratio-based biometric score fusion. IEEE Trans. Pattern Anal. Mach. Intell.
**2008**, 30, 342–347. [Google Scholar] [CrossRef] [PubMed] - Ross, A.; Nandakumar, K.; Jain, A.K. Handbook of Multibiometrics; Springer-Verlag: Berlin, Germany, 2006. [Google Scholar]
- Raghavendra, R.; Dorizzi, B.; Rao, A.; Kumar, G.H. Designing efficient fusion schemes for multimodal biometric system using face and palmprint. Pattern Recognit.
**2011**, 44, 1076–1088. [Google Scholar] [CrossRef] - Lumini, A.; Nanni, L. Over-complete feature generation and feature selection for biometry. Expert Syst. Appl.
**2008**, 35, 2049–2055. [Google Scholar] [CrossRef] - Gökberk, B.; Okan İrfanoğlu, M.; Akarun, L.; Alpaydın, E. Learning the best subset of local features for face recognition. Pattern Recognit.
**2007**, 40, 1520–1532. [Google Scholar] [CrossRef] - Eskandari, M.; Toygar, Ö.; Demirel, H. Feature Extractor Selection for Face-Iris Multimodal Recognition. Signal Image Video Process.
**2014**, 8, 1189–1198. [Google Scholar] [CrossRef] - Zhang, D.; Jing, X.; Yang, J. Biometric Image Discrimination (BID) Technologies; IGI Global: Hershey, PA, USA, 2006. [Google Scholar]
- Nandakumar, K. Integration of Multiple Cues in Biometric Systems. Master’s Thesis, Michigan State University, East Lansing, MI, USA, 2005. [Google Scholar]
- Lam, L.; Suen, C.Y. Application of Majority Voting to Pattern Recognition: An Analysis of Its Behavior and Performance. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum.
**1997**, 27, 553–568. [Google Scholar] [CrossRef] - Lam, L.; Suen, C.Y. Optimal Combination of Pattern Classifiers. Pattern Recognit. Lett.
**1995**, 16, 945–954. [Google Scholar] [CrossRef] - Xu, L.; Krzyzak, A.; Suen, C.Y. Methods for Combining Multiple Classifiers and their Applications to Handwriting Recognition. IEEE Trans. Syst. Man Cybernet.
**1992**, 22, 418–435. [Google Scholar] [CrossRef] - Daugman, J. Combining Multiple Biometrics. Available online: http://www.cl.cam.ac.uk/users/jgd1000/combine/combine.html (assessed on 28 October 2016).
- Eskandari, M.; Toygar, Ö. Fusion of face and iris biometrics using local and global feature extraction methods. Signal Image Video Process.
**2014**, 8, 995–1006. [Google Scholar] [CrossRef] - Wang, F.; Han, J. Multimodal biometric authentication based on score level fusion using support vector machine. Opto Electron. Rev.
**2009**, 17, 59–64. [Google Scholar] [CrossRef] - Vasta, M.; Singh, R.; Noore, A. Integrating image quality in 2v-SVM biometric match score fusion. Int. J. Neural Syst.
**2007**, 17, 343–351. [Google Scholar] - Eskandari, M.; Toygar, Ö.; Demirel, H. A new approach for Face-Iris multimodal biometric recognition using score fusion. Int. J. Pattern Recognit. Artif. Intell.
**2013**, 27. [Google Scholar] [CrossRef] - Wang, Y.; Tan, T.; Wang, Y.; Zhang, D. Combining face and iris biometric for identity verification. In Proceedinmgs of the 4th International Conference on Audio and Video Based Biometric Person Authentication, Guildford, UK, 9–11 June 2003; pp. 805–813.
- Sim, H.M.; Asmunia, H.; Hassan, R.; Othman, R.M. Multimodal biometrics: Weighted score level fusion based on non-ideal iris and face images. Expert Syst. Appl.
**2014**, 41, 5390–5404. [Google Scholar] [CrossRef] - Eskandari, M.; Toygar, Ö. Selection of Optimized features and weights on face-iris fusion using distance images. Comput. Vis. Image Underst.
**2015**, 137, 63–75. [Google Scholar] [CrossRef] - Jing, X.Y.; Yao, Y.F.; Yang, J.Y.; Li, M.; Zhang, D. Face and palmprint pixel level fusion and kernel DCV-RBF classifier for small sample biometric recognition. Pattern Recognit.
**2007**, 40, 3209–3224. [Google Scholar] [CrossRef] - Yao, Y.; Jing, X.; Wong, H. Face and palmprint feature level fusion for single sample biometric recognition. Neurocomputing
**2007**, 70, 1582–1586. [Google Scholar] [CrossRef] - Xiao, Z.; Guo, C.; Yu, M.; Li, Q. Research on log gabor wavelet and its application in image edge detection. In Proceedings of 6th International Conference on Signal Processing (ICSP-2002), Beijing, China, 26–30 August 2002; pp. 592–595.
- Civicioglu, P. Backtracking Search Optimization Algorithm for numerical optimization problems. Appl. Math. Comput.
**2013**, 219, 8121–8144. [Google Scholar] [CrossRef] - Jain, A.K.; Ross, A. Learning User-specific Parameters in a Multibiometric System. In Proceedings of International Conference on Image Processing, New York, NY, USA, 22–25 September 2002; pp. 57–60.
- Tao, Q.; Veldhuis, R. Threshold-Optimized decision-level fusion and its application to biometrics. Pattern Recognit.
**2009**, 42, 823–836. [Google Scholar] [CrossRef] - Biometrics Ideal Test. Available online: http://biometrics.idealtest.org/dbDetailForUser.do?id=4 (assessed on 30 August 2013).
- Patil, H.; Kothari, A.; Bhurchandi, K. 3-D face recognition: Features, databases, algorithms and challenges. Artif. Intell. Rev.
**2015**, 44, 393–441. [Google Scholar] [CrossRef] - Subburaman, V.B.; Marcel, S. Alternative search techniques for face detection using location estimation and binary features. Comput. Vis. Image Underst.
**2013**, 117, 551–570. [Google Scholar] [CrossRef] - Gul, G.; Hou, Z.; Chen, C.; Zhao, Y. A dimensionality reduction method based on structured sparse representation for face recognition. Artif. Intell. Rev.
**2016**. [Google Scholar] [CrossRef] - Bowyer, K.W.; Hollingsworth, K.; Flynn, P.J. Image understanding for iris biometrics: A survey. Comput. Vis. Image Underst.
**2008**, 110, 281–307. [Google Scholar] [CrossRef] - Matey, J.R.; Broussard, R.; Kennell, L. Iris image segmentation and sub-optimal images. Image Vis. Comput.
**2010**, 28, 215–222. [Google Scholar] [CrossRef] - Galbally, J.; Ross, A.; Gomez-Barrero, M.; Fierrez, J.; Ortega-Garcia, J. Iris image reconstruction from binary templates: An efficient probabilistic approach based on genetic algorithms. Comput. Vis. Image Underst.
**2013**, 117, 1512–1525. [Google Scholar] [CrossRef] - Neves, J.; Narducci, F.; Barra, S.; Proença, H. Biometric recognition in surveillance scenarios: A survey. Artif. Intell. Rev.
**2016**. [Google Scholar] [CrossRef] - Huang, C.; Ding, X.; Fang, C. Pose robust face tracking by combining view-based AAMs and temporal filters. Comput. Vis. Image Underst.
**2012**, 116, 777–792. [Google Scholar] [CrossRef] - Active Appearance Modeling. Available online: http://cvsp.cs.ntua.gr/software/AAMtools/ (accessed on 20 April 2013).
- Pujol, P.; Macho, D.; Nadeu, C. On real-time mean-and- variance normalization of speech recognition features. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP2006), Toulouse, France, 14–19 May 2006; pp. 773–776.
- Arora, S.; Londhe, N.D.; Acharya, A.K. Human Identification based on Iris Recognition for Distance Images. Int. J. Comput. Appl.
**2012**, 45, 32–39. [Google Scholar] - Masek, L.; Kovesi, P. MATLAB Source Code for a Biometric Identification System Based on Iris Patterns. Bachelor’s thesis, the School of Computer Science and Software Engineering, The University of Western Australia, Crawley, Australia, 2003. [Google Scholar]
- Tan, C.W.; Kumar, A. A Unified Framework for Automated Iris Segmentation Using Distantly Acquired Face Images. IEEE Trans. Image Process.
**2012**, 21, 4068–4079. [Google Scholar] [CrossRef] [PubMed]

**Figure 1.**The block diagram of face-iris feature level fusion. LDA: Linear Discriminant Analysis; BSA: Backtracking Search Algorithm.

**Figure 2.**The block diagram of face-iris score level fusion. W1, W2 and W3: assigned weights for different modalities.

**Figure 3.**The block diagram of face-iris decision level fusion. ROC: Receiver Operator Characteristics; FRR: False Accept Rate; CRR: Correct Reject Rate

Face | Iris | ||
---|---|---|---|

Pose variations | √ | Occlusion-eyelash | √ |

Facial expressions | √ | Occlusion-eyelid | √ |

Occlusion-glasses | √ | Occlusion-glasses | √ |

Occlusion-mustache | √ | Different noise factors (reflections, contrast, luminosity, off angle, rotation, blurring and focus problems) | √ |

Distance images | √ | Distance images | √ |

**Table 2.**Verification Performance and Minimum Total Error Rates of Unimodal Systems. TER: Total Error Rate; GAR: Genuine Acceptance Rate; FAR: False Acceptance Rate.

Left Iris | Right Iris | Face | |||
---|---|---|---|---|---|

Minimum TER (%) | GAR (at 0.01% FAR) | Minimum TER (%) | GAR (at 0.01% FAR) | Minimum TER (%) | GAR (at 0.01% FAR) |

6.93 ± 1.24 | 69.67 ± 4.10 | 6.24 ± 1.01 | 71.55 ± 3.35 | 2.88 ± 0.75 | 83.22 ± 1.63 |

**Table 3.**Verification Performance and Minimum Total Error Rates of Multimodal Biometric Systems at Feature Level Fusion.

Scheme | Minimum TER (%) | GAR (at 0.01% FAR) |
---|---|---|

Feature fusion scheme implemented in Figure 1a using left iris without BSA | 2.15 ± 0.64 | 87.56 ± 2.94 |

Feature fusion scheme implemented in Figure 1a using left iris with BSA | 2.06 ± 0.71 | 88.37 ± 2.68 |

Feature fusion scheme implemented in Figure 1a using right iris without BSA | 2.06 ± 0.98 | 88.44 ± 3.05 |

Feature fusion scheme implemented in Figure 1a using right iris with BSA | 1.84 ± 0.73 | 90.16 ± 2.88 |

Feature fusion scheme implemented in Figure 1b using both irises without BSA | 1.03 ± 0.46 | 92.87 ± 1.65 |

Feature fusion scheme implemented in Figure 1b using both irises with BSA | 0.86 ± 0.34 | 94.91 ± 1.83 |

**Table 4.**Verification Performance and Minimum Total Error Rates of Multimodal Biometric Systems at Score Level Fusion.

Scheme | Minimum TER (%) | GAR (at 0.01% FAR) |
---|---|---|

Score fusion scheme implemented in Figure 2a using left iris | 2.10 ± 0.63 | 88.17 ± 1.75 |

Score fusion scheme implemented in Figure 2a using right iris | 2.01 ± 0.77 | 88.76 ± 2.18 |

Score fusion scheme implemented in Figure 2b using both irises | 0.81 ± 0.48 | 95.00 ± 2.03 |

**Table 5.**Verification Performance and Minimum Total Error Rates of Multimodal Biometric Systems at Decision Level Fusion.

Scheme | Minimum TER (%) | GAR (at 0.01% FAR) |
---|---|---|

Decision fusion scheme implemented in Figure 3a using left iris | 0.96 ± 0.28 | 91.15 ± 2.38 |

Decision fusion scheme implemented in Figure 3a using right iris | 0.94 ± 0.71 | 92.92 ± 3.01 |

Decision fusion scheme implemented in Figure 3b using both irises | 0.58 ± 0.32 | 96.87 ± 1.83 |

**Table 6.**Verification Performance and Minimum Total Error Rates of Proposed Multimodal Biometric Systems and Different Levels of Fusions.

Scheme | Minimum TER (%) | GAR (at 0.01% FAR) |
---|---|---|

Feature fusion scheme implemented in Figure 1b using both irises-with BSA | 0.86 ± 0.34 | 94.91 ± 1.83 |

Score fusion scheme implemented in Figure 2b using both irises | 0.81 ± 0.48 | 95.00 ± 2.03 |

Decision fusion scheme implemented in Figure 3b using both irises | 0.58 ± 0.32 | 96.87 ± 1.83 |

Proposed Scheme | 0.27 ± 0.41 | 98.93 ± 1.11 |

**Table 7.**Verification Performance and Minimum Total Error Rates of Different Multimodal Biometric Systems and Proposed Scheme. PSO: Particle Swarm Optimization; SVM: Support Vector Machine.

State-of-the-Art Fusion Methods on Face and Iris | Minimum TER (%) | GAR (at 0.01% FAR) |
---|---|---|

Weighted Sum Rule [15] | 2.01 ± 0.77 | 88.76 ± 2.18 |

Score concatenation [18] | 1.49 ± 0.48 | 89.53 ± 1.67 |

SVM [16] | 1.56 ± 0.71 | 91.92 ± 2.03 |

PSO and SVM [1] | 1.06 ± 0.39 | 94.22 ± 1.48 |

Proposed scheme | 0.27 ± 0.41 | 98.93 ± 1.11 |

© 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Sharifi, O.; Eskandari, M.
Optimal Face-Iris Multimodal Fusion Scheme. *Symmetry* **2016**, *8*, 48.
https://doi.org/10.3390/sym8060048

**AMA Style**

Sharifi O, Eskandari M.
Optimal Face-Iris Multimodal Fusion Scheme. *Symmetry*. 2016; 8(6):48.
https://doi.org/10.3390/sym8060048

**Chicago/Turabian Style**

Sharifi, Omid, and Maryam Eskandari.
2016. "Optimal Face-Iris Multimodal Fusion Scheme" *Symmetry* 8, no. 6: 48.
https://doi.org/10.3390/sym8060048