Multibiometrics fusion using $Acz{\acute{e}}l$-Alsina triangular norm

Abstract

Fusing the scores of multibiometrics is a very promising approach to improve the overall system's accuracy and the verification performance. In recent years, there are several approaches towards studying score level fusion of several biometric systems. However, most of them does not consider the genuine and imposter score distributions and result in a higher equal error rate usually. In this paper, a novel score level fusion approach of different biometric systems (dual iris, thermal and visible face traits) based on $Acz{\acute{e}}l$-Alsina triangular norm is proposed. It achieves higher identification performance as well as acquires a closer genuine distance and larger imposter distance. The experimental tests are conducted on a virtual multibiometrics database, which merges the challenging CASIA-Iris-Thousand database with noisy samples and the NVIE face database with visible and thermal face images. The rigorous results suggest that significant performance improvement can be achieved after the implementation of multibiometrics. The comparative experiments also ascertain that the proposed fusion approach outperforms the state-of-art verification performance.

1. Introduction

1.1 Multibiometrics fusion

Unimodal biometric systems have proven their superior performance to adapt the increasing demand of accurate and efficient identification in such a rapid developing society [1]. These systems rely on the person physiological traits such as visible face, voice, thermal face [2], fingerprint [4], iris [3], vein, plamprint [5], lip [6], ear [7], etc. However, unimodal biometric systems have several inherent problems such as large intraclass variations, nonuniversality, and spoofing attacks [8, 9, 10, 11]. The multibiometrics, which is able to effectively overcome most of the above problems, is attracting a significant attention from the researchers in multifaceted disciplines [12, 13, 14]. It has been adopted by some identification systems (e.g., FBI’s IAFIS, the Department of Homeland Security’s US-VISIT, and the Government of India’s UID) [15].

In order to obtain optimal performance, the type of information and the fusion scheme should be selected deliberately in multibiometrics [16]. Iris verification accuracy is affected by some typical noise sources such as reflection spots, eyelash or eyelid obstructions [17], but still could provide an excellent authentication performance. Though visible spectrum face and thermal face imagery recognition performance suffer from uncontrolled operating conditions [18, 19], the acquiring convenience and anti-spoofing property should be paid much more attention. Considering the performance complementation, acquisition convenience and application security, we focus on the fusion of dual iris, visible and thermal face imagery to compose a new multibiometrics system. In our work, the features of iris are represented in binary and face features are in real numbers, so score level fusion is the best choice for this fusion scheme [12].

1.2 Review of the related work

Literature on score level fusion of several biometric systems is quite rich. They can be divided into three categories: (1) Transformation-based fusion: the match scores are first normalized (transformed) into a common domain and then combined using product, sum, max, min and T-norm rules. Chang et al. [20] examine the fusion of face imagery in 2D, 3D and infrared samples of one subject and the weight sum, product and minimum rules are applied. Tchamova et al. [21] propose triangular-conorm/norm (T-conorm/T-norm) based combination rule, which is defined in the framework of Dempster-Shafer Theory [22], to fuse the multibiometrics evidences. Quost et al. [23] investigate the Dempster-Shafer framework and then use the T-norm (Frank) rule to combine non-independent classifiers. Hanmandlu et al. [24] propose different T-norms (Hamacher, Yager, Frank, Schweizer and Sklar, and Einstein product) on three databases to confirm the effectiveness of score level fusion. Mamode Khan [25] combines the scores obtained from the different modalities of dorsal and palmar veins using wight sum rule. The choice of the combination weights is data-dependent and requires extensive empirical evaluation.

(2) Classifier-based fusion: scores from multiple matchers are treated as a feature vector and a classifier is constructed to discriminate genuine and impostor scores.

Wang et al. [26] treat the face and iris recognition scores as a two-dimensional feature vector, and apply the LDA and neural network for classification. He et al. [27] implement sum rule-based and support vector machines (SVM) based methods for the multibiometric system of fingerprint, face and finger vein. They also propose a new robust normalization scheme derived from min-max normalization scheme [28].

(3) Density-based fusion: this is based on the likelihood ratio test and it requires explicit estimation of genuine and impostor match score densities.

Dass et al. [29] estimate generalized densities using copular models, which consider the dependence between the multiple modalities. Nandakumar et al. [30] propose a framework combination, in which the density distributions of genuine and impostor match scores are modeled as finite Gaussian Mixture Model (GMM). Sim and Lee [31] use Logistic Linear Regression (LLR) to estimate the model parameters of multi pieces of speech segments for identification. Nanni et al. [32] also present the GMM to modal the parameters of the genuine and impostor score densities, and test SVM classifier on different biometric verification systems (related to fingerprints, palms, fingers, hand geometry and faces).

1.3 Limitations and scope of the present work

Generally, classifier-based methods need data training and then classification. The unbalanced genuine and imposter score sets result in unbalanced training. Then, the indicates of False Matching Rate (FMR) and False Non Matching Rate (FNMR) are insufficient to capture. Density-based algorithms have the advantage that it directly achieves optimal performance at any desired operating point. However, the density function usually is unknown and it is hard to estimate accurately. We prefer transformation-based fusion approaches ignoring the feature representation. But, traditional transformation-based approaches haven’t consider the distribution properties of genuine and imposter scores, and couldn’t make a closer intra-class and a farther inter-class distance.

Here we aim to elude the drawbacks of the previous works on score level fusion of multibiometrics in order to improve the overall system’s accuracy and the recognition performance.

1.4 Contribution and organization of the paper

In this paper, we explore and study some T-norms for the score level fusion and elaborate them on the merged multibiometric database of dual iris, visible and thermal face imagery. Further, a novel score level fusion approach for multibiometics is proposed. The proposed method based on Acze´l-Alsina triangular norm (AA T-norm), which is suitable for the genuine and imposter score distributions of our multibiometrics database. In addition, it acquires a closer genuine distance and larger imposter distance in accordance with high recognition performance. The comparison suggests that the proposed approach leads to significant results than other approaches proposed in the literatures.

The remains of the paper is organized as follows. Section 2 introduces the proposed score level fusion approach. Section 3 is devoted to experimental setup. In Section 4, the experimental results including comparisons with other approaches and analysis are presented in detail. Finally, in Section 5, the conclusion is drawn.

 

2. The proposed score level fusion approach

2.1 An overview of the proposed Aczél-Alsina T-norm

In this section, we introduce our proposed fusion method. During fusion phase, the fusion function should be basically continuous.

The T-norm is a function defined in fuzzy set/logic theory in order to represent the intersection between two particular fuzzy sets and the AND fuzzy logical operator, respectively. If one extends the T-norm to data fusion, it will be a substitute for the conjunctive and disjunction operators, respectively. These T-norms are the function that map the unit square into the unit interval: T (x, y): [0,1]2 →[0,1].

After investigating the sum rule and these T-norms, we find out that these T-norms can make the fusion of genuine scores (low value) closer comparing with sum rule, but can’t make the imposter scores (high value) farther. The phenomenon can be seen from Fig. 1(a) to Fig. 1(e). If we won’t consider this problem, the fusion performance will be unstable. We need find some function which has some part lower than sum function and some part higher than it. There should be a fusion function F and a critical value t, which makes:

Fig. 1.Graph of different T-norms

and

The parameter t ∈ [th−δ,th+δ], and th can be represented as:

where th1 and th2 are the threshold of the biometric system prepared for fusion, and δ is a bias acquiring from the experiments.

The family of AA T-norm, is introduced in the 1980s by Ja´nos Acze´l and Claudi Alsina [33], is given for 0 ＜ p ＜+∞ by

The AA T-norm TpAA is strict if and only if 0 ＜ p ＜ +∞ (for p=1 it is the product T-norm). The family is strictly increasing and continuous with respect to p. The Acze´l-Alsina T-norm TpAA for 0 ＜ p ＜ +∞ arises from the product T-norm by raising its additive generator to the power of p. An additive generator of TpAA for 0 ＜ p ＜ +∞ is fpAA = (-logx)p.

Based on our sufficient research, the AA T-norm satisfies our requirement Eq. 1 and Eq. 2 showing in Fig. 1(f). After selecting suitable t and p for AA T-norm, we will get a better system performance. We will discuss this point in detail in the experimental comparison section.

2.2 Score level fusion

The whole structure of the proposed score level fusion approach in given in this section. Fig. 2 shows the framework of proposed score level fusion strategy based on AA T-norm. The dual iris, visible and thermal face imageries are merged as a fusion multibiometric system. The iris features are extracted via 1D-log Gabor algorithm [34, 35], and the face features are extracted by (2D)2 FPCA approach [36]. After the matching operation, all the genuine and imposter scores are normalized into the range of [0,1]. Finally, the AA T-norm is implemented for score level fusion.

Fig. 2.The illustration of proposed score level fusion strategy based on AA T-norm

To guarantee a meaningful combination, the different modalities scores (SR,SL,SV,ST) should be first converted to the domain [0, 1]. The normalization criterion used in our work is taken as:

where s stands for the genuine or imposter matching score. Due to all the fusion modalities, the associative manner is taken. That is combining the fusion output of the first two modalities with the third modality, and then with the last modal, until all modalities are finished. According to the associative commutative properties of T-norm, the order of combination is unimportant. The fusion score S is expressed via Eq. 6:

 

3. Experiment setup

The proposed score level fusion algorithm based on AA T-norm is tested on the merged multibiometric database. This database is a virtual multibiometric database, which contains dual iris, visible and thermal face imageries. The iris sub-database named CASIA-Iris-Thousand [37] is a challenging database which includes many noisy iris images. The face sub-database Natural Visible and Infrared facial Expression (NVIE) database which is constructed by The Key Laboratory of Computing and Communication Software of Anhui Province (CCSL). It contains visible and infrared thermal face imagery with six different expressions [38]. In our work, we randomly select 90 classes with every 10 samples to test the proposed approach. The trainning and the testing set are 90*5 samples. There are 40050 intra-class and 450 inter-class comparisons. The evaluation protocols of FMR, FNMR [39], Receiver Operator Characteristic (ROC) curve [40], Expected Performance Curves (EPC) [41], Half Total Error Rate (HTER) [42] and d’ [43] are used in this paper.

 

4. The result comparisons and analysis

In order to specifically measure the effectiveness of proposed approach, different abbreviations of the proposed and conventional methods are compared. Each abbreviation is described as showing in Table. 1. The recognition performance of different approaches are shown in this section. Since having proven that the performance of multibiometrics better than unimodal biometrics system, we mainly pay more attention on fusion systems.

Table 1.The description of different methods

The performance of SVM classification is shown in Table. 2 and Fig. 3(a). The abbreviations LR2, VT2, TLR3, VTL3 and VTLR4 stands for the fusion of dual iris, visible and thermal face, thermal face and dual iris, iris with visible and thermal face and fusion the four biometrics respectively. Caussian radial basis kernel function with a default scaling factor and sigma of 1 are used as the parameters of SVM. Because of the matching style of SVM, we just give the FMR and FNMR for this method.

Table 2.The recognition performance (%) based on SVM approach

Fig. 3.The classification of SVM and Norm2

From Table. 2, we can find out that FMR of LR2 is lowest with highest FNMR. And the lowest FNMR belongs to VTLR4. A biometric system deployed in a security application typically is required to have a lower FMR [45]. Therefore, we need to minimize the FNMR at a specified FMR values rather than minimizing the total error rate. For the fusion strategy in this work, LR2 is best choice for SVM classification.

In Fig. 3(a), the ”＋” stands for the imposter score points and ”*” stands for the genuine score points. The red ”＋” training points are covered by testing points due to its few numbers. The situation of unbalanced numbers of genuine and imposter affects the effect of SVM classification. Fig 3(b) shows the Norm2 classification for feature vector in fusion strategy TLR3. The red ”＋” stands for the imposter score points and blue ”.” stands for the genuine points. In Table 3, it gives the performance for different fusion systems, and it is not as good as Sum method, this is because this method enlarge the distance of intra-class but also the inter-class.

Table 3.The EER(%) performance of score level on multibiometric system

Table 3 shows the EER performance of other different approaches and proposed method. It is based on the testing set. From this table, we can see that the performance of fusion system LR2 is better than VT2. Another, all the methods nearly show the best.

EER with 0.93 for LR2. That is because that the iris data has higher discrimination itself. It is evident that the fusion strategy VT2 can not give better EER than iris, but it can avoid the spoof attack. Comparing with LR2 and TLR3, VT2 and VTL3, the performance turns better as the biometrics modal increasing. This point also be seen after comparing VTLR4. The reason is that the distance of genuine and imposter score distribution could be enlarge after we add more biometrics modal’s discrimination. This also prove that multibiometrics could enhance the system’s recognition performance. The fusion strategy of VTLR4 not only improve the recognition performance, but also avoid the forgery attack. From all the methods, LR-G, Sum, N2 and AApro have a better recognition performance, and the methods of LR-cp, Max, Min and LLR doesn’t show much better. In every fusion strategy, proposed method AApro shows the best recognition performance as its lowest value.

We take the fusion of left and right iris to explain the performance improvement of proposed AA T-norm. The unimodal of left and right iris recognition results are shown in Fig. 4. The indicators of FMR and FNMR are shown in this figure including with threshold and EER. The thresholds of left and right iris are 0.78 and 0.77 respectively. Obey the condition Eq.1, Eq.2 and Eq. 3, the selected critical value t = 0.8 and p = 0.3 for AA T-norm are the best parameters through our experiments.

Fig. 4.The recognition performance of iris

Fig. 5 shows the genuine and imposter score distributions before and after different fusion algorithms. The d’ values are also put in these figures, and the higher the better. Before fusion, unimodal of iris system has lower d’ value. After fusion, the d’ value is enlarged. Using the above parameters, the fusion based on AA T-norm shows the highest d’ among all the approaches from Fig. 5. It is evident that selecting suitable parameters can make the fusion distances of genuine closer and the ones of imposter larger.

Fig. 5.The genuine and imposter score distributions comparison for multibiometrcis system LR2

The EPC curves for the comparison fusion approaches based on VTLR4 are shown in Fig. 6. The lower the values of HTER in the EPC curve, better is the performance for the given cost (controlled by α) [41, 42]. From Fig. 6(a), it is evident that the EPC curves of LLR and LR-Cp are lower than Min, Max and Hm. From Fig. 6(a) shows that score level fusion using AA T-norm significantly better than any other fusion methods for each α ∈ [0, 1]. Fig. 6(a) and Fig. 6(b) implies that Max and Min don’t play very well, and sum rule and LR-G can also give the acceptable and stable identification performance. Table 4 shows the minimum HTER with corresponding α for every fusion system. Obviously, the proposed fusion method gives the excellent recognition performance.

Fig. 6.EPC curves of diffrent approaches for multibiometrics system VTLR4

Table 4.The HTER (%) (with α) performance of score level on multibiometric system

Fig. 7 shows the ROC curves of all the approaches on fusion system VTLR4. For clearly comparison, we put some lower performance methods in Fig. 7(a), and the rest higher ones are shown in Fig. 7(b). The excellent performance of proposed fusion algorithm also been proved through the comparison of ROC.

Fig. 7.ROC curves of diffrent approaches for multibiometrics system VTLR4

After all the above figures and tables, it can be concluded that AA T-norm outperforms over the traditional approaches in terms of EER, d’, HTER, EPC and ROC.

 

5. The result comparisons and analysis

In biometric system, unimodal biometric systems have inherent problems and multibiometric systems can address most of their limitations. Score level fusion can ignore the data representation and just focus on the matching scores. Considering the convenience and security, the fusion of dual iris, visible and thermal face images is proposed in this paper. To larger the inter-class distance and shorter the intra-class distance after fusion, we proposed a novel score level approach Acze´l-Alsina T-norm for fusion. After selecting optimal parameters for AA T-norm to suit the distribution of biometrics data, the recognition performance can be enhanced largely. All the contributions are tested in a virtual multibiometric database which merges CASIA-Iris-Thousand and NVIE face database together from 90 subjects. Various experiments conducted on different biometrics fusion strategies ascertain the efficacy of the proposed approach and show that the proposed method AA T-norm gives the sate-of-the-art recognition performance.