DIU
Heterogeneous Face Recognition (HFR) aims to expand the applicability of Face Recognition (FR) systems to challenging scenarios, enabling the matching of face images across different domains, such as matching thermal images to visible spectra. However, the development of HFR systems is challenging because of the significant domain gap between modalities and the lack of availability of large-scale paired multi-channel data. In this work, we leverage a pretrained face recognition model as a teacher network to learn domaininvariant network layers called Domain-Invariant Units (DIU) to reduce the domain gap. The proposed DIU can be trained effectively even with a limited amount of paired training data, in a contrastive distillation framework. This proposed approach has the potential to enhance pretrained models, making them more adaptable to a wider range of variations in data. We extensively evaluate our approach on multiple challenging benchmarks, demonstrating superior performance compared to state-of-the-art (SOTA) methods.
Overview
- Training: The Domain Invariant Units (DIU) was trained on WebFace12M dataset (it contains over 12M images representing more than 600K identities).
- Backbone: IResNet101
- Parameters: 65.15M
- Task: Heterogeneous Face Recognition Using Domain Invariant Units
- Framework: Pytorch
- Output structure: Batch of face embeddings (ie, features)
Evaluation of Models:
The proposed Domain Invariant Unit (DIU) framework. The lower layers of the student model are trained in contrastive framework to learn invariant features, while supervision from the distillation loss prevents overfitting.
Table1. Pola Thermal - Average Rank-1 Recognition Rate
| Method | Mean (Std. Dev.) |
|---|---|
| DPM in | 75.31% (-) |
| CpNN in | 78.72% (-) |
| PLS in | 53.05% (-) |
| LBPs + DoG in | 36.8% (3.5) |
| ISV in | 23.5% (1.1) |
| DSU (Best Result) | 76.3% (2.1) |
| DSU-Resnet100 | 85.2% (5.8) |
| PDT | 97.1% (1.3) |
| CAIM | 95.0% (1.63) |
| DIU (Proposed) | 97.8% (1.28) |
Table 2. Experimental results on VIS-Thermal protocol of the Tufts Face dataset
| Method | Rank-1 | VR@FAR=1% | VR@FAR=0.1% |
|---|---|---|---|
| LightCNN | 29.4 | 23.0 | 5.3 |
| DVG | 56.1 | 44.3 | 17.1 |
| DVG-Face | 75.7 | 68.5 | 36.5 |
| DSU-Iresnet100 | 49.7 | 49.8 | 28.3 |
| PDT | 65.71 | 69.39 | 45.45 |
| CAIM | 73.07 | 76.81 | 46.94 |
| DIU (Proposed) | 82.94 | 85.9 | 74.95 |
Running Code
- Minimal code to instantiate the model and perform inference:
$ cd bob.paper.icassp2024_diu_hfr
$ conda create --name bob.paper.icassp2024_diu_hfr --file spec-file.txt
$ conda activate bob.paper.icassp2024_diu_hfr # activate the environment
$ pip install pytorch-lightning==1.5.3
$ buildout
$ ./bin/bob bio pipelines vanilla-biometrics --help # test the installation
License
Copyright
(c) 2025, Anjith George, Sébastien Marcel Idiap Research Institute, Martigny 1920, Switzerland.
https://gitlab.idiap.ch/bob/bob.paper.icassp2024_diu_hfr
Please refer to the link for information about the License & Copyright terms and conditions.
Citation
If you find our work useful, please cite the following publication:
@inproceedings{george2024heterogeneous,
title={Heterogeneous face recognition using domain invariant units},
author={George, Anjith and Marcel, S{\'e}bastien},
booktitle={ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages={4780--4784},
year={2024},
organization={IEEE}
}