Unifying Thermal and Visible Face Recognition Through Continuous Convolution Global Weighting Transformers and Fourier Neural Operators

Abstract

Cross-spectrum face recognition, in which thermal and visible images must be jointly analyzed, has long been challenged by discrepancies in illumination, sensor noise, and spectral characteristics. These issues are particularly relevant in security, defense, and healthcare, where robust identification across lighting and environmental conditions is essential. Despite advances in standard convolutional or attention-based networks, many models still struggle with domain adaptation and fail to extract consistent features from thermal and visible inputs. To address this gap, we investigate three alternative architectures: a Continuous Neural Network (CNN) that learns smooth kernel functions, an Attention-Free Transformer (AFT) with global weighting instead of multi-head attention, and a Fourier Neural Operator (FNO) that operates on low-frequency spectral components. Each model was trained on a disjoint set of thermal–visible face images and then evaluated for classification accuracy. Whereas the FNO-based method reached 0.86 (0.85 macro-average F1-scores), our results reveal that the continuous neural network and the attention-free transformer attained 0.98 accuracies (with macro- and weighted-average F1-scores of 0.98).