Paper Detail

Hard Sample-Conditioned Decorrelated Attention for Robust Face Recognition

Tahar Chettaoui, Guray Ozgur, Eduarda Caldeira, Naser Damer, Fadi Boutros

huggingface Score 5.0

Published 2026-10-30 · First seen 2026-11-07

General AI

Abstract

Multi-head attention in Vision Transformers (ViTs) allows independent attention patterns across heads, but without explicit regularization, heads may learn redundant patterns rather than complementary roles. Correlation-based penalties encourage head disentanglement, yet apply uniform regularization across all training samples. In face recognition (FR), hard samples whose embeddings lie close to competing identities are critical for optimization and generalization. Motivated by this, we propose ViT-HarDis, which conditions attention head disentanglement on sample difficulty: samples with high cosine similarity to negative class centers receive stronger head decorrelation, enforcing specialization where identity ambiguity is highest. Experiments show improvements over both the baseline and uniform disentanglement on most benchmarks. Our analysis shows that ViT-HarDis reduces the similarity of hard samples to competing class centers while decreasing inter-head attention similarity, and that applying disentanglement only to the final transformer layers outperforms regularizing the entire network. Trained on WebFace4M, ViT-HarDis achieves SOTA results among ViT-based methods on IJB-B, IJB-C, and TinyFace Rank-1, while achieving the best IJB-B and TinyFace results when trained on MS1MV3, improving FR without architectural modifications or additional inference cost. Code: https://github.com/TaharChettaoui/Hard-Sample-Conditioned-Attention-for-FR.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
later
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@misc{chettaoui2026hard,
  title = {Hard Sample-Conditioned Decorrelated Attention for Robust Face Recognition},
  author = {Tahar Chettaoui and Guray Ozgur and Eduarda Caldeira and Naser Damer and Fadi Boutros},
  year = {2026},
  abstract = {Multi-head attention in Vision Transformers (ViTs) allows independent attention patterns across heads, but without explicit regularization, heads may learn redundant patterns rather than complementary roles. Correlation-based penalties encourage head disentanglement, yet apply uniform regularization across all training samples. In face recognition (FR), hard samples whose embeddings lie close to competing identities are critical for optimization and generalization. Motivated by this, we propose },
  url = {https://huggingface.co/papers/2610.41789},
  keywords = {multi-head attention, Vision Transformers, attention heads, correlation-based penalties, head disentanglement, uniform regularization, hard samples, embeddings, competing identities, cosine similarity, negative class centers, head decorrelation, inter-head attention similarity, transformer layers, sample difficulty, ViT-HarDis, huggingface daily},
  eprint = {2610.41789},
  archiveprefix = {arXiv},
}

Metadata

{}