Paper Detail

FairLLaVA: Fairness-Aware Parameter-Efficient Fine-Tuning for Large Vision-Language Assistants

Mahesh Bhosale, Abdul Wasi, Shantam Srivastava, Shifa Latif, Tianyu Luan, Mingchen Gao, David Doermann, Xuan Gong

Browse

Workflow Queues

arxiv Score 12.8

Published 2026-03-27 · First seen 2026-03-31

General AI

Open paper source

Abstract

While powerful in image-conditioned generation, multimodal large language models (MLLMs) can display uneven performance across demographic groups, highlighting fairness risks. In safety-critical clinical settings, such disparities risk producing unequal diagnostic narratives and eroding trust in AI-assisted decision-making. While fairness has been studied extensively in vision-only and language-only models, its impact on MLLMs remains largely underexplored. To address these biases, we introduce FairLLaVA, a parameter-efficient fine-tuning method that mitigates group disparities in visual instruction tuning without compromising overall performance. By minimizing the mutual information between target attributes, FairLLaVA regularizes the model's representations to be demographic-invariant. The method can be incorporated as a lightweight plug-in, maintaining efficiency with low-rank adapter fine-tuning, and provides an architecture-agnostic approach to fair visual instruction following. Extensive experiments on large-scale chest radiology report generation and dermoscopy visual question answering benchmarks show that FairLLaVA consistently reduces inter-group disparities while improving both equity-scaled clinical performance and natural language generation quality across diverse medical imaging modalities. Code can be accessed at https://github.com/bhosalems/FairLLaVA.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
now
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

Similar Papers

TLoRA+: A Low-Rank Parameter-Efficient Fine-Tuning Method for Large Language Models 2026-04-15 · Score 8.3 General AI ShadowPEFT: Shadow Network for Parameter-Efficient Fine-Tuning 2026-04-21 · Score 11.3 General AI MNAFT: modality neuron-aware fine-tuning of multimodal large language models for image translation 2026-04-18 · Score 14.5 General AI GRASS: Gradient-based Adaptive Layer-wise Importance Sampling for Memory-efficient Large Language Model Fine-tuning 2026-04-09 · Score 11.8 General AI RDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language Models 2026-04-21 · Score 8.3 General AI

BibTeX

@article{bhosale2026fairllava,
  title = {FairLLaVA: Fairness-Aware Parameter-Efficient Fine-Tuning for Large Vision-Language Assistants},
  author = {Mahesh Bhosale and Abdul Wasi and Shantam Srivastava and Shifa Latif and Tianyu Luan and Mingchen Gao and David Doermann and Xuan Gong},
  year = {2026},
  abstract = {While powerful in image-conditioned generation, multimodal large language models (MLLMs) can display uneven performance across demographic groups, highlighting fairness risks. In safety-critical clinical settings, such disparities risk producing unequal diagnostic narratives and eroding trust in AI-assisted decision-making. While fairness has been studied extensively in vision-only and language-only models, its impact on MLLMs remains largely underexplored. To address these biases, we introduce },
  url = {https://arxiv.org/abs/2603.26008},
  keywords = {cs.CV, cs.AI},
  eprint = {2603.26008},
  archiveprefix = {arXiv},
}

Metadata

{}