Paper Detail

Lightweight Multimodal Adaptation of Vision Language Models for Species Recognition and Habitat Context Interpretation in Drone Thermal Imagery

Hao Chen, Fang Qiu, Fangchao Dong, Defei Yang, Eve Bohnett, Li An

Browse

Workflow Queues

arxiv Score 8.8

Published 2026-04-07 · First seen 2026-04-08

General AI

Open paper source

Abstract

This study proposes a lightweight multimodal adaptation framework to bridge the representation gap between RGB-pretrained VLMs and thermal infrared imagery, and demonstrates its practical utility using a real drone-collected dataset. A thermal dataset was developed from drone-collected imagery and was used to fine-tune VLMs through multimodal projector alignment, enabling the transfer of information from RGB-based visual representations to thermal radiometric inputs. Three representative models, including InternVL3-8B-Instruct, Qwen2.5-VL-7B-Instruct, and Qwen3-VL-8B-Instruct, were benchmarked under both closed-set and open-set prompting conditions for species recognition and instance enumeration. Among the tested models, Qwen3-VL-8B-Instruct with open-set prompting achieved the best overall performance, with F1 scores of 0.935 for deer, 0.915 for rhino, and 0.968 for elephant, and within-1 enumeration accuracies of 0.779, 0.982, and 1.000, respectively. In addition, combining thermal imagery with simultaneously collected RGB imagery enabled the model to generate habitat-context information, including land-cover characteristics, key landscape features, and visible human disturbance. Overall, the findings demonstrate that lightweight projector-based adaptation provides an effective and practical route for transferring RGB-pretrained VLMs to thermal drone imagery, expanding their utility from object-level recognition to habitat-context interpretation in ecological monitoring.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
now
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

Similar Papers

SOMA: Strategic Orchestration and Memory-Augmented System for Vision-Language-Action Model Robustness via In-Context Adaptation 2026-03-25 · Score 11.8 Research Track A · General AI When Audio-Language Models Fail to Leverage Multimodal Context for Dysarthric Speech Recognition 2026-05-04 · Score 10.2 General AI Multilingual Training and Evaluation Resources for Vision-Language Models 2026-04-20 · Score 11.3 General AI Information Router for Mitigating Modality Dominance in Vision-Language Models 2026-04-17 · Score 15.3 General AI TalkLoRA: Communication-Aware Mixture of Low-Rank Adaptation for Large Language Models 2026-04-07 · Score 5.8 General AI

BibTeX

@article{chen2026lightweight,
  title = {Lightweight Multimodal Adaptation of Vision Language Models for Species Recognition and Habitat Context Interpretation in Drone Thermal Imagery},
  author = {Hao Chen and Fang Qiu and Fangchao Dong and Defei Yang and Eve Bohnett and Li An},
  year = {2026},
  abstract = {This study proposes a lightweight multimodal adaptation framework to bridge the representation gap between RGB-pretrained VLMs and thermal infrared imagery, and demonstrates its practical utility using a real drone-collected dataset. A thermal dataset was developed from drone-collected imagery and was used to fine-tune VLMs through multimodal projector alignment, enabling the transfer of information from RGB-based visual representations to thermal radiometric inputs. Three representative models,},
  url = {https://arxiv.org/abs/2604.06124},
  keywords = {cs.CV, cs.AI},
  eprint = {2604.06124},
  archiveprefix = {arXiv},
}

Metadata

{}