Paper Detail

Position Anchor Tuning: Towards Efficient Adaptation of Pre-Trained Point Cloud Transformers

Zheng Liu, Xin Gao, Jinchao Zhu, Gao Huang

Browse

Workflow Queues

arxiv Score 5.8

Published 2026-09-16 · First seen 2026-09-17

General AI

Open paper source

Abstract

Parameter-efficient fine-tuning (PEFT) has recently emerged as a pivotal research direction for adapting pre-trained point cloud transformers to diverse downstream tasks. Although existing methods achieve excellent fine-tuning performance with high parameter efficiency, they ignore inference efficiency. To tackle this problem, a novel PEFT method termed position anchor tuning (PAT) is proposed in this paper. As multi-head attention (MHA) and feed-forward network (FFN) are computation-heavy blocks in pre-trained transformers, PAT decreases their computational cost through token aggregation-expansion pairs. Each pair comprises a token aggregation module (TAM) and a token expansion module (TEM). For MHA and FFN blocks, TAMs extract representative tokens from their input tokens based on position anchors in 3D space. These extracted tokens, rather than the original input tokens, are processed by the blocks, thereby reducing the number of tokens involved in computation. Then, TEMs propagate the learned representations back to the original input tokens. Since TAMs are solely responsible for capturing task-specific representations, base-sharing low-rank adaptation (BSLoRA) is further introduced to enable them to learn such representations effectively with only a small number of trainable parameters. Extensive experiments on widely used benchmarks demonstrate that PAT performs comparably to state-of-the-art methods while incurring significantly lower computational overhead and fewer trainable parameters.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
later
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

Similar Papers

NaRA: Noise-Aware LoRA for Parameter-Efficient Fine-Tuning of Diffusion LLMs 2026-05-28 · Score 14.3 General AI Revisiting Human-in-the-Loop Object Retrieval with Pre-Trained Vision Transformers 2026-04-01 · Score 5.3 General AI ACE: Adapter Consolidation across Experts for Parameter-Efficient Fine-Tuning of MoE LLMs 2026-09-05 · Score 27.5 Research Track A · General AI Robust and Interpretable Adaptation of Equivariant Materials Foundation Models via Sparsity-promoting Fine-tuning 2026-06-17 · Score 3.3 General AI One-for-All: A Lightweight Stabilized and Parameter-Efficient Pre-trained LLM for Time Series Forecasting 2026-03-31 · Score 9.3 General AI

BibTeX

@article{liu2026position,
  title = {Position Anchor Tuning: Towards Efficient Adaptation of Pre-Trained Point Cloud Transformers},
  author = {Zheng Liu and Xin Gao and Jinchao Zhu and Gao Huang},
  year = {2026},
  abstract = {Parameter-efficient fine-tuning (PEFT) has recently emerged as a pivotal research direction for adapting pre-trained point cloud transformers to diverse downstream tasks. Although existing methods achieve excellent fine-tuning performance with high parameter efficiency, they ignore inference efficiency. To tackle this problem, a novel PEFT method termed position anchor tuning (PAT) is proposed in this paper. As multi-head attention (MHA) and feed-forward network (FFN) are computation-heavy block},
  url = {https://arxiv.org/abs/2609.18056},
  keywords = {cs.CV},
  eprint = {2609.18056},
  archiveprefix = {arXiv},
}

Metadata

{}