Paper Detail

Scalable Circuit Learning for Interpreting Large Language Models

Naiyu Yin, Dennis Wei, Tian Gao, Amit Dhurandhar, Karthikeyan Natesan Ramamurthy, Yue Yu

Browse

Workflow Queues

arxiv Score 9.3

Published 2026-06-15 · First seen 2026-06-16

General AI

Open paper source

Abstract

A prominent research direction in mechanistic interpretability is learning sparse circuits over LLM components to reveal how they jointly produce model behavior. However, raw neurons are polysemantic, making learned circuits hard to interpret. Sparse autoencoder (SAE) features alleviate this, but their high dimensionality makes existing intervention-based circuit learning methods computationally prohibitive. We propose CircuitLasso, a scalable circuit-learning approach based on sparse linear regression. CircuitLasso recovers circuits whose structural accuracy matches that of state-of-the-art intervention-based methods on the benchmark data, at a fraction of the computational cost. For interpretability, CircuitLasso efficiently uncovers relationships among SAE features, showing how human-interpretable semantic features propagate through the model and influence its predictions. Finally, we validate the utility of our learned circuits by leveraging their insights to achieve comparable performance at substantially lower cost on a domain-generalization task.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
now
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

Similar Papers

Attribution-Guided Continual Learning for Large Language Models 2026-05-06 · Score 19.5 Research Track A · General AI JumpLoRA: Sparse Adapters for Continual Learning in Large Language Models 2026-04-17 · Score 29.0 Research Track A · General AI Black-Box Continual Learning for Vision-Language Models 2026-06-22 · Score 14.4 Research Track A · General AI Hybrid Framework for Robotic Manipulation: Integrating Reinforcement Learning and Large Language Models 2026-03-31 · Score 18.3 General AI IR3DE: A Linear Router for Large Language Models 2026-06-04 · Score 11.5 General AI

BibTeX

@article{yin2026scalable,
  title = {Scalable Circuit Learning for Interpreting Large Language Models},
  author = {Naiyu Yin and Dennis Wei and Tian Gao and Amit Dhurandhar and Karthikeyan Natesan Ramamurthy and Yue Yu},
  year = {2026},
  abstract = {A prominent research direction in mechanistic interpretability is learning sparse circuits over LLM components to reveal how they jointly produce model behavior. However, raw neurons are polysemantic, making learned circuits hard to interpret. Sparse autoencoder (SAE) features alleviate this, but their high dimensionality makes existing intervention-based circuit learning methods computationally prohibitive. We propose CircuitLasso, a scalable circuit-learning approach based on sparse linear reg},
  url = {https://arxiv.org/abs/2606.16939},
  keywords = {cs.LG, cs.AI},
  eprint = {2606.16939},
  archiveprefix = {arXiv},
}

Metadata

{}