Paper Detail

Tree-Ising: Repairing Language Models with Bit-Level Ising Edits

Hadi Salloum, Ammar Ali

huggingface Score 14.5

Published 2026-11-10 · First seen 2026-11-11

General AI

Abstract

Weight-space programming, introduced in recent MIT work by Takehi et al., enables a frontier LLM agent to read a network's weights and write a repair. Their GPT-6 Astra agent corrected the reasoning loops of Qwen3.5-0.8B without gradients, using 158 agent turns, 89.7 million tokens, and 50 forward-pass requests. Our method, Tree-Ising, casts this repair as a sparse discrete optimization problem over the model's own BF16 weights. Forward statistics define an objective that weakens a loop direction, strengthens a concluding direction, and preserves the layer's output on productive tokens. Within a state-tree representation, where lossless BF16 storage acts as a special case, the admissible edits are sign flips and exponent shifts that preserve all shared group exponents and mantissas. We decompose this objective exactly into independent QUBO (Ising) problems, one per output row, which we solve on a GPU using simulated annealing or quantum-inspired simulated bifurcation. On the agent's two layers, Tree-Ising achieves 54.6% on MATH-500, outperforming the agent's edit (51.7%) under the same evaluation protocol, while the base model scores 7.9% as reported. The simulated bifurcation solver yields a statistically indistinguishable repair at 55.1%. Both solvers outperform the agent edit across all five math benchmarks, with statistically significant improvements on MATH-500 and GSM8K after correcting for multiple comparisons. Our approach modifies 13% to 17% fewer weights and reduces BF16 bit flips by a factor of 2.2 to 2.3. It requires only four forward-pass requests and zero LLM tokens, with the annealing recipe costing about $0.90 in GPU time. On Qwen3.5-2B, with layers and dose selected using only the base model, the same recipe increases greedy GSM8K accuracy from 21.6% to 69.4%. As a proof of concept, we execute an eight-variable repair subproblem using QAOA on an IBM Heron processor.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
now
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@misc{salloum2026tree,
  title = {Tree-Ising: Repairing Language Models with Bit-Level Ising Edits},
  author = {Hadi Salloum and Ammar Ali},
  year = {2026},
  abstract = {Weight-space programming, introduced in recent MIT work by Takehi et al., enables a frontier LLM agent to read a network's weights and write a repair. Their GPT-6 Astra agent corrected the reasoning loops of Qwen3.5-0.8B without gradients, using 158 agent turns, 89.7 million tokens, and 50 forward-pass requests. Our method, Tree-Ising, casts this repair as a sparse discrete optimization problem over the model's own BF16 weights. Forward statistics define an objective that weakens a loop directio},
  url = {https://huggingface.co/papers/2611.14210},
  keywords = {weight-space programming, frontier LLM agent, GPT-6 Astra agent, Qwen3.5-0.8B, reasoning loops, gradients, BF16 weights, forward statistics, loop direction, concluding direction, layer output, productive tokens, state-tree representation, lossless BF16 storage, sign flips, exponent shifts, shared group exponents, mantissas, sparse discrete optimization, QUBO, Ising problems, output row, GPU, simulated annealing, quantum-inspired simulated bifurcation, MATH-500, GSM8K, Qwen3.5-2B, greedy GSM8K accuracy, QAOA, IBM Heron processor, code available, huggingface daily},
  eprint = {2611.14210},
  archiveprefix = {arXiv},
}

Metadata

{}