Paper Detail
Ashwin Nedungadi, Stefan Oehmcke, Stefan Lüdtke
Large language models (LLMs) trained only on text and code can sometimes generate programs that draw recognizable images. However, it is unclear whether this reflects an internal representation of 2D spatial layout or simply the ability to translate spatial descriptions into code. We introduce Autoregressive Mosaics (AM-Bench), a benchmark that separates these factors: First, a translation task gives a model a fully specified geometry of a picture in words as a prompt and asks for the code that produces it. Second, a layout task requires the model to compose an image from an underspecified prompt. Across eight open-weight text-and-code-only models, all models reliably translate specified geometry into code, but their open-ended layout performance differs substantially, indicating that these differences are not explained by code-generation ability alone. An output-medium ablation further shows that the interface or medium of expression that the model uses matters: replacing procedural code with raw SVG improves layout scores across all models. Finally, probing model activations shows that a coarse layout plan is present before generation, but reflects only the layout implied by the prompt. During generation, models track the evolving geometric state instead of executing an initially fixed plan. Overall, these results show that 2D spatial performance in text-only LLMs depends on both the model and the output medium, and is not explained by code-generation ability alone.
No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.
No ranking explanation is available yet.
No tags.
@misc{nedungadi2026autoregressive,
title = {Autoregressive Mosaics: Probing 2D Spatial Reasoning in Text-Only Language Models},
author = {Ashwin Nedungadi and Stefan Oehmcke and Stefan Lüdtke},
year = {2026},
abstract = {Large language models (LLMs) trained only on text and code can sometimes generate programs that draw recognizable images. However, it is unclear whether this reflects an internal representation of 2D spatial layout or simply the ability to translate spatial descriptions into code. We introduce Autoregressive Mosaics (AM-Bench), a benchmark that separates these factors: First, a translation task gives a model a fully specified geometry of a picture in words as a prompt and asks for the code that },
url = {https://huggingface.co/papers/2608.30751},
keywords = {Autoregressive Mosaics (AM-Bench), translation task, layout task, open-weight text-and-code-only models, procedural code, raw SVG, probing model activations, coarse layout plan, geometric state, huggingface daily},
eprint = {2608.30751},
archiveprefix = {arXiv},
}
{}