Paper Detail

Can an Agent Debug a Web Corpus? Agentic Auditing of Slovak FineWeb2

Marek Šuppa, Ivan Vykopal, Jaroslav Kopčan

arxiv Score 7.3

Published 2026-10-22 · First seen 2026-10-23

General AI

Abstract

Multilingual pipelines such as FineWeb2 apply one recipe across a thousand languages, leaving the failure modes of individual languages unanticipated. We use an LLM once, as a corpus debugger rather than a classifier: a tool-using agent explores the corpus through a read-only, budgeted API and compiles each finding into an executable test, which a native speaker then validates. On the 26.5M-document Slovak portion of FineWeb2, three runs costing \$2.21-\$2.84 each found seven findings over four phenomena. The largest is a machine-generated content farm of 1,408,437 documents (5.32%) that passed the pipeline's filters, 86.6% of it passing even stricter thresholds. Blinded labels support 6 of the 7 findings (defect rates .53-1.00 against .08 in a random control) and do not support one. The compiled suite scans the full corpus in 0.7 s of wall-clock time without a GPU, against \$12k-\$90k per LLM classification pass. The best static sampling baseline validates more selectors than any agent run, at under half the cost. The agent contributes exact prevalence, a precision estimate checked against sampled documents, an account of why the pipeline missed each slice, and a reusable test. We release the framework, Slovak test suite, validation data and agent trajectories.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
soon
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@article{uppa2026can,
  title = {Can an Agent Debug a Web Corpus? Agentic Auditing of Slovak FineWeb2},
  author = {Marek Šuppa and Ivan Vykopal and Jaroslav Kopčan},
  year = {2026},
  abstract = {Multilingual pipelines such as FineWeb2 apply one recipe across a thousand languages, leaving the failure modes of individual languages unanticipated. We use an LLM once, as a corpus debugger rather than a classifier: a tool-using agent explores the corpus through a read-only, budgeted API and compiles each finding into an executable test, which a native speaker then validates. On the 26.5M-document Slovak portion of FineWeb2, three runs costing \textbackslash{}\$2.21-\textbackslash{}\$2.84 each found seven findings over four },
  url = {https://arxiv.org/abs/2610.31636},
  keywords = {cs.CL},
  eprint = {2610.31636},
  archiveprefix = {arXiv},
}

Metadata

{}