Paper Detail
Qiuyu Tian, Haojie Yin, Yingce Xia, Youyong Kong, Zequn Liu
AI research often requires decisions before future evidence exists: which bottleneck to attack, which direction to pursue, or where a project should be positioned. We introduce ForeSci, a temporally controlled benchmark for evaluating whether LLM agents can make such forward-looking research judgements from historical evidence. ForeSci contains 500 tasks across four fast-moving AI domains and four decision families. Each task is paired with a cutoff-aligned offline knowledge base; post-cutoff papers are hidden during generation and used only for validation. To avoid random future-event prediction, tasks are derived from pre-cutoff taxonomy branches and evidence signals, and answer-generation backbones are selected to precede the task cutoffs. We evaluate native LLMs, Hybrid RAG, and three research-agent adaptations across four backbones. Results show that explicit evidence organization improves traceability and factual support, but gains depend strongly on the decision family. Diagnostics reveal a recurring evidence-decision decoupling: agents may cite relevant evidence while forecasting the wrong research object. ForeSci turns forward-looking AI research judgement into a controlled benchmark for evaluating research agents as decision-making systems.
No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.
No ranking explanation is available yet.
No tags.
@misc{tian2026foresci,
title = {ForeSci: Evaluating LLM Agents for Forward-Looking AI Research Judgment},
author = {Qiuyu Tian and Haojie Yin and Yingce Xia and Youyong Kong and Zequn Liu},
year = {2026},
abstract = {AI research often requires decisions before future evidence exists: which bottleneck to attack, which direction to pursue, or where a project should be positioned. We introduce ForeSci, a temporally controlled benchmark for evaluating whether LLM agents can make such forward-looking research judgements from historical evidence. ForeSci contains 500 tasks across four fast-moving AI domains and four decision families. Each task is paired with a cutoff-aligned offline knowledge base; post-cutoff pa},
url = {https://huggingface.co/papers/2606.00644},
keywords = {LLM agents, forward-looking research judgments, temporal control, benchmark evaluation, hybrid RAG, research-agent adaptations, decision-making systems, huggingface daily},
eprint = {2606.00644},
archiveprefix = {arXiv},
}
{}