Research paper
Design principles, collection construction, the SLE-Agent execution framework, and evaluation methodology.
Publication link to be announcedTHE PROJECT
About Science’s Last Exam
Science’s Last Exam brings five scientific capabilities into one evaluation framework. DiscoveryRSI’s benchmark becomes the Hypothesis Discovery component of this broader project.
Project resources
Release links will appear here when the project’s paper, code, and dataset are ready.
Code & evaluation
SLE-Agent, environment setup, task runners, and capability-specific evaluation tools.
Repository link to be announcedThe public collection
Task inputs, environments, and evaluators for the Representative and Last-Exam subsets.
Dataset release to be announcedContribute to SLE
We welcome scientific task proposals, environment integrations, and model evaluations. The participation guide, draft task catalog and preliminary submission tools are available now. Runnable benchmark downloads remain forthcoming.
Propose scientific tasks
Identify authentic workflows, observable outcomes, and the scientific constraints that make a task worth evaluating.
Build task environments
Build reliable environments, preserve inputs, and connect delivered artifacts to verifiable scientific checks.
Evaluate models
Use capability profiles to investigate where systems succeed and what still prevents reliable scientific work.
Contact
For project inquiries, task proposals, and evaluation collaboration, contact junyangwang@stu.pku.edu.cn.
Frequently asked questions
How does this relate to DiscoveryRSI?
The former discovery benchmark is now part of Science’s Last Exam as Hypothesis Discovery. Its reported metric here is Inspire Rate. The wider project also covers Embodied Perception, World Dynamics, Scientific Coding, and Scientific GUI.
Are the task counts final?
No. The manuscript checked on 24 October 2026 and the repository inventory both contain 9,163 tasks, including 188 Scientific GUI tasks and 198 public tasks. These counts remain provisional. The six research databases are a separate evaluation resource, not six additional tasks.
What results are available now?
The Hypothesis Discovery page contains the project team’s results for five evaluated models, supplied on 20 October 2026, including overall and four research-domain rates. The public capability pages now show three configurations. The DeepSeek V4.1 Flash snapshot, rechecked on 23 October 2026 (UTC+8), remains at 163 of 198 tasks scored, 12 errors, and 23 without a terminal result. Two additional GLM-5.3 Flash configurations report 120 scored / 21 errors / 57 pending and 137 scored / 23 errors / 38 pending. The latter, rechecked on 24 October 2026, uses Harbor ClaudeCode with 92 reused QA results and is stopped with zero running tasks. Its Coding mean is 80.43% over 23 of 48 tasks; its GUI mean is 16.67% over 18 of 54 tasks. These additional configurations remain unranked and have no supplied Representative or Last-Exam breakdown. Coverage and scores stay separate for every run; incomplete runs have no four-capability overall score.
The separate ten-task GPT-6 Astra / medium GUI diagnostic has finished with 9 scored tasks, 5 full-credit results, 1 evaluator error, and no pending tasks. Its mean and full-credit rate are 55.56% over the 9 scored tasks. This selected sample uses a different harness, is not a formal baseline, and is neither ranked nor pooled into Public-198.
Does Inspire Rate measure verified discoveries?
No. It measures whether later literature provides a concrete design, comparator, measurement, or testable evidence gap relevant to developing or testing a generated hypothesis version. It does not show that the hypothesis is true or that researchers adopted it.
Can I download and run the tasks?
The website shows nine examples drawn from the manuscript and the current ScienceBench collection. Each example identifies its source and release status. The runnable task package, code repository, and public dataset download are forthcoming. The participation page provides all 198 draft public task identities, a run guide, and local result-packet validation.
Does every capability cover every discipline?
No. Five capabilities jointly span nine disciplines. The benchmark does not claim complete coverage of every capability–discipline combination.