SLE · Evaluation framework
Evaluate agents on real scientific research.
Science’s Last Exam gives agents scientific tasks, real tools and data, then checks what they produce against withheld references. These docs explain how to run an evaluation and contribute a task.
About SLE
Science’s Last Exam (SLE) is a benchmark for AI agents doing scientific research. It tests whether agents can use scientific tools and produce results that can be independently checked.
Tasks come from real laboratory materials, research code and specialist software across nine disciplines. Capability assessments locate specific weaknesses; End-to-end research (To be expanded) explores whether an agent can carry a research goal through a complete workflow.
Benchmark design
SLE covers six scientific capabilities and one end-to-end research assessment: Hypothesis Discovery, Protocol Planning, Scientific Computing, Scientific Software Operation, Wet-lab Execution and Experiment Observation, plus End-to-end research. Release v1 contains 113 tasks: Hypothesis Discovery 5, Protocol Planning 40, Scientific Computing 23, Scientific Software Operation 14, Experiment Observation 30 and End-to-end research 1. Wet-lab Execution is In construction, with no released task and no score. End-to-end research is To be expanded; it is included in the composite and scores 0 for all currently listed models.
Research code and specialist desktop software.
Laboratory recordings, protocols and scientific inputs.
Withheld references and separate verifiers.
113 tasks in a framework of six scientific capabilities and one end-to-end research assessment. Wet-lab Execution is In construction; End-to-end research is To be expanded.
Evaluation workflow
Each experiment combines three parts:
A pinned Codex or Claude Code profile and its model; Hypothesis Discovery uses the DiscoveryRSI harness.
The task’s Docker container or scientific desktop.
An instruction, inputs, declared outputs and verifier.
- 1 · Prepare the environment
Build or prepare the runtime selected by the task list.
Environments → - 2 · Stage the task
Fetch the matching task inputs and keep references outside the solver.
Task format → - 3 · Run the agent
Use the experiment configuration to launch the selected tasks.
Run an experiment → - 4 · Grade the outputs
Run independent checks on the artifacts the agent submits.
Scoring rules → - 5 · Keep the evidence
Inspect per-task results, native traces and verifier records.
Results and artifacts →
Code and data
Public downloads open on 6 November 2026. The code repository, task cards and agent inputs will be publicly downloadable. Reference solutions and hidden evaluation data require a separate access request and manual approval on Hugging Face before downloading. Until the release date, access remains limited to the private preview. The release includes build recipes; prebuilt SLE images are not distributed.
The website lists every released task with its prompt, inputs and evaluation rules. Use the repository’s versioned task lists to run experiments. Release notes →