Science's Last Exam — result submission template

This is an unfilled template, not an evaluation result.

1. Fill metadata.json
   Replace YOUR_* and VERSION_OR_COMMIT / BENCHMARK_COMMIT_SHA placeholders.
   Record the exact model, harness version, benchmark commit, settings, budgets,
   attempts per task, and reproduction command. Use null only for settings
   that do not apply, and explain any protocol changes in notes.
   Keep only the assessments you evaluated: hd, pp, sc, so, eo, e2e.

2. Fill eval.json
   Replace the zero task counts and null scores with measured results.
   Remove capabilities you did not evaluate. Do not report missing results as 0.
   Scores use a 0–100 scale: pp is mean score; eo, sc and so are pass rates;
   hd is the inspire rate; e2e is the official end-to-end research score.
   The end-to-end progress rubric is a separate diagnostic, not this score.
   Current published e2e scores are 0; this is not a rule for future results.
   task_count is the number of distinct tasks in your declared evaluation set.
   completed_count is the number with a completed evaluation and grader result,
   including graded failures. It is not the number of passing tasks.
   Retain every attempt, timeout and error in runs/ and explain how it was
   treated in scoring. Report repeated trials and the aggregation rule.
   For partial evaluations, leave overall_score null.
   For a complete evaluation, overall_score is
   (pp.score + eo.score + sc.score + so.score + hd.score + e2e.score) / 6.
   Report per-task cost in USD and agent execution time in seconds.
   Leave cost/time null when unmeasured and explain the reason in metadata.
   Report price source/date and the averaging denominator in metadata.notes.
   Cost and time of All equally average per-task means for pp, sc, so and eo; hd and e2e report neither.
   hd uses metric "inspire_rate" on a 0–100 scale: the percentage of hypothesis
   versions that anticipate research published after the model's knowledge
   cutoff, identified by a literature audit.
   Report positive_count and hypothesis_version_count with it, name the
   research harness and databases in metadata.notes, and include the
   hypothesis versions and audit outputs in runs/.
   The end-to-end research score is included in All. The currently listed
   leaderboard models score 0 on this assessment.

3. Populate runs/
   Copy the original run directories, preserving task IDs and trial IDs.
   Include agent trajectories, tool calls, final deliverables, verifier outputs,
   and run configuration. Keep the native runner layout, including failed runs.
   Do not include this empty template as evidence of a completed evaluation.

4. Package from inside the template directory
   zip -r sle-results.zip metadata.json eval.json runs/
   Alternatively:
   tar -czf sle-results.tar.gz metadata.json eval.json runs/
   These three entries must be at the archive root, with no enclosing folder.

5. Email for review
   To: ly1988@princeton.edu, wuyc@stanford.edu
   Subject: [SLE Results] <harness> / <model> / SLE-v1
   Include your name/organization, model and harness versions, evaluated
   capabilities, benchmark commit, and reproduction command.
   Attach the archive if your mail provider permits its size; otherwise include
   a Google Drive, Dropbox, or other download link accessible to the reviewers.
   This page does not upload files or submit a result automatically.
   The team reviews the records and reproduces the evaluation before publication.

Guide: https://sciences-last-exam.org/submit-results
Leaderboard: https://sciences-last-exam.org/leaderboard
