# Organizer review checklist

This is a proposed manual intake process. No review queue or automated acceptance service is running.

1. Acknowledge the email manually and assign the submission ID to an internal review record. Store contact information separately from public score data.
2. Run `node validate.js submission.json`. Archive the submitted file and validation report with SHA-256 hashes. A valid packet is not scientifically verified.
3. Confirm the submitter can provide code, logs, evaluation output and environment evidence. Check evidence archive hashes without executing submitted code on the website host. Review any reproduction code before running it in an isolated evaluation environment.
4. Match the selected task IDs, task-content revision, grader version, model/provider/version, harness commit, budgets, seeds, networking and retry policy. Resolve any unknown fields or reused trials. Check that errors and unfinished tasks are retained. Reject cherry-picked coverage claims.
5. For Coding, check `reward_repair` against the underlying evaluator output. For GUI, inspect delivered artifacts and scoring evidence. For Discovery, inspect version-level provenance, literature cutoff and Inspire Rate judgments; do not accept legacy five-level scores as the primary metric.
6. Reproduce a documented sample or the required full run under the agreed protocol. Record which checks were performed and their outcome. Do not claim complete reproducibility from a format-only check.
7. Record one decision: `needs_changes`, `accepted_unranked`, `accepted_ranked`, or `rejected`, with reviewer, timestamp, evidence references and reason. `accepted_ranked` requires an approved release/protocol and verified comparability. In the current preview, no new external packet is automatically eligible for a formal rank.
8. Prepare a public result projection containing approved model/configuration/scores/coverage/provenance only. Exclude submitter email, private links, keys, hidden references and restricted data. Run website tests and packaging checks, review the diff, then publish the approved snapshot. Keep the original packet private.
9. Notify the submitter manually of the decision and any published result URL. Record later corrections as a new version rather than silently overwriting the evidence.