Add a task

1. Prepare a task

Choose a real scientific problem that requires judgment and has a checkable outcome. You can submit research materials without implementing the entire harness.

Keep inputs and references separate

The agent may access the task materials, but not the answer key or hidden checks. Include data provenance and permission to share.

2. Submit

Website submission

1. Sign in and complete your profile.
2. Create a submission, enter the task details and save a draft.
3. Add files or reviewer-accessible links.
4. Submit when the materials are ready.

Files

Upload up to 50 MB per file. For larger materials, provide a Google Drive or Dropbox link with access for the review team.

3. Follow review

Track status and comments in My submissions.

• Under review: the team checks scientific relevance, reproducibility and evaluation quality.
• Changes requested: update the materials and resubmit.
• Accepted / not selected: the review decision appears with your submission.

Submitting materials does not automatically publish them.

For executable task packages

Package

Use the Harbor task layout: instruction, starting environment, declared outputs and independent verifier.

Validate

Include a reproduction command and expected result. Test the reference solution where available and explain the required resources.

Share

Open a pull request if you have repository access, or attach the code link to the website submission.

Other contributions

We also welcome domain review, datasets, software environments and verification methods. Contact the team to collaborate.

Submitting an agent’s evaluation scores? Follow result submission.