benchmark-publishing
The paper is one surface. The benchmark also needs a citable dataset, a
package name people can search, and a listing only where the site actually
accepts one. Fill the packet once. Paste it. Do not invent a listing on a
site with no public form.
Paper venues, templates, and camera-ready live in
paper-submission. AI-tool directories live
in website-directory-submission.
llms.txt for a site lives in geo-aeo. MCP
click-paths live in
mcp-directory-submission. This
skill is the order and the benchmark-specific rows.
Packet
Write this once, in the benchmark repo, with no placeholders:
- Name, one-line tagline, and a paragraph on what it measures
- Task count, grader, and licence (SPDX)
- Repo URL, npm name, docs URL
- Dataset URL and where the Croissant file will live
- Contact email that a person reads
- Maintainer GitHub usernames
- Citation text (
CITATION.cff)
- Paper URL, filled after arXiv exists
- Provenance: source, licence, and the date you retrieved it
Order
- Packet. Above. Stop if a field is a guess.
- Zenodo. Add
CITATION.cff. Add .zenodo.json only when you need
grants, communities, or access rights. See
references/hosting.md.
- Hugging Face dataset repo. README card, licence, tags. Croissant if
a venue or a user will ask for it.
- Official MCP registry, if the package ships an MCP server. Namespace
and auth first. Directory click-paths stay in the sibling skill.
- Harness or register, if you want the eval runnable inside someone
else's tool. Inspect Evals, lm-evaluation-harness, and HELM are different
doors. See
references/listings.md.
- Paper index, after the arXiv id exists. Hugging Face Papers. Do not
assume Papers with Code still takes submissions.
- GitHub topics and npm
keywords. Cheap, and they are yours.
- On the benchmark's own site:
llms.txt via geo-aeo, and
schema.org Dataset JSON-LD. Then AI-tool directories via
website-directory-submission.
- Aggregators last. Many of them have no public "add my benchmark"
form. Record that. Do not email a stale address from a blog.
Stop
Do not publish to the MCP registry, mint a Zenodo release, or send a
contact form without the owner. Do not print a token. Do not file a harness
PR that claims results you did not run. A paid "we will evaluate your model"
service is not a listing route.
See also
1---2name: benchmark-publishing3description: "List a published or public AI benchmark outside the paper: Hugging Face dataset repo and card, Croissant, the official MCP registry, Zenodo citation files, eval harnesses, paper indexes, and aggregators. Includes the order of operations and a one-page packet filled once. Use when a benchmark needs a dataset card, a DOI, a registry entry, or a leaderboard listing, or when someone asks where to list a benchmark that is not a conference submission."4metadata:5 source: https://skillmd.com/skills/pooriaarab/benchmark-publishing6---78# benchmark-publishing910The paper is one surface. The benchmark also needs a citable dataset, a11package name people can search, and a listing only where the site actually12accepts one. Fill the packet once. Paste it. Do not invent a listing on a13site with no public form.1415Paper venues, templates, and camera-ready live in16[`paper-submission`](../paper-submission/SKILL.md). AI-tool directories live17in [`website-directory-submission`](../website-directory-submission/SKILL.md).18`llms.txt` for a site lives in [`geo-aeo`](../geo-aeo/SKILL.md). MCP19click-paths live in20[`mcp-directory-submission`](../mcp-directory-submission/SKILL.md). This21skill is the order and the benchmark-specific rows.2223## Packet2425Write this once, in the benchmark repo, with no placeholders:2627- Name, one-line tagline, and a paragraph on what it measures28- Task count, grader, and licence (SPDX)29- Repo URL, npm name, docs URL30- Dataset URL and where the Croissant file will live31- Contact email that a person reads32- Maintainer GitHub usernames33- Citation text (`CITATION.cff`)34- Paper URL, filled after arXiv exists35- Provenance: source, licence, and the date you retrieved it3637## Order38391. **Packet.** Above. Stop if a field is a guess.402. **Zenodo.** Add `CITATION.cff`. Add `.zenodo.json` only when you need41 grants, communities, or access rights. See42 [`references/hosting.md`](references/hosting.md).433. **Hugging Face dataset repo.** README card, licence, tags. Croissant if44 a venue or a user will ask for it.454. **Official MCP registry,** if the package ships an MCP server. Namespace46 and auth first. Directory click-paths stay in the sibling skill.475. **Harness or register,** if you want the eval runnable inside someone48 else's tool. Inspect Evals, lm-evaluation-harness, and HELM are different49 doors. See [`references/listings.md`](references/listings.md).506. **Paper index,** after the arXiv id exists. Hugging Face Papers. Do not51 assume Papers with Code still takes submissions.527. **GitHub topics and npm `keywords`.** Cheap, and they are yours.538. **On the benchmark's own site:** `llms.txt` via `geo-aeo`, and54 schema.org `Dataset` JSON-LD. Then AI-tool directories via55 `website-directory-submission`.569. **Aggregators last.** Many of them have no public "add my benchmark"57 form. Record that. Do not email a stale address from a blog.5859## Stop6061Do not publish to the MCP registry, mint a Zenodo release, or send a62contact form without the owner. Do not print a token. Do not file a harness63PR that claims results you did not run. A paid "we will evaluate your model"64service is not a listing route.6566## See also6768- [`references/hosting.md`](references/hosting.md)69- [`references/listings.md`](references/listings.md)70- [`references/shipping-a-family.md`](references/shipping-a-family.md)71- [`../paper-submission/SKILL.md`](../paper-submission/SKILL.md)72- [`../mcp-directory-submission/SKILL.md`](../mcp-directory-submission/SKILL.md)73- [`../website-directory-submission/SKILL.md`](../website-directory-submission/SKILL.md)74- [`../geo-aeo/SKILL.md`](../geo-aeo/SKILL.md)