Review Comet docs: $ARGUMENTS
Pick the mode from the argument:
- PR mode: a number. Review the docs a pull request changes.
- Audit mode: a file or directory that exists in the working tree. Review it as it is now.
- Branch mode: a branch, tag, or commit that
git rev-parse --verifyaccepts. Review what it changes compared withmain, for example before opening a PR. Otherwise, ask which one is meant.
In PR mode, REQUIRED BACKGROUND: use review-comet-pr for its steps 2 to 4 (PR metadata,
existing comments, and the diff) and its CI check. This skill only covers documentation.
This skill gives advice to a human. Never edit files, commit, push, or post comments.
Shell notes. Run every command from the repository root. Each tool call may start a new shell, so save the variables you set in a state file named for this run. Use a different name for each run and mode, and keep it the same in every call of that run:
STATE="$(git rev-parse --git-dir)/review-comet-docs-$(printf %s "<pr-or-name>" | tr / -).env"
{ printf 'R=%q\n' "$R"; printf 'HEAD_SHA=%q\n' "$HEAD_SHA"; printf 'BASE=%q\n' "$BASE";
printf 'TARGET=%q\n' "$TARGET"; printf 'PATHS=%q\n' "$PATHS"; printf 'ADDED=%q\n' "$ADDED"; } \
> "$STATE"
Start every later call by loading it:
STATE="$(git rev-parse --git-dir)/review-comet-docs-$(printf %s "<pr-or-name>" | tr / -).env"; . "$STATE"
In zsh, $HEAD_SHA:spark/... is read as a history modifier, so always write "${HEAD_SHA}:<path>".
Save the state again after step 3 sets PATHS and step 4 sets ADDED. Stop and report the review as
incomplete if a command in steps 1 to 4 reports an error. A search that finds nothing, which makes
git grep exit with status 1, is not an error. Where a step gives a fallback, such as the guide
lookup in step 2 or the docs build in step 4, use it and keep going.
1. Select the Revisions
Find the remote that points at apache/datafusion-comet, because its name differs between clones:
R=$(git remote -v | awk '/apache\/datafusion-comet(\.git)? \(fetch\)/ {print $1; exit}')
If R is empty, use https://github.com/apache/datafusion-comet.git in its place.
PR mode. Read the PR's base branch and head commit, then fetch both without touching the working
tree. The leading + lets the fetch replace a ref after a force push.
gh pr view <pr> --repo apache/datafusion-comet --json baseRefName,headRefOid
git fetch "$R" "+refs/heads/<base>:refs/review/base-<pr>" "+pull/<pr>/head:refs/review/pr-<pr>"
TARGET=refs/review/base-<pr>; HEAD_SHA=$(git rev-parse refs/review/pr-<pr>)
Check that HEAD_SHA equals headRefOid. If it does not, the PR changed while you fetched it: run
both commands again, at most twice. From here on, read everything at HEAD_SHA, not at the ref,
which a later fetch can move.
Branch mode. Fetch main the same way. TARGET=refs/review/main and
HEAD_SHA=$(git rev-parse --verify <branch>):
git fetch "$R" "+refs/heads/main:refs/review/main"
Both modes. Find the merge base. It fails in a shallow clone, so deepen the history and try again, or report the comparison as incomplete:
BASE=$(git merge-base "$TARGET" "$HEAD_SHA") ||
{ git fetch --deepen=1000 "$R" && BASE=$(git merge-base "$TARGET" "$HEAD_SHA"); } ||
{ echo "no merge base: report the comparison as incomplete" >&2; false; }
If git rev-list --count "$BASE..$TARGET" is not zero, the target branch has moved since the change
branched. Check the claims against $TARGET too, and report what has changed there.
Audit mode. There is no base. List the files, including new files that are not committed yet, and
save the list as PATHS, as step 3 does in the other modes. The mechanical checks run on every file
in it; only the judgment checks in step 5 are capped:
PATHS="${STATE%.env}.paths"
git -c core.quotePath=false ls-files --cached --others --exclude-standard -- <path> > "$PATHS"
2. Read the Style Guide
Read docs/source/contributor-guide/style_guide.md in full on every run, from the revision under
review: git show "${HEAD_SHA}:docs/source/contributor-guide/style_guide.md" in PR and branch mode,
or the working tree in audit mode. If it does not exist there, as for a change made before the guide
was added, read it from $TARGET. If it does not exist there either, read the working tree's copy and say in
the report which copy you used. Stop and ask the user for a revision only when there is no copy at
all.
The guide is the rulebook, and its first section says what it covers. Cite its rule IDs (T3, N1,
P16, and so on), and do not restate or reinterpret its rules here. If the guide and this skill
disagree, the guide wins.
3. Select the Files
In PR and branch mode, list the changed files from Git, which is complete even for large PRs:
git -c core.quotePath=false diff --name-only "$BASE" "$HEAD_SHA"
From that list, <paths> below means:
.mdand.rstfiles underdocs/source, andREADME.md;- files that change
.doc(...)strings inCometConf.scalaor in contrib config classes; - files that change doc comments (Scaladoc, Javadoc, and Rust
///comments); AGENTS.mdand.ai/skills/**/SKILL.md;.svgfiles underdocs/source, whose<title>,<desc>, and<text>elements are text readers see.
Save that list to a file, one path per line. Commands read it with tr '\n' '\0' < "$PATHS" | xargs -0 -r,
which passes each path as one argument, so a file name with a space stays one path. Code files are
all kept here; step 4 keeps only their doc comments:
PATHS="${STATE%.env}.paths"
git -c core.quotePath=false diff --name-only "$BASE" "$HEAD_SHA" \
| grep -E '^(docs/source/.*\.(md|rst|svg)|README\.md|AGENTS\.md|\.ai/skills/.*/SKILL\.md|.*\.(scala|java|rs))$' \
| grep -v '/src/test/' > "$PATHS"
For doc comments, AGENTS.md, and skills, check the vocabulary and accuracy rules the guide names
for them, not the page conventions. Check the changed lines and the paragraphs around them, not the
whole page. Report a problem in unchanged text only when the change makes it wrong, or list it
separately as already on the page. Also read the code diff for the behavior the changed docs
describe, because docs are usually only as accurate as the code change beside them.
In every mode:
- Skip the content the guide's Generated Content section lists as generated or as records.
Review the hand-written text around the generated markers. On
mainthe text between the markers is empty, because the docs build fills it in, so any text a change adds between them is a hand edit, and a finding (P5). - Do the judgment checks in step 5 for at most 10 files per run, counting each file that has lines
in
$ADDEDafter the code-file filter in step 4. In audit mode, count the files step 1 lists, in that order. List any pages you skipped so the user can run the skill again on them.
4. Mechanical Checks
These checks produce candidates, not findings. Read each hit in context before reporting it. For the
wording checks, ignore hits inside fenced code blocks, inline code, quoted plan output or logs, and
text that quotes a phrase in order to discuss it (N6), such as the style guide's own examples. The
accuracy checks still apply to code font: config keys, operator names, and code examples are usually
written that way. For each check, record what it checked, the number of hits, and how many of them
are findings.
The commands use git grep -E to find candidate lines and perl to match them. They do not need a
Git built with PCRE, and they avoid two traps: the system grep on macOS has no -P, and \b
misses Markdown italics such as _native execution_, because _ counts as a word character.
Lines to check. In PR and branch mode, list the added lines with their file and line number, and
run the line checks below on this list only. In audit mode, run them on the files with
git grep -n --untracked.
ADDED=$(mktemp "$(git rev-parse --git-dir)/review-added.XXXXXX")
tr '\n' '\0' < "$PATHS" \
| xargs -0 -r git -c core.quotePath=false diff --no-ext-diff --src-prefix=a/ --dst-prefix=b/ -U0 \
"$BASE" "$HEAD_SHA" -- \
| awk '/^--- /{h=1; next}
h && /^\+\+\+ b\//{f=substr($0,7); sub(/\t$/,"",f); h=0; next}
/^@@/{h=0; split($3,a,/[+,]/); n=a[2]; next}
/^\+/{print f":"n": "substr($0,2); n++}' > "$ADDED"
In a code file, only doc comments and .doc(...) strings are text to check. Before the wording checks,
leave out its other added lines:
perl -i -ne 'print if !/\.(scala|java|rs):\d+: / || /:\d+: \s*(\/\*\*|\*|\/\/[\/!]|\.doc\(|")/' "$ADDED"
"Native" and "vectorized". Check each hit against N1-N6 and T10. Which page the hit is on
matters, because N3 allows more in the contributor guide:
P='(?:(?<![A-Za-z0-9])native(?:ly)?(?![A-Za-z0-9])|vectori[sz]ed)'
perl -ne "print if /^.+?:\d+: .*$P/i" "$ADDED"
tr '\n' '\0' < "$PATHS" \
| xargs -0 -r git -c core.quotePath=false grep -n -i --untracked -E 'native|vectori[sz]ed' -- \
| perl -ne "print if /^.+?:\d+:.*$P/i" # audit
Config keys. Every spark.comet.* key a page names must exist (P13). List the keys the text
names, then the keys the code defines at the revision under review. <rev> is "$HEAD_SHA". In
audit mode, leave <rev> out entirely: an empty "$HEAD_SHA" is an empty revision, which Git
rejects. The second command reads definitions that span two lines, expands
createExecEnabledConfig, and leaves out deprecated alternative names and unexpanded templates:
perl -ne 'print "$1\n" while /(spark\.comet\.[\w.<>]*[\w>])/g' "$ADDED" | sort -u
tr '\n' '\0' < "$PATHS" \
| xargs -0 -r git -c core.quotePath=false grep -h --untracked -o \
-E 'spark\.comet\.[A-Za-z0-9_.<>]*[A-Za-z0-9_>]' -- | sort -u # audit
git grep -h -A1 -E '(conf|ConfigBuilder|createExecEnabledConfig)\(' <rev> -- spark/src/main contrib \
| perl -0777 -ne '
while (/\b(?:conf|ConfigBuilder)\(\s*s?"([^"]+)"/g) {
($k = $1) =~ s/\$\{?COMET_EXEC_CONFIG_PREFIX\}?/spark.comet.exec/;
$k =~ s/\$\{?COMET_PREFIX\}?/spark.comet/;
print "$k\n" if $k =~ /^spark\.comet\./ && $k !~ /\$/;
}
while (/\bcreateExecEnabledConfig\(\s*"([^"]+)"/g) { print "spark.comet.exec.$1.enabled\n" }' \
| sort -u
If the second command prints nothing, it is broken: stop and report that, rather than flagging every key. Some keys are built from a name at run time:
spark.comet.expression.<Name>.enabledand.allowIncompatible, also written with an uppercase placeholder such asEXPRNAME:<Name>is the Spark expression class name, as in theQueryPlanSerdeserde maps, not a Comet or protobuf name. Some expressions use their own opt-in key instead, set bynativeOptInConfigKeyOverride, such asspark.comet.caseConversion.enabled.spark.comet.operator.<Name>.allowIncompatible:<Name>is the Spark operator class name, such asProjectExec.spark.comet.datafusion.<key>: not a Comet config. Comet passes it to DataFusion only whenspark.comet.exec.respectDataFusionConfigs=true, which is off by default. Check the key against DataFusion.
Before reporting a missing key, search the code for it, because a few keys are defined in other
ways, such as spark.comet.version in Plugins.scala. A deprecated key that a page still uses is a
finding (P16).
Operator names. Check operator names against O1, class names against O2, and expressions
against O4. List the concrete operator classes and the classes that override their printed name:
git grep -h -E 'case class (Comet|Iceberg)' <rev> -- spark/src/main contrib \
| perl -ne 'print "$1\n" if /case class ((?:Comet|Iceberg)[A-Za-z]+Exec)\b/' | sort -u
git grep -n -E 'override (def|val|lazy val) (nodeName|simpleString)' <rev> -- spark/src/main contrib
spark/src/main includes the per-Spark-version source directories. If the first command prints
nothing, it is broken: stop and report that. A class's nodeName is its name without Exec, unless
it overrides nodeName. Compare each name in the text with these, as O1 describes, not with a rule
about suffixes. When the page is an operator reference, also report concrete operators that it
leaves out. Leave out planning markers such as CometScanExec, which never appear in a final plan,
and contrib operators, which belong on their contrib page. A name the text calls a protobuf message,
such as ToJson, is not an operator: check it in native/proto instead. When you search the code
for a class name, git grep -E has no \b, so end the pattern with ([^A-Za-z0-9_]|$).
Docs build. Links, anchors, images, toctrees, and page structure, in Markdown and
reStructuredText, are checked by building the docs the way the published site is built. This
catches what a reading of the changed lines cannot, such as a link on an unchanged page that a
renamed heading breaks, or a page that no toctree includes. Each revision builds with its own
docs/requirements.txt, so a change that updates a Sphinx extension is built with the new version.
venv installs those requirements into a virtual environment named after their hash and prints its
path, so revisions with the same requirements share one environment. Each tool call may start a new
shell, so define venv and build in every call that uses them:
venv() { # $1 = revision, or worktree in audit mode; prints the environment's path
if [ "$1" = worktree ]; then req=$(cat docs/requirements.txt) || return 1
else req=$(git show "$1:docs/requirements.txt") || return 1; fi
V="$(git rev-parse --git-dir)/review-comet-docs-venv-$(printf '%s\n' "$req" | git hash-object --stdin | cut -c1-12)"
[ -x "$V/bin/sphinx-build" ] || {
PY=$(for p in python3.13 python3.12 python3.11 python3.10 python3; do
command -v $p >/dev/null && $p -c 'import sys; sys.exit(sys.version_info < (3, 10))' && { echo $p; break; }
done)
[ -n "$PY" ] && "$PY" -m venv --clear "$V" >&2 && printf '%s\n' "$req" > "$V.txt" &&
"$V/bin/pip" install -q --disable-pip-version-check -r "$V.txt" >&2
} || { rm -rf "$V"; echo "docs build not checked: needs Python 3.10 or later" >&2; return 1; }
printf '%s\n' "$V"
}
Then build the base and the head and compare their warnings. build copies the expression templates
into the per-version pages, as docs/build.sh does, so links to those pages resolve. It returns an
error if any step fails, including Sphinx itself, so a failed build is never read as a clean one:
build() { # $1 = revision, or worktree in audit mode; $2 = output directory
VENV=$(venv "$1") || return 1
mkdir -p "$2/src/docs" || return 1
if [ "$1" = worktree ]; then
cp -R docs/source "$2/src/docs/" && profiles=$(cat docs/build.sh) || return 1
else
git archive "$1" docs/source | tar -x -C "$2/src" && profiles=$(git show "$1:docs/build.sh") ||
return 1
fi
T="$2/src/docs/source/user-guide/latest/compatibility/expressions"
for p in $(printf '%s\n' "$profiles" | sed -n 's/^SPARK_PROFILES=(\(.*\))/\1/p'); do
mkdir -p "$T/$p" && cp "$T/_category_template"/*.md "$T/$p/" || return 1
done
rm -rf "$T/_category_template"
"$VENV/bin/sphinx-build" -q -b html -n -D mermaid_output_format=raw "$2/src/docs/source" \
"$2/html" > "$2/log.txt" 2>&1 || { echo "sphinx-build failed: $2/log.txt" >&2; return 1; }
sed -e "s#$2/src/##" -e 's/:[0-9]*: /: /' "$2/log.txt" | sort > "$2/warnings.txt"
}
WORK=$(cd "$(mktemp -d)" && pwd -P)
build "$BASE" "$WORK/base" && build "$HEAD_SHA" "$WORK/head" &&
comm -13 "$WORK/base/warnings.txt" "$WORK/head/warnings.txt"
build worktree "$WORK/head" && cat "$WORK/head/warnings.txt" # audit
Every warning that only the head build has is a finding: a broken link, anchor, or image, a page
missing from a toctree (P6), or a heading problem (P2). One exception: the published per-version
pages under compatibility/expressions/spark-*/ also get a ## <Expression> heading from
GenerateDocs for each expression whose serde returns compatibility notes, incompatible reasons, or
unsupported reasons (renderExpressionCompatNotes). This build copies only the templates, so a link
to one of those anchors warns here even when the published page has it. Before reporting such a link
as checked by hand, confirm both: the serde map that GenerateDocs.categoryPages uses for the
linked page (string.md reads QueryPlanSerde.stringExpressions, math.md reads
mathExpressions, and so on) contains the expression's Spark class at the linked Spark version,
since some are registered only in a version shim; and that class's serde returns one of those notes.
The heading is the class's simple name, such as ## Concat, so a link to that heading on another
category page is a broken link. In audit mode, report the warnings for the audited pages. If the
docs requirements cannot be installed or a build fails, say that the build was not checked, and
check that each changed link's target file exists.
Sphinx does not count H1s. Check that each new or changed Markdown page has one H1 with the parser
the docs build uses, which knows where code blocks start and end. It uses venv from above;
without an environment, count the H1s by reading the page and say so:
H1='import sys; from markdown_it import MarkdownIt
print(sum(t.type == "heading_open" and t.tag == "h1" for t in MarkdownIt("commonmark").parse(sys.stdin.read())))'
git show "${HEAD_SHA}:<file>" | "$(venv "$HEAD_SHA")/bin/python" -c "$H1"
"$(venv worktree)/bin/python" -c "$H1" < <file> # audit
Other structure checks. A new file has the ASF header (P1). A link to Comet's published docs by
URL breaks P4 on pages under docs/source, but not in README.md. List the pages the change
renames or deletes; each one needs an entry in redirects in docs/source/conf.py (P17). This
check needs a base, so audit mode skips it:
git diff --name-status --diff-filter=DR "$BASE" "$HEAD_SHA" -- docs/source
External links. For external links on the lines under review (at most 20 in audit mode), check
the status with curl -s -o /dev/null -w '%{http_code}' -I -L --max-time 10 <url>. Report a 404 as
a finding. If the request fails or the network is unavailable, say the link was not checked.
Version claims and markers. Find version claims to check against P8, and TODO or XXX
markers, which break P7:
V='(?:(?:Spark|Iceberg) [0-9]\.[0-9x]+\+?|\b(?:TODO|XXX)\b)'
perl -ne "print if /^.+?:\d+: .*$V/" "$ADDED"
tr '\n' '\0' < "$PATHS" \
| xargs -0 -r git -c core.quotePath=false grep -n --untracked -E 'Spark [0-9]|Iceberg [0-9]|TODO|XXX' -- \
| perl -ne "print if /$V/" # audit
CI runs prettier and the Apache RAT license check on PRs, so do not repeat them. If CI has not run them yet, for example on a draft, say they were not checked. Mention them otherwise only when CI has failed on them.
5. Judgment Checks
- Is it true? For every sentence that names a class, rule, config, default, or file, read the
code it describes at
HEAD_SHA, or in the working tree in audit mode. A sentence can follow every wording rule and still be wrong: a scan that no longer exists, a config default that has changed, or an operator described as Rust-implemented that runs on the JVM. Check numbers, versions, and names first, and check that a condition in the text is the whole condition in the code. Pages that describe architecture or test infrastructure go stale most often. A false or outdated claim breaksP16. This is the most valuable check in this skill. - Does it agree with the rest of the docs? A new sentence can contradict an older one on the
same page, or a count such as "five configs" can stop matching the list below it (
P16). When a change rewrites the description of a config or class, searchdocs/sourcefor the same name and check the other descriptions. - Which axis? When you suggest a replacement for vague "native", choose the term the code
supports: in the Comet pipeline (
T1), Rust-implemented (T3), or JVM-side (T4). Never suggest Rust-implemented without checking the code. - Page conventions. Check
P3,P7,P8,P9,P10,P11,P12,P14, andP18by reading the page. For plan output (P12,P16), check that the operator names exist, that the tree shape is one Spark could print, and that messages in it match the strings in the code. Do not ask for new plan output to be invented. - Same-change updates. If the change adds an operator or expression, check the pages
P15lists. If it adds a config, check that its.doc(...)string follows the guide, sinceGenerateDocspublishes it, and that any page that documents the config by hand mentions it.
6. Severity
- Must-fix. The page is wrong or broken: a false or outdated claim (
P16), a config key, class, or file that does not exist (P13), a broken link, anchor, or image, a hand edit to generated content, a new file without the ASF header, or a new page missing from a toctree. When a sentence is both false and badly worded, report it once, as must-fix, with a rewrite that fixes both. - Should-fix. The page breaks a style guide rule, but what it says is true. Most
T,N,O, andPfindings belong here. - Nit. A one-off change that would make the text clearer, but that no rule requires. Nits carry no rule ID.
A problem that keeps coming up and that no rule covers is a guide gap, not a finding. Report it so the guide can be extended.
7. Output
When this skill runs as part of a review-comet-pr review, put its findings into that skill's
Findings and Suggested Review Comments, and drop the nits. That skill's review bar has no
third tier, and its rules win for anything posted to a PR. On its own, use this format:
- Scope. The mode, the revisions reviewed (the base and head commits, and which guide version), the files reviewed, and the files skipped, with the reason.
- Mechanical checks. A table with each check, its hit count, and how many hits are findings.
- Findings, grouped as must-fix, should-fix, and nit. For each one give the
path:line, the rule ID, the text quoted from the page, and a suggested rewrite. Keep rewrites short and plain, and change only what the rule requires. - Guide gaps.
If there are no findings, say so.
Checking This Skill
After changing this skill or the style guide, run it on these cases and compare the results. The
expected results were checked on main in October 2026, so a later fix to a page can change them.
Each row lists results the run must report and false positives it must not. A run can also report
other real findings.
| Run | Expected |
|---|---|
PR mode on #6786, which predates the guide, so step 2 reads it from $TARGET or the working tree |
Must-fix: datasources.md and the new .doc string claim more than shouldBridgeUnsupportedScan does |
Audit docs/source/user-guide/latest/understanding-comet-plans.md |
No finding for Comet's native shuffle, CometNativeScan, CometWindowExec, or config keys |
Audit docs/source/contributor-guide/timezones.md |
No finding for native code or native plan, or for ToJson and ToCsv, which are protobuf message names |
Audit docs/source/user-guide/latest/compatibility/expressions |
Lists all 16 files, reviews 10 of the templates and index pages, names the skipped ones, and skips only text between markers |
Audit a scratch page containing _native execution_ and "Spark's native shuffle" |
N1 and N5 findings |