CI Review

CI Review - Automated PR Code Review

Goal: every PR gets an automated, structured review comment.

Preflight

Ask before editing files:

1. Tool access: Claude Code, Codex, or both?
2. Reviewer topology: single reviewer or dual cross-validation?
3. Existing assets: .github/workflows/, .claude/agents/, AGENTS.md already present?
4. Endpoint type: official API vs relay/proxy? If Codex is used, read ~/.codex/config.toml and ~/.codex/auth.json.
5. Output language?
6. Review policy: focus (security/correctness/perf/style), severity scheme, comment-only vs merge-blocking, agent mode (full-context, slower) vs diff-only (faster).

Report scope and wait for confirmation.

Architecture Options

A) Claude only (simplest)

PR event -> install @anthropic-ai/claude-code -> claude --agent code-reviewer -p ... -> gh pr comment

Requires: ANTHROPIC_API_KEY (+ ANTHROPIC_BASE_URL if custom endpoint).

B) Codex only (verified; default recommendation)

PR event -> install @openai/codex -> codex review --base origin/$BASE -> filter output -> gh pr comment

Requires: OPENAI_API_KEY, ~/.codex/config.toml, ~/.codex/auth.json, and CODEX_RELAY_BASE_URL when using relay.

Mandatory: use codex review --base, not codex -a full-auto -p.

C) Claude + Codex in parallel

Use two independent jobs and one instruction source:

.claude/agents/code-reviewer.md  # source of truth
AGENTS.md -> symlink             # Codex reads this

Implementation

1) Agent definition

Create .claude/agents/code-reviewer.md:

---
name: code-reviewer
description: PR code reviewer. Autonomously explores codebase and outputs structured review.
tools: Read, Grep, Glob, Bash
model: opus
permissionMode: bypassPermissions
---

[Review instructions: checks, severity, output format]

Guidelines:

• model: opus (quality), sonnet (cost), haiku only for trivial repos.
• permissionMode: bypassPermissions is required in CI.
• Keep tools read-only unless explicitly needed.
• memory: project is optional for cross-PR memory.

Codex compatibility:

ln -s .claude/agents/code-reviewer.md AGENTS.md

YAML frontmatter in AGENTS.md is safe; Codex treats it as plain text.

2) Workflow essentials

Claude job

- uses: actions/checkout@v4
  with:
    ref: ${{ github.event.pull_request.head.sha }}
    fetch-depth: 0

- run: npm install -g @anthropic-ai/claude-code

- name: Fetch base branch
  run: git fetch origin ${{ github.event.pull_request.base.ref }}

- name: Run Claude Review
  env:
    BASE_BRANCH: ${{ github.event.pull_request.base.ref }}
    ANTHROPIC_BASE_URL: ${{ secrets.ANTHROPIC_BASE_URL }}
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
  run: |
    claude --agent code-reviewer \
      -p "Review changes on branch ${{ github.head_ref }} against origin/${BASE_BRANCH}" \
      > /tmp/review.txt 2>&1

Codex config (choose one)

Relay endpoint:

- name: Configure Codex (relay)
  env:
    CODEX_BASE_URL: ${{ secrets.CODEX_RELAY_BASE_URL }}
    CODEX_API_KEY: ${{ secrets.OPENAI_API_KEY }}
  run: |
    mkdir -p ~/.codex
    cat > ~/.codex/config.toml << EOF
    model_provider = "relay"
    model = "gpt-5.3-codex-spark"
    model_reasoning_effort = "xhigh"
    disable_response_storage = true

    [model_providers.relay]
    name = "relay"
    base_url = "${CODEX_BASE_URL}"
    wire_api = "responses"
    requires_openai_auth = true
    EOF
    printf '{"OPENAI_API_KEY": "%s"}' "$CODEX_API_KEY" > ~/.codex/auth.json

Official OpenAI endpoint:

- name: Configure Codex (official)
  env:
    CODEX_API_KEY: ${{ secrets.OPENAI_API_KEY }}
  run: |
    mkdir -p ~/.codex
    cat > ~/.codex/config.toml << EOF
    model = "codex-mini"
    disable_response_storage = true
    EOF
    printf '{"OPENAI_API_KEY": "%s"}' "$CODEX_API_KEY" > ~/.codex/auth.json

Run Codex review, extract final text, truncate if needed:

- name: Run Codex Review
  env:
    BASE_BRANCH: ${{ github.event.pull_request.base.ref }}
    PR_TITLE: ${{ github.event.pull_request.title }}
  run: |
    codex review \
      --base "origin/${BASE_BRANCH}" \
      --title "${PR_TITLE}" \
      > /tmp/raw.txt 2>&1 || true

    if grep -q "^codex$" /tmp/raw.txt; then
      tac /tmp/raw.txt | sed '/^codex$/q' | tac | sed '1d' > /tmp/review.txt
    else
      cp /tmp/raw.txt /tmp/review.txt
    fi

    if [ "$(wc -c < /tmp/review.txt)" -gt 60000 ]; then
      head -c 60000 /tmp/review.txt > /tmp/review_truncated.txt
      printf '\n\n---\n*Review truncated due to length.*\n' >> /tmp/review_truncated.txt
      mv /tmp/review_truncated.txt /tmp/review.txt
    fi

Post comment:

- name: Post Review Comment
  if: always()
  env:
    GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    PR_NUMBER: ${{ github.event.pull_request.number }}
  run: |
    if [ ! -s /tmp/review.txt ]; then
      echo "Review output is empty, skipping comment."
      exit 0
    fi

    gh pr comment "$PR_NUMBER" --body "$(cat <<EOF
    ## Codex CR: PR #${PR_NUMBER}

    $(cat /tmp/review.txt)

    ---
    *Automated review by [Codex](https://openai.com/codex) using [CR Agent](.claude/agents/code-reviewer.md)*
    EOF
    )"

Non-negotiable rules:

• ref: head.sha and fetch-depth: 0 for reliable diff context.
• Redirect order is > file 2>&1.
• codex review --base and prompt arg cannot be combined; use AGENTS.md for custom instructions.
• Pass PR title with --title; Codex does not read PR body/comments/issues.
• Codex requires both config.toml and auth.json.
• Use printf for auth.json, not echo.
• Use unquoted heredoc << EOF when shell var expansion is needed.
• Keep || true so review step non-zero does not suppress comment posting.

3) Secrets

Read local Codex config first (~/.codex/config.toml, ~/.codex/auth.json), then set repo secrets:

Set with gh secret set <NAME> --body "<VALUE>".

4) Testing and quality calibration

Local spike before pushing CI:

SPIKE_DIR="/tmp/spike-$(date +%s)"
mkdir -p "$SPIKE_DIR" && cd "$SPIKE_DIR"
git init && git checkout -b main
echo "console.log('hello')" > index.js
git add . && git commit -m "init"

git checkout -b test-pr
echo "const x = 1/0;" >> index.js
git add . && git commit -m "add bug"

echo "# Review Instructions\nFind bugs in the code." > AGENTS.md
codex review --base main

Then:

1. Push workflow to base branch first (main/staging). pull_request uses workflow from base branch.
2. Open a PR with intentional defects.
3. Verify: trigger -> reviewer run -> comment.
4. If failed: gh run view <RUN_ID> --log-failed.

Calibrate expected detection:

Observed baseline: Codex is strong on runtime/security issues; convention-only checks should stay in lint/guardrails.

Codex Data Boundaries

Output Parsing

Raw output includes exec logs; final review starts after the last line exactly equal to codex.

Example:

OpenAI Codex ...
...
exec ... git diff ...
exec ... sed ...
codex
- [P1] ...
- [P2] ...

Filter:

• Ubuntu/CI: tac /tmp/raw.txt | sed '/^codex$/q' | tac | sed '1d' > /tmp/review.txt
• macOS/local: tail -r /tmp/raw.txt | sed '/^codex$/q' | tail -r | sed '1d' > /tmp/review.txt

Pitfalls