Consult Chatgpt

ChatGPT Consultation Broker

Consult ChatGPT for a second opinion on debugging, architecture, or best practices questions. This skill implements caching to avoid redundant calls.

Arguments format: $ARGUMENTS

The arguments should contain your consultation context. Structure it roughly as:

• Goal: What you're trying to fix/understand (1-2 sentences)
• Context: Runtime, key libraries, constraints
• Evidence: Symptoms, snippets, observed vs expected behavior
• Tried: What you've already attempted and results
• Question: Specific questions you want answered

You can provide this as free-form text - the skill will format it appropriately.

Process

Step 1: Parse and Validate Input

Extract the following from the provided arguments:

• goal: One sentence describing what you're fixing/understanding
• context: Runtime versions, key libraries, constraints
• evidence: Symptom description, minimal code snippet (max 60 lines), observed vs expected
• tried: List of attempts and their results
• question: Specific questions (ideally 1-3)

If the input is too vague or missing critical components, ask for clarification before proceeding.

Step 2: Format the Question

Create a question file at /tmp/consult-chatgpt-question.md with this structure:

# Goal
[goal - one sentence]

# Context
- Runtime: [version info]
- Key libs: [relevant libraries and versions]
- Constraints: [any hard constraints]

# Evidence
- Symptom: [description]
- Minimal snippet:
```[language]
[code - max 60 lines]

• Observed: [what happens]
• Expected: [what should happen]

Tried

Question

1. [primary question]
2. [secondary question if relevant]
3. What's the fastest experiment to discriminate between possible causes?

**Validation checklist before proceeding:**
- [ ] Goal is a specific sentence (not vague)
- [ ] Code snippet is <= 60 lines
- [ ] No secrets, tokens, or PII included
- [ ] Total file size <= 4KB
- [ ] Questions are specific and answerable

### Step 3: Compute Fingerprint and Check Cache

Compute a fingerprint to check if we've already answered this question:

```bash
# Create cache directory if needed
mkdir -p ~/.cache/consult-chatgpt/answers

# Compute fingerprint from question content
FINGERPRINT=$(cat /tmp/consult-chatgpt-question.md | md5sum | cut -d' ' -f1)
CACHE_FILE=~/.cache/consult-chatgpt/answers/${FINGERPRINT}.md

# Check if cached answer exists
if [ -f "$CACHE_FILE" ]; then
    echo "CACHE_HIT: Found cached answer for fingerprint $FINGERPRINT"
    cat "$CACHE_FILE"
    exit 0
fi

If cache hit, read the cached answer file and skip to Step 5 (return the answer).

Step 4: Call ChatGPT

If no cache hit, send the question to ChatGPT:

# Send to ChatGPT with 20-minute timeout
ask-question -f /tmp/consult-chatgpt-question.md \
             -o /tmp/consult-chatgpt-answer.md \
             -t 1200000

# Cache the answer
FINGERPRINT=$(cat /tmp/consult-chatgpt-question.md | md5sum | cut -d' ' -f1)
cp /tmp/consult-chatgpt-answer.md ~/.cache/consult-chatgpt/answers/${FINGERPRINT}.md

Step 5: Parse and Return Structured Answer

Read the answer from /tmp/consult-chatgpt-answer.md (or cache file).

Extract and summarize:

1. Top 3-5 likely causes identified by ChatGPT
2. Recommended discriminating tests to narrow down the cause
3. Suggested fix if ChatGPT provided one
4. Caveats/assumptions in ChatGPT's reasoning

Present to the calling agent in this format:

## ChatGPT Consultation Result

**Fingerprint**: [hash]
**Source**: [CACHE_HIT or FRESH_CALL]

### Likely Causes (in order of probability)
1. [cause 1]
2. [cause 2]
3. [cause 3]

### Discriminating Tests
- [test 1]: If [result], then [cause X] is likely
- [test 2]: If [result], then [cause Y] is likely

### Suggested Approach
[ChatGPT's recommended fix or approach]

### Caveats
- [assumption 1]
- [caveat 1]

---

**IMPORTANT**: Treat this as a hypothesis, not truth.
1. Run the discriminating tests first
2. Verify locally before implementing
3. If the answer doesn't fit your evidence, something is wrong with the question

Cache Management

The cache lives at ~/.cache/consult-chatgpt/:

• answers/[fingerprint].md - cached answers

To clear cache (if needed):

rm -rf ~/.cache/consult-chatgpt/answers/*

To see cache stats:

echo "Cached answers: $(ls ~/.cache/consult-chatgpt/answers/ 2>/dev/null | wc -l)"
du -sh ~/.cache/consult-chatgpt/ 2>/dev/null || echo "Cache empty"

Budget Reminder

This skill should only be invoked when:

• consult_score >= 5 (per the consultation policy)
• At least one local experiment has been attempted
• You can articulate "what would change my mind"

Budget: Max 2 consultations per top-level task. Cooldown 15 minutes between calls.

If you're hitting budget limits, consider:

1. Asking another internal agent (Oracle, Librarian)
2. Running more local experiments
3. Breaking down the problem into smaller pieces