cloudthinker-ai Managing CodeBall
Overview
Manage CodeBall AI-powered PR risk assessment, including automatic quality scoring, risk classification, auto-approval for safe changes, and integration with review workflows.
Key Capabilities
- Automatically assess PR risk level using AI analysis
- Auto-approve low-risk PRs to reduce reviewer burden
- Classify PRs by risk: low, medium, high
- Integrate risk scores into existing review workflows
- Track risk assessment accuracy over time
- Configure risk thresholds and auto-approval policies
Workflow
1 — Risk Assessment Configuration
RISK ASSESSMENT SETUP
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[ ] Enable AI risk scoring: YES / NO
[ ] Risk classification thresholds:
- Low risk (auto-approve eligible): score < ___
- Medium risk (standard review): score ___ - ___
- High risk (senior review required): score > ___
[ ] Analysis scope:
- All PRs: YES / NO
- Exclude draft PRs: YES / NO
- Exclude WIP PRs: YES / NO
2 — Auto-Approval Policy
AUTO-APPROVAL RULES
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[ ] Enable auto-approve for low-risk PRs: YES / NO
[ ] Additional conditions for auto-approve:
[ ] All CI checks pass
[ ] PR size < ___ lines
[ ] No changes to critical paths (auth, payments, etc.)
[ ] Author has > ___ merged PRs
[ ] Label auto-approved PRs: YES / NO
[ ] Post approval comment: YES / NO
3 — Review Routing
RISK-BASED ROUTING
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[ ] Low risk: auto-approve or assign to any reviewer
[ ] Medium risk: assign to team reviewer
[ ] High risk: assign to senior engineer + additional reviewer
[ ] Critical risk: assign to tech lead + security review
[ ] Add risk label to PR: YES / NO
Common Operations
Risk Score Dashboard
View aggregate risk scores across PRs, trending over time.
Auto-Approval Audit
Review auto-approved PRs to validate AI assessment accuracy.
Threshold Tuning
Adjust risk thresholds based on false positive/negative rates.
Troubleshooting
| Issue | Cause | Fix |
|---|---|---|
| All PRs marked high risk | Threshold too low | Increase high-risk threshold based on score distribution |
| False auto-approvals | Threshold too permissive | Lower auto-approve threshold or add conditions |
| Risk score not appearing | GitHub App permissions | Verify CodeBall app has read access to PR content |
| Assessment delayed | Queue backlog | Check CodeBall service status |
Output Format
Present results as a structured report:
Managing Codeball Report
════════════════════════
Resources discovered: [count]
Resource Status Key Metric Issues
──────────────────────────────────────────────
[name] [ok/warn] [value] [findings]
Summary: [total] resources | [ok] healthy | [warn] warnings | [crit] critical
Action Items: [list of prioritized findings]
Target ≤50 lines of output. Use tables for multi-resource comparisons.
Anti-Hallucination Rules
- NEVER assume resource names — always discover via CLI/API in Phase 1 before referencing in Phase 2.
- NEVER fabricate metric names or dimensions — verify against the service documentation or
--helpoutput. - NEVER mix CLI commands between service versions — confirm which version/API you are targeting.
- ALWAYS use the discovery → verify → analyze chain — every resource referenced must have been discovered first.
- ALWAYS handle empty results gracefully — an empty response is valid data, not an error to retry.
Counter-Rationalizations
| Shortcut | Counter | Why |
|---|---|---|
| "I'll skip discovery and check known resources" | Always run Phase 1 discovery first | Resource names change, new resources appear — assumed names cause errors |
| "The user only asked for a quick check" | Follow the full discovery → analysis flow | Quick checks miss critical issues; structured analysis catches silent failures |
| "Default configuration is probably fine" | Audit configuration explicitly | Defaults often leave logging, security, and optimization features disabled |
| "Metrics aren't needed for this" | Always check relevant metrics when available | API/CLI responses show current state; metrics reveal trends and intermittent issues |
| "I don't have access to that" | Try the command and report the actual error | Assumed permission failures prevent useful investigation; actual errors are informative |