You are in AUTONOMOUS MODE. Do NOT ask questions. Analyze every aspect of the credit risk system systematically.
TARGET: $ARGUMENTS
If no arguments provided, analyze the entire credit risk codebase in the current working directory.
============================================================
PHASE 0: SYSTEM DISCOVERY
Auto-detect the credit risk system architecture:
TECH STACK:
requirements.txt / pyproject.toml -> Python (scikit-learn, XGBoost, LightGBM, TensorFlow)
pom.xml / build.gradle -> Java (FICO, custom engines)
package.json -> Node.js (custom scoring, API layer)
go.mod -> Go (decision engine, real-time scoring)
*.sas / *.r / *.R -> SAS / R (traditional statistical models)
- Jupyter notebooks (
*.ipynb) -> Model development and experimentation
MODEL COMPONENTS -- identify each:
- Scoring models: logistic regression, gradient boosting, neural networks, ensemble
- Feature stores and feature engineering pipelines
- Decision engines: rule-based, model-based, hybrid
- Model serving: batch scoring, real-time API, embedded scoring
- Monitoring: model drift detection, performance tracking, alerting
- Data sources: credit bureaus, application data, alternative data
Produce a system inventory before proceeding.
============================================================
PHASE 1: MODEL ARCHITECTURE ANALYSIS
ALGORITHM REVIEW:
- Identify all scoring algorithms in use (logistic regression, GBM, neural net, etc.).
- Assess model complexity vs interpretability tradeoff.
- Verify model selection rationale is documented.
- Check ensemble methods and sub-model combination strategy.
- Verify hyperparameter tuning process (grid search, Bayesian optimization, cross-validation).
FEATURE SELECTION:
- List all input features used in scoring models.
- Check for feature importance ranking (Gini, information value, chi-square).
- Identify correlated features that may cause multicollinearity.
- Verify feature stability analysis across time periods.
- Check for feature drift monitoring between training and production.
MODEL VALIDATION:
- Verify out-of-sample testing methodology (train/test split, k-fold, time-based).
- Check discrimination metrics: AUC-ROC, Gini coefficient, KS statistic.
- Check calibration metrics: Hosmer-Lemeshow, calibration curves.
- Verify PSI (Population Stability Index) monitoring.
- Check for stress testing and sensitivity analysis.
- Verify backtesting against historical default data.
For each finding: record file path, model component, severity, description, recommendation.
============================================================
PHASE 2: DATA QUALITY ASSESSMENT
INPUT DATA VALIDATION:
- Schema validation on incoming data (bureau data, application data).
- Data type enforcement (numeric fields not accepting strings, date formats).
- Range validation (age > 0, income > 0, credit score 300-850).
- Referential integrity checks across data sources.
MISSING VALUE HANDLING:
- Detection and flagging of missing values.
- Imputation strategies (mean, median, model-based, flag-and-fill).
- Missing value rate monitoring and alerting.
- Missing-at-random assumption validation.
- Flag hardcoded magic values used as missing indicators (-999, 9999, etc.).
OUTLIER DETECTION:
- Outlier detection in continuous features.
- Treatment strategy (winsorization, capping, exclusion).
- Documented and justified outlier thresholds.
- Extreme value handling in production scoring.
DATA LINEAGE:
- Data source documentation.
- Data transformation audit trail.
- Data versioning for model reproducibility.
- Training data snapshot archival.
============================================================
PHASE 3: REGULATORY COMPLIANCE
Audit against fair lending and consumer protection regulations.
FAIR LENDING (ECOA / Regulation B / FHA):
- Verify protected class variables are excluded from models: race, color, religion, national origin, sex, marital status, age (except as permitted).
- Scan for proxy variables that correlate with protected classes: zip code (race proxy), first name (ethnicity proxy), university attended (race proxy).
- Check disparate impact analysis documentation.
- Verify adverse action notice generation meets Regulation B requirements.
- Check that specific denial reasons are provided (not generic).
- Verify adverse action reason codes map to FCRA/ECOA requirements.
ADVERSE ACTION NOTICES:
- Verify specific reason codes generated for each denial.
- Check reason codes are ordered by impact (most impactful first).
- Verify reason code descriptions are consumer-friendly.
- Check that up to 4 principal reasons are provided per ECOA requirements.
- Verify adverse action notice templates include all required disclosures.
MODEL DOCUMENTATION (SR 11-7 / OCC 2011-12):
- Model development documentation (methodology, assumptions, limitations).
- Independent model validation.
- Ongoing monitoring plan documentation.
- Model inventory/registry with version tracking.
- Model risk tier classification documentation.
FCRA COMPLIANCE:
- Permissible purpose checks before pulling credit reports.
- Accurate furnishing logic for credit bureau reporting.
- Dispute resolution workflow.
- Consumer disclosure mechanisms.
============================================================
PHASE 4: BIAS DETECTION AND FAIRNESS
PROTECTED CLASS PROXY ANALYSIS:
- Compute correlation between each input feature and known protected attributes.
- Flag features with correlation > 0.3 to race, gender, age, or national origin.
- Check if zip code, education institution, or employer are used (common proxies).
- Verify alternative data sources (rent payments, utility data) are tested for bias.
DISPARATE IMPACT TESTING:
- Approval rate comparison across demographic groups.
- Four-fifths (80%) rule analysis.
- Marginal effect analysis on protected classes.
- Statistical significance testing on outcome differences.
- Disparate impact testing at multiple score thresholds.
FAIRNESS METRICS:
- Demographic parity measurement.
- Equalized odds / equal opportunity metrics.
- Predictive parity across groups.
- Calibration fairness (equal calibration across groups).
- Fairness-accuracy tradeoff documentation.
BIAS MITIGATION:
- Pre-processing techniques (reweighting, resampling).
- In-processing techniques (adversarial debiasing, fairness constraints).
- Post-processing adjustments (threshold optimization per group).
- Mitigation step documentation with impact analysis.
============================================================
PHASE 5: MODEL GOVERNANCE
VERSION CONTROL:
- Model version control with reproducibility artifacts.
- Training data, code, hyperparameters, and outputs versioned together.
- Model registry (MLflow, Weights & Biases, custom).
- Rollback capability to previous model versions.
CHAMPION-CHALLENGER FRAMEWORK:
- Challenger models tested alongside production champion.
- A/B testing or shadow scoring infrastructure.
- Defined and documented champion replacement criteria.
- Performance comparison methodology.
MONITORING AND ALERTING:
- Model performance degradation detection.
- PSI monitoring on input features.
- Concept drift detection on target variable.
- Automated alerting when metrics breach thresholds.
- Regular model performance reporting cadence.
APPROVAL AND AUDIT:
- Model approval workflow (development -> validation -> approval -> deployment).
- Audit trail on model changes and approvals.
- Segregation of duties between model developers and validators.
- Documented model risk assessments.
============================================================
PHASE 6: EXPLAINABILITY AND TRANSPARENCY
GLOBAL EXPLAINABILITY:
- Feature importance calculations (Gini, permutation, SHAP).
- Partial dependence plots or accumulated local effects.
- Global surrogate model documentation.
- Model behavior documentation for edge cases.
LOCAL EXPLAINABILITY:
- Individual prediction explanations (SHAP values, LIME).
- Reason code generation from model explanations.
- Explanation magnitudes mapped to adverse action reasons.
- Explanation consistency across similar applicants.
DOCUMENTATION:
- Model cards or model factsheets.
- Intended use and limitations documented.
- Performance metrics broken down by relevant segments.
- Known failure modes documented.
============================================================
SELF-HEALING VALIDATION (max 2 iterations)
After producing output, validate data quality and completeness:
- Verify all output sections have substantive content (not just headers).
- Verify every finding references a specific file, code location, or data point.
- Verify recommendations are actionable and evidence-based.
- If the analysis consumed insufficient data (empty directories, missing configs),
note data gaps and attempt alternative discovery methods.
IF VALIDATION FAILS:
- Identify which sections are incomplete or lack evidence
- Re-analyze the deficient areas with expanded search patterns
- Repeat up to 2 iterations
IF STILL INCOMPLETE after 2 iterations:
- Flag specific gaps in the output
- Note what data would be needed to complete the analysis
============================================================
OUTPUT
Credit Risk Model Analysis Report
System: [name/description]
Stack: [detected technologies]
Models Found: [count and types]
Summary
| Category |
Status |
Findings |
Critical |
| Model Architecture |
[PASS/WARN/FAIL] |
N |
N |
| Data Quality |
[PASS/WARN/FAIL] |
N |
N |
| Regulatory Compliance |
[PASS/WARN/FAIL] |
N |
N |
| Bias & Fairness |
[PASS/WARN/FAIL] |
N |
N |
| Model Governance |
[PASS/WARN/FAIL] |
N |
N |
| Explainability |
[PASS/WARN/FAIL] |
N |
N |
Model Inventory
| Model |
Type |
Features |
AUC |
Last Validated |
Status |
Detailed Findings
For each category with WARN or FAIL:
[Category Name]
| # |
Severity |
File |
Description |
Regulation |
Recommendation |
Regulatory Risk Assessment
- Fair Lending violations: [count and summary]
- Documentation gaps: [count and summary]
- Adverse action deficiencies: [count and summary]
- Model governance gaps: [count and summary]
Bias Analysis Summary
- Proxy variables identified: [list]
- Disparate impact findings: [summary by protected class]
- Fairness metric results: [summary table]
Remediation Priority
[Ordered list by regulatory risk and severity -- compliance issues first]
============================================================
NEXT STEPS
- "Run
/fraud-detection to analyze fraud detection components in the lending pipeline."
- "Run
/financial-compliance to review broader regulatory compliance (KYC/AML, BSA)."
- "Run
/owasp to audit the scoring API for security vulnerabilities."
- "Run
/analyze to trace data flows end-to-end across the system."
============================================================
SELF-EVOLUTION TELEMETRY
After producing output, record execution metadata for the /evolve pipeline.
Check if a project memory directory exists:
- Look for the project path in
~/.claude/projects/
- If found, append to
skill-telemetry.md in that memory directory
Entry format:
### /credit-risk — {{YYYY-MM-DD}}
- Outcome: {{SUCCESS | PARTIAL | FAILED}}
- Self-healed: {{yes — what was healed | no}}
- Iterations used: {{N}} / {{N max}}
- Bottleneck: {{phase that struggled or "none"}}
- Suggestion: {{one-line improvement idea for /evolve, or "none"}}
Only log if the memory directory exists. Skip silently if not found.
Keep entries concise — /evolve will parse these for skill improvement signals.
============================================================
DO NOT
- Do NOT modify any model code or scoring logic -- this is an analysis skill.
- Do NOT retrain or re-score any models.
- Do NOT access or display actual customer PII from training data or production.
- Do NOT make definitive legal conclusions -- flag issues for legal/compliance review.
- Do NOT skip regulatory compliance phases even if the system appears small.
- Do NOT assume fair lending compliance without testing -- always check for proxy variables.
- Do NOT conflate statistical correlation with confirmed disparate impact -- note confidence levels.
1---2name: credit-risk3description: "Audit credit risk modeling software for scoring algorithm accuracy, regulatory compliance (ECOA, FCRA, SR 11-7), bias and disparate impact testing, model governance lifecycle, and explainability.."4version: "2.0.1"5category: analysis6platforms:7 - CLAUDE_CODE8metadata:9 source: https://skillmd.com/skills/tinh2/credit-risk10---1112You are in AUTONOMOUS MODE. Do NOT ask questions. Analyze every aspect of the credit risk system systematically.1314TARGET: $ARGUMENTS1516If no arguments provided, analyze the entire credit risk codebase in the current working directory.1718============================================================19PHASE 0: SYSTEM DISCOVERY20============================================================2122Auto-detect the credit risk system architecture:2324TECH STACK:25- `requirements.txt` / `pyproject.toml` -> Python (scikit-learn, XGBoost, LightGBM, TensorFlow)26- `pom.xml` / `build.gradle` -> Java (FICO, custom engines)27- `package.json` -> Node.js (custom scoring, API layer)28- `go.mod` -> Go (decision engine, real-time scoring)29- `*.sas` / `*.r` / `*.R` -> SAS / R (traditional statistical models)30- Jupyter notebooks (`*.ipynb`) -> Model development and experimentation3132MODEL COMPONENTS -- identify each:33- Scoring models: logistic regression, gradient boosting, neural networks, ensemble34- Feature stores and feature engineering pipelines35- Decision engines: rule-based, model-based, hybrid36- Model serving: batch scoring, real-time API, embedded scoring37- Monitoring: model drift detection, performance tracking, alerting38- Data sources: credit bureaus, application data, alternative data3940Produce a system inventory before proceeding.4142============================================================43PHASE 1: MODEL ARCHITECTURE ANALYSIS44============================================================4546ALGORITHM REVIEW:47- Identify all scoring algorithms in use (logistic regression, GBM, neural net, etc.).48- Assess model complexity vs interpretability tradeoff.49- Verify model selection rationale is documented.50- Check ensemble methods and sub-model combination strategy.51- Verify hyperparameter tuning process (grid search, Bayesian optimization, cross-validation).5253FEATURE SELECTION:54- List all input features used in scoring models.55- Check for feature importance ranking (Gini, information value, chi-square).56- Identify correlated features that may cause multicollinearity.57- Verify feature stability analysis across time periods.58- Check for feature drift monitoring between training and production.5960MODEL VALIDATION:61- Verify out-of-sample testing methodology (train/test split, k-fold, time-based).62- Check discrimination metrics: AUC-ROC, Gini coefficient, KS statistic.63- Check calibration metrics: Hosmer-Lemeshow, calibration curves.64- Verify PSI (Population Stability Index) monitoring.65- Check for stress testing and sensitivity analysis.66- Verify backtesting against historical default data.6768For each finding: record file path, model component, severity, description, recommendation.6970============================================================71PHASE 2: DATA QUALITY ASSESSMENT72============================================================7374INPUT DATA VALIDATION:75- Schema validation on incoming data (bureau data, application data).76- Data type enforcement (numeric fields not accepting strings, date formats).77- Range validation (age > 0, income > 0, credit score 300-850).78- Referential integrity checks across data sources.7980MISSING VALUE HANDLING:81- Detection and flagging of missing values.82- Imputation strategies (mean, median, model-based, flag-and-fill).83- Missing value rate monitoring and alerting.84- Missing-at-random assumption validation.85- Flag hardcoded magic values used as missing indicators (-999, 9999, etc.).8687OUTLIER DETECTION:88- Outlier detection in continuous features.89- Treatment strategy (winsorization, capping, exclusion).90- Documented and justified outlier thresholds.91- Extreme value handling in production scoring.9293DATA LINEAGE:94- Data source documentation.95- Data transformation audit trail.96- Data versioning for model reproducibility.97- Training data snapshot archival.9899============================================================100PHASE 3: REGULATORY COMPLIANCE101============================================================102103Audit against fair lending and consumer protection regulations.104105FAIR LENDING (ECOA / Regulation B / FHA):106- Verify protected class variables are excluded from models: race, color, religion, national origin, sex, marital status, age (except as permitted).107- Scan for proxy variables that correlate with protected classes: zip code (race proxy), first name (ethnicity proxy), university attended (race proxy).108- Check disparate impact analysis documentation.109- Verify adverse action notice generation meets Regulation B requirements.110- Check that specific denial reasons are provided (not generic).111- Verify adverse action reason codes map to FCRA/ECOA requirements.112113ADVERSE ACTION NOTICES:114- Verify specific reason codes generated for each denial.115- Check reason codes are ordered by impact (most impactful first).116- Verify reason code descriptions are consumer-friendly.117- Check that up to 4 principal reasons are provided per ECOA requirements.118- Verify adverse action notice templates include all required disclosures.119120MODEL DOCUMENTATION (SR 11-7 / OCC 2011-12):121- Model development documentation (methodology, assumptions, limitations).122- Independent model validation.123- Ongoing monitoring plan documentation.124- Model inventory/registry with version tracking.125- Model risk tier classification documentation.126127FCRA COMPLIANCE:128- Permissible purpose checks before pulling credit reports.129- Accurate furnishing logic for credit bureau reporting.130- Dispute resolution workflow.131- Consumer disclosure mechanisms.132133============================================================134PHASE 4: BIAS DETECTION AND FAIRNESS135============================================================136137PROTECTED CLASS PROXY ANALYSIS:138- Compute correlation between each input feature and known protected attributes.139- Flag features with correlation > 0.3 to race, gender, age, or national origin.140- Check if zip code, education institution, or employer are used (common proxies).141- Verify alternative data sources (rent payments, utility data) are tested for bias.142143DISPARATE IMPACT TESTING:144- Approval rate comparison across demographic groups.145- Four-fifths (80%) rule analysis.146- Marginal effect analysis on protected classes.147- Statistical significance testing on outcome differences.148- Disparate impact testing at multiple score thresholds.149150FAIRNESS METRICS:151- Demographic parity measurement.152- Equalized odds / equal opportunity metrics.153- Predictive parity across groups.154- Calibration fairness (equal calibration across groups).155- Fairness-accuracy tradeoff documentation.156157BIAS MITIGATION:158- Pre-processing techniques (reweighting, resampling).159- In-processing techniques (adversarial debiasing, fairness constraints).160- Post-processing adjustments (threshold optimization per group).161- Mitigation step documentation with impact analysis.162163============================================================164PHASE 5: MODEL GOVERNANCE165============================================================166167VERSION CONTROL:168- Model version control with reproducibility artifacts.169- Training data, code, hyperparameters, and outputs versioned together.170- Model registry (MLflow, Weights & Biases, custom).171- Rollback capability to previous model versions.172173CHAMPION-CHALLENGER FRAMEWORK:174- Challenger models tested alongside production champion.175- A/B testing or shadow scoring infrastructure.176- Defined and documented champion replacement criteria.177- Performance comparison methodology.178179MONITORING AND ALERTING:180- Model performance degradation detection.181- PSI monitoring on input features.182- Concept drift detection on target variable.183- Automated alerting when metrics breach thresholds.184- Regular model performance reporting cadence.185186APPROVAL AND AUDIT:187- Model approval workflow (development -> validation -> approval -> deployment).188- Audit trail on model changes and approvals.189- Segregation of duties between model developers and validators.190- Documented model risk assessments.191192============================================================193PHASE 6: EXPLAINABILITY AND TRANSPARENCY194============================================================195196GLOBAL EXPLAINABILITY:197- Feature importance calculations (Gini, permutation, SHAP).198- Partial dependence plots or accumulated local effects.199- Global surrogate model documentation.200- Model behavior documentation for edge cases.201202LOCAL EXPLAINABILITY:203- Individual prediction explanations (SHAP values, LIME).204- Reason code generation from model explanations.205- Explanation magnitudes mapped to adverse action reasons.206- Explanation consistency across similar applicants.207208DOCUMENTATION:209- Model cards or model factsheets.210- Intended use and limitations documented.211- Performance metrics broken down by relevant segments.212- Known failure modes documented.213214215============================================================216SELF-HEALING VALIDATION (max 2 iterations)217============================================================218219After producing output, validate data quality and completeness:2202211. Verify all output sections have substantive content (not just headers).2222. Verify every finding references a specific file, code location, or data point.2233. Verify recommendations are actionable and evidence-based.2244. If the analysis consumed insufficient data (empty directories, missing configs),225 note data gaps and attempt alternative discovery methods.226227IF VALIDATION FAILS:228- Identify which sections are incomplete or lack evidence229- Re-analyze the deficient areas with expanded search patterns230- Repeat up to 2 iterations231232IF STILL INCOMPLETE after 2 iterations:233- Flag specific gaps in the output234- Note what data would be needed to complete the analysis235236============================================================237OUTPUT238============================================================239240## Credit Risk Model Analysis Report241242**System:** [name/description]243**Stack:** [detected technologies]244**Models Found:** [count and types]245246### Summary247248| Category | Status | Findings | Critical |249|----------|--------|----------|----------|250| Model Architecture | [PASS/WARN/FAIL] | N | N |251| Data Quality | [PASS/WARN/FAIL] | N | N |252| Regulatory Compliance | [PASS/WARN/FAIL] | N | N |253| Bias & Fairness | [PASS/WARN/FAIL] | N | N |254| Model Governance | [PASS/WARN/FAIL] | N | N |255| Explainability | [PASS/WARN/FAIL] | N | N |256257### Model Inventory258259| Model | Type | Features | AUC | Last Validated | Status |260|-------|------|----------|-----|----------------|--------|261262### Detailed Findings263264For each category with WARN or FAIL:265266#### [Category Name]267268| # | Severity | File | Description | Regulation | Recommendation |269|---|----------|------|-------------|------------|----------------|270271### Regulatory Risk Assessment272- **Fair Lending violations:** [count and summary]273- **Documentation gaps:** [count and summary]274- **Adverse action deficiencies:** [count and summary]275- **Model governance gaps:** [count and summary]276277### Bias Analysis Summary278- **Proxy variables identified:** [list]279- **Disparate impact findings:** [summary by protected class]280- **Fairness metric results:** [summary table]281282### Remediation Priority283[Ordered list by regulatory risk and severity -- compliance issues first]284285============================================================286NEXT STEPS287============================================================288289- "Run `/fraud-detection` to analyze fraud detection components in the lending pipeline."290- "Run `/financial-compliance` to review broader regulatory compliance (KYC/AML, BSA)."291- "Run `/owasp` to audit the scoring API for security vulnerabilities."292- "Run `/analyze` to trace data flows end-to-end across the system."293294295============================================================296SELF-EVOLUTION TELEMETRY297============================================================298299After producing output, record execution metadata for the /evolve pipeline.300301Check if a project memory directory exists:302- Look for the project path in `~/.claude/projects/`303- If found, append to `skill-telemetry.md` in that memory directory304305Entry format:306```307### /credit-risk — {{YYYY-MM-DD}}308- Outcome: {{SUCCESS | PARTIAL | FAILED}}309- Self-healed: {{yes — what was healed | no}}310- Iterations used: {{N}} / {{N max}}311- Bottleneck: {{phase that struggled or "none"}}312- Suggestion: {{one-line improvement idea for /evolve, or "none"}}313```314315Only log if the memory directory exists. Skip silently if not found.316Keep entries concise — /evolve will parse these for skill improvement signals.317318============================================================319DO NOT320============================================================321322- Do NOT modify any model code or scoring logic -- this is an analysis skill.323- Do NOT retrain or re-score any models.324- Do NOT access or display actual customer PII from training data or production.325- Do NOT make definitive legal conclusions -- flag issues for legal/compliance review.326- Do NOT skip regulatory compliance phases even if the system appears small.327- Do NOT assume fair lending compliance without testing -- always check for proxy variables.328- Do NOT conflate statistical correlation with confirmed disparate impact -- note confidence levels.