Regulatory Templates Gate1

Regulatory Templates - Gate 1: Placeholder Mapping (Post Gate 0)

Overview

UPDATED: Gate 1 now maps placeholders from Gate 0 template to data sources. NO structure creation, NO logic addition.

Parent skill: regulatory-templates

Prerequisites:

• Context from regulatory-templates-setup
• Template base from regulatory-templates-setup

Output: Mapping of placeholders to backend data sources

Foundational Principle

Field mapping errors compound through Gates 2-3 and into production.

Gate 1 is the foundation of regulatory template accuracy:

• snake_case conversion: Python/Django ecosystem standard (PEP 8) - mixed conventions cause maintenance nightmares
• Data source prefixes: BACEN audits require data lineage traceability - "where did this value come from?"
• Interactive validation: No dictionary = no ground truth - user approval prevents assumption errors
• Confidence thresholds: Quality gates prevent low-confidence mappings from reaching production
• Dictionary checks: Consistency across team, audit trail for regulatory reviews

Every "shortcut" in Gate 1 multiplies through downstream gates:

• Skip snake_case → Gate 3 templates have mixed conventions → maintenance debt
• Skip prefixes → Gate 2 cannot trace data sources → debugging nightmares
• Auto-approve mappings → Gate 2 validates wrong assumptions → compliance violations
• Skip optional fields → Gate 1 fails confidence threshold → rework loops
• Lower thresholds → Low-confidence fields reach Gate 3 → production errors

Technical correctness in Gate 1 = foundation for compliance in production.

When to Use

Called by: regulatory-templates skill after Gate 0 template structure copy

Purpose: Map each placeholder to its data source - structure already defined in Gate 0

NO EXCEPTIONS - Technical Requirements Are Mandatory

Gate 1 field mapping requirements have ZERO exceptions. Every requirement exists to prevent specific failure modes.

Common Pressures You Must Resist

Technical Requirements (Non-Negotiable)

snake_case Conversion:

• ✅ REQUIRED: Convert ALL field names to snake_case
• ❌ FORBIDDEN: Use camelCase, PascalCase, or mixed conventions
• Why: Python/Django PEP 8 standard, grep-able patterns, maintenance

Data Source Prefixes:

• ✅ REQUIRED: {{ midaz_onboarding.organization.0.legal_document }}
• ❌ FORBIDDEN: {{ organization.legal_document }}
• Why: Data lineage traceability, multi-source disambiguation, audit compliance

Interactive Validation:

• ✅ REQUIRED: AskUserQuestion for EACH field mapping
• ❌ FORBIDDEN: Auto-approve HIGH confidence fields
• Why: No dictionary = no ground truth, user provides domain knowledge

Confidence Threshold:

• ✅ REQUIRED: Overall confidence ≥ 80%
• ❌ FORBIDDEN: Lower threshold or skip fields
• Why: Quality gate for Gate 2/3, prevents low-confidence mappings in production

Dictionary Check:

• ✅ REQUIRED: Check ~/.claude/docs/regulatory/dictionaries/ first
• ❌ FORBIDDEN: Skip check and use memory
• Why: Consistency, audit trail, error prevention

The Bottom Line

Shortcuts in field mapping = errors in production regulatory submissions.

Gate 1 creates the foundation for Gates 2-3. Technical correctness here prevents compliance violations downstream.

If you're tempted to skip ANY requirement, ask yourself: Am I willing to debug production BACEN submission failures caused by this shortcut?

Rationalization Table - Know the Excuses

Every rationalization below has been used to justify skipping requirements. ALL are invalid.

If You Find Yourself Making These Excuses

STOP. You are rationalizing.

The requirements exist to prevent these exact thoughts from causing errors. If a requirement seems "unnecessary," that's evidence it's working - preventing shortcuts that seem reasonable but create risk.

CRITICAL CHANGE

❌ OLD Gate 1 (Over-engineering)

• Created complex field mappings
• Added transformation logic
• Built nested structures
• Result: 90+ line templates

✅ NEW Gate 1 (Simple)

• Takes template from Gate 0
• Maps placeholders to single data source
• NO structural changes
• Result: <20 line templates

🔴 CRITICAL: NAMING CONVENTION - SNAKE_CASE STANDARD

ALL field names MUST be converted to snake_case:

• ✅ If API returns legalDocument → convert to legal_document
• ✅ If API returns taxId → convert to tax_id
• ✅ If API returns openingDate → convert to opening_date
• ✅ If API returns naturalPerson → convert to natural_person
• ✅ If API returns tax_id → keep as tax_id (already snake_case)

ALWAYS convert camelCase, PascalCase, or any other convention to snake_case.

🔴 CRITICAL: DATA SOURCES - ALWAYS USE CORRECT DOMAIN PREFIX

REFERENCE: See /docs/regulatory/DATA_SOURCES.md for complete documentation.

Available Data Sources (Reporter Platform):

Field Path Format: {data_source}.{entity}.{index?}.{field}

Examples: {{ midaz_onboarding.organization.0.legal_document }} | {{ midaz_transaction.operation_route.code }} | {{ midaz_transaction.balance.available }}

Common Mappings: CNPJ→organization.0.legal_document, COSIF→operation_route.code, Saldo→balance.available

RULE: Always prefix with data source! ❌ {{ organization.legal_document }} → ✅ {{ midaz_onboarding.organization.0.legal_document }}

Gate 1 Process

STEP 1: Check for Data Dictionary (FROM/TO Mappings)

HIERARCHICAL SEARCH - Dictionary first, Interactive Validation second:

Dictionary Path: ~/.claude/docs/regulatory/dictionaries/{category}-{code}.yaml

Dictionary contains: field_mappings (FROM→TO), transformations, pitfalls, validation_rules

🔴 CRITICAL: INTERACTIVE VALIDATION FOR TEMPLATES WITHOUT DICTIONARY

Data Dictionaries Location

Dicionários de dados disponíveis em: ~/.claude/docs/regulatory/dictionaries/

Consulte os dicionários existentes antes de iniciar o mapeamento de campos.

Interactive Validation Process (MANDATORY for templates without dictionary)

AskUserQuestion Implementation for Field Mapping

CRITICAL: Use AskUserQuestion tool with these patterns:

Note: "Other" option automatically added by AskUserQuestion for custom input.

Complete Interactive Validation Flow

Process: Read spec → Query MCP schemas → For EACH field: AskUserQuestion → Process response → Track approved/skipped

Output: { approvedMappings: [...], skippedFields: [...] }

Validation Rules for User Input

Valid path patterns for custom input:

Validation: If path doesn't match patterns → warn user but allow (may be valid custom path).

NAMING CONVENTION IN FIELD DISCOVERY

CRITICAL: ALWAYS CONVERT TO SNAKE_CASE!

Search patterns help FIND fields. Once found, CONVERT TO SNAKE_CASE!

Hierarchical Search Strategy

CRITICAL: Convert ALL discovered fields to snake_case!

Confidence Scoring System

Formula: Score = Base(30) + NameMatch(0-25) + SystemMatch(0-25) + TypeMatch(0-20) + ValidationMatch(0-20)

Validation with Examples

Process: Fetch sample → Apply transformation → Validate format

Example: CNPJ Base: "12345678000190" → slice:':8' → "12345678" → /^\d{8}$/ → ✓ valid (+20 confidence)

Agent Dispatch

Dispatch: Task(subagent_type: "finops-analyzer", model: "opus")

Pre-dispatch: Check dictionary at ~/.claude/docs/regulatory/dictionaries/{category}-{code}.yaml

Prompt includes: Template info, dictionary status/content (if exists), snake_case requirement, validation steps, output format

CRITICAL REQUIREMENTS:

Report Output: dictionary_status, field_mappings (code, name, required, source, transformation, confidence, validated, examples), validation_summary (total, mapped, coverage%, avg_confidence)

COMPLETION STATUS: COMPLETE, INCOMPLETE, or NEEDS_DISCUSSION

Capture Gate 1 Response

Response structure:

Documentation Sources

Official Regulatory Sources (SOURCE OF TRUTH)

Red Flags - STOP Immediately

If you catch yourself thinking ANY of these, STOP and re-read the NO EXCEPTIONS section:

Skip Patterns

• "Skip snake_case conversion for..."
• "Omit prefix for obvious fields"
• "Use camelCase this time"
• "Mixed conventions are fine"
• "Dictionary check is ceremony"

Partial Compliance

• "Convert only mandatory fields"
• "Prefix only ambiguous fields"
• "Auto-approve HIGH confidence"
• "Map only mandatory fields"
• "75% is close enough to 80%"

Experience-Based Shortcuts

• "I memorized these mappings"
• "I know where this comes from"
• "We've done this 50 times"
• "The pattern is obvious"
• "Dictionary won't exist anyway"

Justification Language

• "Unnecessary busywork"
• "Verbose and ugly"
• "Wasting user time"
• "Process over outcome"
• "Being pragmatic"
• "Close enough"
• "Everyone knows"

If You See These Red Flags

1. Acknowledge the rationalization ("I'm trying to skip snake_case")
2. Read the NO EXCEPTIONS section (understand why it's required)
3. Read the Rationalization Table (see your exact excuse refuted)
4. Follow the requirement completely (no modifications)

Technical requirements are not negotiable. Field mapping errors compound through Gates 2-3.

Pass/Fail Criteria

PASS Criteria

• ✅ COMPLETION STATUS: COMPLETE
• ✅ 0 Critical gaps (unmapped mandatory fields)
• ✅ Overall confidence score ≥ 80%
• ✅ All mandatory fields mapped (even if LOW confidence)
• ✅ < 10% of fields with LOW confidence
• ✅ Dynamic discovery via MCP executed
• ✅ Documentation was consulted (both official and implementation)
• ✅ CRM checked first for banking/personal data

FAIL Criteria

• ❌ COMPLETION STATUS: INCOMPLETE
• ❌ Critical gaps exist (mandatory fields unmapped)
• ❌ Overall confidence score < 60%
• ❌ > 20% fields with LOW confidence
• ❌ Documentation not consulted
• ❌ MCP discovery not performed
• ❌ Only checked one system (didn't check CRM + Midaz)

State Tracking

Critical Validations

Ensure these patterns are followed:

• Use EXACT patterns from Lerian documentation
• Apply filters like slice, floatformat as shown in docs
• Follow tipoRemessa rules: "I" for new/rejected, "S" for approved only
• Date formats must match regulatory requirements (YYYY/MM, YYYY-MM-DD)
• CNPJ/CPF formatting rules must be exact

Output to Parent Skill

Return to regulatory-templates: { gate1_passed: bool, gate1_context: {...}, uncertainties_count: N, critical_gaps: [], next_action: "proceed_to_gate2" | "fix_gaps_and_retry" }

Common Issues and Solutions

Dynamic Discovery Example

Finding "Agência" field for CADOC 4010:

Remember

1. CONVERT TO SNAKE_CASE - All fields must be snake_case (legal_document not legalDocument)
2. Use MCP for dynamic discovery - Never hardcode field paths
3. CRM first for banking/personal data - It has the most complete holder info
4. Official specs are SOURCE OF TRUTH - Regulatory requirements from government
5. Lerian docs show IMPLEMENTATION - How to create templates in their system
6. Template-specific knowledge is valuable - Always check for existing sub-skills
7. Confidence scoring is key - Always calculate and document confidence
8. Be conservative with mappings - Mark uncertain rather than guess
9. Capture everything - Gate 2 needs complete context with all attempted mappings
10. Reference both sources - Note official specs AND implementation examples
11. Risk assessment based on confidence - Low confidence = higher compliance risk

Important Distinction

⚠️ Regulatory Compliance vs Implementation

• WHAT (Requirements) = Official government documentation
• HOW (Implementation) = Lerian documentation examples
• When validating compliance → Use official specs
• When creating templates → Use Lerian patterns
• Never confuse implementation examples with regulatory requirements