Prowler Compliance

When to Use

Use this skill when:

• Creating a new compliance framework for any provider — decide universal vs legacy first (see below)
• Syncing an existing framework with an upstream source of truth (CIS, FINOS CCC, CSA CCM, NIST, ENS, etc.)
• Adding requirements to existing frameworks, or extending a universal framework to a new provider
• Mapping checks to compliance controls
• Adding ConfigRequirements guardrails so configurable checks can't silently satisfy a requirement with a loosened config
• Auditing existing check mappings as a cloud auditor ("are these mappings correct?", "which checks apply?", "review the mappings")
• Adding a new legacy output formatter (table dispatcher + per-provider classes + CSV models)
• Fixing JSON bugs: duplicate IDs, empty Version, wrong Section, stale check refs, inconsistent FamilyName, padded tangential check mappings
• Investigating why a finding/check isn't showing under the expected compliance framework in the UI
• Understanding compliance framework structures and attributes

The authoritative contributor doc is docs/developer-guide/security-compliance-framework.mdx — keep this skill and that doc consistent when either changes. For reviewing a compliance PR, use the sister skill prowler-compliance-review instead.

Universal vs Legacy: The First Decision

Prowler supports two JSON schemas. Choosing wrong means unnecessary Python code, so decide this before anything else. At load time both converge: legacy files are adapted into the universal ComplianceFramework model (adapt_legacy_to_universal()), so the difference is about authoring cost and capabilities, not about what the rest of Prowler sees.

Side-by-side comparison

When to use which

Use universal when (any of these):

• The framework is new to Prowler — no existing attribute class, no existing formatter. This is the default: zero Python changes needed.
• The framework spans (or will span) more than one provider — DORA, CSA CCM, CIS Controls. One file covers all providers; extending to a new provider is a one-line checks edit.
• The attribute shape is unique to this framework — declare it in attributes_metadata instead of adding a Pydantic class to the Union.

Use legacy only when extending an existing legacy family:

• A new version of a shipped legacy framework (CIS 8.0 for AWS → new cis_8.0_aws.json, same CIS_Requirement_Attribute, same cis/ formatter).
• An existing legacy framework for a new provider (ENS for m365 → new ens_rd2022_m365.json + ens_m365.py transformer).
• Consistency with the family matters more than the universal benefits — a lone cis_8.0_aws in universal format while 20+ CIS files stay legacy would fragment the family.

Never: start a brand-new single-provider framework as legacy "because it's only AWS today". Universal handles single-provider fine (the checks dict just has one key) and you skip 3 output files + 3 registrations.

The same requirement in both schemas

Universal (prowler/compliance/my_framework_1.0.json):

{
  "framework": "My-Framework",
  "name": "My Framework 1.0",
  "version": "1.0",
  "description": "...",
  "attributes_metadata": [
    {"key": "Section", "type": "str", "required": true},
    {"key": "Service", "type": "str"}
  ],
  "outputs": {"table_config": {"group_by": "Section"}},
  "requirements": [
    {
      "id": "MF-1.1",
      "name": "Root MFA",
      "description": "Root account must have MFA enabled.",
      "attributes": {"Section": "IAM", "Service": "iam"},
      "checks": {
        "aws": ["iam_root_mfa_enabled"],
        "azure": []
      }
    }
  ]
}

Legacy (prowler/compliance/aws/my_framework_1.0_aws.json — plus a second file per extra provider, plus formatter + registrations):

{
  "Framework": "My-Framework",
  "Name": "My Framework 1.0 for AWS",
  "Version": "1.0",
  "Provider": "AWS",
  "Description": "...",
  "Requirements": [
    {
      "Id": "MF-1.1",
      "Name": "Root MFA",
      "Description": "Root account must have MFA enabled.",
      "Attributes": [
        {"ItemId": "MF-1.1", "Section": "IAM", "Service": "iam"}
      ],
      "Checks": ["iam_root_mfa_enabled"]
    }
  ]
}

Same control, but the universal file already covers Azure, validates its own attribute schema, and renders table/CSV/OCSF with no code. Field-by-field references for each schema follow below.

Architecture (Mental Model)

Prowler compliance is a four-layer system. Bugs usually happen where one layer doesn't match another, so know all four before touching anything.

Layer 1: SDK / Core Models — prowler/lib/check/

All in Pydantic v1 (from pydantic.v1 import ...). Three model groups live in compliance_models.py:

Legacy tree — Compliance → Compliance_Requirement / Mitre_Requirement:

• One *_Requirement_Attribute class per framework family. Registered today (Union order matters): ASDEssentialEight, CIS, ENS, ISO27001_2013, AWS_Well_Architected, KISA_ISMSP, Prowler_ThreatScore, CCC, C5Germany, CSA_CCM, STIG (Okta IDaaS), and Generic_Compliance_Requirement_Attribute as fallback.
• Generic MUST stay LAST in Compliance_Requirement.Attributes: list[Union[...]] — Pydantic v1 tries union members in order; Generic first would swallow every framework-specific attribute. NIST 800-53/CSF, PCI DSS, GDPR, HIPAA, SOC2, FedRAMP, SecNumCloud etc. intentionally use Generic.
• A root_validator rejects empty Framework, Provider or Name.
• MITRE uses the separate Mitre_Requirement model (Tactics, SubTechniques, Platforms, TechniqueURL at requirement top level, per-provider Mitre_Requirement_Attribute_{AWS,Azure,GCP}).

Universal tree — ComplianceFramework → UniversalComplianceRequirement:

• Flat attributes: dict per requirement, schema declared in attributes_metadata (key, label, type, enum, required, enum_display, enum_order, output_formats). A root_validator rejects missing required keys, unknown keys (drift guard), enum violations, and int/float/bool type mismatches. If attributes_metadata is omitted, no validation runs.
• checks: dict[provider, list[check_id]] — the provider list of the framework is derived from these keys (get_providers() / supports_provider()); the top-level provider field is only a fallback.
• outputs.table_config (group_by, split_by, scoring, labels) drives the CLI table; outputs.pdf_config exists in the model but is not consumed by the API PDF pipeline yet (see Layer 4).

Guardrails — Compliance_Requirement_ConfigConstraint:

• Fields Check, ConfigKey, Operator (lte|gte|eq|in|subset|superset), Value, optional Provider (required in universal multi-provider files).
• A root_validator rejects Value/Operator type mismatches at load time.
• Evaluation is centralized in prowler/lib/check/compliance_config_eval.py (evaluate_config_constraints, apply_config_status, get_effective_status, CONFIG_NOT_VALID_PREFIX = "Configuration not valid for this requirement."), shared by CSV/OCSF/table outputs and the API backend. A violated constraint forces the requirement to FAIL and prepends the reason to status_extended. Constraints whose ConfigKey is absent from audit_config are skipped (defaults assumed compliant).

Loaders:

• Compliance.get_bulk(provider) — legacy: scans only prowler/compliance/{provider}/ (+ external JSONs via the prowler.compliance entry-point group). Does NOT see top-level universal files.
• get_bulk_compliance_frameworks_universal(provider) — scans both the top-level prowler/compliance/ and every provider subdirectory, adapting legacy files via adapt_legacy_to_universal() (flattens Attributes[0] to a dict, wraps Checks as {provider: [...]}, infers attributes_metadata). Also loads external universal frameworks via the prowler.compliance.universal entry-point group (built-ins win collisions).
• get_check_compliance(finding, provider_type, bulk_checks_metadata) lives in prowler/lib/outputs/compliance/compliance_check.py (not in lib/check/compliance.py). It builds the per-finding dict keyed f"{Framework}-{Version}" only when Version is non-empty — an empty Version silently produces the key "{Framework}" and breaks downstream filters and tests.
• prowler/lib/check/compliance.py now contains only update_checks_metadata_with_compliance().

Layer 2: JSON Catalogs — prowler/compliance/

See "Compliance Catalog Coverage" below.

Layer 3: Output Formatters — prowler/lib/outputs/compliance/

Universal path (no Python needed per framework):

• universal/universal_table.py — get_universal_table(), renders the CLI table from outputs.table_config + attributes_metadata.
• universal/universal_output.py — UniversalComplianceOutput, builds the CSV Pydantic model dynamically from attributes_metadata.
• universal/ocsf_compliance.py — OCSFComplianceOutput; OCSF output is always generated for universal frameworks regardless of --output-formats.
• Orchestrated by process_universal_compliance_frameworks() in compliance.py, which runs before any legacy dispatch and removes the processed frameworks from the set.

Legacy path — per-framework directory, usually:

{framework}/
├── __init__.py
├── {framework}.py            # get_{framework}_table() summary-table function
├── {framework}_{provider}.py # One ComplianceOutput subclass per provider
└── models.py                 # One Pydantic CSV row model per provider

Directories today: asd_essential_eight, aws_well_architected, c5, ccc, cis, cisa_scuba, ens, generic, iso27001, kisa_ismsp, mitre_attack, okta_idaas_stig, prowler_threatscore, universal. Known deviations (don't "fix" them without a reason): iso27001/ has no table file (falls to the generic table), aws_well_architected/ has no per-provider files, cisa_scuba/ only ships googleworkspace.

• CSV writers emit ;-delimited files with UPPERCASE headers (ComplianceOutput.batch_write_data_to_file). Field names in models.py are public API — renaming breaks downstream consumers.
• Circular import rule: the table file ({framework}.py) must not import Finding directly or transitively (compliance.compliance → table module → ComplianceOutput → Finding → get_check_compliance → cycle). Keep table files bare (colorama, tabulate, prowler.config.config); when a module genuinely needs both, use if TYPE_CHECKING: or function-local imports (see universal_output.py / process_universal_compliance_frameworks).
• Legacy table functions have no docstrings; the universal ones do. Match the style of the file family you're touching.
• Dispatcher display_compliance_table() in compliance.py order: universal (table_config) first → cis_ → ens_ → mitre_attack → kisa → prowler_threatscore_ → c5_ → ccc_ → asd_essential_eight (substring) → okta_idaas_stig → else provider hook (provider.display_compliance_table(), may raise NotImplementedError) → get_generic_compliance_table(). iso27001, aws_well_architected and cisa_scuba ride the fallback on purpose.

Layer 4: API / UI

• API lazy loaders: api/src/backend/api/compliance.py — LazyComplianceTemplate / LazyChecksMapping (per-provider lazy caches over get_bulk_compliance_frameworks_universal, with Gunicorn background warm-up).
• API CSV export dispatch: COMPLIANCE_CLASS_MAP in api/src/backend/tasks/jobs/export.py, consumed from tasks/tasks.py. It is a dict provider → [(predicate, exporter_class)] with GenericCompliance as fallback. Predicates mix startswith for multi-version families (cis_, ens_, iso27001_, ccc_, cisa_scuba_, ...) and exact name == ... for true singletons (mitre_attack_aws, prowler_threatscore_*, asd_essential_eight_aws — and inconsistently c5_azure/c5_gcp, while aws uses startswith("c5_")). Rule of thumb: if the framework can ever grow versions or variants, use startswith.
• API overview ingestion: create_compliance_requirements() in api/src/backend/tasks/jobs/scan.py builds per-region rows from the lazy template and persists ComplianceRequirementOverview (COPY with bulk-create fallback) plus ComplianceOverviewSummary.
• API PDF reports: api/src/backend/tasks/jobs/reports/ — hardcoded FRAMEWORK_REGISTRY (own FrameworkConfig dataclass, NOT the SDK PDFConfig) with one generator class per framework. Only prowler_threatscore, ens, nis2, csa_ccm and cis have PDFs today; adding one means a generator class + registry entry + wiring in report.py.
• UI mapper routing: ui/lib/compliance/compliance-mapper.ts — getComplianceMappers() keyed by the JSON's framework value (e.g. "CIS", "CIS-Controls", "DORA", "Okta-IDaaS-STIG"). Unregistered frameworks fall back to the generic mapper + GenericCustomDetails automatically — a dedicated mapper/detail panel is a first-class upgrade, not a requirement to render.
• UI grouping varies per mapper: generic/cis group by Section/SubSection, iso by Category, ccc by FamilyName. All read attributes[0] — inconsistent values within one JSON become separate tree branches, so normalize before shipping.
• UI types: ui/types/compliance.ts — one *AttributesMetadata interface per framework, added to the AttributesItemData metadata union.
• UI icons: ui/components/icons/compliance/ + IconCompliance.tsx. Registration is an ordered substring match (COMPLIANCE_LOGOS): put framework-specific keywords before generic ones (nist before nis2, cisa before cis; aws deliberately last).

The CLI Pipeline (end-to-end)

prowler aws --compliance cis_7.0_aws          # framework key = JSON basename
    ↓
Compliance.get_bulk("aws")                      # legacy frameworks
get_bulk_compliance_frameworks_universal("aws") # legacy (adapted) + universal
    ↓
update_checks_metadata_with_compliance()        # attaches compliance to CheckMetadata
    ↓
execute_checks() → Finding objects
    ↓
get_check_compliance(finding, "aws", bulk)      # dict "{Framework}-{Version}" → [req_ids]
    ↓
process_universal_compliance_frameworks()       # universal: CSV + OCSF, then removed from set
per-provider elif branches in __main__.py       # legacy: AWSCIS(...).batch_write_data_to_file()
    ↓
display_compliance_table()                      # universal table first, then legacy elifs,
                                                # then generic fallback

Compliance Catalog Coverage

Counts as of 2026-07 (109 JSON files). Regenerate before trusting them:

for d in prowler/compliance/*/; do printf "%s: %s\n" "$(basename $d)" "$(ls $d*.json 2>/dev/null | wc -l)"; done
ls prowler/compliance/*.json   # universal, top-level

Universal (top-level, multi-provider): cis_controls_8.1.json (18 providers), csa_ccm_4.0.json (aws/azure/gcp/alibabacloud/oraclecloud), dora_2022_2554.json (aws/azure/gcp/alibabacloud/cloudflare).

Legacy per-provider (families, not exhaustive versions):

Providers with a compliance directory but no frameworks yet: cloudflare, iac, linode, llm, mongodbatlas, openstack, stackit. Provider keys inside universal checks dicts must match directory names under prowler/providers/ (lowercase).

Universal Schema Reference

Full spec in docs/developer-guide/security-compliance-framework.mdx. Skeleton:

{
  "framework": "DORA",
  "name": "Digital Operational Resilience Act (DORA) 2022/2554",
  "version": "2022/2554",
  "description": "Shown in --list-compliance and PDF reports.",
  "icon": "dora",
  "attributes_metadata": [
    {"key": "Pillar", "label": "Pillar", "type": "str", "required": true,
     "enum": ["ICT Risk Management", "..."],
     "output_formats": {"csv": true, "ocsf": true}},
    {"key": "Article", "type": "str", "required": true}
  ],
  "outputs": {
    "table_config": {"group_by": "Pillar"},
    "pdf_config": {"group_by_field": "Pillar", "charts": ["..."]}
  },
  "requirements": [
    {
      "id": "DORA-Art5",
      "name": "Governance and organisation",
      "description": "Requirement text verbatim from the source.",
      "attributes": {"Pillar": "ICT Risk Management", "Article": "Article 5"},
      "checks": {
        "aws": ["iam_no_root_access_key"],
        "azure": [],
        "gcp": []
      },
      "config_requirements": [
        {"Check": "iam_user_accesskey_unused", "Provider": "aws",
         "ConfigKey": "max_unused_access_keys_days", "Operator": "lte", "Value": 45}
      ]
    }
  ]
}

Universal fields, top level (ComplianceFramework)

Universal fields, per requirement (UniversalComplianceRequirement)

attributes_metadata entry fields (AttributeMetadata)

Key rules:

• --compliance key = JSON basename without .json (dora_2022_2554).
• Auto-discovered: no __init__.py, no formatter, no dispatcher registration.
• table_config.group_by, pdf_config.group_by_field and every charts[].group_by must reference a key declared in attributes_metadata.
• Runtime type validation only covers int/float/bool; str/list_str/ list_dict are documentation-only.
• Extending to a new provider = adding a key to requirement.checks. Nothing else.
• No automatic check-existence validation at load time — a typo'd check id silently produces a requirement with no findings. Always run the check-existence cross-check (see Validation).
• In universal files, always set Provider on every config constraint so a guardrail authored for an AWS check never affects Azure/GCP scans of the same requirement.

Legacy Schema Reference

Base legacy file structure:

{
  "Framework": "FRAMEWORK_NAME",
  "Name": "Full Framework Name with Version",
  "Version": "X.X",
  "Provider": "AWS",
  "Description": "Framework description...",
  "Requirements": [
    {
      "Id": "requirement_id",
      "Name": "Optional requirement name",
      "Description": "Requirement description",
      "Attributes": [ ... ],
      "Checks": ["check_name_1"],
      "ConfigRequirements": [ ... ]
    }
  ]
}

Legacy fields, top level (Compliance)

Legacy fields, per requirement (Compliance_Requirement)

MITRE files use Mitre_Requirement instead, which adds Tactics, SubTechniques, Platforms, TechniqueURL at the requirement top level.

Attribute shapes per framework family

Unlike universal (schema in-file), a legacy requirement's Attributes must match one of the Pydantic classes registered in Compliance_Requirement.Attributes — a shape matching no class silently falls through to Generic, dropping its specific fields. The most common shapes (full field sets in compliance_models.py):

CIS — cis_{version}_{provider}

{
  "Section": "1 Identity and Access Management",
  "SubSection": "Optional subsection",
  "Profile": "Level 1",
  "AssessmentStatus": "Automated",
  "Description": "...", "RationaleStatement": "...", "ImpactStatement": "...",
  "RemediationProcedure": "...", "AuditProcedure": "...",
  "AdditionalInformation": "...", "DefaultValue": "...", "References": "https://..."
}

Profile: Level 1|Level 2|E3 Level 1|E3 Level 2|E5 Level 1|E5 Level 2. AssessmentStatus: Automated|Manual.

ENS — ens_rd2022_{provider}

{
  "IdGrupoControl": "op.acc.1", "Marco": "operacional",
  "Categoria": "control de acceso", "DescripcionControl": "...",
  "Nivel": "alto", "Tipo": "requisito",
  "Dimensiones": ["trazabilidad", "autenticidad"],
  "ModoEjecucion": "automatico", "Dependencias": []
}

Nivel: opcional|bajo|medio|alto. Tipo: refuerzo|requisito|recomendacion|medida. Dimensiones: confidencialidad|integridad|trazabilidad|autenticidad|disponibilidad.

ISO 27001 — iso27001_{year}_{provider}

{
  "Category": "A.5 Organizational controls",
  "Objetive_ID": "A.5.1", "Objetive_Name": "Policies for information security",
  "Check_Summary": "Summary of what is being checked"
}

Note: Objetive_ID / Objetive_Name use this exact (mis)spelling.

MITRE ATT&CK — mitre_attack_{provider} (separate requirement model)

{
  "Name": "Exploit Public-Facing Application", "Id": "T1190",
  "Tactics": ["Initial Access"], "SubTechniques": [],
  "Platforms": ["IaaS"], "Description": "...",
  "TechniqueURL": "https://attack.mitre.org/techniques/T1190/",
  "Checks": ["guardduty_is_enabled"],
  "Attributes": [
    {"AWSService": "Amazon GuardDuty", "Category": "Detect",
     "Value": "Minimal", "Comment": "..."}
  ]
}

AzureService/GCPService for the other providers. Category: Detect|Protect|Respond. Value: Minimal|Partial|Significant.

CCC — ccc_{provider}

{
  "FamilyName": "Data", "FamilyDescription": "...",
  "Section": "CCC.Core.CN01 Encrypt Data for Transmission", "SubSection": "",
  "SubSectionObjective": "...",
  "Applicability": ["tlp-green", "tlp-amber", "tlp-red"],
  "Recommendation": "...",
  "SectionThreatMappings": [{"ReferenceId": "CCC", "Identifiers": ["CCC.Core.TH02"]}],
  "SectionGuidelineMappings": [{"ReferenceId": "NIST-CSF", "Identifiers": ["PR.DS-02"]}]
}

Applicability holds TLP tags (tlp-clear|tlp-green|tlp-amber|tlp-red).

ASD Essential Eight — asd_essential_eight_aws

{
  "Section": "Patch applications", "MaturityLevel": "ML1",
  "AssessmentStatus": "Automated", "CloudApplicability": "partial",
  "MitigatedThreats": ["..."], "Description": "...",
  "RationaleStatement": "...", "ImpactStatement": "...",
  "RemediationProcedure": "...", "AuditProcedure": "...",
  "AdditionalInformation": "...", "References": "..."
}

MaturityLevel: ML1|ML2|ML3. CloudApplicability: full|partial|limited|non-applicable.

DISA STIG — okta_idaas_stig_v1r2_okta

{
  "Section": "...", "Severity": "high", "RuleID": "...", "StigID": "...",
  "CCI": ["CCI-000015"], "CheckText": "...", "FixText": "..."
}

Severity: high|medium|low (maps to CAT I/II/III).

Other registered shapes

• AWS Well-Architected (aws_well_architected_framework_{pillar}_pillar_aws): Name, WellArchitectedQuestionId, WellArchitectedPracticeId, Section, SubSection, LevelOfRisk, AssessmentMethod, Description, ImplementationGuidanceUrl.
• KISA ISMS-P (kisa_isms_p_2023_{provider}): Domain, Subdomain, Section, AuditChecklist, RelatedRegulations, AuditEvidence, NonComplianceCases.
• C5 (c5_{provider}): Section, SubSection, Type, AboutCriteria, ComplementaryCriteria.
• CSA CCM (legacy shape; the shipped CSA CCM 4.0 is universal): Section, CCMLite, IaaS, PaaS, SaaS, ScopeApplicability.
• Prowler ThreatScore (prowler_threatscore_{provider}): Title, Section, SubSection, AttributeDescription, AdditionalInformation, LevelOfRisk (1–5), Weight (1/8/10/100/1000). Pillars: 1 IAM, 2 Attack Surface, 3 Logging and Monitoring, 4 Encryption. Available for aws, azure, gcp, kubernetes, m365, alibabacloud.
• Generic (fallback): ItemId, Section, SubSection, SubGroup, Service, Type, Comment — all optional. Used by NIST, PCI, GDPR, HIPAA, SOC2, FedRAMP, CISA, FFIEC, RBI, NIS2, GxP, SecNumCloud, etc.

Config Guardrails (ConfigRequirements)

Requirements backed by configurable checks can be silently "satisfied" by a loosened audit_config (e.g. CIS demands 45-day unused credentials but the scan ran with max_unused_access_keys_days: 120). Guardrails force such requirements to FAIL:

"ConfigRequirements": [
  {"Check": "iam_user_accesskey_unused",
   "ConfigKey": "max_unused_access_keys_days", "Operator": "lte", "Value": 45}
]

• Operators: lte/gte (numeric thresholds), eq (toggles/exact — use JSON booleans, not 0/1), in (scalar in allowed set), subset (allowlists — widening breaks it), superset (denylists — removing an entry breaks it).
• Value must be the strictest setting the control text tolerates.
• ConfigKey must be spelled exactly as the check reads it; unknown keys are silently skipped (defaults assumed OK).
• Guardrails only tighten (PASS→FAIL), never relax.
• Universal files: lowercase config_requirements + mandatory Provider per constraint.
• Tests: tests/lib/check/compliance_config_eval_test.py, compliance_config_constraint_model_test.py, compliance_config_requirements_data_test.py, plus per-output tests under tests/lib/outputs/compliance/.

Workflow A: Sync a Framework With an Upstream Catalog

Use when the framework is maintained upstream (CIS Benchmarks, FINOS CCC, CSA CCM, NIST, ENS, etc.) and Prowler needs to catch up.

Step 1 — Cache the upstream source

Download every upstream file to a local cache so iterations don't hit the network. For FINOS CCC:

mkdir -p /tmp/ccc_upstream
catalogs="core/ccc storage/object management/auditlog management/logging ..."
for p in $catalogs; do
  safe=$(echo "$p" | tr '/' '_')
  gh api "repos/finos/common-cloud-controls/contents/catalogs/$p/controls.yaml" \
    -H "Accept: application/vnd.github.raw" > "/tmp/ccc_upstream/${safe}.yaml"
done

Step 2 — Run the generic sync runner against a framework config

The sync tooling is three layers, so adding a framework only takes a YAML config (plus a parser module for an unfamiliar upstream format):

skills/prowler-compliance/assets/
├── sync_framework.py          # generic runner — works for any framework
├── configs/ccc.yaml           # per-framework config (canonical example)
└── parsers/finos_ccc.py       # parser module for FINOS CCC YAML

python skills/prowler-compliance/assets/sync_framework.py \
       skills/prowler-compliance/assets/configs/ccc.yaml

The runner loads the config, dynamically imports parser.module, calls parse_upstream(config) -> list[dict], then applies generic post-processing (id-uniqueness safety net, FamilyName normalization, legacy check-mapping preservation with config-driven fallback keys) and writes the provider JSONs with Pydantic post-validation.

To add a new framework sync:

1. Write assets/configs/{framework}.yaml (see ccc.yaml). Required sections:
   • framework — name, display_name, version (never empty — the runner refuses to start, because empty Version breaks the get_check_compliance() key), description_template.
   • providers — list of {key, display} pairs.
   • output.path_template — e.g. "prowler/compliance/{provider}/cis_{version}_{provider}.json".
   • upstream.dir — local cache (Step 1).
   • parser.module — module under parsers/; the rest of parser. is passed through opaque.
   • post_processing.check_preservation.primary_key (almost always Id) and fallback_keys — lists of Attributes[0] field names composed into tuples for recovering mappings when ids change. CCC: - [Section, Applicability]; CIS: - [Section, Profile]; NIST: - [ItemId]. List-valued fields are frozen to frozenset automatically.
   • post_processing.family_name_normalization (optional) — raw → canonical map; the UI groups by the exact attribute value, so upstream variants otherwise become separate tree branches.
2. Reuse an existing parser or write parsers/{name}.py implementing parse_upstream(config) -> list[dict] returning Prowler-format requirements with guaranteed-unique ids. The runner raises on duplicates — it never silently renumbers, because mutating a canonical upstream id (CIS 1.1.1, NIST AC-2(1)) would be catastrophic. The parser owns all upstream quirks: foreign-prefix rewriting, genuine collision renumbering, multi-shape handling.

Gotchas the runner already handles (from the FINOS CCC v2025.10 sync):

• Multiple upstream YAML shapes. Most FINOS CCC catalogs use control-families: [...] but storage/object uses top-level controls: [...]. A single-shape parser silently drops entire catalogs — this exact bug dropped ObjStor for a full iteration. Test with one file of each shape.
• Whitespace collapse. Upstream | block scalars keep newlines; Prowler stores single-line. Collapse with " ".join(value.split()).
• Foreign-prefix id rewriting. Upstream aliases requirements across catalogs keeping the original prefix (CCC.AuditLog.CN08.AR01 nested under CCC.Logging.CN03) — rewrite to fit the parent (CCC.Logging.CN03.AR01).
• Genuine upstream collisions. Two different requirements sharing one id (upstream typo): renumber the second to the next free number; check-mapping preservation recovers by the fallback keys.
• Populate Version — fail-fast beats the silent broken-key bug.

Step 3 — Validate before committing

Run the full Validation section below (universal loader + check existence + CLI smoke + pytest).

Step 4 — Add an attribute model if needed

Only if the framework has fields beyond Generic_Compliance_Requirement_Attribute and must stay legacy. Add the class to compliance_models.py and register it in the Compliance_Requirement.Attributes Union before Generic (Generic stays last). For new frameworks, prefer universal attributes_metadata instead.

Workflow B: Audit Check Mappings as a Cloud Auditor

Use when the user asks to review existing mappings. This is the highest-value compliance task — it surfaces padded mappings with zero actual coverage and missing mappings for legitimate coverage.

The golden rule

A Prowler check's title/risk MUST literally describe what the requirement text says. "Related" is not enough. If no check actually addresses the requirement, leave the checks list empty (MANUAL) — honest MANUAL is worth more than padded coverage.

Audit process

1. Build a per-provider check inventory — assets/build_inventory.py (writes /tmp/checks_{provider}.json for every provider discovered under prowler/providers/).

2. Query it — assets/query_checks.py (run from the repository root):

   python skills/prowler-compliance/assets/query_checks.py aws encryption transit  # keyword AND-search
   python skills/prowler-compliance/assets/query_checks.py aws --service iam       # all iam checks
   python skills/prowler-compliance/assets/query_checks.py aws --id kms_cmk_rotation_enabled
   

3. Dump a framework section with current mappings — assets/dump_section.py:

   python skills/prowler-compliance/assets/dump_section.py ccc "CCC.Core."
   python skills/prowler-compliance/assets/dump_section.py cis_5.0_aws "1."
   

4. Encode explicit REPLACE decisions — assets/audit_framework_template.py:

   DECISIONS = {}
   DECISIONS["CCC.Core.CN01.AR01"] = {
       "aws": ["cloudfront_distributions_https_enabled", ...],
       "azure": ["storage_secure_transfer_required_is_enabled", ...],
       "gcp": ["cloudsql_instance_ssl_connections"],
       # Missing provider key = leave the legacy mapping untouched
   }
   # Empty list = EXPLICITLY MANUAL (overwrites legacy)
   DECISIONS["CCC.Core.CN01.AR07"] = {"aws": [], "azure": [], "gcp": []}
   

   REPLACE, not PATCH. Full lists make the audit reproducible and surface hidden assumptions in the legacy data.

5. Pre-validate every check id against the inventory; the script MUST abort with stderr listing typos (real audits caught storage_secure_transfer_required_enabled → storage_secure_transfer_required_is_enabled, sqlserver_minimum_tls_version_12 → sqlserver_recommended_minimal_tls_version, and several checks that simply don't exist).

6. Apply + validate + test:

   python /path/to/audit_script.py
   uv run pytest -n auto tests/lib/outputs/compliance/ tests/lib/check/ -q
   

For the curated mapping table (requirement text → AWS/Azure/GCP checks) and the list of controls Prowler genuinely cannot verify, see references/check-mapping-reference.md.

Workflow C: Add a New Universal Framework

1. Author prowler/compliance/{framework}_{version}.json following the Universal Schema Reference above (use dora_2022_2554.json or csa_ccm_4.0.json as template).
2. Declare every attribute in attributes_metadata (with required/enum where possible — that's your load-time validation) and a outputs.table_config.group_by.
3. Map checks per provide

…(truncated)