ashfordeOU CFD Validation (aerodynamics/cfd/cfd-validation)
Use when a CFD result must be judged against authoritative reference data before it can be trusted: which reference case to run, which comparison metrics to use, whether the agreement is within the acceptance band, and how to report it. Follows AIAA G-077-1998 (Guide for the Verification and Validation of CFD Simulations) and ASME V&V 20-2009 (Standard for Verification and Validation in CFD and Heat Transfer); see references/vv-guidance.md.
Domain quick reference
- Verification solves the equations right (code and discretization errors); validation solves the right equations (model vs physical reality). This skill covers validation; pair it with cfd-convergence for residual and mesh checks.
- Validation compares computed quantities against reference data of known quality and states PASS or FAIL within a tolerance band. It never proves a code, it bounds it for a stated application and condition range.
- Case selection by flow regime and application (implemented in select_validation_case):
| Flow regime | Application | Reference case | Key reference values |
|---|---|---|---|
| incompressible | airfoil | NACA 0012 (alt: NACA 4412) | Cd 0.0081 at M 0.30, Re 6e6 (NACA TR 824) |
| transonic | wing | ONERA M6 | CL 0.266, CD 0.0163 at M 0.84, Re 11.72e6, alpha 3.06 deg (AGARD-AR-138) |
| transonic / transport | wing-body | DLR-F6 | CD ~ 0.0299 at CL 0.5, M 0.75, Re 3e6 (AIAA Drag Prediction Workshop) |
| incompressible | flat plate | ZPG flat plate | Cf = 1.328/sqrt(Rex) laminar (Blasius), Cf = 0.074/Rex^0.2 turbulent (Schlichting) |
- Comparison metrics: relative_error for integrated quantities (Cd, Cl); rms_error and max_error for distributed quantities (Cp, Cf, u profiles).
- Richardson extrapolation on 3 meshes gives the apparent order p and an infinite-grid estimate; the Roache grid convergence index (GCI, safety factor 1.25) is the discretization uncertainty on the finest mesh.
- Typical acceptance bands: section drag within 5% of reference, wing-body drag within 10% (DPW participant scatter is of this order), Cp within 0.02 or 5% of local magnitude. State the band before judging.
- Validation uncertainty U_val combines the identified error sources (discretization, modeling, numerical, experimental) in quadrature.
Workflow
- Select the reference case with select_validation_case(flow_regime, application), e.g. ("incompressible", "airfoil") -> NACA 0012.
- Run the CFD analysis on at least three mesh levels and extract the quantities of interest at the documented conditions.
- Compute relative_error on integrated quantities and rms_error plus max_error on distributed quantities against the reference.
- Run richardson_extrapolation on the 3 mesh values; a monotone sequence with positive apparent order is required for a sane grid convergence statement.
- Judge with validation_verdict(computed, reference, tolerance); drag within 5% of the reference is the default band for section cases.
- List the error sources and combine them with validation_uncertainty(sources) into U_val.
- Assemble the report with report_skeleton(case, metrics, verdict, uncertainty).
Worked example
NACA 0012 at M 0.30, Re 6e6, alpha 0 deg. Reference Cd 0.0081 (NACA TR 824 classic data).
- Computed Cd 0.0085: relative_error = 0.0004/0.0081 = 0.0494, under the 5% band. validation_verdict(0.0085, 0.0081, 0.05) -> PASS.
- Computed Cd 0.010: relative_error = 0.2346, over the 5% band. validation_verdict(0.010, 0.0081, 0.05) -> FAIL.
- Grid convergence: values [0.0085, 0.0090, 0.0100] on meshes refined by r = 2 give apparent order p = 1.0, extrapolated Cd 0.0080, GCI 0.000625. The extrapolated value is closer to the reference than the finest mesh value, which supports the 5% PASS.
- Uncertainty: sources discretization 0.0002, modeling 0.0003, numerical 0.0001 combine to U_val = 0.000374, dominated by modeling. The verdict band is wider than U_val, so the PASS is robust.
Pitfalls
- Calling a code validated because one case passes: validation never proves a code, it bounds it for a stated application and condition range - the verdict from validation_verdict belongs to the case, the band, and the flow regime that was run, and a clean residual (cfd-convergence) does not make a 23% drag error acceptable.
- Selecting the reference case by convenience instead of regime: the incompressible NACA 0012, transonic ONERA M6 and DLR-F6 cases each anchor one flow regime and application, so comparing a transonic wing-body run against the NACA 0012 Cd band is meaningless.
- Choosing the acceptance band after seeing the error: the 5% section / 10% wing-body / 0.02 Cp bands are documented typicals that must be stated before the comparison, not tuned to force a PASS.
- Using the wrong metric for the quantity: relative_error is for integrated quantities (Cd, Cl) while distributed quantities (Cp, Cf, u profiles) need rms_error and max_error - a single-point Cp relative error hides the profile mismatch.
- Running Richardson extrapolation on a non-monotone mesh sequence: three mesh values with no clear trend or a non-positive apparent order give no sane grid-convergence statement, so the GCI on the finest mesh is not trustworthy.
- Reporting a verdict whose band is narrower than the uncertainty: when U_val (0.000374 in the worked example) approaches the tolerance band, the PASS is not robust - widen the band or reduce the dominant error source before claiming validation.
Behavior contract (gate 3)
The behavior contract is scripts/test_cfd_validation.py against scripts/cfd_validation_logic.py (stdlib unittest, offline, deterministic). Run:
python3 scripts/test_cfd_validation.py
It asserts: NACA 0012 Cd reference 0.0081 at M 0.3 Re 6e6; computed Cd 0.0085 passes the 5% band; computed Cd 0.010 fails; Richardson extrapolation on 3 meshes yields a sensible extrapolated value (closer to the reference than the finest mesh); every invalid input raises ValueError.
References
- references/vv-guidance.md: AIAA G-077-1998 and ASME V&V 20-2009 summary, case data sources, acceptance-band guidance.
- scripts/cfd_validation_logic.py: the logic module (pure Python, stdlib only).
- scripts/test_cfd_validation.py: the behavior contract test.
Related skills
- cfd-convergence: residual, CFL and mesh refinement verification (the verification half of V&V).
- cfd-turbulence-modeling: model choice feeds the modeling uncertainty term of U_val.
- cfd-mesh-generation: mesh quality feeds the discretization uncertainty term.
- xfoil-analysis: section polars validated against NACA TR 824 data.
- boundary-layer-theory: flat plate skin friction anchors the zero-pressure-gradient boundary layer case.
Compliance
- NACA TR 824 is US government work (public domain); summary and physics values only, per standards-map.yaml.
- AIAA G-077-1998 and ASME V&V 20-2009 are referenced as guidance; paraphrased summary only in references/vv-guidance.md.
- compliance: STANDARDS-REF, gated: false.