SA-15(1)_quality-metrics

SA-15(1) Quality Metrics

Enhancement of: SA-15

High-Level Description

Family: System and Services Acquisition (SA) Framework: NIST SP 800-53 Rev 5

Organizations use quality metrics to establish acceptable levels of system quality. Metrics can include quality gates, which are collections of completion criteria or sufficiency standards that represent the satisfactory execution of specific phases of the system development project. For example, a quality gate may require the elimination of all compiler warnings or a determination that such warnings have no impact on the effectiveness of required security or privacy capabilities. During the execution phases of development projects, quality gates provide clear, unambiguous indications of progress. Other metrics apply to the entire development project. Metrics can include defining the severity thresholds of vulnerabilities in accordance with organizational risk tolerance, such as requiring no known vulnerabilities in the delivered system with a Common Vulnerability Scoring System (CVSS) severity of medium or high.

What to Check

• Verify SA-15(1) Quality Metrics is documented in SSP
• Validate all 2 control requirements are implemented
• Confirm control is operating effectively
• Review evidence of continuous monitoring for SA-15(1)
• Verify enhancement builds upon base control SA-15

How to Test

Step 1: Review Documentation

Examine the System Security Plan (SSP) and related artifacts for SA-15(1) implementation details. Verify the organization has documented how this control is satisfied.

Step 2: Validate Implementation

# For cloud environments, use cloud-audit-mcp tools
# For on-premises, review system configurations directly

# Example: Check if account management policies exist
grep -r "account.management\|access.control" /etc/security/ 2>/dev/null

Step 3: Test Operating Effectiveness

Verify the control is actively functioning, not just documented. Check logs, configurations, and operational evidence.

Tools

Remediation Guide

Control Statement

Require the developer of the system, system component, or system service to: Define quality metrics at the beginning of the development process; and Provide evidence of meeting the quality metrics [organization-defined].

Implementation Guidance

Organizations use quality metrics to establish acceptable levels of system quality. Metrics can include quality gates, which are collections of completion criteria or sufficiency standards that represent the satisfactory execution of specific phases of the system development project. For example, a quality gate may require the elimination of all compiler warnings or a determination that such warnings have no impact on the effectiveness of required security or privacy capabilities. During the execution phases of development projects, quality gates provide clear, unambiguous indications of progress. Other metrics apply to the entire development project. Metrics can include defining the severity thresholds of vulnerabilities in accordance with organizational risk tolerance, such as requiring no known vulnerabilities in the delivered system with a Common Vulnerability Scoring System (CVSS) severity of medium or high.

Risk Assessment

CWE Categories

References

• NIST SP 800-53 Rev 5 - SA-15(1)
• NIST SP 800-53A Rev 5 (Assessment Procedures)
• NIST SP 800-53 Rev 5 Full Catalog

Checklist

• Control documented in SSP
• Implementation evidence collected
• Operating effectiveness validated
• Continuous monitoring in place
• Related controls (none) reviewed