Newman Runner

Newman Runner

This skill provides tools to run Newman (Postman CLI) tests and analyze the results for API testing and validation.

Instructions

Running Newman Tests

1. Verify Newman Installation

   • Check if Newman is installed: which newman
   • If not installed, install with: npm install -g newman

2. Run Collection

   • Use script: scripts/run-newman.sh <collection.json>
   • Or manually: newman run collection.json --reporters cli,json --reporter-json-export output.json

3. Analyze Results

   • Parse JSON output with: scripts/analyze-newman-results.py output.json
   • Extract: Pass/fail status, response times, error messages, assertions

4. Run in CI/CD

   • Use scripts/run-newman-ci.sh <collection.json> for CI-optimized execution
   • Produces JUnit XML for CI reporting + JSON for analysis
   • Uses GitHub Actions annotation format for inline error reporting

Available Scripts

• scripts/run-newman.sh - Run Newman with standard options
• scripts/run-newman-ci.sh - CI-optimized runner with JUnit output
• scripts/analyze-newman-results.py - Parse Newman JSON output
• scripts/validate-collection.sh - Validate Postman collection structure

Advanced Patterns

For OpenAPI-to-Newman pipeline, auth injection, and collection management, see the api-contract-testing skill which builds on this runner.

Examples

Example 1: Run API Tests

# Run Newman tests on Postman collection
./scripts/run-newman.sh my-api-tests.json

# Analyze results
./scripts/analyze-newman-results.py newman-results.json

Example 2: Validate Collection

# Check collection is valid before running
./scripts/validate-collection.sh my-collection.json

Example 3: Run in CI

# CI-optimized run with JUnit output
./scripts/run-newman-ci.sh my-api-tests.json

# Results in:
# test-results/api-contract/newman-results.json  (JSON)
# test-results/api-contract/newman-junit.xml     (JUnit XML)

Requirements

• Newman installed globally: npm install -g newman
• Valid Postman collection JSON file
• Python 3.7+ for analysis scripts

Success Criteria

• Newman tests run successfully
• Results parsed and analyzed
• Pass/fail status clearly reported
• Error details extracted for failures