Run Tests

Robot Framework Test Execution Guide

Claude Instructions

IMPORTANT: When user asks a simple question like "How do I run Oracle tests?", provide a concise answer first with just the command(s), then offer to explain more if needed. Do NOT dump all documentation.

Response format for simple questions:

1. Give the direct command(s) first
2. Add a brief note if relevant
3. Offer "Want me to explain more?" only if appropriate

Quick Command Reference

Note: Use PROJECT_SPACE_SETUP=True for first run, omit for subsequent runs.

Usage Examples

Why Make Commands Instead of Robot Command?

Important: In a standard Robot Framework setup, you would run tests directly using:

robot --include oracle test/suite/

However, this project uses a dockerized environment. This means:

1. Tests run inside Docker containers - Not on your local machine
2. Services (databases, Kafka, etc.) run in containers - They communicate via Docker networks
3. Environment variables are managed - Loaded from .env files into containers
4. Groundplex runs in a container - For SnapLogic pipeline execution

The make commands handle all this complexity for you:

• Start the correct Docker containers
• Set up networking between services
• Load environment variables
• Execute Robot Framework inside the tools container
• Manage Groundplex lifecycle

Quick Start

First Time Setup (Full Workflow)

# Complete workflow: create project space, launch Groundplex, run tests
make robot-run-all-tests TAGS="snowflake_demo" PROJECT_SPACE_SETUP=True

Subsequent Runs (Project Space Exists)

# Run tests using existing project space
make robot-run-all-tests TAGS="snowflake_demo"

Quick Iteration (Groundplex Already Running)

# Run tests without Groundplex management
make robot-run-tests-no-gp TAGS="snowflake_demo"

Understanding the Make Targets

1. robot-run-all-tests - Full Workflow with Groundplex

Use this when: First time setup, CI/CD pipelines, or you need Groundplex managed automatically.

make robot-run-all-tests TAGS="your_tags" PROJECT_SPACE_SETUP=True|False

Execution Flow:

Phase 1: Project Space Setup
        ↓
Phase 2: Launch Groundplex
        ↓
Phase 2.1: Set Permissions (Travis CI)
        ↓
Phase 3: Run Tests

Behavior with PROJECT_SPACE_SETUP:

2. robot-run-tests-no-gp - Without Groundplex Launch

Use this when: Groundplex is already running, or tests don't need Groundplex.

make robot-run-tests-no-gp TAGS="your_tags" PROJECT_SPACE_SETUP=True|False

Execution Flow:

Phase 1: Project Space Setup
        ↓
Phase 2: SKIPPED (No Groundplex)
        ↓
Phase 2.1: Set Permissions (Travis CI)
        ↓
Phase 3: Run Tests

Behavior with PROJECT_SPACE_SETUP:

3. robot-run-tests - Base Target (Direct Execution)

Use this when: You need fine-grained control without any environment setup.

make robot-run-tests TAGS="your_tags" PROJECT_SPACE_SETUP=True|False

Note: This is the base target that the other two call internally. It only executes tests without project space or Groundplex management.

Comparison Table

When to Use Which Target

Understanding Robot Framework Tags

Tags are labels attached to test cases that allow you to selectively run tests.

How Tags Work

*** Test Cases ***
Create Triggered Task
    [Documentation]    Creates a triggered task for pipeline execution
    [Tags]    snowflake_demo    task_creation    smoke
    [Template]    Create Triggered Task From Template
    ${unique_id}    ${PIPELINES_LOCATION_PATH}    ${pipeline_name}    ${task_name}

Using Tags with Make Commands

# Run only tests with 'snowflake_demo' tag
make robot-run-all-tests TAGS="snowflake_demo"

# Run tests with 'smoke' tag
make robot-run-all-tests TAGS="smoke"

# Run tests with multiple tags (OR logic - runs if ANY tag matches)
make robot-run-all-tests TAGS="snowflake_demo OR postgres_demo"

# Run tests with multiple tags (AND logic - runs only if ALL tags match)
make robot-run-all-tests TAGS="snowflake_demo AND task_creation"

# Exclude tests with specific tag
make robot-run-all-tests TAGS="NOT cleanup"

Common Tag Categories

Special Framework Tags

Available Test Tags by System

Step-by-Step Examples

Example 1: Running Snowflake Tests (First Time)

# 1. Start required services
make start-services

# 2. Run tests with full setup
make robot-run-all-tests TAGS="snowflake_demo" PROJECT_SPACE_SETUP=True

# 3. View results
open test/robot_output/report-*.html

Example 2: Running Oracle Tests

# 1. Start Oracle database
make oracle-start

# 2. Wait for Oracle to be ready (check logs)
make oracle-logs

# 3. Load test data (if needed)
make oracle-load-data

# 4. Run Oracle tests
make robot-run-all-tests TAGS="oracle" PROJECT_SPACE_SETUP=True

# 5. View results
open test/robot_output/report-*.html

Example 3: Running Kafka Tests

# 1. Start Kafka cluster
make kafka-start

# 2. Create test topics
make kafka-create-topics

# 3. Run Kafka tests
make robot-run-all-tests TAGS="kafka" PROJECT_SPACE_SETUP=True

Example 4: Running Multiple System Tests

# Start required services
make oracle-start
make postgres-start

# Run tests for both (OR logic)
make robot-run-all-tests TAGS="oracle OR postgres" PROJECT_SPACE_SETUP=True

Example 5: Quick Test Iteration (After Initial Setup)

# Groundplex is already running, just re-run tests
make robot-run-tests-no-gp TAGS="snowflake_demo"

Pre-requisites Checklist

Before running tests, ensure:

1. Environment Configuration

# Verify .env file exists and is configured
make check-env

# Or manually check key variables
cat .env | grep -E "^(URL|ORG_|PROJECT_)"

2. Required Services Running

# Check overall status
make status

# Check specific service
make oracle-status  # or postgres-status, etc.

3. Docker Environment

# Verify Docker is running
docker ps

# Check project containers
make show-running

Test Results

Location

test/robot_output/
├── output-YYYYMMDD-HHMMSS.xml   # Raw results (for CI/CD)
├── log-YYYYMMDD-HHMMSS.html     # Detailed execution log
└── report-YYYYMMDD-HHMMSS.html  # Summary report (open this)

Viewing Results

# Open the latest report (macOS)
open test/robot_output/report-*.html

# Find the latest report
ls -lt test/robot_output/report-*.html | head -1

Sharing Results

# Send to Slack
make slack-notify

# Upload to S3
make upload-test-results

Troubleshooting

Common Issues

Debug Steps

1. Check Test Logs:

   open test/robot_output/log-*.html
   

2. Check Container Logs:

   make logs-tools        # Test container logs
   make oracle-logs       # Database logs (replace with your system)
   

3. Verify Environment:

   make check-env
   make status
   

4. Re-run with Fresh Setup:

   make stop-groundplex
   make robot-run-all-tests TAGS="..." PROJECT_SPACE_SETUP=True
   

Quick Reference Commands

# Full setup with new project space and plex
make robot-run-all-tests TAGS="snowflake_demo" PROJECT_SPACE_SETUP=True

# Run tests using existing project space (launch GP)
make robot-run-all-tests TAGS="snowflake_demo"

# Run tests when GP is already running
make robot-run-tests-no-gp TAGS="snowflake_demo"

# Create project space only (no plex, no GP launch)
make robot-run-tests-no-gp TAGS="oracle" PROJECT_SPACE_SETUP=True

# Check environment
make check-env

# View running containers
make status

# Stop Groundplex
make stop-groundplex

# View test results
open test/robot_output/report-*.html

Architecture Overview

┌─────────────────────────────────────────────────────────────────────────┐
│                         User Commands                                    │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                         │
│   ┌───────────────────────┐       ┌───────────────────────┐            │
│   │  robot-run-all-tests  │       │  robot-run-tests-no-gp │            │
│   │  (Full Workflow)      │       │  (No GP Workflow)      │            │
│   └───────────┬───────────┘       └───────────┬───────────┘            │
│               │                               │                         │
│               │   Phase 1: Project Space      │                         │
│               │   Phase 2: Launch GP          │   Phase 2: SKIPPED      │
│               │   Phase 2.1: Permissions      │   Phase 2.1: Permissions│
│               │   Phase 3: Run Tests          │   Phase 3: Run Tests    │
│               │                               │                         │
│               └───────────────┬───────────────┘                         │
│                               │                                         │
│                               ▼                                         │
│               ┌───────────────────────────────┐                         │
│               │       robot-run-tests         │                         │
│               │       (Base Target)           │                         │
│               │                               │                         │
│               │  - Executes Robot Framework   │                         │
│               │  - Inside Docker container    │                         │
│               │  - With specified TAGS        │                         │
│               └───────────────────────────────┘                         │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘