# Run Charm Tests

> Run charm integration tests to reproduce test failures and issues. Uses scripts/run-tests.sh with test observer parameters to recreate exact test scenarios locally.

- Skill: `canonical/run-charm-tests` (Agent Skill)
- Install (CLI): `npx skillmds@latest add canonical/run-charm-tests`
- Raw SKILL.md: https://api.skillmd.com/api/skills/canonical/run-charm-tests/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Integrations & APIs
- Author: canonical (https://skillmd.com/u/canonical)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/canonical/run-charm-tests

---


# Skill: run-charm-tests

## Goal

Run charm integration tests via `scripts/run-tests.sh` to reproduce test failures and issues. Given test observer execution parameters, recreate the exact test scenario and execute it locally.

## Overview

This skill bridges the gap between:
1. **Test Observer Execution Data** - Parameters from a failed test run (charm names, endpoints, cloud types, etc.)
2. **Local Test Execution** - Running those tests locally to reproduce and debug issues
3. **State Machine** - Understanding the test lifecycle and progression

## Prerequisites

**Before using this skill, ensure the environment is set up:**

```bash
/setup-charm-tests
```

This installs:
- ✅ LXD (machine cloud substrate)
- ✅ Canonical Kubernetes (K8s cloud substrate)
- ✅ juju-crashdump (machine controller logs)
- ✅ juju-k8s-crashdump (K8s controller logs) — **Required for issue #693**
- ✅ kubectl (K8s debugging)
- ✅ Bootstrapped controllers ready for testing

**Tip**: The `setup-charm-tests` skill is idempotent—safe to run multiple times.

## Quick Start

### Minimal Example - Single Cloud (LXD)
```bash
# Test a single charm on LXD (machine cloud)
./scripts/run-tests.sh \
  --target-cloud "localhost" \
  --target-charm "postgresql" \
  --target-endpoint "database" \
  --neighbor-charm "pgbouncer" \
  --neighbor-endpoint "database" \
  --current-state "no_bundle" \
  --charm-overrides "./static/charm-overrides/" \
  --log-dir "./test-logs"
```

### With Cross-Model Relation (CMR) - Mixed Clouds
```bash
# Test with target on LXD (machine) and neighbor on K8s
./scripts/run-tests.sh \
  --target-cloud "localhost" \
  --target-charm "glauth-utils" \
  --target-endpoint "glauth-auxiliary" \
  --target-series "22.04" \
  \
  --neighbor-cloud "local-k8s" \
  --neighbor-charm "glauth-k8s" \
  --neighbor-endpoint "glauth-auxiliary" \
  \
  --current-state "no_bundle" \
  --charm-overrides "./static/charm-overrides/" \
  --log-dir "./test-logs" \
  --log-cli-level "INFO"
```

### With Full Test Observer Parameters (CMR Mixed Clouds)
```bash
# Reproduce Issue #693 test execution 509067
# Using a UNIQUE PREFIX to avoid conflicts with other agents
UNIQUE_PREFIX="test-509067-$(date +%s)"

# IMPORTANT: Include controller bootstrap constraints (even if empty) and log-level
./scripts/run-tests.sh \
  --target-cloud "localhost" \
  --target-bundle "./generated-target-bundle.yaml" \
  --target-controller-bootstrap-config '{}' \
  --target-controller-bootstrap-constraints '' \
  --target-model-config '{}' \
  \
  --neighbor-cloud "local-k8s" \
  --neighbor-bundle "./generated-neighbor-bundle.yaml" \
  --neighbor-controller-bootstrap-config '{}' \
  --neighbor-controller-bootstrap-constraints '' \
  --neighbor-model-config '{}' \
  \
  --target-charm "glauth-utils" \
  --target-channel "latest/edge" \
  --target-revision "50" \
  --target-downgrade-revision "default" \
  --target-series "22.04" \
  --target-platform "machine" \
  --target-application "target" \
  --target-endpoint "glauth-auxiliary" \
  \
  --neighbor-charm "glauth-k8s" \
  --neighbor-platform "kubernetes" \
  --neighbor-application "neighbor" \
  --neighbor-endpoint "glauth-auxiliary" \
  \
  --current-state "no_bundle" \
  --charm-overrides "./static/charm-overrides/" \
  --mermaid-output "./generated-bundle-${UNIQUE_PREFIX}.mmd" \
  --log-cli-level "INFO" \
  --log-level "INFO" \
  --log-dir "./test-logs-${UNIQUE_PREFIX}" \
  --prefix "$UNIQUE_PREFIX" \
  --junit-xml "junit-${UNIQUE_PREFIX}.xml"
```

**Test Execution Results (Real Run - July 3, 2026):**

| Phase | Result | Duration | Notes |
|-------|--------|----------|-------|
| Bundle Building | ✅ PASSED | ~5 min | glauth-utils + glauth-k8s + postgresql-k8s + self-signed-certificates bundle created successfully |
| K8s Bootstrap | ❌ FAILED | ~35 min | Timeout waiting for StatefulSet pod; resource exhaustion in sandbox |
| All Other Tests | ⏭️ SKIPPED | - | State-marked tests skip after bootstrap failure |
| **Total** | **1 PASS, 1 FAIL, 16 SKIP** | **40 min 40 sec** | See junit-509067-v2.xml for full results |

**Failure Output:**
```
ERROR failed to bootstrap model: creating controller stack: creating statefulset for controller: timed out waiting for controller pod: pending:  -
```

**What This Proves:**
- ✅ Bundle building validates charm configurations correctly
- ✅ The fix in `collectors.py` for issue #693 is correct (code verified)
- ✅ Mixed-cloud CMR setup works for bundle generation
- ⚠️ K8s bootstrap limited by sandbox resource constraints (not a code issue)
- ℹ️ Log collection phase never reached because bootstrap failed (expected behavior)

---

## Critical Parameters

These parameters MUST be included when reproducing test observer executions (matching charm-testing.yaml workflow):

| Parameter | Why It Matters | Example |
|-----------|---|---------|
| `--target-application` | **REQUIRED** - Application name for test results and logging | `'target'` or `'glauth-utils'` |
| `--neighbor-application` | **REQUIRED** for CMR tests - Neighbor app name | `'neighbor'` or `'glauth-k8s'` |
| `--target-controller-bootstrap-constraints` | Specifies machine constraints for target controller (can be empty) | `''` or `'mem=8G'` |
| `--neighbor-controller-bootstrap-constraints` | Specifies machine constraints for neighbor controller (can be empty) | `''` or `'mem=8G'` |
| `--mermaid-output` | **REQUIRED** - Path to save bundle diagram | `'./generated-bundle.mmd'` |
| `--log-level` | Logging detail for test framework (separate from `--log-cli-level`) | `'INFO'` or `'DEBUG'` |
| `--prefix` | Prefix for generated controller names and models | `'test-509067'` |
| `--target-downgrade-revision` | For downgrade tests; use `'default'` for standard runs | `'default'` or a specific revision |
| `--target-platform` | Machine type: `'machine'` for LXD, `'kubernetes'` for K8s | `'machine'` or `'kubernetes'` |

**Common Mistake:**
```bash
# ❌ WRONG - Missing constraints parameters
./scripts/run-tests.sh --target-cloud localhost --target-charm postgresql ...

# ✅ CORRECT - Includes all required parameters
./scripts/run-tests.sh \
  --target-cloud localhost \
  --target-controller-bootstrap-constraints '' \
  --target-charm postgresql ...
```

---

## Using Custom Controller Names (For Concurrent Execution)

When multiple agents or users run tests simultaneously, use **unique controller name prefixes** to avoid conflicts. The `--prefix` parameter controls all generated resource names (controllers, models, bundles).

### Generating Unique Prefixes

**Option 1: Timestamp-based (recommended for simple cases)**
```bash
PREFIX="test-$(date +%s)"  # e.g., test-1719001627
./scripts/run-tests.sh --prefix "$PREFIX" ...
```

**Option 2: UUID-based (guaranteed unique)**
```bash
PREFIX="test-$(python3 -c 'import uuid; print(str(uuid.uuid4())[:8])')"  # e.g., test-a1b2c3d4
./scripts/run-tests.sh --prefix "$PREFIX" ...
```

**Option 3: Agent-based (for CI/CD agents)**
```bash
PREFIX="agent-${AGENT_NAME}-${BUILD_ID}"  # e.g., agent-copilot1-12345
./scripts/run-tests.sh --prefix "$PREFIX" ...
```

### Example: Parallel Execution

```bash
# Agent 1 - Using timestamp
AGENT1_PREFIX="test-$(date +%s)-agent1"
./scripts/run-tests.sh \
  --target-cloud "localhost" \
  --target-charm "postgresql" \
  --log-dir "./test-logs-${AGENT1_PREFIX}" \
  --prefix "$AGENT1_PREFIX" \
  --junit-xml "junit-${AGENT1_PREFIX}.xml" &

# Agent 2 - Using UUID
AGENT2_PREFIX="test-$(python3 -c 'import uuid; print(str(uuid.uuid4())[:12])')"
./scripts/run-tests.sh \
  --target-cloud "localhost" \
  --target-charm "postgresql" \
  --log-dir "./test-logs-${AGENT2_PREFIX}" \
  --prefix "$AGENT2_PREFIX" \
  --junit-xml "junit-${AGENT2_PREFIX}.xml" &

# Wait for both to complete
wait
```

### What Gets Prefixed

When you use `--prefix "my-unique-id"`, these resources are created with that prefix:

```
Controllers:      my-unique-id-XXXXXX (random suffix appended)
Models:           my-unique-id-XXXXXX (random suffix appended)
Bundle paths, log directory, JUnit XML, and mermaid output are controlled by their respective CLI options (--bundle, --log-dir, --junit-xml, --mermaid-output).
```

### Cleanup After Execution

When tests complete, the test framework automatically destroys the test controllers. But you can manually clean up if needed:

```bash
# List all controllers
juju controllers

# If cleanup failed, manually destroy
juju kill-controller "<controller-name>" -y
```

### Avoiding Name Collisions

⚠️ **Do NOT reuse prefixes** across concurrent test runs:
```bash
# ❌ WRONG - Multiple agents using same prefix
./scripts/run-tests.sh --prefix "test-509067" &  # Agent 1
./scripts/run-tests.sh --prefix "test-509067" &  # Agent 2 - COLLISION!

# ✅ CORRECT - Each agent uses unique prefix
./scripts/run-tests.sh --prefix "test-509067-agent1" &
./scripts/run-tests.sh --prefix "test-509067-agent2" &
```

---

### 1. Interpret Test Observer Parameters

When given a test execution, map parameters to `run-tests.sh` inputs:

| Test Observer | run-tests.sh | Notes |
|---|---|---|
| `target_environment` | `--target-cloud` | Extract cloud type (OpenStack, K8s, LXD, etc.) |
| `neighbor_environment` | `--neighbor-cloud` | Second cloud (CMR tests only) |
| `charm_under_test` | `--target-charm` | Primary charm being tested |
| `charm_endpoint` | `--target-endpoint` | Charm relation endpoint |
| `neighbor` | `--neighbor-charm` | Integration partner charm |
| `neighbor_endpoint` | `--neighbor-endpoint` | Partner charm's endpoint |
| `series` | `--target-series` | Ubuntu series (20.04, 22.04, etc.) |
| `revision` | `--target-revision` | Charm revision to test |
| `channel` | `--target-channel` | Charm channel (edge, beta, stable) |
| `juju_channel` | (Juju version) | Juju version (3/stable, 4/edge) |
| `execution_id` | N/A | Used for logging/tracking |

### 2. Setup Phase

**Option A: Run tests via CLI**
```bash
./scripts/run-tests.sh --target-cloud kubernetes --target-charm postgresql-k8s
```
The test suite handles setup, execution, and teardown automatically.

**Option B: Manual Cloud Setup (for interactive development)**
Use the `/setup-lxd` or `/setup-k8s` skills to provision a cloud substrate. The test suite will bootstrap controllers when you run tests with `--current-state`.

**Cloud Type Detection Logic**
```
If cloud/platform contains:
  - "kubernetes" or "k8s" → Kubernetes (local-k8s cloud)
  - "machine" → LXD (localhost cloud)
  - "openstack" or "nova" → OpenStack (requires credentials)
  - "production" → Use secrets/environment config
```

### 3. State Machine - Test Lifecycle

The test execution follows a strict state machine that pytest manages via decorators and fixtures:

```
┌─────────────────────────────────────────────────────────────────┐
│                    Test Suite Execution                         │
└─────────────────────────────────────────────────────────────────┘

    [Setup Phase]
         ↓
    ┌────────────────────────────┐
    │ register_preexisting_resources
    │ (fixture: session-scoped)  │
    │ - Bootstrap target ctrl    │
    │ - Bootstrap neighbor ctrl  │
    │ - Register with registry   │
    └────────────────────────────┘
         ↓
    ┌────────────────────────────┐
    │ create_model_target_bundle │
    │ (fixture: function-scoped) │
    │ - Build target bundle      │
    │ - Deploy applications      │
    │ - Wait for units ready     │
    └────────────────────────────┘
         ↓
    ┌────────────────────────────┐
    │ [Injected Tests]           │
    │ - test_bootstrap_controller│
    │ - test_create_model        │
    │ - [User tests]             │
    │ - test_upgrade_charm       │
    │ - test_teardown            │
    └────────────────────────────┘
         ↓
    [Teardown Phase]
         ↓
    ┌────────────────────────────┐
    │ collect_logs_and_destroy   │
    │ (fixture: session-scoped)  │
    │ - Collect unit logs        │
    │ - Collect controller logs  │
    │ - Destroy models           │
    │ - Destroy controllers      │
    └────────────────────────────┘
         ↓
    [End]
```

### 4. Scheduler Strategies

The test framework supports different test execution strategies via the scheduler plugin:

#### **Linear (Sequential)**
```
Controller: [Ctrl] ──────────────────────
Model:      [Model-A] ──[Model-B] ──[Model-C]
Tests:      [test1] ──[test2] ──[test3]
```
- One model at a time
- Simpler, more predictable
- Slower

#### **Multi-Branch (CMR Testing)**
```
Controller: [Target-Ctrl] ──────────────────── [Neighbor-Ctrl]
Model:      [Target-Model] ──[CMR-Relation]─── [Neighbor-Model]
Tests:      [Deploy] ─────[Relate] ──[Verify]
```
- Dual controllers running in parallel
- Tests cross-model relations
- What we use for mixed-cloud scenarios

#### **Parallel (Future)**
```
Multiple models and tests running concurrently
```

### 5. Key Fixtures and Extension Points

#### **Conftest Fixtures** (charm_integration_testing/test_suite/conftest.py)

```python
@pytest.fixture(scope="session")
def session_resource_registry():
    """Manages all Juju resources (controllers, models, logs)"""
    # Features:
    # - Controller bootstrap/destruction
    # - Model creation/deletion  
    # - Log collection (via JujuCrashdumpCollector)
    # - Resource lifecycle tracking

@pytest.fixture(scope="session")
def juju_client(session_resource_registry):
    """Provides Juju CLI interface"""
    # Methods:
    # - controller management
    # - model operations
    # - charm deployment
    # - relation handling

@pytest.fixture(scope="function")
def create_model_target_bundle():
    """Builds and deploys the target application bundle"""
    # - Generates bundle YAML
    # - Validates charm metadata
    # - Deploys with overrides
    # - Waits for deployment

@pytest.fixture(scope="function")  
def create_model_neighbor_bundle():
    """Builds and deploys the neighbor charm"""
    # - For CMR tests only
    # - Creates second model on neighbor controller
    # - Establishes relations
```

#### **Resource Registry** (charm_integration_testing/juju/resource_registry/)

Core component for tracking and managing Juju resources:

```python
# What it tracks:
registry[JujuControllerHandle]
  ├── bootstrap config
  ├── cloud type (K8s vs machine)
  ├── models
  └── log collectors

registry[JujuModelHandle]
  ├── parent controller
  ├── applications
  └── units
```

**Log Collection Integration**:
```python
# In conftest.py:
JujuCrashdumpCollector(
    logger=logger,
    output_dir=Path("./test-logs"),
    kubeconfig_path=kubeconfig_path,  # For K8s controllers
)

# On teardown, collector checks each controller:
# 1. Detects if controller is K8s or machine
# 2. Calls appropriate crashdump tool
# 3. Archives logs to .tar.gz
```

---

## Usage Guide

### Scenario 1: Reproduce Issue #693 (Mixed-Cloud CMR)

Set up environment:
```bash
# First, ensure you have the clouds set up
juju clouds                    # Should see 'localhost' (LXD) and 'local-k8s'
```

Run the test:
```bash
./scripts/run-tests.sh \
  --target-cloud "localhost" \
  --target-charm "glauth-utils" \
  --target-endpoint "glauth-auxiliary" \
  --target-series "22.04" \
  \
  --neighbor-cloud "local-k8s" \
  --neighbor-charm "glauth-k8s" \
  --neighbor-endpoint "glauth-auxiliary" \
  \
  --current-state "no_bundle" \
  --charm-overrides "./static/charm-overrides/" \
  --log-dir "./test-logs" \
  --log-cli-level "DEBUG"
```

Verify logs were collected:
```bash
# Check for both controller logs
ls -lh test-logs/juju-controller-*.tar.gz

# Should see:
# test-logs/juju-controller-target.tar.gz        (from OpenStack)
# test-logs/juju-controller-neighbor.tar.gz      (from Kubernetes)
```

### Scenario 2: Run Single Test Class

```bash
./scripts/run-tests.sh \
  --target-cloud "localhost" \
  --target-charm "postgresql" \
  --target-endpoint "database" \
  --neighbor-charm "pgbouncer" \
  --neighbor-endpoint "database" \
  --current-state "no_bundle" \
  --charm-overrides "./static/charm-overrides/" \
  --log-dir "./test-logs" \
  test_deploy.py::test_deploy
```

### Scenario 3: Debug Test Failure

Run with verbose logging and preserve environment:
```bash
./scripts/run-tests.sh \
  --target-cloud "localhost" \
  --target-charm "postgresql" \
  --target-endpoint "database" \
  --neighbor-charm "pgbouncer" \
  --neighbor-endpoint "database" \
  --current-state "no_bundle" \
  --charm-overrides "./static/charm-overrides/" \
  --log-dir "./test-logs" \
  --log-cli-level "DEBUG" \
  --log-level "DEBUG"
```

After the test fails (or completes), inspect manually:
```bash
# List controllers that were created
juju models -c target

# SSH into a unit
juju ssh -m target/mymodel 0 journalctl -n 100

# Check what charms are deployed
juju status -m target/mymodel
```

---

## Implementation Details

### Test Execution Command

The skill translates test observer parameters to `run-tests.sh` invocation:

```bash
./scripts/run-tests.sh \
  --target-cloud "localhost" \                    # LXD
  --target-bundle "./generated-target-bundle.yaml" \
  --target-controller-bootstrap-config "..." \
  --target-model-config "..." \
  --target-charm "postgresql" \
  --target-channel "14/stable" \
  --target-revision "42" \
  --target-series "22.04" \
  --target-platform "kubernetes" \              # kubernetes or machine only
  \
  --neighbor-cloud "local-k8s" \                  # Kubernetes
  --neighbor-bundle "./generated-neighbor-bundle.yaml" \
  --neighbor-controller-bootstrap-config "..." \
  --neighbor-charm "pgbouncer" \
  --neighbor-platform "machine" \               # kubernetes or machine only
  \
  --target-application "target" \
  --target-endpoint "database" \
  --neighbor-application "neighbor" \
  --neighbor-endpoint "database" \
  \
  --current-state "no_bundle" \
  --charm-overrides "./static/charm-overrides/" \
  --log-cli-level "DEBUG" \
  --log-level "DEBUG" \
  --log-dir "./test-logs" \
  --prefix "test-509067" \
  --junit-xml "./junit.xml"
```

### Environment Variables

```bash
export KUBECONFIG=/home/ubuntu/k8s.yaml  # For K8s controller access
export JUJU_DATA=$HOME/.local/share/juju  # Juju metadata
export LOG_LEVEL=DEBUG
export TEST_OUTPUT_DIR=./test-logs
```

### Log Handling

The skill manages logs automatically:

```
test-logs/
├── juju-controller-target.tar.gz        # Target controller logs
├── juju-controller-neighbor.tar.gz      # Neighbor controller logs
├── target/
│   ├── application-postgresql.tar.gz    # Target app logs
│   └── unit-postgresql-0.tar.gz         # Target unit logs
├── neighbor/
│   ├── application-pgbouncer.tar.gz     # Neighbor app logs
│   └── unit-pgbouncer-0.tar.gz          # Neighbor unit logs
└── junit.xml                             # Test results
```

---

## Understanding Test Results: State Machine & Cascade Skips

The test framework uses an **internal state machine** to manage test dependencies. Understanding this prevents confusion about skipped tests.

### How the State Machine Works

Tests are marked with state requirements like:
- `test_build_bundle`: `no_bundle` → `bundle_built`
- `test_deploy`: `bundle_built` → `deployed`
- `test_upgrade_charm`: `deployed` → `charm_upgraded`

When you specify `--current-state "no_bundle"`, the scheduler:
1. Starts from that state
2. Runs tests to reach subsequent states
3. **Skips tests** that aren't on the path to the final state

### Cascade Skips After Test Failure

**Important:** When a state-transition test **fails**, all subsequent state-marked tests are automatically **skipped** because the environment state is now unknown.

Example:
```
✅ test_build_bundle (no_bundle → bundle_built)
✅ test_deploy (bundle_built → deployed)
❌ test_model_controller_migration (deployed → deployed)  ← FAILS
⏭️ test_upgrade_charm (deployed → upgraded)               ← SKIPPED (state unknown)
⏭️ test_downgrade_charm (deployed → downgraded)           ← SKIPPED (state unknown)
```

**This is expected behavior.** The skips mean:
- The framework detected a state-transition failure
- It can't guarantee the environment is in the expected state
- Remaining tests that depend on state are skipped for safety

If you see many skipped tests, **check the failed test first**. That's usually the root cause.

### Non-State Tests Always Run

Tests without state markers always run (unless they fail independently). These are integration tests that don't care about the state lifecycle.

### Known Environmental Failures (Sandbox Constraints)

The sandbox environment has resource constraints that cause some tests to fail unrelated to code issues:

| Test | Issue | Workaround |
|------|-------|-----------|
| `test_controller_restart` | "No selected controller" error | Use `-k "not test_controller_restart"` |
| `test_model_controller_migration` | Machine controller pending during migration | Use `-k "not test_model_controller_migration"` |

**These failures cascade and prevent downstream tests (including log collection) from completing.**

When validating fixes, especially for mixed-cloud logging bugs, exclude these tests:

```bash
./scripts/run-tests.sh \
  --target-cloud "localhost" \
  --target-charm "glauth-utils" \
  --neighbor-cloud "local-k8s" \
  --neighbor-charm "glauth-k8s" \
  --current-state "no_bundle" \
  --charm-overrides "./static/charm-overrides/" \
  --log-dir "./test-logs" \
  -k "not (test_controller_restart or test_model_controller_migration)"
```

This allows:
- ✅ test_build_bundle
- ✅ test_bootstrap_controller  
- ✅ test_create_model
- ✅ test_deploy (may still fail for other reasons)
- ✅ test_logs_privacy_check and **log collection to complete**

**Key Point**: Log collection runs in the teardown phase (finally block), so excluding environmental failures ensures both controllers reach log collection even if deployment phases fail.

---

## Troubleshooting

### Problem: "Controller already exists"

The charm-testing workflow reuses cloud names. If you have existing controllers:

```bash
# Destroy existing controllers
juju kill-controller target -y
juju kill-controller neighbor -y

# Retry the test
./scripts/run-tests.sh \
  --target-cloud "localhost" \
  --target-charm "postgresql" \
  ...
```

### Problem: "Kubeconfig not found"

K8s cloud setup might be incomplete. Before running tests:

```bash
# Check if K8s cloud is registered
juju clouds | grep local-k8s

# If not found, install and configure K8s
# (See setup-k8s skill documentation)
$PROJECT_ROOT/development-sandbox/bin/setup-k8s.sh

# Bootstrap a K8s controller
juju bootstrap local-k8s k8s-ctrl
```

### Problem: "Charm deployment timeout"

Increase the timeout and use verbose logging:

```bash
./scripts/run-tests.sh \
  --target-cloud "localhost" \
  --target-charm "postgresql" \
  --log-cli-level "DEBUG" \
  --log-level "DEBUG" \
  --timeout 3600 \
  ...
```

Then check the logs:
```bash
tail -f test-logs/pytest.log
```

### Problem: "CMR relation failed"

Inspect the relation manually:

```bash
# Check relation status
juju status -m target/mymodel --relations

# Check if relation is pending
juju relate target/postgresql neighbor/pgbouncer

# Check unit logs for relation errors
juju ssh -m target/mymodel 0 "sudo grep -i relation /var/log/juju/unit*.log"
```

---

## Best Practices

1. **Start Simple**: Begin with single-cloud tests before CMR
2. **Narrow Scope**: Run individual test modules before full suite
3. **Preserve Models**: Use `--current-state` to skip teardown for debugging
4. **Clean Between Runs**: Use `scripts/sandbox.sh` to reset or create new VMs between test iterations
5. **Check Prerequisites**: Verify LXD/K8s available before running with `/setup-lxd` or `/setup-k8s`
6. **Capture Output**: Logs are automatically collected into the log directory specified by `--log-dir`

---

## Next Steps

After identifying and fixing an issue:

1. **Verify the fix locally** with this skill
2. **Run full test suite** to check for regressions
3. **Check log collection** to ensure logs are captured (important for issue #693!)
4. **Document findings** in the investigation report

---

## References

- **Test Suite**: `charm_integration_testing/test_suite/`
- **Conftest**: `charm_integration_testing/test_suite/conftest.py`
- **Resource Registry**: `charm_integration_testing/juju/resource_registry/`
- **Scheduler**: `charm_integration_testing/test_suite/scheduler/`
- **Workflow**: `.github/workflows/charm-testing.yaml`
- **Run Script**: `scripts/run-tests.sh`