Python Debugging

Python Debugging Methodology

Systematic problem-solving framework for Python code debugging.

When to Use This Skill

• Investigating exceptions and stack traces
• Debugging asynchronous code (async/await)
• Resolving logging configuration issues
• Troubleshooting MCP server communication errors
• Diagnosing test failures (pytest, unittest)
• Analyzing performance problems

Debugging Workflow (推論プロセス)

1. Interpret Error Messages

Objective: Extract actionable information from error output

Error message anatomy:

Traceback (most recent call last):
  File "script.py", line 42, in main        ← Calling context
    result = process_data(input)            ← Line that triggered error
  File "utils.py", line 15, in process_data ← Where exception was raised
    return data['key']                      ← Failing operation
KeyError: 'key'                             ← Exception type and message

Questions to answer:

• What is the exception type? (KeyError, AttributeError, TypeError, etc.)
• What was the last successful operation?
• What input triggered the failure?

2. Read Stack Traces

Objective: Understand execution path to failure point

Reading strategy:

Top-down (most common):

• Start from outermost frame (entry point)
• Trace execution flow downward
• Understand calling context

Bottom-up (for root cause):

• Start from innermost frame (where exception was raised)
• Identify the specific operation that failed
• Trace why inputs were invalid

Focus points:

• Function arguments at each level
• Local variables at failure point
• Library code vs application code boundaries

3. Reproduce Consistently

Objective: Create minimal failing test case

Simplification steps:

1. Isolate the failing function
2. Extract minimal input that triggers failure
3. Remove dependencies (use mocks if needed)
4. Create standalone script (~10 lines)

Example:

# Minimal reproducer
data = {'wrong_key': 123}
result = data['key']  # KeyError

4. Form Hypotheses

Objective: List plausible causes based on exception type

Hypothesis by exception type:

For asynchronous code:

• Is await missing? (RuntimeWarning: coroutine never awaited)
• Is blocking I/O used in async function?
• Are multiple tasks accessing shared state?

For MCP/RPC issues:

• Is JSON-RPC request malformed?
• Is server process running?
• Are method names spelled correctly?
• Is timeout too short?

5. Select Debugging Strategy

Objective: Choose appropriate tool for the problem

Decision tree:

Is the error location obvious from stack trace?
├─ Yes → Use print() or logging
│        Fast iteration, minimal setup
└─ No → Use pdb debugger
         Interactive inspection needed

Is the problem in async code?
├─ Yes → Use logging with timestamps
│        await points are not debugger-friendly
└─ No → pdb works well

Is it a test failure?
├─ Yes → pytest --pdb
│        Drops into debugger at failure point
└─ No → Standard debugging approach

6. Apply Debugging Tool

Objective: Gather evidence to confirm/reject hypothesis

Tool selection:

print() / logging: Quick hypothesis testing

print(f"DEBUG: variable={variable}, type={type(variable)}")

pdb: Interactive investigation

import pdb; pdb.set_trace()  # Breakpoint here

logging: Persistent diagnostics

logger.debug(f"Processing {count} items")
logger.warning(f"Unexpected value: {value}")

Debugging Strategy Selection

When to Use print()

Use for:

• Quick value inspection
• Hypothesis: "If I print X here, I'll see Y"
• Short-lived debugging sessions

Limitations:

• Clutters code
• Must re-run for each new hypothesis
• No interactive inspection

When to Use pdb

Use for:

• Multiple hypotheses to test
• Need to inspect many variables
• Control flow is unclear
• Complex data structures

Basic commands:

(Pdb) l          # List code around current line
(Pdb) n          # Next line (step over)
(Pdb) s          # Step into function
(Pdb) c          # Continue to next breakpoint
(Pdb) p variable # Print variable value
(Pdb) pp obj     # Pretty-print object
(Pdb) w          # Where am I? (stack trace)

When to Use logging

Use for:

• Production code debugging
• Long-running processes
• Intermittent failures
• Timeline reconstruction

Configuration:

import logging
logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)

Error Pattern Recognition

Common Python Patterns

Pattern: "NoneType has no attribute X"

# Cause: Function returned None instead of expected object
result = get_data()  # Returns None on error
result.process()     # AttributeError: 'NoneType' has no attribute 'process'

# Investigation: Why did get_data() return None?

Pattern: "Dictionary changed size during iteration"

# Cause: Modifying dict while iterating
for key in data:
    if condition:
        del data[key]  # RuntimeError

# Fix: Iterate over copy
for key in list(data.keys()):
    if condition:
        del data[key]

Pattern: "JSON decode error: Expecting value"

# Cause: Empty or non-JSON response
response = requests.get(url)
data = response.json()  # JSONDecodeError if response is HTML or empty

# Investigation: Print response.text to see actual content

Async/Await Patterns

Pattern: "Coroutine was never awaited"

# Wrong: Calling async function without await
result = async_function()  # Returns coroutine object, doesn't execute

# Right: Await the coroutine
result = await async_function()

Pattern: "Event loop already running"

# Cause: Nested asyncio.run() calls
async def outer():
    asyncio.run(inner())  # Error: loop already running

# Fix: Just await
async def outer():
    await inner()

Hypothesis Testing Techniques

Binary Search Debugging

Narrow down the failing region:

# Add checkpoints to partition code
def process_data(data):
    print("CHECKPOINT 1: Input valid")  # Check 1
    
    intermediate = transform(data)
    print("CHECKPOINT 2: Transform OK")  # Check 2
    
    result = finalize(intermediate)
    print("CHECKPOINT 3: Finalize OK")  # Check 3
    
    return result

# Run: Which checkpoint is NOT printed? Problem is before it.

Differential Analysis

Compare working vs failing cases:

# Working case
process_data({'key': 'value'})  # ✓ Success

# Failing case  
process_data({'wrong_key': 'value'})  # ✗ KeyError

# Insight: Code assumes 'key' always exists
# Fix: Add default or validation

Isolation Testing

Test components independently:

# Hypothesis: Function X works correctly in isolation
def test_function_x():
    input_data = {'key': 'value'}
    result = function_x(input_data)
    assert result == expected_value

# If test passes: Problem is in calling context or inputs
# If test fails: Problem is in function_x implementation

Logging Strategies

Log Level Guidelines

Structured Logging

Include context in log messages:

# Good: Context-rich logging
logger.info(f"Test {test_name} compilation completed in {duration}s")

# Bad: Ambiguous logging  
logger.info("Compilation completed")

Async Code Logging

Log await points to trace execution:

async def process_request(request_id):
    logger.debug(f"[{request_id}] Starting")
    
    data = await fetch_data()
    logger.debug(f"[{request_id}] Data fetched")
    
    result = await process(data)
    logger.debug(f"[{request_id}] Processing complete")
    
    return result

MCP Server Debugging

Common MCP Issues

Pattern: "Method not found"

# Cause: Typo in method name or not registered
# Investigation: Check server tool registry, verify spelling

Pattern: "Connection refused"

# Cause: Server not running or wrong port
# Investigation: Check server process, verify port number

Pattern: "Timeout waiting for response"

# Cause: Server hung, processing too slow, timeout too short
# Investigation: Check server logs, increase timeout, add progress logging

MCP Debugging Workflow

1. Verify server is running

   # Check process list for MCP server
   # Check server startup logs
   

2. Check request format

   logger.debug(f"Sending request: {json.dumps(request, indent=2)}")
   

3. Monitor server response

   logger.debug(f"Received response: {json.dumps(response, indent=2)}")
   

4. Validate JSON-RPC structure

   • Is "jsonrpc": "2.0" present?
   • Is "method" spelled correctly?
   • Are "params" in correct format?

Test Debugging

pytest Integration

Run test with debugger:

pytest test_file.py::test_function --pdb
# Drops into pdb on first failure

Add breakpoint in test:

def test_processing():
    data = load_test_data()
    import pdb; pdb.set_trace()  # Inspect here
    result = process(data)
    assert result == expected

Debugging Test Isolation

Problem: Test passes alone, fails in suite

Investigation:

• Check for shared state (module-level variables)
• Check for fixture side effects
• Run tests in different orders to isolate interaction

Common Pitfalls

Don't Ignore Warnings

# RuntimeWarning: coroutine 'func' was never awaited
# ↑ This WILL cause bugs, fix immediately

Don't Assume Exception Type

# Wrong: Catching generic exception
try:
    data = json.loads(text)
except:  # Catches everything, including KeyboardInterrupt
    pass

# Right: Catch specific exception
try:
    data = json.loads(text)
except json.JSONDecodeError as e:
    logger.error(f"Invalid JSON: {e}")

Don't Debug in Production

# Wrong: Leaving debug code in production
import pdb; pdb.set_trace()  # Will hang production server

# Right: Use logging with appropriate level
logger.debug("Debug information")  # Only in dev mode

Integration with Other Skills

• mcp-workflow: Use when debugging MCP server integration issues
• dsim-debugging: Coordinate when Python MCP server controls DSIM simulator
• uvm-verification: Debug UVM test invocation from Python scripts

Summary

Python debugging is systematic problem-solving:

1. Interpret error messages for exception type and context
2. Read stack traces to understand execution path
3. Reproduce with minimal test case
4. Form hypotheses based on exception patterns
5. Select appropriate debugging tool (print/pdb/logging)
6. Test hypotheses systematically

Key principle: Understand the error, then investigate the cause. Don't guess—use evidence from stack traces, logs, and debugger inspection.