# Explain Sample

> explain-sample

- Skill: `tankygranny05/explain-sample` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add tankygranny05/explain-sample`
- Raw SKILL.md: https://api.skillmd.com/api/skills/tankygranny05/explain-sample/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: tankygranny05 (https://skillmd.com/u/tankygranny05)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/tankygranny05/explain-sample

---

# explain-sample

<!-- [Created by Claude: 04db570d-16de-40b4-9c6d-760224b34357] -->

Use this skill when the user asks about the macOS `sample` command, process profiling, diagnosing high CPU usage, understanding why a process is spinning, or wants to know what a running process is doing internally.

---

## What is `sample`?

`sample` is a **built-in macOS command-line profiler** that answers the question: *"What is this process doing right now?"*

It works by taking **stack trace snapshots** of a running process at regular intervals (default: 1ms), then aggregating them into a call graph showing where CPU time is spent.

```bash
sample <PID> <duration_seconds> [-file output.txt]
```

---

## How It Works (The Magic)

Every running program uses a **call stack** - it's fundamental to how CPUs execute code:

```
main() calls → processData() calls → parseJSON() calls → readBuffer()
```

Each function call pushes a return address onto the stack. `sample` uses macOS Mach kernel APIs (`task_for_pid`) to:

1. **Pause** the process briefly (~1ms)
2. **Walk the stack** - read the chain of return addresses
3. **Resume** the process
4. **Repeat** ~1000 times per second
5. **Aggregate** - tally which functions appeared most often

If a function appears in 900 out of 1000 samples, it's using ~90% of CPU time.

---

## Basic Usage

```bash
# Sample process for 1 second, output to terminal
sample 12345 1

# Sample for 5 seconds, save to file
sample 12345 5 -file /tmp/profile.txt

# Sample by process name (if unique)
sample -p "node" 1
```

---

## Reading the Output

### The Call Graph

```
Call graph:
    1000 Thread_12345   ← 1000 samples from this thread
    + 1000 main
    +   950 doWork      ← 95% of time here
    +   ! 800 innerLoop ← 80% in this sub-function
    +   ! ! 800 spin    ← the actual culprit
    +   50 cleanup      ← only 5% here
```

**Key insight:** Follow the numbers. The deepest function with high counts is usually your bottleneck.

### Symbols vs Addresses

| What you see | Meaning |
|--------------|---------|
| `functionName (in libfoo.dylib)` | Symbols available - readable |
| `0x100003f40 (in binary)` | Stripped binary - just addresses |
| `??? (in <unknown binary>)` | JIT-compiled code (JavaScript, Java, etc.) |

---

## Real-World Example: Diagnosing a Spinning Process

### The Problem

A Claude Code process (PID 39779) was consuming 100% CPU despite being "idle":

```bash
ps -Ao pid,pcpu,comm | head -5
  PID  %CPU COMM
39779 102.1 claude   ← Why is this spinning?
```

### The Investigation

```bash
sample 39779 1 -file /tmp/claude-sample.txt
```

### The Output (abbreviated)

```
Call graph:
    805 Thread_223980213
    + 805 node::Start
    +   805 node::NodeMainInstance::Run()
    +     805 node::SpinEventLoopInternal
    +       733 uv_run                        ← libuv event loop
    +         733 uv__run_check               ← "check" phase
    +           730 node::CheckImmediate      ← setImmediate callbacks
    +             729 v8::Function::Call      ← executing JavaScript
    +               726 ???                   ← JIT code (no symbols)
```

### The Diagnosis

- **733/805 samples (91%)** in `uv__run_check` → `CheckImmediate`
- The process is **busy-looping** in `setImmediate` callbacks
- Combined with `lsof -p 39779` showing stdin/stdout as `(revoked)`
- **Root cause:** The tmux session was killed, but the process didn't exit - it's spinning trying to read from a disconnected terminal

### The Fix

```bash
kill 39779
```

---

## What `sample` Can and Cannot Do

### ✅ Works On

| Target | Notes |
|--------|-------|
| **Any compiled binary** | C, C++, Rust, Go, Swift, etc. |
| **Interpreted languages** | Python, Ruby, Node.js (shows interpreter frames) |
| **JIT languages** | JavaScript, Java (shows runtime frames, but JIT code is `???`) |
| **System processes** | Most, unless SIP-protected |
| **GUI apps** | Absolutely |
| **Daemons/services** | Yes |

### ❌ Limitations

| Limitation | Workaround |
|------------|------------|
| **JIT code has no symbols** | Use language-specific profilers (node --prof, py-spy, async-profiler) |
| **Can't sample Apple-signed system binaries** (SIP) | Disable SIP (not recommended) or use Instruments |
| **Need same user or root** | `sudo sample` |
| **macOS only** | Use `perf` on Linux |
| **Sleeping/blocked threads show less** | They're not on CPU, so fewer samples |

### 🔑 Key Non-Limitation

**You don't need source code, debug builds, or special compilation flags.** `sample` works on any running process. Stripped binary? You still get the call stack - just addresses instead of names.

---

## Pro Techniques

### 1. Quick CPU Hog Diagnosis

```bash
# Find top CPU process and sample it immediately
TOP_PID=$(ps -Ao pid,%cpu -r | awk 'NR==2 {print $1}')
sample $TOP_PID 3 -file /tmp/hot.txt && head -80 /tmp/hot.txt
```

### 2. Compare Before/After

```bash
# Baseline
sample $PID 5 -file /tmp/before.txt

# ... make changes ...

# After
sample $PID 5 -file /tmp/after.txt

# Diff the call graphs
diff <(grep -E '^\s+[0-9]+' /tmp/before.txt | head -50) \
     <(grep -E '^\s+[0-9]+' /tmp/after.txt | head -50)
```

### 3. Sample During Specific Operation

```bash
# Start sampling in background
sample $PID 30 -file /tmp/during-operation.txt &

# Trigger the slow operation
curl http://localhost:8080/slow-endpoint

# Wait for sample to finish
wait
```

### 4. Focus on Specific Thread

```bash
# Sample outputs all threads - grep for the hot one
sample $PID 5 | grep -A 100 "Thread_.*DispatchQueue.*main"
```

### 5. Combine with Other Tools

```bash
# Step 1: Find the suspect
ps -Ao pid,%cpu,command -r | head -10

# Step 2: Check what files it has open
lsof -p $PID | head -20

# Step 3: See its call stack
sample $PID 1

# Step 4: Watch syscalls (if SIP allows)
sudo dtruss -p $PID 2>&1 | head -50
```

### 6. Symbolicate Addresses Later

If you have addresses from a stripped binary:

```bash
# Get the load address
sample $PID 1 | grep "Load Address"
# Load Address:    0x100000000

# Symbolicate an address
atos -o /path/to/binary -l 0x100000000 0x100003f40
```

---

## Related Tools

| Tool | Platform | Best For |
|------|----------|----------|
| `sample` | macOS | Quick CLI profiling |
| `Instruments.app` | macOS | Deep GUI profiling, Time Profiler |
| `spindump` | macOS | Hang/freeze diagnosis |
| `perf record/report` | Linux | Equivalent to sample |
| `py-spy` | Any | Python-specific, resolves Python frames |
| `rbspy` | Any | Ruby-specific |
| `node --prof` | Any | V8/Node.js profiling |
| `async-profiler` | Any | Java/JVM profiling |

---

## Quick Reference

```bash
# Basic: sample PID for N seconds
sample <PID> <seconds>

# Save to file
sample <PID> <seconds> -file /tmp/output.txt

# Sample with more frequent snapshots (microseconds between samples)
sample <PID> <seconds> -wait 500  # 500 microseconds = 2000 samples/sec

# Just see the summary
sample <PID> 1 2>&1 | head -50
```

---

## TL;DR

`sample` answers **"why is this process using CPU?"** by showing you the call stack. It works on any process, any language, any binary - you just might get addresses instead of function names if symbols are stripped. When you see a process spinning at 100% CPU, `sample` is your first tool.