tomevault-io Signal E2E Test Skill
Verify coop features work end-to-end over real Signal messaging. Catches bugs that unit tests miss: protocol edge cases, timing issues, presage quirks, message ordering, LLM API failures, tool execution through the full stack.
When to use: After implementing or fixing anything that touches Signal message handling, agent turns, tool execution, provider calls, prompt building, compaction, or slash commands.
Architecture
Two Signal accounts are registered to signal-cli on this VM. Coop is linked as a presage secondary device to one. The other sends test messages simulating a human user.
signal-cli (Alice) signal-cli (Bob — primary)
Test sender ↕ shared identity
│ coop/presage (Bob — secondary)
│ Signal Protocol COOP_TRACE_FILE=traces.jsonl
└─────────────────────▶ │
▼
traces.jsonl ← you read this
The scripts auto-detect which account is coop (multi-device) and which is the sender.
Scripts
All scripts are in scripts/ relative to this skill:
SKILL_DIR=/root/coop/main/.claude/skills/signal-e2e-test
discover-accounts.sh
Auto-detects the coop account and test sender from local signal-cli. Probes that the sender can actually deliver.
bash $SKILL_DIR/scripts/discover-accounts.sh
# Outputs: COOP_NUMBER, SENDER_CMD, SENDER_NUMBER, SENDER_UUID
preflight.sh
Checks coop is running, signal channel is active, traces are flowing.
bash $SKILL_DIR/scripts/preflight.sh traces.jsonl
send-and-verify.sh
Sends a message, polls traces for completion, reports PASS/FAIL. Automatically patches coop.toml with the sender's UUID for the test and restores the placeholder on exit.
bash $SKILL_DIR/scripts/send-and-verify.sh "What is 2+2?"
bash $SKILL_DIR/scripts/send-and-verify.sh "/status"
bash $SKILL_DIR/scripts/send-and-verify.sh "Read SOUL.md and summarize it" --timeout 60
bash $SKILL_DIR/scripts/send-and-verify.sh "Run 'uname -a'" --expect-tool bash --timeout 60
Options: --timeout N, --expect-tool NAME, --expect-command, --no-reply-check, --trace-file FILE, --sender-cmd CMD, --target NUM.
Verification Workflow
1. Build
cargo fmt
cargo build --features signal
2. Start or restart coop in tmux
# Kill existing, clear traces, restart
tmux send-keys -t coop C-c 2>/dev/null; sleep 2
> traces.jsonl
tmux send-keys -t coop 'COOP_TRACE_FILE=traces.jsonl cargo run --features signal --bin coop -- start' Enter
# Or create fresh session
tmux new-session -d -s coop
tmux send-keys -t coop 'cd /root/coop/main && COOP_TRACE_FILE=traces.jsonl cargo run --features signal --bin coop -- start' Enter
Wait ~10s, then:
bash $SKILL_DIR/scripts/preflight.sh
3. Send and verify
bash $SKILL_DIR/scripts/send-and-verify.sh "YOUR TEST MESSAGE"
4. If FAIL — read traces, fix, repeat from step 1
The script prints the trace tail on failure. For deeper analysis:
grep '"level":"ERROR"' traces.jsonl | tail -10
grep -E 'signal_receive|signal inbound|route_message|agent_turn|signal_action' traces.jsonl | tail -20
Diagnosing Failures
| Checklist item that failed | Likely cause | Fix |
|---|---|---|
| Message received by coop | Presage not connected | Restart coop, check signal.db exists |
| Message dispatched | Parse error in signal channel | Check inbound_from_content, look for panics |
| Slash command handled | Command not recognized | Check command dispatch in signal loop |
| Agent turn started | Routing failure, trust mismatch | Check coop.toml users match sender UUID |
| Reply sent via Signal | Turn completed but reply not routed | Check signal loop reply path |
| Tool 'X' executed | Agent didn't call the expected tool | Check tool definitions, prompt, trust level |
| No errors | Something threw an error | Read the error from traces |
| Completed within timeout | Processing hung or too slow | Increase --timeout, check for infinite loops |
First-time setup
Sender not recognized (trust mismatch): After Alice's first message, get her UUID from traces and add to coop.toml:
grep '"signal inbound dispatched"' traces.jsonl | tail -1
# Look for signal.sender field — that's Alice's UUID
# Add to coop.toml [[users]] match list. Coop hot-reloads.
Common Test Messages
| Feature changed | Test message | Extra flags |
|---|---|---|
| Basic message flow | "What is 2+2?" |
|
| Slash commands | "/status" |
|
| Session management | "/new" |
|
| File tools | "Read SOUL.md and summarize it" |
--timeout 60 --expect-tool read_file |
| Bash tool | "Run 'uname -a'" |
--timeout 60 --expect-tool bash |
| Memory | "Remember: test value. Save to memory." |
--expect-tool memory_write |
| Signal history | "Use signal_history to see recent messages" |
--expect-tool signal_history |
Reactions
eval "$(bash $SKILL_DIR/scripts/discover-accounts.sh 2>/dev/null)"
RESULT=$(${SENDER_CMD} send -m "React to this" ${COOP_NUMBER})
TS=$(echo "$RESULT" | python3 -c "import sys,json; print(json.load(sys.stdin)['timestamp'])")
${SENDER_CMD} sendReaction -t ${COOP_NUMBER} -e "👍" -a ${SENDER_NUMBER} --target-timestamp "$TS"
sleep 8
grep 'signal inbound dispatched' traces.jsonl | tail -3
# Pass if: inbound_kind contains "reaction", no crash
Rapid messages (turn serialization)
eval "$(bash $SKILL_DIR/scripts/discover-accounts.sh 2>/dev/null)"
${SENDER_CMD} send -m "First" ${COOP_NUMBER}; sleep 1
${SENDER_CMD} send -m "Second" ${COOP_NUMBER}; sleep 1
${SENDER_CMD} send -m "Third" ${COOP_NUMBER}
sleep 30
# Pass if: 3 signal_receive_event, 3 signal_action_send, no panics
grep -c 'signal_action_send' traces.jsonl
Full Test Plan
For comprehensive pre-release validation (19 scenarios including compaction), see docs/prompts/signal-e2e-trace-loop.md.
Bug Logging
ls docs/bugs/ | sort -n | tail -1— find next bug number- Create
docs/bugs/NNN-short-description.mdwith symptom + trace evidence - Fix → test → rebuild → restart → re-verify with
send-and-verify.sh - Update bug file with root cause, fix, mark
Fixed - Append to
docs/bugs/SESSION-LOG.md
Template in docs/prompts/signal-e2e-trace-loop.md.
Converted and distributed by TomeVault — claim your Tome and manage your conversions.