py-libp2p Master Operational Suite
A dedicated engineering toolkit for developing, auditing, and maintaining py-libp2p — the official Python implementation of the libp2p modular peer-to-peer networking stack.
🏛️ py-libp2p Architectural Layers
┌────────────────────────────────────────────────────────┐
│ Application Protocols: Ping, Identify, DHT, Bitswap │
├────────────────────────────────────────────────────────┤
│ PubSub Subsystems: GossipSub, FloodSub │
├────────────────────────────────────────────────────────┤
│ Stream Multiplexing: Yamux, Mplex │
├────────────────────────────────────────────────────────┤
│ Secure Channel: Noise (XX pattern), TLS │
├────────────────────────────────────────────────────────┤
│ Transports: TCP, QUIC, WebSocket │
├────────────────────────────────────────────────────────┤
│ Network Swarm, Peerstore & PeerID │
└────────────────────────────────────────────────────────┘
Golden Architectural Rule: Lower layers must NEVER import from or depend on upper layers. Any upward dependency (e.g., Transport importing from PubSub or Host) is a critical architecture violation.
🛠️ Specialized py-libp2p Skills
| Skill |
Path |
Primary Purpose |
| Spec Compliance Guard |
pylibp2p-spec-guard |
Enforces 100% official libp2p spec compliance by default; intercepts any out-of-spec changes with explicit warning and requires user confirmation. |
| PR Pre-Flight & Newsfragments |
pylibp2p-pr-prep |
Validates mandatory towncrier newsfragments, executes make pr, checks Sphinx docs, and verifies interface ABI compatibility. |
| Concurrency & Lifecycle Auditor |
pylibp2p-concurrency-audit |
Audits Trio structured concurrency, task leaks, cancellation swallow traps (trio.Cancelled), and stream short-read bugs (read_exactly). |
| Protocol Scaffolder |
pylibp2p-protocol-scaffolder |
Scaffolds new libp2p protocols: Protocol ID, stream handlers, protobuf framing, client methods, and @pytest.mark.trio integration tests. |
| Wire Protocol & Interop Debugger |
pylibp2p-wire-debug |
Decodes raw packets, multistream-select varint headers, Noise XX handshakes, Yamux 12-byte frames, and runs go-libp2p interop tests. |
⚡ Daily Development Cheat Sheet
1. Build, Lint & CI Pre-Flight
# Activate virtual environment
source .venv/bin/activate || source venv/bin/activate
# Full PR validation (clean -> fix -> lint -> typecheck -> test)
make pr
# Autoformat code with ruff
make fix
# Validate mandatory newsfragments before pushing
python ./newsfragments/validate_files.py
towncrier build --draft --version preview
# Recompile protocol buffers if .proto files changed
make protobufs
2. Fast Targeted Testing
# Run a specific test module
pytest tests/core/host/test_ping.py -v
# Run with full async debug logs to trace packet flows
pytest tests/core/stream_muxer/test_yamux.py -v -s --log-cli-level=DEBUG
# Run cross-implementation interop tests against go-libp2p
pytest tests/interop/ -v
3. Concurrency & Stream Golden Invariants
- Always use
read_exactly(stream, n) when expecting a fixed-length header; stream.read(n) is a short read.
- Never swallow
trio.Cancelled or asyncio.CancelledError — always re-raise.
- Always handle
StreamEOF and StreamReset gracefully in protocol loops.
1---2name: pylibp2p3description: Master router and architectural reference for developing, debugging, and maintaining the py-libp2p repository. Guides py-libp2p PR pre-flight validation, newsfragments, Trio structured concurrency auditing, protocol scaffolding, multistream-select, Noise security handshakes, Yamux stream muxing, and interop testing. Use when the user asks for "py-libp2p help", "work on py-libp2p", "prepare py-libp2p PR", "debug py-libp2p", "audit py-libp2p concurrency", or any task inside the py-libp2p codebase.4license: MIT5---67# py-libp2p Master Operational Suite89A dedicated engineering toolkit for developing, auditing, and maintaining **`py-libp2p`** — the official Python implementation of the libp2p modular peer-to-peer networking stack.1011---1213## 🏛️ py-libp2p Architectural Layers1415```16┌────────────────────────────────────────────────────────┐17│ Application Protocols: Ping, Identify, DHT, Bitswap │18├────────────────────────────────────────────────────────┤19│ PubSub Subsystems: GossipSub, FloodSub │20├────────────────────────────────────────────────────────┤21│ Stream Multiplexing: Yamux, Mplex │22├────────────────────────────────────────────────────────┤23│ Secure Channel: Noise (XX pattern), TLS │24├────────────────────────────────────────────────────────┤25│ Transports: TCP, QUIC, WebSocket │26├────────────────────────────────────────────────────────┤27│ Network Swarm, Peerstore & PeerID │28└────────────────────────────────────────────────────────┘29```3031> **Golden Architectural Rule:** Lower layers must NEVER import from or depend on upper layers. Any upward dependency (e.g., Transport importing from PubSub or Host) is a critical architecture violation.3233---3435## 🛠️ Specialized py-libp2p Skills3637| Skill | Path | Primary Purpose |38|---|---|---|39| **Spec Compliance Guard** | [pylibp2p-spec-guard](./pylibp2p-spec-guard/SKILL.md) | Enforces 100% official libp2p spec compliance by default; intercepts any out-of-spec changes with explicit warning and requires user confirmation. |40| **PR Pre-Flight & Newsfragments** | [pylibp2p-pr-prep](./pylibp2p-pr-prep/SKILL.md) | Validates mandatory towncrier newsfragments, executes `make pr`, checks Sphinx docs, and verifies interface ABI compatibility. |41| **Concurrency & Lifecycle Auditor** | [pylibp2p-concurrency-audit](./pylibp2p-concurrency-audit/SKILL.md) | Audits Trio structured concurrency, task leaks, cancellation swallow traps (`trio.Cancelled`), and stream short-read bugs (`read_exactly`). |42| **Protocol Scaffolder** | [pylibp2p-protocol-scaffolder](./pylibp2p-protocol-scaffolder/SKILL.md) | Scaffolds new libp2p protocols: Protocol ID, stream handlers, protobuf framing, client methods, and `@pytest.mark.trio` integration tests. |43| **Wire Protocol & Interop Debugger** | [pylibp2p-wire-debug](./pylibp2p-wire-debug/SKILL.md) | Decodes raw packets, multistream-select varint headers, Noise XX handshakes, Yamux 12-byte frames, and runs `go-libp2p` interop tests. |4445---4647## ⚡ Daily Development Cheat Sheet4849### 1. Build, Lint & CI Pre-Flight50```bash51# Activate virtual environment52source .venv/bin/activate || source venv/bin/activate5354# Full PR validation (clean -> fix -> lint -> typecheck -> test)55make pr5657# Autoformat code with ruff58make fix5960# Validate mandatory newsfragments before pushing61python ./newsfragments/validate_files.py62towncrier build --draft --version preview6364# Recompile protocol buffers if .proto files changed65make protobufs66```6768### 2. Fast Targeted Testing69```bash70# Run a specific test module71pytest tests/core/host/test_ping.py -v7273# Run with full async debug logs to trace packet flows74pytest tests/core/stream_muxer/test_yamux.py -v -s --log-cli-level=DEBUG7576# Run cross-implementation interop tests against go-libp2p77pytest tests/interop/ -v78```7980### 3. Concurrency & Stream Golden Invariants81- **Always use `read_exactly(stream, n)`** when expecting a fixed-length header; `stream.read(n)` is a short read.82- **Never swallow `trio.Cancelled` or `asyncio.CancelledError`** — always re-raise.83- **Always handle `StreamEOF` and `StreamReset`** gracefully in protocol loops.
sumanjeet0012