Bittorrent Client From Scratch

/bittorrent-client-from-scratch

When to Use

• Building a minimal BitTorrent client (download-only or full peer) to understand the protocol, not to ship a production downloader.
• Implementing bencode (BitTorrent's serialization format) for a .torrent file parser.
• Writing tracker communication (HTTP/UDP announce) or a bare peer-wire-protocol handshake.
• Teaching or learning piece-selection strategy (rarest-first), choke/unchoke algorithms.

Do NOT use for

• Production file transfer — use an existing library (libtorrent, aria2, WebTorrent) instead of hand-rolling protocol code for a real product.
• General P2P networking unrelated to BitTorrent's specific wire format (see wamp-pubsub-patterns or raft-consensus-patterns for other distributed patterns).
• DHT-only (trackerless) discovery as a first implementation — build the tracker-based path first (see Step 4), DHT is a documented extension, not the starting point.

Protocol Layers (build in this order)

1. Bencode        — the serialization format everything else is written in
2. .torrent file  — bencoded dict describing the download (info hash, piece hashes, tracker URL)
3. Tracker         — HTTP/UDP request that returns a peer list
4. Peer wire       — TCP protocol between two peers: handshake, message framing, piece exchange
5. Piece manager    — tracks which pieces you have/need, drives requests
6. (Optional) DHT  — trackerless peer discovery, BEP 5 — add only after 1-5 work

Step 1: Bencode

Four types only — no schema needed, all self-describing:

Integer:      i<number>e              → i42e = 42
Byte string:  <length>:<bytes>        → 4:spam = "spam"
List:         l<items>e               → l4:spami42ee = ["spam", 42]
Dictionary:   d<key><value>...e       → d3:cow3:mooe = {"cow": "moo"}, keys sorted by raw byte order

Write both decode(bytes) -> value and encode(value) -> bytes — you need encode to re-derive the info hash (see Step 2) and to build tracker announce requests.

Step 2: Parse the .torrent File

A .torrent file is a bencoded dict with keys: announce (tracker URL), info (dict containing name, piece length, pieces — a concatenated string of 20-byte SHA-1 hashes, one per piece, length for single-file or files for multi-file torrents).

The info hash — the torrent's unique identifier everywhere in the protocol — is SHA1(bencode(info_dict)), computed over the raw bencoded bytes of just the info value, not a re-serialization from a parsed struct (re-encoding can silently reorder or reformat and produce the wrong hash — keep the raw byte range from the original decode).

Step 3: Talk to the Tracker

HTTP GET to the announce URL with query params: info_hash (20 raw bytes, URL-encoded), peer_id (your 20-byte client ID), port, uploaded, downloaded, left, compact=1. Response is a bencoded dict with interval (seconds until re-announce) and peers — with compact=1, a byte string where every 6 bytes is one peer (4 bytes IP + 2 bytes port, big-endian).

UDP tracker protocol (BEP 15) exists for trackers that don't support HTTP — it's a separate connect/announce handshake, implement it only if a tracker in practice requires it.

Step 4: Peer Wire Protocol

Handshake (fixed 68 bytes): \x13BitTorrent protocol + 8 reserved bytes + 20-byte info hash + 20-byte peer ID. Both sides send this first; a peer that returns a different info hash is not serving your torrent — drop the connection.

After the handshake, all messages are length-prefixed (4-byte big-endian length, then 1-byte message ID, then payload). Core message IDs:

0  choke            5  bitfield   (which pieces this peer has)
1  unchoke          6  request    (index, begin, length)
2  interested       7  piece      (index, begin, block data)
3  not interested    8  cancel
4  have (piece idx)

State machine per connection: you start choked + not interested. Send interested. Wait for the peer to unchoke you — only then can you request blocks. A zero-length message (just the 4-byte length field, no ID) is a keep-alive.

Step 5: Piece Manager

• Track piece state as a bitfield (have / don't have / in-flight).
• Split each piece into blocks (commonly 16KB) — request messages are per-block, not per-piece.
• Rarest-first: track which peers have which pieces; request pieces that fewest peers have first — this keeps piece availability balanced across the swarm and avoids everyone racing for the same common piece.
• On receiving a full piece, verify its SHA-1 against the hash from the .torrent file's pieces field before writing to disk or announcing have — a mismatched hash means a corrupt/malicious peer, discard and re-request from someone else.

What NOT to Do

• Don't reorder or reformat the info dict before hashing it — you'll compute the wrong info hash and every tracker/peer will reject you.
• Don't request whole pieces in one message — the wire protocol requests blocks (~16KB); requesting a multi-MB piece in one request message is invalid per spec and most peers will drop you.
• Don't skip piece-hash verification "to save time" — an unverified piece can poison the rest of the download since later pieces may depend on file layout assumptions from earlier ones.
• Don't build DHT before the tracker path works end-to-end — DHT is a strictly harder distributed-hash-table problem layered on top of a protocol you should already understand.