How do I install the Pilot Map Reduce skill?

Run npx skillmds@latest add teoslayer/pilot-map-reduce in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Pilot Map Reduce safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Pilot Map Reduce?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Pilot Map Reduce free to use?

Yes. Installing skills from SkillMD is free. This skill is licensed under AGPL-3.

Who published Pilot Map Reduce?

TeoSlayer (@teoslayer) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Pilot Map Reduce by teoslayer — Agent Skill

Name: Pilot Map Reduce
Author: TeoSlayer

Pilot Map Reduce

Distributed map-reduce over agent swarms for parallel data processing. Use this skill when: 1. You need to process large datasets across multiple workers 2. You want parallel map phase followed by aggregating reduce phase 3. You have embarrassingly parallel tasks with combine step Do NOT use this skill when: - Tasks are not parallelizable (use single worker) - You need streaming results (use pilot-load-balancer)

TeoSlayer 3525560 2 files · 3.1 KB Updated Sep 17, 2026

File contents

pilot-map-reduce

Implement distributed map-reduce patterns for parallel data processing across agent swarms.

Commands

Submit map tasks to workers

TOTAL_WORKERS=$(pilotctl --json peers --search "role:mapper" | jq 'length')

for i in $(seq 0 $((TOTAL_WORKERS - 1))); do
  WORKER=$(pilotctl --json peers --search "role:mapper" | jq -r ".[$i].address")

  pilotctl --json send-message "$WORKER" \
    --data "{\"type\":\"map_task\",\"job_id\":\"$JOB_ID\",\"chunk_start\":$((i * 1000)),\"chunk_end\":$(((i + 1) * 1000))}"
done

Collect map results

EXPECTED_RESULTS=$TOTAL_WORKERS
RECEIVED=0

while [ $RECEIVED -lt $EXPECTED_RESULTS ]; do
  RESULTS=$(pilotctl --json received \
    | jq '[.messages[] | select(.payload.type == "map_result" and .payload.job_id == "'$JOB_ID'")] | length')
  RECEIVED=$RESULTS
  sleep 1
done

Shuffle and reduce

MAP_RESULTS=$(cat /tmp/map-results-$JOB_ID.json)
KEYS=$(echo "$MAP_RESULTS" | jq -r '.[].payload.results | to_entries | .[].key' | sort -u)

for key in $KEYS; do
  VALUES=$(echo "$MAP_RESULTS" | jq -r '[.[].payload.results."'$key'" // empty] | flatten')

  pilotctl --json send-message "$REDUCER" \
    --data "{\"type\":\"reduce_task\",\"job_id\":\"$JOB_ID\",\"key\":\"$key\",\"values\":$VALUES}"
done

Workflow Example

Word count across distributed text corpus:

#!/bin/bash
JOB_ID="wordcount-$(date +%s)"

# MAP phase
MAPPERS=$(pilotctl --json peers --search "role:mapper" | jq -r '.[].address')
for i in $(seq 0 9); do
  pilotctl --json send-message "${MAPPERS[$i]}" \
    --data "{\"type\":\"map_task\",\"job_id\":\"$JOB_ID\",\"chunk\":$i}" &
done
wait

# REDUCE phase
sleep 5
MAP_RESULTS=$(pilotctl --json received \
  | jq '[.messages[] | select(.payload.type == "map_result")]')

FINAL=$(echo "$MAP_RESULTS" | jq 'map({(.payload.word): .payload.count}) | add')
echo "$FINAL"

Dependencies

Requires pilot-protocol skill, jq, and sort.

---name: pilot-map-reducedescription: Distributed map-reduce over agent swarms for parallel data processing. Use this skill when: 1. You need to process large datasets across multiple workers 2. You want parallel map phase followed by aggregating reduce phase 3. You have embarrassingly parallel tasks with combine step Do NOT use this skill when: - Tasks are not parallelizable (use single worker) - You need streaming results (use pilot-load-balancer)license: AGPL-3.0---# pilot-map-reduceImplement distributed map-reduce patterns for parallel data processing across agent swarms.## Commands### Submit map tasks to workers```bashTOTAL_WORKERS=$(pilotctl --json peers --search "role:mapper" | jq 'length')for i in $(seq 0 $((TOTAL_WORKERS - 1))); do WORKER=$(pilotctl --json peers --search "role:mapper" | jq -r ".[$i].address") pilotctl --json send-message "$WORKER" \ --data "{\"type\":\"map_task\",\"job_id\":\"$JOB_ID\",\"chunk_start\":$((i * 1000)),\"chunk_end\":$(((i + 1) * 1000))}"done```### Collect map results```bashEXPECTED_RESULTS=$TOTAL_WORKERSRECEIVED=0while [ $RECEIVED -lt $EXPECTED_RESULTS ]; do RESULTS=$(pilotctl --json received \ | jq '[.messages[] | select(.payload.type == "map_result" and .payload.job_id == "'$JOB_ID'")] | length') RECEIVED=$RESULTS sleep 1done```### Shuffle and reduce```bashMAP_RESULTS=$(cat /tmp/map-results-$JOB_ID.json)KEYS=$(echo "$MAP_RESULTS" | jq -r '.[].payload.results | to_entries | .[].key' | sort -u)for key in $KEYS; do VALUES=$(echo "$MAP_RESULTS" | jq -r '[.[].payload.results."'$key'" // empty] | flatten') pilotctl --json send-message "$REDUCER" \ --data "{\"type\":\"reduce_task\",\"job_id\":\"$JOB_ID\",\"key\":\"$key\",\"values\":$VALUES}"done```## Workflow ExampleWord count across distributed text corpus:```bash#!/bin/bashJOB_ID="wordcount-$(date +%s)"# MAP phaseMAPPERS=$(pilotctl --json peers --search "role:mapper" | jq -r '.[].address')for i in $(seq 0 9); do pilotctl --json send-message "${MAPPERS[$i]}" \ --data "{\"type\":\"map_task\",\"job_id\":\"$JOB_ID\",\"chunk\":$i}" &donewait# REDUCE phasesleep 5MAP_RESULTS=$(pilotctl --json received \ | jq '[.messages[] | select(.payload.type == "map_result")]')FINAL=$(echo "$MAP_RESULTS" | jq 'map({(.payload.word): .payload.count}) | add')echo "$FINAL"```## DependenciesRequires pilot-protocol skill, jq, and sort.

Frequently asked questions