Muennighoff Code Eval Octopack

Design, tune, and evaluate production RAG pipelines with deterministic tools for chunking, pipeline design, and retrieval evaluation.

Guides usage of the Gemini API on Agent Platform with the Google Gen AI SDK, covering SDK usage (Python, JS/TS, Go, Java, C#), capabilities like multimodal inputs, tools, media generation, caching, batch prediction, and Live API.

Spawns multiple parallel AI agents that compete on the same task using isolated git worktrees, evaluates results, and merges the best solution.

Design multi-agent system architectures, generate tool schemas for Anthropic and OpenAI formats, and evaluate execution logs for cost, latency, and failure bottlenecks.

Defines an inter-agent communication protocol for C-suite agent teams, including invocation syntax, loop prevention, isolation rules, and response formats to coordinate cross-functional analysis and board meetings.

Design and ship production-ready MCP servers from OpenAPI contracts instead of hand-written tool wrappers. Supports Python and TypeScript, schema validation, and safe evolution.