Research: Pydantic-AI to Alchemy-RS Migration

Research – Pydantic-AI to Alchemy-RS Migration

Date: 2026-01-27 Owner: Claude (Opus 4.5) Phase: Research

Goal

Map out what needs to change to migrate from pydantic-ai to alchemy-rs for LLM API calls, while keeping Python for state management and agent loop.

Architecture Summary

alchemy-rs provides:

• Unified LLM API abstraction (8+ providers)
• Streaming-first async design
• Tool/function calling support
• Cross-provider message transformation

Current tunacode uses pydantic-ai for:

1. LLM API calls (what alchemy-rs will replace)
2. Message types (ModelRequest, ModelResponse, parts)
3. Agent iteration loop (agent.iter() streaming)
4. Tool registration & execution (Tool class, ModelRetry)

Migration Difficulty Ranking

Tier 1: EASY - Direct Replacements

Why Easy: alchemy-rs is a drop-in replacement for the model layer. Same inputs (model name, API key), same outputs (streaming tokens).

Tier 2: MEDIUM - Need Adapter Layer

Why Medium: The message format will change. You already have a canonical message layer (types/canonical.py, utils/messaging/adapter.py) that isolates format changes. Extend this pattern for alchemy-rs.

Tier 3: KEEP IN PYTHON - No Migration Needed

Why Keep: State management has zero coupling to pydantic-ai's LLM features. It uses pydantic-ai message types for storage, but those are just dataclasses.

Tier 4: KEEP - Agent Loop (Python Orchestration)

Why Keep: The agent loop is control flow, not LLM protocol. Replace agent.iter() call with alchemy-rs streaming, keep the surrounding Python orchestration.

Concrete Changes Required

Phase 1: Create Alchemy-RS Bridge (Python bindings)

src/tunacode/core/llm/
├── __init__.py
├── alchemy_client.py      # PyO3 bindings to alchemy-rs
├── message_types.py       # Python wrappers for Rust types
└── streaming.py           # Async stream adapter

What this does:

1. Compile alchemy-rs as Python extension (PyO3/maturin)
2. Expose get_model(), stream(), Context to Python
3. Convert Rust stream events to Python async generator

Phase 2: Replace Model Layer

File: src/tunacode/core/agents/agent_components/agent_config.py

Before:

from pydantic_ai import Agent
from pydantic_ai.providers.anthropic import AnthropicModel, AnthropicProvider
from pydantic_ai.providers.openai import OpenAIChatModel, OpenAIProvider

def create_model_instance(...):
    if is_anthropic:
        return AnthropicModel(model, provider=AnthropicProvider(...))
    else:
        return OpenAIChatModel(model, provider=OpenAIProvider(...))

After:

from tunacode.core.llm.alchemy_client import get_model

def create_model_instance(...):
    return get_model(provider, model_name, api_key=api_key, base_url=base_url)

Phase 3: Update Streaming

File: src/tunacode/core/agents/agent_components/streaming.py

Before:

async with node.stream(agent_run_ctx) as request_stream:
    async for event in request_stream:
        if isinstance(event, PartDeltaEvent):
            delta_text = event.delta.content_delta

After:

from tunacode.core.llm.streaming import stream_completion

async for chunk in stream_completion(model, context):
    delta_text = chunk.content

Phase 4: Update Message Adapter

File: src/tunacode/utils/messaging/adapter.py

Add conversion functions:

def alchemy_to_canonical(msg: AlchemyMessage) -> CanonicalMessage:
    ...

def canonical_to_alchemy(msg: CanonicalMessage) -> AlchemyMessage:
    ...

Phase 5: Replace ModelRetry

Before (8 files):

from pydantic_ai.exceptions import ModelRetry
raise ModelRetry("error message")

After:

from tunacode.exceptions import ToolRetryError
raise ToolRetryError("error message")

Files Affected by Migration

MUST CHANGE (11 files)

MIGHT CHANGE (Adapter Layer)

NO CHANGES NEEDED (16+ files)

Estimated Complexity

Total: ~1 week of focused work

Key Patterns That Help

1. Canonical Message Layer - Already isolates format changes. Just add alchemy converters.

2. Protocol-Based Types - StateManagerProtocol means state code doesn't import concrete LLM types.

3. Wrapper Pattern - types/pydantic_ai.py provides single import point. Replace with alchemy types.

4. Callback Architecture - Streaming and tool callbacks decouple UI from agent internals.

Risks & Mitigations

Recommendation

Yes, this migration is achievable and relatively easy because:

1. Clear boundaries - LLM API is isolated in agent_config.py and streaming.py
2. Adapter pattern exists - utils/messaging/adapter.py already abstracts message formats
3. State is decoupled - StateManager has no LLM dependencies beyond type annotations
4. Agent loop is Python - Only replace the agent.iter() call, keep the orchestration

Suggested approach:

1. Build alchemy-rs Python bindings first (PyO3)
2. Run both pydantic-ai and alchemy-rs in parallel during transition
3. Switch one model at a time (start with Anthropic)
4. Verify message round-trips through canonical layer
5. Remove pydantic-ai after all providers migrated

References

• alchemy-rs repo: https://github.com/tunahorse/alchemy-rs
• pydantic-ai usage: /home/tuna/tunacode/src/tunacode/types/pydantic_ai.py
• Message adapter: /home/tuna/tunacode/src/tunacode/utils/messaging/adapter.py
• Agent config: /home/tuna/tunacode/src/tunacode/core/agents/agent_components/agent_config.py
• Streaming: /home/tuna/tunacode/src/tunacode/core/agents/agent_components/streaming.py
• State manager: /home/tuna/tunacode/src/tunacode/core/state.py