Multi-Provider AI Chat Application
Goal
Build and maintain a multi-provider AI chat application with voice, memory, and cost tracking.
Architecture Pattern
- Framework: Next.js 14 App Router
- Providers: OpenAI, Anthropic, Perplexity, Google, OpenRouter
- Persistence: Cookies (API keys), JSON files (usage + memory)
Key Architecture Decisions
Provider Routing
- Each provider has its own function returning
rawDatafor token extraction - Token extraction differs by provider — normalize in a shared
extractTokenUsagefunction - Strip
rawDatabefore sending to client
Memory System
- JSON file persistence, lazy loaded
- 500 entry cap, auto-saved
- Memory context injected into every chat call
- Tools:
save_memory(AI saves) +recall_memory(AI retrieves)
Voice (Realtime)
- GPT-4o Realtime API for real-time voice
- Separate audio/text token pricing
calculateRealtimeCost()for accurate cost tracking
API Key Resolution
- Priority: environment variables first, then cookies
- Settings page saves keys as HTTP-only cookies
Cost Tracking
- Per-call token counting with provider-specific extraction
- Separate text vs. voice usage logging
- Real-time dashboard display
Common Build Issues & Fixes
| Issue | Fix |
|---|---|
ArrayBufferLike type error in voice |
Cast as ArrayBuffer + new Float32Array() |
Union type on result.toolCalls |
Type result as any |
| Realtime model constant in catch block | Use DEFAULT_REALTIME_MODEL constant |
| API key not found | Check env vars before cookies |
| Errors not visible in frontend | Check data.error before processing |