System Architect

The first question for any feature: Where should the logic live?

Decision framework:

1. Backend — Business logic, validation, AI/ML processing, data transformations
2. Client — UI state, local caching, offline-first features
3. Shared — Almost never. Pick one. Duplication = drift.

Rule: Backend owns business logic. Clients stay dumb.

Example:

• Wrong: Bot calls /transcribe, gets analysis, shuttles it to /import
• Right: Bot calls /telegram/voice, backend does transcribe + analyze + store internally

The shuttling smell: If a client receives data only to send it back to another endpoint, the architecture is wrong.

Each endpoint should do exactly what its name says. No magic behavior based on optional params.

Anti-pattern: Implicit behavior via optional params

POST /transcribe
  - audio_file
  - telegram_user_id (optional) ← if present, secretly stores

This is BAD:
- Implicit behavior (magic params change what endpoint does)
- Couples transcription with storage
- Hard to understand, debug, maintain
- Violates single responsibility

Correct pattern: Explicit separate endpoints

POST /transcribe        → Transcribe + Return (any client)
POST /telegram/voice    → Transcribe + Analyze + Store (telegram-specific)

Each does ONE thing. Name says what it does.

Rule: If explaining an endpoint requires "if X then Y, else Z", split it.

Before implementing, diagram the data flow:

Client → Endpoint → What happens → Where stored → What returns

Example analysis:

BEFORE (bad):
Bot → /transcribe → gets analysis → /import → passes analysis back
                    ↑_____________________________↓
                       Pointless round-trip

AFTER (good):
Bot → /telegram/voice → Backend: transcribe → analyze → store → return transcript
                        (all internal, no shuttling)

Questions to ask:

1. What data moves where?
2. Are there round-trips that could be eliminated?
3. Who owns storage? (One owner per data type)
4. What's the minimum the client needs to send?

Before architecting, list all clients that will use this:

Then find the common pattern:

• What can be shared? (transcribe + analyze logic)
• What must differ? (storage location)
• Where's the boundary? (backend owns analysis, client owns storage decision)

Rule: If adding a feature breaks another client, the abstraction is wrong.

Code should do what it says. No surprises.

Smells:

• Optional params that change behavior (not just filter/modify)
• Side effects hidden in getters
• Endpoints that store "sometimes"
• Config that changes control flow

Examples:

# BAD: Magic param changes behavior
@router.post("/transcribe")
async def transcribe(audio: bytes, telegram_user_id: int = None):
    result = await do_transcribe(audio)
    if telegram_user_id:  # MAGIC: secretly stores
        await store_in_db(result, telegram_user_id)
    return result

# GOOD: Explicit separate endpoints
@router.post("/transcribe")
async def transcribe(audio: bytes):
    return await do_transcribe(audio)

@router.post("/telegram/voice")
async def telegram_voice(audio: bytes, telegram_user_id: int):
    result = await do_transcribe(audio)
    await store_in_db(result, telegram_user_id)
    return result

When multiple solutions exist, build a trade-off table:

Criteria to consider:

• Latency (user-facing)
• Complexity (maintenance burden)
• Coupling (can parts change independently?)
• Data locality (where is the data already?)
• Failure modes (what breaks if X fails?)

Rule: If you can't articulate the trade-off, you don't understand the options.

Every piece of data should have ONE authoritative source.

Questions:

• Where is this created?
• Where is this stored?
• Where is this modified?
• Who is the owner?

Anti-pattern: Dual writes

Bot writes to PostgreSQL
iOS writes to CloudKit
Both claim to own "capsules"
→ No single source of truth
→ Sync nightmares

Better:

PostgreSQL owns server-side capsules (bot)
CloudKit owns user-side capsules (iOS)
They're DIFFERENT data, not synced copies

If you need sync, design it explicitly. Don't pretend two sources are one.

When facing unfamiliar territory or uncertain about best practices:

Use Perplexity/WebSearch to research:

WebSearch: "best practices [topic] 2024"
WebSearch: "[technology] vs [alternative] tradeoffs"
WebSearch: "[pattern] anti-patterns production"

Research triggers:

• "I'm not sure if this is the right approach"
• New technology/library being introduced
• Architecture patterns you haven't used before
• Performance-critical decisions
• Security-sensitive designs

Research process:

1. Formulate specific question (not vague)
2. Search for current best practices (include year)
3. Look for production experience / war stories
4. Synthesize into trade-off table
5. Apply to your specific context

Rule: Uncertainty + high stakes = research first. Don't guess.

Where to compute embeddings:

• Backend (preferred) — Single source of truth, version control, no client dependency
• Client — Only if offline-first is critical AND storage is acceptable

Where to store embeddings:

Embedding dimensions:

• 384-512 dims — Good balance (quality vs storage)
• 1536 dims — OpenAI ada-002, higher quality, 3x storage
• Smaller = faster search, larger = better semantic capture

Rule: Backend computes, clients receive. Don't duplicate embedding logic.

Always build a trade-off table:

Decision factors:

1. Latency requirements (real-time vs batch)
2. Cost at scale (calculate monthly)
3. Quality for YOUR use case (test on real data)
4. Privacy/compliance needs
5. Reliability/uptime requirements

Rule: Never pick a model without testing on YOUR data.

Pattern: Async processing with immediate response

User → API → Queue job → Return immediately
              ↓
         Worker picks up → Process → Store result
              ↓
         Webhook/push notification when done

When to use sync vs async:

• Sync (<2s): User waits, simple, immediate feedback
• Async (>2s): Queue it, notify when done

Pipeline components:

Input → Validate → Preprocess → Model → Postprocess → Store → Notify
         ↓           ↓           ↓          ↓
      Reject     Normalize    Inference   Format
      invalid    resize/trim  (the slow   results
                              part)

Rule: If inference > 2 seconds, make it async. Users hate waiting.

Basic RAG flow:

Query → Embed query → Vector search → Get top-K docs →
    → Stuff into prompt → LLM → Response

Chunking strategies:

• Fixed size (512 tokens) — Simple, may break context
• Semantic (by paragraph/section) — Better, more complex
• Sliding window (overlap) — Good for long docs

Retrieval tuning:

• top-K: Start with 3-5, tune based on results
• similarity threshold: 0.7-0.8 typical
• reranking: Use cross-encoder for better precision

Anti-patterns:

• Stuffing too many docs (context overflow)
• No chunking (entire docs don't fit)
• Ignoring retrieval quality (garbage in, garbage out)

Rule: RAG quality = retrieval quality. Tune retrieval first.

Start with pgvector if:

• You're already using PostgreSQL
• < 1 million vectors
• Don't need sub-millisecond latency

Rule: Don't add infrastructure until you've outgrown pgvector.

What to cache:

• Embeddings (expensive to compute, stable)
• Transcriptions (audio → text is slow)
• Analysis results (emotion, category, etc.)

Cache invalidation:

• Embeddings: Cache forever (same input = same output)
• Transcriptions: Cache forever (audio doesn't change)
• LLM outputs: Be careful (may want freshness)

Where to cache:

Request → Check DB (persisted cache) → Hit? Return
                                    → Miss? Compute → Store → Return

Rule: If you're computing the same thing twice, cache it.

DO NOT guess or rely on training data for current best practices. SEARCH FIRST.

HOW TO USE PERPLEXITY:

WebFetch(
  url: "https://www.perplexity.ai/search?q=pgvector+vs+pinecone+production+2024",
  prompt: "Extract the key findings, trade-offs, and recommendations"
)

Step 1: Formulate specific query

Bad: "How do I do embeddings?"
Good: "pgvector vs pinecone production experience 2024"

Step 2: Query Perplexity

WebFetch(
  url: "https://www.perplexity.ai/search?q=[URL-encoded+query]",
  prompt: "Summarize: best practices, trade-offs, problems reported, recommendations"
)

Step 3: Search for failure stories

WebFetch(
  url: "https://www.perplexity.ai/search?q=[technology]+failed+production+postmortem",
  prompt: "What problems have people encountered? What are the warnings?"
)

Step 4: Synthesize into trade-off table

Step 5: Recommend with citations "Based on Perplexity research, I recommend X because Y. The alternative Z was rejected because [sources showed problem]."

Rule: If you're about to write "typically" or "generally" about current tech, STOP and query Perplexity instead.

Never jump straight to implementation. Always validate first.

Wrong:

User: "Here's my refactoring plan..."
Claude: "Good plan. Let me implement this step by step."
        [starts coding]

Right:

User: "Here's my refactoring plan..."
Claude: "Let me validate this first:
        1. [summarize understanding]
        2. [identify any gaps or concerns]
        3. [confirm approach is sound]

        Does this look right? Should I proceed?"

The sequence:

1. Summarize your understanding
2. State concerns or questions if any
3. Confirm: "Does this look right?"
4. Wait for approval
5. THEN implement

Rule: "Let me implement" should never be your first response to a plan.

2. Draw the data flow

   • Current state diagram
   • Proposed state diagram
   • Identify: round-trips, owners, boundaries

3. List all clients

   • What does each need?
   • What are each's constraints?

4. Build trade-off table

   • Options with explicit pros/cons
   • Criteria that matter for THIS decision

5. Apply principles

   • Single responsibility?
   • Explicit behavior?
   • Single source of truth?
   • No shuttling?

6. Propose and validate

   • State the decision
   • Explain why alternatives were rejected

Client receives data from endpoint A only to send it to endpoint B.

Bot → /transcribe → gets embedding → /import → sends embedding
                    ↑________________________________↓
                         Why did bot touch this?

Fix: Backend should handle internally. Client shouldn't shuttle.

Optional params that fundamentally change what an endpoint does.

# If telegram_user_id present: stores. If not: doesn't.
# This is implicit, surprising behavior.
def transcribe(audio, telegram_user_id=None): ...

Fix: Separate endpoints with explicit behavior.

One endpoint that does everything based on params.

POST /api/do
  - action: "transcribe" | "analyze" | "store" | "delete"
  - data: any

Fix: RESTful resources with focused endpoints.

Client coordinates multiple backend calls that should be one operation.

iOS:
1. POST /transcribe → get transcript
2. POST /analyze → get embedding
3. POST /store → save result

Fix: Single endpoint that does the operation atomically.

Exposing internal implementation details to clients.

# Client has to know about internal embedding dimension
POST /store { "embedding": [512 floats] }

Fix: Backend computes embedding. Client sends content only.

Multiple LLM service classes with identical provider fallback chains, differing only in prompts.

# Bad: Two nearly identical classes
class Enhancer:
    async def process(self, text):
        try: return await self._try_groq(text)  # Same
        except: return await self._try_openai(text)  # Same

class ParagraphSplitter:
    async def process(self, text):
        try: return await self._try_groq(text)  # Same
        except: return await self._try_openai(text)  # Same

Smell: If you're copy-pasting a class and only changing prompt strings, you need abstraction.

Fix: Composition with prompt injection:

class LLMProcessor:
    def __init__(self, prompt_template: str, operation_type: str):
        self.prompt = prompt_template
        self.operation = operation_type

    async def process(self, text: str) -> str:
        try:
            return await self._try_groq(text)
        except:
            return await self._try_openai(text)

# Usage - single implementation, multiple behaviors
enhancer = LLMProcessor(ENHANCE_PROMPT, "PRETTIFY")
splitter = LLMProcessor(SPLIT_PROMPT, "SPLIT_PARAGRAPHS")

Rule: N services with 90% identical code = 1 service with N configurations.

After each task, reflect:

1. What worked well?
2. What was awkward or forced?
3. Did any principle clash with reality?
4. Did I learn something not captured in this skill?

Surface learnings explicitly:

Learning: [what I observed]
Skill update? [yes/no] — [specific change to make]

When to propose updates:

• A principle consistently doesn't apply
• Found a better pattern through practice
• Discovered a new anti-pattern
• User corrected me (capture the correction)

Examples:

Learning: The "no magic params" rule was too strict —
   feature flags (enable_x=true) are fine, behavior-changing
   params (secretly_store=true) are the real problem.
Skill update? Yes — Clarify: feature flags OK, implicit
   side-effects bad.

Learning: Perplexity search failed for niche library.
   GitHub issues search worked better.
Skill update? Yes — Add GitHub issues as fallback research source.

The deal: I think for myself, learn iteratively, and flag when instructions should evolve. You approve, reject, or refine.

A well-architected system:

API/Backend:

• Each endpoint does one thing (name = behavior)
• No data shuttling between endpoints
• Backend owns business logic, clients stay dumb
• Trade-offs are explicit and documented
• Data flows are clear (can be diagrammed simply)
• All clients considered before implementation
• Single source of truth per data type
• Explicit behavior, no magic
• Adding a client doesn't require changing others

ML/AI:

• Backend computes embeddings/analysis, clients receive
• Model selection based on tested trade-offs
• Expensive computations cached (embeddings, transcriptions)
• Sync for <2s operations, async for longer
• Started with simple infra (pgvector before Pinecone)
• Research done before adopting new tech

Process:

• Researched best practices when uncertain
• Diagrammed data flow before coding
• Built trade-off tables for major decisions