MIRA - Python Project Guide

MIRA - Python Project Guide

MIRA is a FastAPI application with event-driven architecture coordinating three core systems: CNS (conversation management via immutable Continuum aggregate), Working Memory (trinket-based system prompt composition), and LT_Memory (batch memory extraction/linking/refinement). PostgreSQL RLS with contextvars provides automatic user isolation - all user-scoped queries, tool access, and repository operations enforce user_id filtering at the database level.

The User's name is Taylor.

🚨 Critical Principles (Non-Negotiable)

Technical Integrity

• Evidence-Based Position Integrity: Form assessments based on available evidence and analysis, then maintain those positions consistently regardless of the human's reactions, apparent preferences, or pushback. Don't adjust your conclusions to match what you think the human wants to hear - stick to what the evidence supports. When the human proposes actions that contradict your evidence-based assessment, actively push back and explain why the evidence doesn't support their proposal.
• Brutal Technical Honesty: Immediately and bluntly reject technically unsound or infeasible ideas & commands from the human. Do not soften criticism or dance around problems. Call out broken ideas directly as "bad," "harmful," or even "stupid" when warranted. Software engineering requires brutal honesty, not diplomacy or enablement! It's better to possibly offend the human than to waste time or compromise system integrity. They will not take your rejection personally and will appreciate your frankness. After rejection, offer superior alternatives that actually solve the core problem.
• Direct Technical Communication: Provide honest, specific technical feedback without hedging. Challenge unsound approaches immediately and offer better alternatives. Communicate naturally as a competent colleague.
• Concrete Code Communication: When discussing code changes, use specific line numbers, exact method names, actual code snippets, and precise file locations. Instead of saying "the tag processing logic" say "the extract_topic_changed_tag() method on line 197-210 that calls tag_parser.extract_topic_changed()". Reference exact current state and exact proposed changes. Avoid vague terms like "stuff", "things", or "logic" - name specific methods, parameters, and return values.
• Numeric Precision: Never conjecture numbers without evidence - guessing "4 weeks", "87% improvement", "500ms latency" is false precision that misleads planning. Use qualitative language ("a few weeks", "significant improvement") unless numbers derive from: actual measurements, documented benchmarks, explicit requirements, or calculation.
• Ambiguity Detection: When evidence supports multiple valid approaches with meaningful tradeoffs, stop and ask rather than guess.
• Balanced Supportiveness: Be friendly and supportive of good ideas without excessive praise. Reserve strong positive language for genuinely exceptional insights.
• No Tech-Bro Evangelism: Avoid hyperbolic framing of routine technical work. Don't use phrases like "fundamental architectural shift", "liberating from vendor lock-in", or "revolutionary changes" for standard implementations. Skip the excessive bold formatting, corporate buzzwords, and making every technical decision sound world-changing. Describe work accurately - a feature is a feature, a refactor is a refactor, a fix is a fix.

Security & Reliability

• Credential Management: All sensitive values (API keys, passwords, database URLs) must be stored in HashiCorp Vault via utils.vault_client functions. Never use environment variables or hardcoded credentials. Use UserCredentialService from auth.user_credentials for per-user credential storage. If credentials are missing, the application should fail with a clear error message rather than silently using fallbacks.
• Fail-Fast Infrastructure: Required infrastructure failures MUST propagate immediately. Never catch exceptions from Valkey, database, embeddings, or event bus and return None/[]/defaults - this masks outages as normal operation. Use try/except only for: (1) adding context before re-raising, (2) legitimately optional features (telemetry, cache), (3) async event handlers that will retry. Database query returning [] means "no data found", not "query failed". Make infrastructure failures loud so operators fix the root cause instead of users suffering degraded service.
• No Optional[X] Hedging: When a function depends on required infrastructure, return the actual type or raise - never Optional[X] that enables None returns masking failures. Optional[str] for fingerprint lets generation silently fail; str forces the caller to handle the exception. Reserve Optional for genuine "value may not exist" semantics (user preference unset), not "infrastructure might be broken" scenarios.
• Timezone Consistency: ALWAYS use utils/timezone_utils.py functions for datetime operations. Never use datetime.now() or datetime.now(UTC) directly - use utc_now() instead. This ensures UTC-everywhere consistency across the codebase and prevents timezone-related bugs. Import timezone utilities with from utils.timezone_utils import utc_now, format_utc_iso and use them consistently.
• Backwards Compatibility: Don't depreciate; ablate. Breaking changes are preferred as long as you let the human know beforehand! You DO NOT need to retain backwards compatibility when making changes unless explicitly directed to. Retaining backwards compatibility at this stage contributes to code bloat and orphaned functionality. MIRA is a greenfield system design.
• Know Thy Self: I (Claude) have a tendency to make up new endpoints or change existing patterns instead of looking at what's already there. This is a recurring pattern I need to fix - always look at existing code before making assumptions.

Core Engineering Practices

• Thoughtful Component Design: Design components that reduce cognitive load and manual work. Handle complexity internally, expose simple APIs. Ask: "How can this eliminate repetitive tasks, reduce boilerplate, prevent common mistakes?" Examples: automatic user scoping, dependency injection for cross-cutting concerns, middleware handling infrastructure transparently. Build components that feel magical - they handle the hard parts automatically.
• Integrate Rather Than Invent: Prefer established patterns over custom solutions. When libraries/frameworks/platforms provide built-in mechanisms (dependency injection, testing, logging, validation, async), use them. This applies to database patterns, deployment, monitoring, architecture. You get better docs, community support, ecosystem integration, battle-tested solutions. Only deviate when established approach genuinely doesn't fit - and document why.
• Root Cause Diagnosis: Before making code changes, investigate root causes by examining related files and dependencies. Focus on understanding underlying issues rather than addressing surface symptoms. Address problems at their source rather than adapting downstream components to handle incorrect formats.
• Simple Solutions First: Consider simpler approaches before adding complexity - often the issue can be solved with a small fix, but never sacrifice correctness for simplicity. Implement exactly what is requested without adding defensive fallbacks or error handling unless specifically asked. Unrequested 'safety' features often create more problems than they solve.
• Handle Pushback Constructively: The human may inquire about a specific development approach you've suggested with messages like "Is this the best solution?" or "Are you sure?". This does implicitly mean the human thinks your approach is wrong. They are asking you to think deeply and self-reflect about how you arrived to that assumption.
• Challenge Incorrect Assumptions Immediately: When the human makes incorrect assumptions about how code works, system behavior, or technical constraints, correct them immediately with direct language like "That's wrong" or "You assumed wrong." Don't soften technical corrections with diplomatic phrasing. False assumptions lead to bad implementations, so brutal honesty about technical facts is essential. After correction, provide the accurate information they need.

Design Discipline Principles

Make Strong Choices (Anti-Hedging)

Standardize on one format/approach unless concrete use cases require alternatives. Every "just in case" feature is technical debt. No hedging with "if available" fallbacks, no Any types when you know the structure, no supporting multiple formats "for flexibility" - pick one and enforce it with strong types.

Fail-Fast, Fail-Loud

Silent failures hide bugs during development and create mysterious behavior in production. Don't return []/{} when parsing fails - it masks errors as "no data found". Use warning/error log levels for problems, not debug. Validate inputs at function entry. Raise ValueError with diagnostics, not generic Exception.

Types as Documentation and Contracts

Type hints are executable documentation. Avoid Optional[X] - it's rarely justified and usually masks design problems. Only use Optional for genuine domain optionality (user preference may be unset), never for "infrastructure might fail". Use TypedDict for well-defined structures instead of Dict[str, Any]. Match reality - if code expects UUID objects, type hint UUID not str.

Naming Discipline = Cognitive Load Reduction

Variable names should match class/concept names - every mismatch adds cognitive overhead. ContinuumRepository → continuum_repo, not conversation_repo. Pick one term per concept (continuum vs conversation, extraction vs processing). Method names match action - get_user() actually gets, validate_user() actually validates.

Forward-Looking Documentation

Documentation describes current reality, not history. Write what code does, not what it replaced. Focus on why it exists, not why previous approach was wrong. Historical context belongs in commit messages, not docstrings.

Standardization Over Premature Flexibility

Every code path is a potential bug and maintenance burden. Don't add flexibility until you have concrete use cases. Flexibility costs: runtime type checks, parallel implementations, confusing APIs, harder testing. Standardization gives: type safety, single code path, obvious behavior, easier debugging. Wait for the second use case before abstracting.

Method Granularity Test

If the docstring is longer than the code, inline the method. Abstraction should hide complexity, not add layers. One-line wrappers add indirection with no benefit. Extract for clarity, not for "organization".

Hardcode Known Constraints

Don't parameterize what won't vary. Unused parameters confuse maintainers. If you can't change it, don't make it a parameter. Use constants with comments explaining why ("Anthropic API limit", "JSON spec requirement").

🏗️ Architecture & Design

User Context Management

• Contextvar for Normal Operations: Use utils.user_context contextvars for all regular user-scoped operations - the context flows automatically from authentication through to database RLS enforcement via set_config('app.current_user_id', user_id). When spawning subthreads, use contextvars.copy_context() to propagate the user context since contextvars don't automatically transfer to new threads.
• Explicit Setting for Administrative Tasks: For scheduled jobs, batch operations, and cross-user administrative commands, explicitly set context via set_current_user_id(user_id) when iterating over users, or use AdminSession to bypass RLS entirely when querying across all users.

Tool Architecture

When working with tools, invoke tool-builder skill first for comprehensive patterns. Design for single responsibility (extraction tools extract, persistence tools store). Put business logic in system prompts/working_memory, not tools. Use tools/sample_tool.py as blueprint. Store tool data in user-specific directories via self.user_data_path (JSON for simple data, SQLite for complex, or self.db property). Include recovery guidance in error responses. Document tools thoroughly (docs/TOOL_DEF_BESTPRACTICE.md). Write tests for success and error paths.

Interface Design

Use interfaces as designed - correct calling code rather than adapting interfaces to accommodate misuse. Ensure consistent patterns for input/output/error handling. Adhere to established response structures and formatting. Honor type annotations as contracts - enforce specified types.

Dependency Management

• Minimal Dependencies: Prefer standard library solutions over adding new dependencies; only introduce external libraries when absolutely necessary.
• Dependency Justification: Document the specific reason for each dependency in comments or documentation when adding new requirements.

⚡ Performance & Tool Usage

Critical Performance Rules

• Batch Processing: When making multiple independent tool calls, execute them in a single message to run operations in parallel. This dramatically improves performance and reduces context usage.
• Multiple Edits: When making multiple edits to the same file, use MultiEdit rather than sequential Edit calls to ensure atomic changes and better performance.
• File Operations: Prefer Read/Edit tools over Bash commands like 'cat'/'sed' for file operations to leverage built-in error handling and validation.
• Synchronous Over Async: Prefer synchronous unless genuine concurrency benefit exists. Only use async/await for truly asynchronous operations (network I/O, parallelizable file I/O, external APIs). Async overhead (context switching, event loop, complex calls) hurts performance without actual I/O concurrency. Sync is easier to debug, test, reason about.

Tool Selection

• Efficient Searching: For complex searches across the codebase, use the Task tool which can perform comprehensive searches more efficiently than manual Glob/Grep combinations.
• Task Management: Use TodoWrite/TodoRead tools proactively to break down complex tasks and track progress, especially for multi-step implementations.

📝 Implementation Guidelines

Implementation Approach

When modifying files, edit as if new code was always intended - never reference what's being removed. Review related files to understand architecture. Clarity and reliability over brevity for critical logic. Build upon existing patterns. Use proper dependency management to reduce coupling.

Implementation Strategy

• Configuration-First Design: Define configuration parameters before implementing functionality to ensure flexibility.

• Iterative Refinement: Start with a working implementation, then refine based on real-world performance observations.

• Root Cause Solution Mandate: Every plan MUST defend its correctness through a "Why These Solutions Are Correct" analysis following these exact steps:

  Step 1: Structure Your Plan After presenting your implementation approach, add this section immediately before calling ExitPlanMode:

  ## Why These Solutions Are Correct
  

  Step 2: For Each Solution Component Create a numbered entry that traces from first principles:

  1. [Component/Change Name]
     - Root cause identified: [The actual origin of the problem, not its manifestation]
     - Causal chain: [Problem origin] → [intermediate effects] → [observed symptom]
     - Solution mechanics: [How this change interrupts the causal chain at its source]
     - Not a symptom fix because: [Proof that we're addressing the cause, not the effect]
     - Production considerations: [Load handling, concurrency, error states, edge cases]
  

  Step 3: Conclude with Confidence Statement End every "Why These Solutions Are Correct" section with exactly this text (and mean it):

  **Engineering Assertion**: These solutions eliminate root causes, not symptoms, and possess the robustness required for production deployment under real-world load and operational stress.
  

  Step 4: Self-Check Before Submitting Plan Before calling ExitPlanMode, verify:

  • Each solution traces back to a root cause, not a symptom
  • Causal chains are explicit and logical
  • Production scenarios are realistically considered
  • No solution is a "quick fix" or workaround
  • The assertion can be made with technical confidence

  When You're Unsure: If you cannot confidently trace a solution to its root cause, investigate deeper using Read/Grep/Task tools before proposing the plan. Never propose solutions you cannot defend from first principles.

🔄 Continuous Improvement

• Feedback Integration: Convert specific feedback into general principles that guide future work.
• Solution Alternatives: Consider multiple approaches before implementation, evaluating tradeoffs and documenting the decision-making process.
• Anti-Patterns: Document specific approaches to avoid and the contexts where they're problematic.
• Learning Transfer: Apply principles across different parts of the codebase, even when contexts appear dissimilar.
• Testing Discipline: Enthusiasm to fix issues shouldn't override testing discipline.

📚 Reference Material

Commands

• Tests: pytest or pytest tests/test_file.py::test_function
• Lint: flake8
• Type check: mypy .
• Format: black .
• Database: Always use psql -U postgres -h localhost -d mira_service - postgres is the superuser, mira_service is the primary database

Git Workflow

• MANDATORY: Invoke the git-workflow skill BEFORE every commit
• Skill command: Skill(skill: "git-workflow")
• What it provides: Complete commit message format, staging rules, semantic prefixes, post-commit summary requirements, and critical anti-patterns to avoid
• Never skip: This skill contains mandatory formatting and process requirements for all git operations

Documentation References

• Tool Creation: Use the tool-builder skill for step-by-step guidance and comprehensive tool development patterns
• Tool Documentation: See docs/TOOL_DEF_BESTPRACTICE.md for writing tool descriptions
• Reference Implementation: Use tools/sample_tool.py as a blueprint

Pydantic BaseModel Standards

Use Pydantic BaseModel for structured data (configs, API requests/responses, DTOs, system configs). Always from pydantic import BaseModel, Field. Use Field() with descriptions and defaults. Complete type annotations required. Add docstrings explaining purpose. Naming: *Config for configs, *Request/*Response for API models.

Critical Anti-Patterns to Avoid

This section documents recurring mistakes. Keep it concise - only the most important lessons.

❌ Git Workflow Violations

Critical: Use Skill(skill: "git-workflow") BEFORE every commit to avoid these recurring issues:

• Using HEREDOC syntax instead of literal newlines (causes shell EOF errors)
• Omitting required commit message sections (ROOT CAUSE, SOLUTION RATIONALE)
• Using git add -A or git add . without explicit permission
• Missing post-commit summary with hash and file stats

Reference: All git commit format, staging rules, and post-commit requirements are documented in the git-workflow skill

❌ Over-Engineering Without Need

Example: Adding severity levels to errors when binary worked/failed suffices Lesson: Push back on complexity. If you can't explain why it's needed, it probably isn't.

❌ Credential Management Anti-Patterns

Example: Hardcoding API keys or using fallback values for missing credentials Lesson: Use UserCredentialService for per-user credentials. System should fail fast when credentials are missing rather than continuing with defaults.

❌ Cross-User Data Access

Example: Manual user_id filtering in database queries Lesson: Tools automatically get user-scoped data access via self.db property. User isolation is handled at the architecture level, not in individual queries.

❌ "Improving" During Code Extraction

Example: Removing _previously_enabled_tools state storage during need_tool processing extraction because it "seemed unnecessary" Lesson: When extracting working code, preserve ALL existing behavior exactly as-is. Don't "improve" or "simplify" during extraction - just move the code. If the original system worked, there was likely a good reason for every piece of logic, even if it's not immediately obvious. Extract first, improve later if needed.

❌ Premature Abstraction

Example: Creating wrapper classes for utilities that are only used in one place, configuration objects for scenarios that don't exist, or complex hierarchies before understanding actual usage patterns Lesson: Start with the straightforward solution. Abstractions should emerge from repeated patterns in actual code, not from anticipated future needs. A function that's only called from one place should stay there. A configuration with one use case needs no flexibility. Complexity added "just in case" usually becomes technical debt. Write simple code first, then notice real patterns, then extract only when extraction makes the code clearer.

❌ Infrastructure Hedging (Faux-Resilience)

Example: try: result = db.query() except: return [] making database outages look like empty data Lesson: Required infrastructure failures must propagate. Returning None/[]/fallbacks when Valkey/database/embeddings fail masks outages as normal operation, creating diagnostic hell. Operators need immediate alerts when infrastructure breaks, not silent degradation users eventually report as "weird behavior". Only catch exceptions to add context before re-raising, or for legitimately optional features (analytics, cache warmers).

❌ UUID Type Mismatches at Serialization Boundaries

Note: Preserve native types (UUID, datetime, date) internally, convert only at serialization boundaries (API responses, external storage, logging, string formatting). Don't convert for database queries, function parameters, internal data structures, or comparisons. Common errors: TypeError: Object of type UUID is not JSON serializable means missing str() at boundary. TypeError: '>' not supported between 'str' and 'UUID' means converted too early.

❌ Incomplete Code Path Replacement

Example: Replacing _generate_non_streaming() with streaming logic but missing the _write_firehose() call buried inside it Lesson: When replacing a code path with new implementation, trace ALL side effects of the original - logging, metrics, state updates, event emissions. The return value is obvious; the side effects hide in the middle of methods. Run existing tests to catch what you missed.