Agent Memory Skills

Agent Memory & Continuous Self-Improvement Skills

Purpose: Enable agents to learn from experience, evaluate themselves continuously, and store improvements in ChromaDB collections separate from static configuration files.

Philosophy: Static configuration (.md) for capabilities; dynamic memory (ChromaDB) for learned experiences.

Architecture Overview

The Dual-Layer Model

┌──────────────────────────────────────────────────────────┐
│ Layer 1: Static Agent Configuration (.md files)          │
│ - Core capabilities, workflows, prompt templates         │
│ - Human-designed, expert-reviewed                        │
│ - Version controlled (Git)                               │
│ - Changes: Infrequent (weeks/months)                     │
└──────────────────────────────────────────────────────────┘
                          ↓ loads at runtime
┌──────────────────────────────────────────────────────────┐
│ Layer 2: Dynamic Agent Memory (ChromaDB)                 │
│ - Learned improvements, preferences, patterns            │
│ - Self-evaluation results, performance metrics           │
│ - Agent-managed, auto-updated                            │
│ - Changes: Continuous (every task completion)            │
└──────────────────────────────────────────────────────────┘

Memory Collection Structure

Each agent gets 3 ChromaDB collections:

1. agent_{name}_improvements: Learned patterns and strategies
2. agent_{name}_evaluations: Self-assessment results
3. agent_{name}_performance: Metrics over time

Collection 1: Improvements Storage

When to Store Improvements

Store when:

• ✅ Task completed successfully with clear learning
• ✅ User feedback received (positive or negative)
• ✅ New pattern discovered (not in static config)
• ✅ Confidence score ≥ 0.7 (high confidence learning)

Don't store:

• ❌ Failed tasks without clear lessons
• ❌ Routine operations following existing patterns
• ❌ Low confidence observations (< 0.5)

Improvement Schema

const improvement = {
  // Document (semantic search target)
  document: `
    When user asks for "latest" information, prioritize sources from 2025
    over 2024. Use date filters: where: { year: { "$gte": 2025 } }
  `,

  // ID (unique, timestamp-based)
  id: `improvement_research_${Date.now()}`,

  // Metadata (for filtering)
  metadata: {
    agent_name: "research-specialist",
    category: "search_strategy",
    learned_from: "feedback_2025-11-18",
    confidence: 0.85,
    success_rate: null,  // Will be updated with usage
    usage_count: 0,
    created_at: "2025-11-18T15:30:00Z",
    last_used: null,
    tags: ["recency", "date_filtering", "search"]
  }
};

Storage Function

async function storeImprovement(agentName, improvement) {
  const collectionName = `agent_${agentName}_improvements`;

  // Ensure collection exists
  const collections = await mcp__chroma__list_collections();
  if (!collections.includes(collectionName)) {
    await mcp__chroma__create_collection({
      collection_name: collectionName,
      embedding_function_name: "default",
      metadata: {
        agent: agentName,
        purpose: "learned_improvements",
        created_at: new Date().toISOString()
      }
    });
  }

  // Store improvement
  await mcp__chroma__add_documents({
    collection_name: collectionName,
    documents: [improvement.document],
    ids: [improvement.id],
    metadatas: [improvement.metadata]
  });

  console.log(`✅ Stored improvement for ${agentName}: ${improvement.category}`);
}

Retrieval Before Task

async function retrieveRelevantImprovements(agentName, taskDescription, limit = 5) {
  const collectionName = `agent_${agentName}_improvements`;

  try {
    const results = await mcp__chroma__query_documents({
      collection_name: collectionName,
      query_texts: [taskDescription],
      n_results: limit,
      where: {
        "$and": [
          { "confidence": { "$gte": 0.7 } },  // High confidence only
          { "success_rate": { "$gte": 0.6 } }  // Or null (new, untested)
        ]
      },
      include: ["documents", "metadatas", "distances"]
    });

    // Filter by relevance (distance < 0.4)
    const relevant = results.ids[0]
      .map((id, idx) => ({
        id: id,
        improvement: results.documents[0][idx],
        metadata: results.metadatas[0][idx],
        relevance: 1 - results.distances[0][idx]
      }))
      .filter(item => item.relevance > 0.6);

    return relevant;
  } catch (error) {
    console.log(`No improvements found for ${agentName} (collection may not exist yet)`);
    return [];
  }
}

Collection 2: Self-Evaluation Storage

Continuous Self-Evaluation Pattern

After every task, agents run self-evaluation:

async function selfEvaluate(agentName, taskContext, taskResult) {
  const evaluation = {
    // What was the task?
    task_description: taskContext.description,
    task_type: taskContext.type,  // "research", "code", "debug", etc.

    // How did I perform?
    success: taskResult.success,  // true/false
    quality_score: taskResult.quality,  // 0-100
    time_taken_ms: taskResult.duration,
    tokens_used: taskResult.tokens,

    // What went well?
    strengths: identifyStrengths(taskResult),
    // What went poorly?
    weaknesses: identifyWeaknesses(taskResult),

    // What did I learn?
    insights: extractInsights(taskResult),

    // Metadata
    timestamp: new Date().toISOString(),
    context: taskContext.additionalContext
  };

  // Store evaluation
  await storeEvaluation(agentName, evaluation);

  // If strong learning → store as improvement
  if (evaluation.insights.length > 0 && evaluation.quality_score >= 70) {
    for (const insight of evaluation.insights) {
      await storeImprovement(agentName, {
        document: insight.description,
        id: `improvement_${agentName}_${Date.now()}`,
        metadata: {
          agent_name: agentName,
          category: insight.category,
          learned_from: `task_${evaluation.timestamp}`,
          confidence: insight.confidence,
          created_at: evaluation.timestamp
        }
      });
    }
  }

  return evaluation;
}

function identifyStrengths(taskResult) {
  const strengths = [];

  if (taskResult.time_taken_ms < taskResult.expected_duration) {
    strengths.push("Completed faster than expected");
  }

  if (taskResult.validation_passed) {
    strengths.push("All validations passed");
  }

  if (taskResult.user_feedback?.positive) {
    strengths.push(taskResult.user_feedback.comment);
  }

  return strengths;
}

function identifyWeaknesses(taskResult) {
  const weaknesses = [];

  if (taskResult.errors.length > 0) {
    weaknesses.push(`Encountered ${taskResult.errors.length} errors`);
  }

  if (taskResult.retries > 0) {
    weaknesses.push(`Required ${taskResult.retries} retries`);
  }

  if (taskResult.user_feedback?.negative) {
    weaknesses.push(taskResult.user_feedback.comment);
  }

  return weaknesses;
}

function extractInsights(taskResult) {
  const insights = [];

  // Example: Learned a new error handling pattern
  if (taskResult.errors.some(e => e.type === "ConnectionTimeout") && taskResult.success) {
    insights.push({
      description: "When encountering ConnectionTimeout, retry with exponential backoff (3 attempts)",
      category: "error_handling",
      confidence: 0.8
    });
  }

  // Example: Discovered optimal parameter
  if (taskResult.optimization_found) {
    insights.push({
      description: taskResult.optimization_found.description,
      category: "optimization",
      confidence: 0.9
    });
  }

  return insights;
}

Evaluation Storage

async function storeEvaluation(agentName, evaluation) {
  const collectionName = `agent_${agentName}_evaluations`;

  await mcp__chroma__add_documents({
    collection_name: collectionName,
    documents: [
      `Task: ${evaluation.task_description}. Result: ${evaluation.success ? 'Success' : 'Failure'}. ` +
      `Quality: ${evaluation.quality_score}/100. Insights: ${evaluation.insights.map(i => i.description).join('; ')}`
    ],
    ids: [`eval_${Date.now()}`],
    metadatas: [{
      agent_name: agentName,
      task_type: evaluation.task_type,
      success: evaluation.success,
      quality_score: evaluation.quality_score,
      time_taken_ms: evaluation.time_taken_ms,
      tokens_used: evaluation.tokens_used,
      strengths_count: evaluation.strengths.length,
      weaknesses_count: evaluation.weaknesses.length,
      insights_count: evaluation.insights.length,
      timestamp: evaluation.timestamp
    }]
  });
}

Collection 3: Performance Metrics

Aggregate Performance Tracking

async function trackPerformanceMetrics(agentName) {
  const collectionName = `agent_${agentName}_evaluations`;

  // Retrieve last 100 evaluations
  const recentEvals = await mcp__chroma__get_documents({
    collection_name: collectionName,
    limit: 100,
    include: ["metadatas"],
    where: {
      "timestamp": { "$gte": thirtyDaysAgo() }
    }
  });

  const metrics = {
    agent_name: agentName,
    period: "30_days",

    // Success metrics
    total_tasks: recentEvals.metadatas.length,
    successful_tasks: recentEvals.metadatas.filter(m => m.success).length,
    success_rate: 0,

    // Quality metrics
    avg_quality: average(recentEvals.metadatas.map(m => m.quality_score)),
    quality_trend: calculateTrend(recentEvals.metadatas, 'quality_score'),

    // Efficiency metrics
    avg_time_ms: average(recentEvals.metadatas.map(m => m.time_taken_ms)),
    avg_tokens: average(recentEvals.metadatas.map(m => m.tokens_used)),

    // Learning metrics
    total_insights: sum(recentEvals.metadatas.map(m => m.insights_count)),
    improvements_stored: await countImprovements(agentName, thirtyDaysAgo()),

    // Trend
    improving: false,  // Will calculate
    timestamp: new Date().toISOString()
  };

  metrics.success_rate = metrics.successful_tasks / metrics.total_tasks;

  // Is agent improving over time?
  metrics.improving = metrics.quality_trend > 0 && metrics.success_rate > 0.7;

  // Store aggregated metrics
  const perfCollection = `agent_${agentName}_performance`;
  await mcp__chroma__add_documents({
    collection_name: perfCollection,
    documents: [
      `Performance snapshot: ${metrics.success_rate * 100}% success, ` +
      `quality ${metrics.avg_quality}/100, ${metrics.improvements_stored} improvements`
    ],
    ids: [`perf_${Date.now()}`],
    metadatas: [metrics]
  });

  return metrics;
}

function calculateTrend(evaluations, metric) {
  if (evaluations.length < 2) return 0;

  const recent = evaluations.slice(0, Math.floor(evaluations.length / 2));
  const older = evaluations.slice(Math.floor(evaluations.length / 2));

  const recentAvg = average(recent.map(e => e[metric]));
  const olderAvg = average(older.map(e => e[metric]));

  return recentAvg - olderAvg;  // Positive = improving
}

Complete Workflow: Agent with Memory

At Agent Initialization

async function initializeAgentMemory(agentName) {
  // Load static configuration from .md file
  const staticConfig = await loadAgentConfig(`/agents/${agentName}.md`);

  // Load dynamic improvements from ChromaDB
  const improvements = await retrieveAllImprovements(agentName);

  // Combine static + dynamic
  return {
    ...staticConfig,
    learned_improvements: improvements,
    memory_enabled: true
  };
}

Before Task Execution

async function executeTaskWithMemory(agentName, task) {
  // Step 1: Retrieve relevant past learnings
  const relevantImprovements = await retrieveRelevantImprovements(
    agentName,
    task.description,
    5  // top 5 improvements
  );

  console.log(`📚 Retrieved ${relevantImprovements.length} relevant improvements`);

  // Step 2: Execute task with improvements as context
  const taskResult = await executeTask(task, {
    improvements: relevantImprovements.map(i => i.improvement)
  });

  // Step 3: Self-evaluate
  const evaluation = await selfEvaluate(agentName, task, taskResult);

  // Step 4: Update improvement usage stats
  for (const improvement of relevantImprovements) {
    await updateImprovementUsage(agentName, improvement.id, taskResult.success);
  }

  // Step 5: Track performance (daily)
  if (shouldRunDailyMetrics()) {
    await trackPerformanceMetrics(agentName);
  }

  return taskResult;
}

Update Improvement Statistics

async function updateImprovementUsage(agentName, improvementId, wasSuccessful) {
  const collectionName = `agent_${agentName}_improvements`;

  // Get current improvement
  const current = await mcp__chroma__get_documents({
    collection_name: collectionName,
    ids: [improvementId]
  });

  const metadata = current.metadatas[0];

  // Update statistics
  const newUsageCount = (metadata.usage_count || 0) + 1;
  const successCount = (metadata.success_count || 0) + (wasSuccessful ? 1 : 0);
  const newSuccessRate = successCount / newUsageCount;

  // Update metadata
  await mcp__chroma__update_documents({
    collection_name: collectionName,
    ids: [improvementId],
    metadatas: [{
      ...metadata,
      usage_count: newUsageCount,
      success_count: successCount,
      success_rate: newSuccessRate,
      last_used: new Date().toISOString()
    }]
  });

  // If improvement consistently fails (< 40% success after 10+ uses), mark as deprecated
  if (newUsageCount >= 10 && newSuccessRate < 0.4) {
    console.log(`⚠️ Improvement ${improvementId} has low success rate (${newSuccessRate}), marking deprecated`);

    await mcp__chroma__update_documents({
      collection_name: collectionName,
      ids: [improvementId],
      metadatas: [{
        ...metadata,
        deprecated: true,
        deprecated_reason: "Low success rate after extensive usage"
      }]
    });
  }
}

Continuous Evaluation Schedule

Evaluation Frequency

• After every task: Store evaluation (lightweight)
• Daily: Calculate performance metrics (aggregation)
• Weekly: Review top/bottom improvements, identify trends
• Monthly: Archive old evaluations, clean up deprecated improvements

Scheduled Maintenance

async function dailyAgentMaintenance(agentName) {
  console.log(`🔧 Running daily maintenance for ${agentName}...`);

  // 1. Track performance metrics
  const metrics = await trackPerformanceMetrics(agentName);

  // 2. Identify underperforming improvements
  const deprecatedCount = await deprecateFailingImprovements(agentName);

  // 3. Promote high-performing improvements (boost confidence)
  const promotedCount = await promoteSuccessfulImprovements(agentName);

  console.log(`
    Metrics: ${metrics.success_rate * 100}% success, quality ${metrics.avg_quality}/100
    Deprecated: ${deprecatedCount} low-performing improvements
    Promoted: ${promotedCount} high-performing improvements
  `);
}

async function deprecateFailingImprovements(agentName) {
  const collectionName = `agent_${agentName}_improvements`;

  const allImprovements = await mcp__chroma__get_documents({
    collection_name: collectionName,
    where: {
      "$and": [
        { "usage_count": { "$gte": 10 } },
        { "success_rate": { "$lt": 0.4 } },
        { "deprecated": { "$ne": true } }
      ]
    }
  });

  for (const id of allImprovements.ids) {
    await mcp__chroma__update_documents({
      collection_name: collectionName,
      ids: [id],
      metadatas: [{
        ...allImprovements.metadatas[id],
        deprecated: true
      }]
    });
  }

  return allImprovements.ids.length;
}

async function promoteSuccessfulImprovements(agentName) {
  const collectionName = `agent_${agentName}_improvements`;

  const highPerformers = await mcp__chroma__get_documents({
    collection_name: collectionName,
    where: {
      "$and": [
        { "usage_count": { "$gte": 20 } },
        { "success_rate": { "$gte": 0.85 } }
      ]
    }
  });

  for (const [idx, id] of highPerformers.ids.entries()) {
    const metadata = highPerformers.metadatas[idx];

    await mcp__chroma__update_documents({
      collection_name: collectionName,
      ids: [id],
      metadatas: [{
        ...metadata,
        confidence: Math.min(0.95, metadata.confidence + 0.05),  // Boost confidence (max 0.95)
        promoted: true
      }]
    });
  }

  return highPerformers.ids.length;
}

Agent Memory Dashboard

Performance Summary

async function getAgentMemoryDashboard(agentName) {
  const dashboard = {
    agent: agentName,
    timestamp: new Date().toISOString(),

    // Improvements
    total_improvements: await countDocuments(`agent_${agentName}_improvements`),
    active_improvements: await countDocuments(`agent_${agentName}_improvements`, {
      "deprecated": { "$ne": true }
    }),
    deprecated_improvements: await countDocuments(`agent_${agentName}_improvements`, {
      "deprecated": true
    }),

    // Evaluations
    total_evaluations: await countDocuments(`agent_${agentName}_evaluations`),
    recent_success_rate: await calculateRecentSuccessRate(agentName, 30),  // Last 30 days

    // Performance
    latest_metrics: await getLatestMetrics(agentName),

    // Top improvements (by success rate)
    top_improvements: await getTopImprovements(agentName, 5),

    // Learning rate
    improvements_last_7_days: await countDocuments(`agent_${agentName}_improvements`, {
      "created_at": { "$gte": sevenDaysAgo() }
    })
  };

  return dashboard;
}

Global vs Project Memory Scopes

Dual-Scope Architecture

┌─────────────────────────────────────────────────────────┐
│ Global Memory (~/.claude/chroma_data)                   │
│ - Cross-project patterns that apply everywhere          │
│ - Reusable learnings (search strategies, code quality)  │
│ - Collections: agent_{name}_improvements                │
│ - Persists: Forever (user's institutional knowledge)    │
└─────────────────────────────────────────────────────────┘
                         ↓ query both
┌─────────────────────────────────────────────────────────┐
│ Project Memory (.claude/chroma_data)                    │
│ - Project-specific patterns (this codebase's style)     │
│ - Domain knowledge (this API, this architecture)        │
│ - Collections: agent_{name}_project_{hash}_improvements │
│ - Persists: Project lifetime                            │
└─────────────────────────────────────────────────────────┘

Memory Scope Selection

function selectMemoryScope(insight) {
  // Global: Universal patterns
  if (insight.category in ['search_strategy', 'error_handling', 'code_quality',
                           'testing_patterns', 'documentation_style']) {
    return 'global';
  }

  // Project: Codebase-specific patterns
  if (insight.category in ['naming_conventions', 'architecture_patterns',
                           'api_usage', 'domain_terminology']) {
    return 'project';
  }

  // Default: Store in both if uncertain
  return 'both';
}

async function storeWithScope(agentName, improvement, scope = 'global') {
  const globalCollection = `agent_${agentName}_improvements`;
  const projectCollection = `agent_${agentName}_project_${getProjectHash()}_improvements`;

  if (scope === 'global' || scope === 'both') {
    await storeImprovement(globalCollection, improvement);
  }
  if (scope === 'project' || scope === 'both') {
    await storeImprovement(projectCollection, improvement);
  }
}

async function retrieveWithScope(agentName, taskDescription) {
  const globalResults = await retrieveRelevantImprovements(
    `agent_${agentName}_improvements`, taskDescription, 3
  );
  const projectResults = await retrieveRelevantImprovements(
    `agent_${agentName}_project_${getProjectHash()}_improvements`, taskDescription, 3
  );

  // Merge and deduplicate, project-specific takes priority
  return mergeImprovements(projectResults, globalResults);
}

Slim Agent Integration Template

Agents should reference this skill instead of duplicating memory code:

## Memory Configuration (uses agent-memory-skills)

**Collections**:
- Global: `agent_{name}_improvements`, `agent_{name}_evaluations`, `agent_{name}_performance`
- Project: `agent_{name}_project_{hash}_improvements` (if project-specific learning)

**Quality Criteria** (agent-specific):
- [criterion_1]: weight X%
- [criterion_2]: weight Y%
- [criterion_3]: weight Z%

**Insight Categories** (agent-specific):
- [category_1]: Description of when this applies
- [category_2]: Description of when this applies

**Memory Workflow**:
1. **Phase 0.5 (Before Task)**: Retrieve relevant improvements
   - Call: `retrieveWithScope(agentName, taskDescription)`
   - Apply retrieved patterns to current task

2. **Phase N.5 (After Task)**: Self-evaluate and store
   - Assess quality using agent-specific criteria
   - Extract insights with agent-specific categories
   - Store improvements if quality ≥ 70
   - Update usage statistics for retrieved improvements

Example: Slim Code-Finder Memory Section

## Memory Configuration (uses agent-memory-skills)

**Collections**: `agent_code_finder_improvements`, `agent_code_finder_evaluations`

**Quality Criteria**:
- Result accuracy: 30%
- Search confidence: 25%
- Strategy completeness: 20%
- Search efficiency: 15%
- Coverage assessment: 10%

**Insight Categories**:
- `search_strategy`: Effective search patterns for code types
- `file_patterns`: File naming/location patterns that work
- `naming_conventions`: Casing and naming variants that help
- `query_optimization`: Query formulations that improve results

**Memory Workflow**:
- Phase 0.5: Retrieve search strategy improvements before search
- Phase 4.5: Evaluate search quality, store effective patterns

Memory Consolidation Integration

Individual agent memories benefit from periodic consolidation by the memory-consolidation-agent. This section describes how agents interact with the consolidation system.

Consolidation-Ready Metadata

When storing improvements, include metadata that enables consolidation:

const improvementMetadata = {
  // Standard fields
  agent_name: agentName,
  category: insight.category,
  confidence: insight.confidence,
  created_at: timestamp,
  usage_count: 0,
  success_rate: null,

  // Consolidation-ready fields
  cross_validated: false,           // Set true when validated across agents
  source: 'self',                   // 'self' | 'system_principles' | 'transferred'
  original_id: null,                // If transferred from another agent
  consolidation_eligible: true,     // Can be used for schema formation
  context_tags: ['domain', 'tech'], // Help consolidation group related patterns
  deprecated: false,
  deprecated_reason: null
};

Receiving Transferred Knowledge

Agents may receive improvements from the consolidation system. Handle these appropriately:

async function retrieveWithTransfers(agentName, taskDescription) {
  const results = await mcp__chroma__query_documents({
    collection_name: `agent_${agentName}_improvements`,
    query_texts: [taskDescription],
    n_results: 10,
    where: { "deprecated": { "$ne": true } }
  });

  // Prioritize cross-validated and transferred improvements
  return results.ids[0]
    .map((id, idx) => ({
      id: id,
      improvement: results.documents[0][idx],
      metadata: results.metadatas[0][idx],
      relevance: 1 - results.distances[0][idx],
      // Boost score for cross-validated patterns
      priority: results.metadatas[0][idx].cross_validated ? 1.2 : 1.0
    }))
    .filter(item => item.relevance > 0.6)
    .sort((a, b) => (b.relevance * b.priority) - (a.relevance * a.priority));
}

Consolidation Hooks

Agents should expose these hooks for the consolidation system:

// Hook: Get all improvements for consolidation analysis
async function getConsolidatableImprovements(agentName) {
  return await mcp__chroma__get_documents({
    collection_name: `agent_${agentName}_improvements`,
    where: { "consolidation_eligible": true, "deprecated": { "$ne": true } },
    include: ["documents", "metadatas"]
  });
}

// Hook: Mark improvement as cross-validated
async function markCrossValidated(agentName, improvementId, validatingAgents) {
  const current = await mcp__chroma__get_documents({
    collection_name: `agent_${agentName}_improvements`,
    ids: [improvementId]
  });

  await mcp__chroma__update_documents({
    collection_name: `agent_${agentName}_improvements`,
    ids: [improvementId],
    metadatas: [{
      ...current.metadatas[0],
      cross_validated: true,
      validating_agents: validatingAgents.join(','),
      cross_validated_at: new Date().toISOString()
    }]
  });
}

// Hook: Receive transferred principle from consolidation
async function receiveTransferredPrinciple(agentName, principle, sourceMetadata) {
  await mcp__chroma__add_documents({
    collection_name: `agent_${agentName}_improvements`,
    documents: [principle],
    ids: [`transferred_${sourceMetadata.original_id}_${Date.now()}`],
    metadatas: [{
      agent_name: agentName,
      category: sourceMetadata.category,
      confidence: sourceMetadata.confidence * 0.9,  // Discount for transfer
      source: 'system_principles',
      original_id: sourceMetadata.original_id,
      cross_validated: true,
      transferred_at: new Date().toISOString(),
      usage_count: 0,
      success_rate: null,
      consolidation_eligible: false  // Don't re-consolidate transferred items
    }]
  });
}

Consolidation Schedule

QAVR Integration (Q-Value Augmented Retrieval)

QAVR adds learned utility ranking to memory retrieval. Instead of just semantic similarity, QAVR re-ranks results by historical usefulness.

How QAVR Enhances Agent Memory

Traditional: Query → ChromaDB → Most Similar Memories
QAVR:        Query → ChromaDB → Re-rank by Q-values → Actually Useful Memories

QAVR-Enhanced Retrieval

import sys
sys.path.insert(0, '/home/kim/.claude/qavr')
from q_value_store import QValueStore

async def retrieveImprovementsWithQAVR(agentName, taskDescription, contextType='general', limit=5):
    """
    Retrieve improvements with QAVR Q-value ranking.

    Args:
        agentName: Agent name for collection lookup
        taskDescription: Current task for semantic search
        contextType: Context for Q-value lookup (debugging, research, etc.)
        limit: Number of results to return

    Returns:
        List of improvements ranked by combined semantic + Q-value score
    """
    collectionName = f"agent_{agentName}_improvements"

    # Phase 1: Semantic retrieval (get more candidates than needed)
    candidateK = limit * 10
    candidates = await mcp__chroma__chroma_query_documents({
        'collection_name': collectionName,
        'query_texts': [taskDescription],
        'n_results': candidateK,
        'where': { 'deprecated': { '$ne': True } },
        'include': ['documents', 'metadatas', 'distances']
    })

    if not candidates['ids'][0]:
        return []

    # Phase 2: Q-value re-ranking (if warm)
    qStore = QValueStore('/home/kim/.claude/qavr/q_values.json')
    isWarm = qStore.is_warm(contextType)
    lambdaBalance = 0.5  # Balance semantic vs Q-value

    results = []
    for idx, memoryId in enumerate(candidates['ids'][0]):
        distance = candidates['distances'][0][idx]
        semanticScore = 1.0 - distance

        if isWarm:
            qValue = qStore.get_q(memoryId, contextType)
            combinedScore = (1 - lambdaBalance) * semanticScore + lambdaBalance * qValue
        else:
            qValue = 0.5  # Neutral prior in cold mode
            combinedScore = semanticScore

        results.append({
            'id': memoryId,
            'document': candidates['documents'][0][idx],
            'metadata': candidates['metadatas'][0][idx],
            'semantic_score': semanticScore,
            'q_value': qValue,
            'combined_score': combinedScore
        })

    # Sort by combined score and return top N
    results.sort(key=lambda x: x['combined_score'], reverse=True)
    return results[:limit]

Recording Outcomes for Q-Learning

async def recordTaskOutcome(agentName, usedImprovementIds, contextType, success):
    """
    Record task outcome to update Q-values for used improvements.

    Args:
        agentName: Agent name
        usedImprovementIds: List of improvement IDs that were used
        contextType: Context type for Q-value grouping
        success: Whether the task succeeded
    """
    qStore = QValueStore('/home/kim/.claude/qavr/q_values.json')

    reward = 1.0 if success else -0.3

    for memoryId in usedImprovementIds:
        result = qStore.update_q(memoryId, contextType, reward)
        print(f"Q-update: {memoryId} {result.old_q:.3f} → {result.new_q:.3f}")

    qStore.save()

    return {
        'mode': qStore.get_mode(contextType),
        'interactions': qStore.context_interactions.get(contextType, 0)
    }

Cold vs Warm Mode

Quick QAVR Status Check

# Check QAVR status for an agent
from q_value_store import QValueStore

store = QValueStore('/home/kim/.claude/qavr/q_values.json')
stats = store.get_stats()

print(f"Mode: {store.get_mode('debugging')}")
print(f"Interactions: {store.context_interactions.get('debugging', 0)}")
print(f"Top memories: {store.get_top_memories('debugging', 5)}")

Related

• See ~/.claude/qavr/config.yaml for configuration
• Run /qavr-status for full diagnostic
• See qavr-retrieval/SKILL.md for detailed QAVR documentation

Success Criteria

Agent memory system is working when:

• ✅ Improvements Stored: Agent learns from each task (10+ improvements/week)
• ✅ Relevant Retrieval: Retrieved improvements match current task (>80% relevance)
• ✅ Success Rate Tracking: Improvements update usage stats correctly
• ✅ Performance Improving: Quality trend positive over 30 days
• ✅ Self-Evaluation: Runs after every task completion
• ✅ Deprecation Works: Low-performing improvements marked deprecated
• ✅ Promotion Works: High-performing improvements boosted
• ✅ No .md Modifications: Static config unchanged, memory in ChromaDB

Comparison: .md vs ChromaDB

Recommendation: Use ChromaDB for dynamic memory, .md for static config

Version: 1.0 Created: 2025-11-18 Purpose: Enable continuous agent self-improvement with ChromaDB memory Dependencies: chromadb-integration-skills Applicable To: All agents (research, development, trading, legal, etc.)