Feature Implement

Feature Implementation

Systematic approach to implementing new features in the Rust memory system.

Purpose

Add new functionality following project conventions, maintaining code quality and test coverage.

Implementation Process

Phase 1: Planning

1. Understand Requirements

• What is the feature?
• Why is it needed?
• Who will use it?
• What are the acceptance criteria?

2. Design Approach

• How does it fit into existing architecture?
• What modules need changes?
• What new modules are needed?
• What are the data structures?
• What are the API signatures?

3. Check Constraints

• File size limit: ≤ 500 LOC per file
• Async/Tokio patterns for I/O
• Error handling with anyhow::Result
• Storage: Turso (durable) + redb (cache)

Phase 2: Implementation

1. Create Module Structure

# For new module
touch src/new_feature/mod.rs
touch src/new_feature/core.rs
touch src/new_feature/storage.rs

# Update src/lib.rs
# pub mod new_feature;

Example structure:

src/
├── new_feature/
│   ├── mod.rs          # Public API exports
│   ├── core.rs         # Core logic (<500 LOC)
│   ├── storage.rs      # Storage operations (<500 LOC)
│   └── types.rs        # Data structures (<500 LOC)

2. Define Types

// src/new_feature/types.rs
use serde::{Deserialize, Serialize};

/// Feature data structure
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct FeatureData {
    pub id: String,
    pub created_at: i64,
    pub data: serde_json::Value,
}

/// Configuration for feature
#[derive(Debug, Clone)]
pub struct FeatureConfig {
    pub enabled: bool,
    pub max_items: usize,
}

3. Implement Core Logic

// src/new_feature/core.rs
use anyhow::Result;

/// Main feature implementation
pub struct Feature {
    config: FeatureConfig,
}

impl Feature {
    /// Create new feature instance
    pub fn new(config: FeatureConfig) -> Self {
        Self { config }
    }

    /// Core feature operation
    pub async fn process(&self, input: FeatureData) -> Result<FeatureData> {
        // Validate input
        self.validate(&input)?;

        // Process
        let processed = self.process_internal(input).await?;

        // Store result
        self.store(&processed).await?;

        Ok(processed)
    }

    fn validate(&self, data: &FeatureData) -> Result<()> {
        // Validation logic
        Ok(())
    }

    async fn process_internal(&self, data: FeatureData) -> Result<FeatureData> {
        // Processing logic
        Ok(data)
    }

    async fn store(&self, data: &FeatureData) -> Result<()> {
        // Storage logic
        Ok(())
    }
}

4. Add Storage Layer

// src/new_feature/storage.rs
use super::types::FeatureData;
use anyhow::Result;

/// Turso storage for feature
pub struct FeatureStorage {
    turso: TursoClient,
}

impl FeatureStorage {
    pub async fn save(&self, data: &FeatureData) -> Result<()> {
        let sql = "INSERT OR REPLACE INTO feature_table (id, data, created_at) VALUES (?, ?, ?)";
        self.turso
            .execute(sql)
            .bind(&data.id)
            .bind(serde_json::to_string(&data.data)?)
            .bind(data.created_at)
            .await?;
        Ok(())
    }

    pub async fn get(&self, id: &str) -> Result<Option<FeatureData>> {
        let sql = "SELECT id, data, created_at FROM feature_table WHERE id = ?";
        let row = self.turso.query(sql).bind(id).await?;

        // Parse and return
        Ok(None) // Placeholder
    }
}

5. Implement Public API

// src/new_feature/mod.rs
mod core;
mod storage;
mod types;

pub use types::{FeatureConfig, FeatureData};
pub use core::Feature;

/// Convenience function for common use case
pub async fn quick_process(data: FeatureData) -> anyhow::Result<FeatureData> {
    let feature = Feature::new(FeatureConfig::default());
    feature.process(data).await
}

Phase 3: Testing

1. Unit Tests

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_feature_creation() {
        let config = FeatureConfig {
            enabled: true,
            max_items: 100,
        };
        let feature = Feature::new(config);
        assert!(feature.config.enabled);
    }

    #[test]
    fn test_validation() {
        let feature = Feature::new(FeatureConfig::default());
        let data = FeatureData {
            id: "test".to_string(),
            created_at: 0,
            data: serde_json::json!({}),
        };
        assert!(feature.validate(&data).is_ok());
    }

    #[tokio::test]
    async fn test_process() {
        let feature = Feature::new(FeatureConfig::default());
        let input = create_test_data();
        let result = feature.process(input).await;
        assert!(result.is_ok());
    }
}

2. Integration Tests

// tests/integration/feature_test.rs
use memory_core::new_feature::*;

#[tokio::test]
async fn test_end_to_end_feature() {
    // Setup
    let memory = create_test_memory().await;

    // Execute feature
    let data = FeatureData { /* ... */ };
    let result = memory.feature_operation(data).await;

    // Verify
    assert!(result.is_ok());

    // Check storage
    let stored = memory.get_feature_data("id").await.unwrap();
    assert_eq!(stored.id, "id");
}

Phase 4: Integration

1. Wire into Main API

// src/lib.rs
pub mod new_feature;

use new_feature::Feature;

pub struct SelfLearningMemory {
    // Existing fields...
    feature: Feature,
}

impl SelfLearningMemory {
    pub async fn feature_operation(&self, data: FeatureData) -> Result<FeatureData> {
        self.feature.process(data).await
    }
}

2. Update Database Schema

-- migrations/003_add_feature_table.sql
CREATE TABLE IF NOT EXISTS feature_table (
    id TEXT PRIMARY KEY,
    data TEXT NOT NULL,
    created_at INTEGER NOT NULL
);

CREATE INDEX idx_feature_created ON feature_table(created_at DESC);

3. Add Configuration

// src/config.rs
pub struct Config {
    // Existing config...
    pub feature_config: FeatureConfig,
}

Phase 5: Documentation

1. Code Documentation

/// Process feature data through the memory system.
///
/// This function takes raw feature data, validates it, processes it according
/// to learned patterns, and stores the result in both Turso and redb.
///
/// # Arguments
///
/// * `data` - The feature data to process
///
/// # Returns
///
/// Processed feature data with enriched information
///
/// # Errors
///
/// Returns error if:
/// - Data validation fails
/// - Storage operations fail
/// - Processing encounters unrecoverable error
///
/// # Example
///
/// ```no_run
/// use memory_core::new_feature::*;
///
/// #[tokio::main]
/// async fn main() -> anyhow::Result<()> {
///     let data = FeatureData {
///         id: "example".to_string(),
///         created_at: 1234567890,
///         data: serde_json::json!({"key": "value"}),
///     };
///
///     let result = quick_process(data).await?;
///     println!("Processed: {:?}", result);
///     Ok(())
/// }
/// ```
pub async fn feature_operation(&self, data: FeatureData) -> Result<FeatureData>

2. Update README

Add feature to main README.md:

## Features

- Episodic memory storage
- Pattern extraction and learning
- Context retrieval
- **NEW: Feature name** - Brief description

Phase 6: Quality Checks

# Format
cargo fmt

# Lint
cargo clippy --all -- -D warnings

# Build
cargo build --all

# Test
cargo test --all

# Documentation
cargo doc --no-deps

Phase 7: Commit

# Stage changes
git add src/new_feature/ tests/integration/feature_test.rs

# Commit with clear message
git commit -m "[feature] add new_feature module

- Implemented core Feature struct with process logic
- Added Turso storage layer with save/get operations
- Created comprehensive unit and integration tests
- Updated main API to expose feature_operation
- Added database migration for feature_table

Closes: #123
"

Best Practices

Code Organization

• One feature per module
• Split large modules (< 500 LOC per file)
• Clear separation: types, core logic, storage, API

Error Handling

// Good: Propagate errors
pub async fn operation(&self) -> Result<Data> {
    let data = self.fetch().await?;
    self.process(data).await?;
    Ok(data)
}

// Bad: Swallow errors
pub async fn operation(&self) -> Option<Data> {
    let data = self.fetch().await.ok()?;  // Error info lost
    Some(data)
}

Async Patterns

// Good: Concurrent operations
let (result1, result2) = tokio::join!(
    operation1(),
    operation2(),
);

// Bad: Sequential when could be concurrent
let result1 = operation1().await;
let result2 = operation2().await;

Testing

• Unit tests for each function
• Integration tests for workflows
• Test error cases
• Test edge cases (empty, max, invalid)

Documentation

• Public APIs must be documented
• Include examples for complex APIs
• Document errors and edge cases
• Keep docs up to date with code

Feature Checklist

• Requirements understood
• Design reviewed
• Module structure created
• Types defined
• Core logic implemented
• Storage layer added
• Public API exposed
• Unit tests written (>80% coverage)
• Integration tests added
• Documentation written
• Code formatted (cargo fmt)
• Lints pass (cargo clippy)
• All tests pass (cargo test --all)
• Build succeeds (cargo build --release)
• Committed with clear message
• PR created (if applicable)

Common Pitfalls

1. Forgetting .await

// Wrong
let data = async_function();  // Returns Future, not Data

// Right
let data = async_function().await?;

2. Blocking in Async Context

// Wrong
async fn process() {
    std::thread::sleep(Duration::from_secs(1));  // Blocks executor!
}

// Right
async fn process() {
    tokio::time::sleep(Duration::from_secs(1)).await;
}

3. Not Testing Error Cases

#[tokio::test]
async fn test_error_handling() {
    let result = operation_with_invalid_input().await;
    assert!(result.is_err());
    assert!(matches!(result.unwrap_err(), Error::InvalidInput(_)));
}

4. Ignoring Performance

// Consider batch operations
for item in items {
    storage.save(item).await?;  // N database calls
}

// Better
storage.save_batch(items).await?;  // 1 database call

Examples