Ivx Cf Performance Optimization

Performance Optimization Skill

Purpose

Make systems fast, efficient, and cost-effective without sacrificing correctness.

Optimization Loop

1. Measure: Profile to find actual bottlenecks
2. Hypothesize: Identify root cause
3. Implement: Make targeted change
4. Validate: Measure improvement (or lack thereof)
5. Document: Record learnings

Profiling Tools

Common Bottlenecks

API Latency

• N+1 queries → Batch / JOIN
• Missing indexes → Add covering indexes
• Large payloads → Pagination, field selection
• Sync external calls → Async + caching

GPU Inference

• Small batches → Continuous batching
• Memory bandwidth → Quantization (FP8/INT8)
• Kernel overhead → TensorRT compilation
• Cold start → Warm pools, pre-loading

Memory

• Large objects → Streaming, generators
• Memory leaks → Profile allocations
• Redis bloat → Key TTLs, eviction policies
• Task store → Don't store large blobs in result

CF-Specific

• Use model_gateway.get_pricing_summary() for cost estimates
• Batch pipeline stages when independent
• Cache prompt registry lookups
• Stream large files, don't load into memory
• Use Redis pipeline for multiple operations
• Profile pipeline runs: utils/pipeline/run_memory.py

Targets