Results for “data-lake-storage-gen2”

Create and validate Earth2Studio data source wrappers (DataSource, ForecastSource, DataFrameSource, ForecastFrameSource) from remote stores like S3, GCS, Azure, HTTP, or HuggingFace.

Agent D2 - Data Collection Specialist - Interviews, Focus Groups & Observation. Covers protocol development, question design, probing strategies, transcription conventions, and systematic observation. Absorbed D3 (Observation Protocol Designer) capabilities.

Designs and implements big data architectures, processes large-scale datasets with distributed systems, and optimizes data pipelines for throughput using Hadoop, Spark, and cloud platforms.

Local RAG indexing with 97% storage reduction via anchor-based lazy recomputation. Graph-based selective embedding storage for memory-efficient semantic code search.

Caching strategies across the RAG stack. Semantic caching with GPTCache and LangChain, Redis-based embedding-similarity cache, cache key design, TTL/invalidation, partial caching (cache retrieval only), provider-native prompt caching (Anthropic, OpenAI), and hierarchical L1/L2 caches. USE WHEN: user mentions "semantic cache", "GPTCache", "LLM cache", "prompt caching", "Redis vector cache", "cache invalidation for RAG", "reduce LLM cost", "latency reduction LLM" DO NOT USE FOR: retrieval accuracy - use `rag-patterns`; groundedness checks - use `rag-guardrails`; incremental indexing - use `rag-production`

Persistent memory CLI for LLM agents. Store facts, recall past knowledge, link related memories, manage lifecycle.

Process large ML datasets in parallel across CPU or GPU clusters, with streaming execution, multi-format I/O, and integration with Ray Train, PyTorch, and TensorFlow for batch inference and preprocessing pipelines.