Results for “bulk-processing”

Designs and implements big data architectures, processes large-scale datasets with distributed systems, and optimizes data pipelines for throughput using Hadoop, Spark, and cloud platforms.

Process large-scale ML datasets with distributed streaming execution across CPU/GPU, supporting Parquet, CSV, JSON, images, and integration with PyTorch, TensorFlow, and Ray Train.

Foundational theory for process-oriented concurrency through synchronous message-passing, applicable to multi-agent coordination and parallel decomposition

Scale pandas and NumPy workflows to larger-than-memory datasets using parallel and distributed computing.

Expert in building Retrieval-Augmented Generation systems. Masters embedding models, vector databases, chunking strategies, and retrieval optimization for LLM applications.

Build and debug Retrieval-Augmented Generation pipelines — chunking, embedding, retrieval, reranking

Optimizes long-running agent sessions with structured context compression, summarization, and durable handoff summaries that preserve decisions, files, risks, and next actions.

Quantize large language models to 4-bit with minimal accuracy loss using GPTQ, enabling deployment of 70B+ models on consumer GPUs with 4× memory reduction and 3-4× faster inference.

Evaluates computational throughput and real-time efficiency of embedded CPU and GPU platforms by measuring peak FLOPS via a matrix rotation kernel and assessing inference latency and power consumption on a robotic vision pipeline.

Fast tokenization for NLP using Rust-based tokenizers supporting BPE, WordPiece, and Unigram algorithms, with training, alignment tracking, and padding/truncation.

Guides analysis of Neuropixels recordings from raw data to curated units, covering preprocessing, motion correction, spike sorting, quality metrics, and export.

Transform the agent into a performance engineer. Apply methodologies for measuring, profiling, and optimizing code (caching, algorithm complexity, resource usage).