Results for “bulk-processing”

Bulk-retouch a folder of portrait photos using Adobe tools — designed for wedding photographers and event photographers who need fast, walk-away batch processing.

Designs and implements big data architectures, processes large-scale datasets with distributed systems, and optimizes data pipelines for throughput using Hadoop, Spark, and cloud platforms.

Compresses long K-line (candlestick) data into a fixed-length sequence using OHLC aggregation rules and applies min-max normalization.

Archive multiple completed changes at once. Use when archiving several parallel changes.

Use this when the system needs queue-backed processing design, message handling rules, delivery semantics, dead-letter strategy, or throughput planning. Trigger on requests about queues, brokers, message processing, and consumer coordination.

Apache Spark, Hadoop, distributed computing, and large-scale data processing for petabyte-scale workloads

Implements Bulk API 2.0 REST calls for ingest and query jobs, covering CSV format requirements, locator pagination, and v1 vs v2 selection.

Diagnose and optimize slow-loading, high-memory, or low-FPS USD scenes using a structured workflow with profiling, validation, and mutation phases.

Diagnoses and resolves slow Qdrant indexing and data ingestion by optimizing batching, sharding, HNSW parameters, and payload indexing strategies.

Runs batch processing and high-performance computing (HPC) workloads on Google Kubernetes Engine (GKE), including job queues, parallel processing, and MPI workloads.

Process and manipulate images using ImageMagick: resize, convert formats, batch process, and retrieve metadata.

Redesign Salesforce Flows to handle bulk data loads without hitting transaction limits, using collection-first patterns and safe transaction scope.

Guides when to use Salesforce Bulk API 2.0 and provides commands for importing, updating, upserting, deleting, and exporting records, along with CSV format requirements, limits, and error handling.

Data pipeline specialist for ML experiments. Use when the user asks to talk to Gekko, requests the data engineer, or needs DataLoader optimization.

Orchestrates a complete bulk RNA-seq differential-expression study from raw FASTQ reads through QC, alignment, quantification, differential expression, pathway enrichment, and publication figures.

Process large-scale ML datasets with distributed streaming execution across CPU/GPU, supporting Parquet, CSV, JSON, images, and integration with PyTorch, TensorFlow, and Ray Train.

Foundational theory for process-oriented concurrency through synchronous message-passing, applicable to multi-agent coordination and parallel decomposition

Scale pandas and NumPy workflows to larger-than-memory datasets using parallel and distributed computing.

Designs Elasticsearch indexes and mappings, tunes queries, sizes clusters, and handles operations like shard/replica strategy, ILM, monitoring, troubleshooting, and safe reindexing or upgrades.

Expert in building Retrieval-Augmented Generation systems. Masters embedding models, vector databases, chunking strategies, and retrieval optimization for LLM applications.

Web Workers

Build and debug Retrieval-Augmented Generation pipelines — chunking, embedding, retrieval, reranking

Compress bloated project or global memory while preserving decisions, rationale, revisit triggers, provenance, and active user preferences. Load when memory exceeds budget, global memory is too large, session logs are repetitive, or before appending to an over-budget memory file.

Scalable data processing for ML workloads. Streaming execution across CPU/GPU, supports Parquet/CSV/JSON/images. Integrates with Ray Train, PyTorch, TensorFlow. Scales from single machine to 100s of nodes. Use for batch inference, data preprocessing, multi-modal data loading, or distributed ETL pipelines.

Evaluates a containerized framework for deploying distributed big data workloads, measuring execution time and cloud cost scaling from four to eight nodes.

Optimizes long-running agent sessions with structured context compression, summarization, and durable handoff summaries that preserve decisions, files, risks, and next actions.

Quantize large language models to 4-bit with minimal accuracy loss using GPTQ, enabling deployment of 70B+ models on consumer GPUs with 4× memory reduction and 3-4× faster inference.

Perform differential gene expression analysis for bulk RNA-seq data using PyDESeq2, supporting formulaic designs, Wald tests, FDR correction, LFC shrinkage, and result visualization.

Runs NCBI BLAST+ 2.17.0 sequence searches from the command line: makeblastdb (with -parse_seqids), blastn/blastp/blastx/tblastn with tabular -outfmt 6/7 for parsing, correct -task choice (megablast vs blastn vs blastn-short), -taxids/-negative_taxids taxonomic scoping, and -mt_mode multithreading; plus a DIAMOND blastp --ultra-sensitive path for large protein searches. Warns that -max_target_seqs is a heuristic keep-count, not a top-N best-hits filter. Use when the user wants command-line BLAST, makeblastdb, a local BLAST database, blastn/blastp/blastx/tblastn searches, or DIAMOND protein search. For the Bio.Blast web NCBIWWW API prefer alterlab-biopython; for quick one-liner database lookups prefer alterlab-gget. Part of the AlterLab Academic Skills suite.

Multi-project signal isolation with cascading recipe resolution

Evaluates computational throughput and real-time efficiency of embedded CPU and GPU platforms by measuring peak FLOPS via a matrix rotation kernel and assessing inference latency and power consumption on a robotic vision pipeline.

Fast tokenization for NLP using Rust-based tokenizers supporting BPE, WordPiece, and Unigram algorithms, with training, alignment tracking, and padding/truncation.

This skill provides guidance for video analysis and processing tasks using computer vision techniques. It should be used when analyzing video frames, detecting motion or events, tracking objects, extracting temporal data (e.g., identifying specific frames like takeoff/landing moments), or performing frame-by-frame processing with OpenCV or similar libraries.

Generates OMG-compliant BPMN 2.0 XML and SVG diagrams from natural language process descriptions, with validation, automatic layout, and optional process optimization advisories.

Use this skill for processing and analyzing large tabular datasets (billions of rows) that exceed available RAM. Vaex excels at out-of-core DataFrame operations, lazy evaluation, fast aggregations, efficient visualization of big data, and machine learning on large datasets. Apply when users need to work with large CSV/HDF5/Arrow/Parquet files, perform fast statistics on massive datasets, create visualizations of big data, or build ML pipelines that don't fit in memory.