Results for “scaling”

Evaluates a containerized framework for deploying distributed big data workloads, measuring execution time and cloud cost scaling from four to eight nodes.

Assesses NumPy code for cuPyNumeric migration readiness by analyzing source code against an API support manifest and GPU-scaling idioms, producing a structured verdict with per-finding reasoning.

Guides scaling Qdrant vector search with time-based data rotation using shard rotation, collection rotation, or filter-and-delete strategies.

Optimizes Qdrant query performance for large limits across multiple shards by using Poisson-distributed subsampling to reduce inter-shard data transfer.

Scales pandas/NumPy workflows beyond memory with Dask distributed computing — parallel DataFrames, arrays, delayed task graphs, and cluster execution. Use when existing pandas/NumPy code must run on larger-than-RAM data or across clusters, for parallel file processing, distributed ML, or integration with existing pandas code. For out-of-core analytics on a single machine prefer vaex; for in-memory speed prefer polars. Part of the AlterLab Academic Skills suite.

Scalable data processing for ML workloads. Streaming execution across CPU/GPU, supports Parquet/CSV/JSON/images. Integrates with Ray Train, PyTorch, TensorFlow. Scales from single machine to 100s of nodes. Use for batch inference, data preprocessing, multi-modal data loading, or distributed ETL pipelines.

Scale pandas and NumPy workflows to larger-than-memory datasets using parallel and distributed computing.

Compute TAM/SAM/SOM with top-down and bottoms-up methods, plan survey sample sizes with per-segment minimums, and score candidate segments against Kotler's five criteria.

Guides database design and operations to avoid common scaling, reliability, and data integrity pitfalls.

Process large ML datasets in parallel across CPU or GPU clusters, with streaming execution, multi-format I/O, and integration with Ray Train, PyTorch, and TensorFlow for batch inference and preprocessing pipelines.

Dask parallel computing reference for Python. Covers Dask DataFrame (parallel Pandas), Dask Array (parallel NumPy), Dask Delayed for custom parallelism, Dask Bag, distributed clusters, dashboard monitoring, and scaling best practices.

Manages Neon serverless Postgres databases via the neonctl CLI and Neon API, covering projects, branches, databases, roles, endpoints, and compute scaling with a discovery-first workflow.

Build production RAG and semantic search systems with a high-performance vector database written in Rust, supporting hybrid search, filtering, and horizontal scaling.

Provides battle-tested PyTorch training recipes for LLMs, vision, diffusion, and biomedical domains, covering training loops, optimizer selection, LR scheduling, mixed precision, and debugging.

Use for writing, reviewing, debugging, testing, or diagnosing Bayesian regression and hierarchical models built with Bambi formulas, Model, Family/Likelihood/Link, Prior, fit, prior predictive, and predict. Trigger on common versus group-specific terms, categorical coding, family/link choice, automatic prior scaling, missing rows, PyMC backend settings, and InferenceData predictions. Do not use for hand-built PyMC graphs, NumPyro programs, ArviZ-only analysis of existing draws, frequentist statsmodels formulas, or generic pandas work.