Results for “distributed-compute”

Scales pandas and NumPy workflows to datasets larger than memory using parallel and distributed computing, with support for dataframes, arrays, bags, and custom task graphs.

Scale pandas and NumPy workflows to larger-than-memory datasets using parallel and distributed computing.

Computação paralela/distribuída. Escale pandas/NumPy além da memória disponível, DataFrames/Arrays paralelos, processamento multi-arquivo, grafos de tarefas, para datasets maiores que RAM e workflows paralelos.

Expert guidance for distributed NoSQL databases (Cassandra, DynamoDB). Focuses on mental models, query-first modeling, single-table design, and avoiding hot partitions in high-scale systems.

Apache Spark, Hadoop, distributed computing, and large-scale data processing for petabyte-scale workloads

Distributed computing for larger-than-RAM pandas/NumPy workflows. Use when you need to scale existing pandas/NumPy code beyond memory or across clusters. Best for parallel file processing, distributed ML, integration with existing pandas code. For out-of-core analytics on single machine use vaex; for in-memory speed use polars.

Distributed computing for larger-than-RAM pandas/NumPy workflows. Use when you need to scale existing pandas/NumPy code beyond memory or across clusters. Best for parallel file processing, distributed ML, integration with existing pandas code. For out-of-core analytics on single machine use vaex; for in-memory speed use polars.

Orchestrates polyglot distributed applications using Aspire's AppHost, CLI, and MCP server for local development, testing, and deployment.

Parallel/distributed computing. Scale pandas/NumPy beyond memory, parallel DataFrames/Arrays, multi-file processing, task graphs, for larger-than-RAM datasets and parallel workflows.

Parallel/distributed computing. Scale pandas/NumPy beyond memory, parallel DataFrames/Arrays, multi-file processing, task graphs, for larger-than-RAM datasets and parallel workflows.

Design scalable distributed systems using structured approaches for load balancing, caching, database scaling, and message queues.

Expert guidance for distributed NoSQL databases (Cassandra, DynamoDB). Focuses on mental models, query-first modeling, single-table design, and avoiding hot partitions in high-scale systems.

Framework for scaling Python applications from a laptop to a cluster. Includes Ray Core for distributed computing, Ray Serve for model serving, Ray Tune for hyperparameter optimization, and Ray Data for distributed data processing.

Evaluates a containerized framework for deploying distributed big data workloads, measuring execution time and cloud cost scaling from four to eight nodes.

Design reliable, scalable, and maintainable data systems by applying principles from storage engines, replication, partitioning, transactions, and consistency models.

Designs and implements big data architectures, processes large-scale datasets with distributed systems, and optimizes data pipelines for throughput using Hadoop, Spark, and cloud platforms.

etcd distributed key-value store reference. Backbone of Kubernetes. Covers key-value operations, watch for real-time updates, leases with TTL, atomic transactions, cluster setup, backup/restore, authentication, TLS, and Prometheus monitoring.

Compute, object storage, block storage, a managed relational database, a message

Distributed training orchestration across clusters. Scales PyTorch/TensorFlow/HuggingFace from laptop to 1000s of nodes. Built-in hyperparameter tuning with Ray Tune, fault tolerance, elastic scaling. Use when training massive models across multiple machines or running distributed hyperparameter sweeps.

Use when setting up CI/CD pipelines, containerizing applications, or managing infrastructure as code. Invoke for pipelines, Docker, Kubernetes, cloud platforms, GitOps.

Scales pandas/NumPy workflows beyond memory with Dask distributed computing — parallel DataFrames, arrays, delayed task graphs, and cluster execution. Use when existing pandas/NumPy code must run on larger-than-RAM data or across clusters, for parallel file processing, distributed ML, or integration with existing pandas code. For out-of-core analytics on a single machine prefer vaex; for in-memory speed prefer polars. Part of the AlterLab Academic Skills suite.

Designs distributed system architectures, decomposes monoliths into bounded-context services, recommends communication patterns, and produces service boundary diagrams and resilience strategies.

Parallel/distributed computing. Scale pandas/NumPy beyond memory, parallel DataFrames/Arrays, multi-file processing, task graphs, for larger-than-RAM datasets and parallel workflows.

CAP Theorem

Scales AI and Python applications across clusters with distributed computing primitives for ML workloads.

Write, optimize, and debug Apache Spark jobs for high-performance distributed data processing, ETL pipelines, and big data workloads.

Distributed control system manager

Dask parallel computing reference for Python. Covers Dask DataFrame (parallel Pandas), Dask Array (parallel NumPy), Dask Delayed for custom parallelism, Dask Bag, distributed clusters, dashboard monitoring, and scaling best practices.

Expert guidance for distributed NoSQL databases (Cassandra, DynamoDB). Focuses on mental models, query-first modeling, single-table design, and avoiding hot partitions in high-scale systems.

Orchestrate token-heavy research, coding, and testing by delegating bounded tasks to cheaper subagents while reserving Claude Fable for architecture, synthesis, and final review.

Designs chaos experiments, creates failure injection frameworks, and facilitates game day exercises for distributed systems — producing runbooks, experiment manifests, rollback procedures, and post-mortem templates.

Deploy, monitor, and manage GCP services with battle-tested patterns.

DSPy: declarative LM programs, auto-optimize prompts, RAG.

Distributed training orchestration across clusters. Scales PyTorch/TensorFlow/HuggingFace from laptop to 1000s of nodes. Built-in hyperparameter tuning with Ray Tune, fault tolerance, elastic scaling. Use when training massive models across multiple machines or running distributed hyperparameter sweeps.