How do I install the Coreweave Performance Tuning skill?

Run npx skillmds@latest add jeremylongshore/coreweave-performance-tuning in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Coreweave Performance Tuning safe to use?

This skill has not completed SkillMD's automated safety review yet. Capability flags: makes network calls. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Coreweave Performance Tuning?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Coreweave Performance Tuning free to use?

Yes. Installing skills from SkillMD is free. This skill is licensed under MIT.

Who published Coreweave Performance Tuning?

jeremylongshore (@jeremylongshore) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Coreweave Performance Tuning by jeremylongshore

Name: Coreweave Performance Tuning
Rating: 4.666666666666667 (3 reviews)
Author: jeremylongshore

Coreweave Performance Tuning

Optimize CoreWeave GPU inference latency and throughput. Use when reducing inference latency, maximizing GPU utilization, or tuning batch sizes and concurrency. Trigger with phrases like "coreweave performance", "coreweave latency", "coreweave throughput", "optimize coreweave inference".

File contents

CoreWeave Performance Tuning

GPU Selection by Workload

Workload	Recommended GPU	Why
LLM inference (7-13B)	A100 80GB	Good balance of memory and cost
LLM inference (70B+)	8xH100	NVLink for tensor parallelism
Image generation	L40	Good for diffusion models
Training (large models)	8xH100 SXM5	Fastest interconnect
Batch processing	A100 40GB	Cost-effective

Inference Optimization

# Continuous batching with vLLM
containers:
  - name: vllm
    args:
      - "--model=meta-llama/Llama-3.1-8B-Instruct"
      - "--max-num-batched-tokens=8192"
      - "--max-num-seqs=256"
      - "--gpu-memory-utilization=0.90"
      - "--enable-prefix-caching"
      - "--dtype=float16"

Autoscaling Tuning

# HPA based on GPU utilization
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: inference-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: inference-server
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Pods
      pods:
        metric:
          name: DCGM_FI_DEV_GPU_UTIL
        target:
          type: AverageValue
          averageValue: "70"

Performance Benchmarks

Metric	A100-80GB	H100-80GB
Llama-8B tokens/sec	~2,000	~4,500
Llama-70B tokens/sec	~200 (4x)	~500 (4x)
Cold start (vLLM)	30-60s	20-40s

Resources

Next Steps

For cost optimization, see coreweave-cost-tuning.

---name: coreweave-performance-tuningdescription: Optimize CoreWeave GPU inference latency and throughput. Use when reducing inference latency, maximizing GPU utilization, or tuning batch sizes and concurrency. Trigger with phrases like "coreweave performance", "coreweave latency", "coreweave throughput", "optimize coreweave inference".license: MIT---# CoreWeave Performance Tuning## GPU Selection by Workload| Workload | Recommended GPU | Why ||----------|----------------|-----|| LLM inference (7-13B) | A100 80GB | Good balance of memory and cost || LLM inference (70B+) | 8xH100 | NVLink for tensor parallelism || Image generation | L40 | Good for diffusion models || Training (large models) | 8xH100 SXM5 | Fastest interconnect || Batch processing | A100 40GB | Cost-effective |## Inference Optimization```yaml# Continuous batching with vLLMcontainers: - name: vllm args: - "--model=meta-llama/Llama-3.1-8B-Instruct" - "--max-num-batched-tokens=8192" - "--max-num-seqs=256" - "--gpu-memory-utilization=0.90" - "--enable-prefix-caching" - "--dtype=float16"```## Autoscaling Tuning```yaml# HPA based on GPU utilizationapiVersion: autoscaling/v2kind: HorizontalPodAutoscalermetadata: name: inference-hpaspec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: inference-server minReplicas: 2 maxReplicas: 10 metrics: - type: Pods pods: metric: name: DCGM_FI_DEV_GPU_UTIL target: type: AverageValue averageValue: "70"```## Performance Benchmarks| Metric | A100-80GB | H100-80GB ||--------|-----------|-----------|| Llama-8B tokens/sec | ~2,000 | ~4,500 || Llama-70B tokens/sec | ~200 (4x) | ~500 (4x) || Cold start (vLLM) | 30-60s | 20-40s |## Resources- [CoreWeave Inference](https://www.coreweave.com/solutions/ai-inference)- [vLLM Documentation](https://docs.vllm.ai)## Next StepsFor cost optimization, see `coreweave-cost-tuning`.

Frequently asked questions