Optimizing R
This skill covers profiling, benchmarking, parallelization, and performance best practices for R.
Core Principle
Profile before optimizing - Use profvis and bench to identify real bottlenecks. Write readable code first, optimize only when necessary.
Profiling Tools Decision Matrix
| Tool |
Use When |
Don't Use When |
What It Shows |
profvis |
Complex code, unknown bottlenecks |
Simple functions, known issues |
Time per line, call stack |
bench::mark() |
Comparing alternatives |
Single approach |
Relative performance, memory |
system.time() |
Quick checks |
Detailed analysis |
Total runtime only |
Rprof() |
Base R only environments |
When profvis available |
Raw profiling data |
Performance Workflow
- Profile first - Find the actual bottlenecks
- Focus on the slowest parts - 80/20 rule
- Benchmark alternatives - For hot spots only
- Consider tool trade-offs - Based on bottleneck type
See profiling-workflow.md for the complete workflow.
When Each Tool Helps vs Hurts
Parallel Processing (in_parallel())
Helps when:
- CPU-intensive computations
- Embarrassingly parallel problems
- Large datasets with independent operations
- I/O bound operations (file reading, API calls)
Hurts when:
- Simple, fast operations (overhead > benefit)
- Memory-intensive operations (may cause thrashing)
- Operations requiring shared state
- Small datasets
See parallel-examples.md for decision points.
Data Backend Selection
| Backend |
Use When |
| data.table |
Very large datasets (>1GB), complex grouping, maximum performance critical |
| dplyr |
Readability priority, complex joins/window functions, moderate data (<100MB) |
| base R |
No dependencies allowed, simple operations, teaching/learning |
See backend-selection.md for guidance.
Profiling Best Practices
- Profile realistic data sizes - Not toy examples
- Profile multiple runs - For stability
- Check memory usage too - Not just time
- Profile realistic usage patterns - Not isolated calls
See profiling-best-practices.md for examples.
Performance Anti-Patterns to Avoid
- Don't optimize without measuring - Profile first
- Don't over-engineer - Complex optimizations for 1% gains
- Don't assume - "for loops are always slow" is a myth
- Don't ignore readability costs - Readable code with targeted optimizations
See performance-anti-patterns.md for examples.
Modern purrr Patterns
Data Frame Binding (purrr 1.0+)
| Superseded |
Modern Replacement |
map_dfr(x, f) |
map(x, f) |> list_rbind() |
map_dfc(x, f) |
map(x, f) |> list_cbind() |
map2_dfr(x, y, f) |
map2(x, y, f) |> list_rbind() |
Side Effects with walk()
Use walk() and walk2() for side effects (file writing, plotting).
Parallel Processing (purrr 1.1.0+)
Use in_parallel() with mirai for scaling across cores.
See purrr-patterns.md for all patterns.
Backend Tools for Performance
When speed is critical, consider:
- vctrs - Type-stable vector operations
- rlang - Metaprogramming
- data.table - Large data operations
Profile to identify whether these tools will help your specific bottleneck.
source: Sarah Johnson's gist https://gist.github.com/sj-io/3828d64d0969f2a0f05297e59e6c15ad
1---2name: optimizing-r3description: R performance profiling, benchmarking, and optimization strategies. Use this skill when code is running slowly, comparing alternative implementations, deciding between dplyr/data.table/base R, or implementing parallel processing. Covers profvis and bench usage, performance workflow, parallel processing with in_parallel(), data backend selection, modern purrr patterns (list_rbind, walk), and common performance anti-patterns to avoid.4---56# Optimizing R78This skill covers profiling, benchmarking, parallelization, and performance best practices for R.910## Core Principle1112**Profile before optimizing** - Use profvis and bench to identify real bottlenecks. Write readable code first, optimize only when necessary.1314## Profiling Tools Decision Matrix1516| Tool | Use When | Don't Use When | What It Shows |17|------|----------|----------------|---------------|18| **`profvis`** | Complex code, unknown bottlenecks | Simple functions, known issues | Time per line, call stack |19| **`bench::mark()`** | Comparing alternatives | Single approach | Relative performance, memory |20| **`system.time()`** | Quick checks | Detailed analysis | Total runtime only |21| **`Rprof()`** | Base R only environments | When profvis available | Raw profiling data |2223## Performance Workflow24251. **Profile first** - Find the actual bottlenecks262. **Focus on the slowest parts** - 80/20 rule273. **Benchmark alternatives** - For hot spots only284. **Consider tool trade-offs** - Based on bottleneck type2930See [profiling-workflow.md](references/profiling-workflow.md) for the complete workflow.3132## When Each Tool Helps vs Hurts3334### Parallel Processing (`in_parallel()`)3536**Helps when:**37- CPU-intensive computations38- Embarrassingly parallel problems39- Large datasets with independent operations40- I/O bound operations (file reading, API calls)4142**Hurts when:**43- Simple, fast operations (overhead > benefit)44- Memory-intensive operations (may cause thrashing)45- Operations requiring shared state46- Small datasets4748See [parallel-examples.md](references/parallel-examples.md) for decision points.4950### Data Backend Selection5152| Backend | Use When |53|---------|----------|54| **data.table** | Very large datasets (>1GB), complex grouping, maximum performance critical |55| **dplyr** | Readability priority, complex joins/window functions, moderate data (<100MB) |56| **base R** | No dependencies allowed, simple operations, teaching/learning |5758See [backend-selection.md](references/backend-selection.md) for guidance.5960## Profiling Best Practices61621. **Profile realistic data sizes** - Not toy examples632. **Profile multiple runs** - For stability643. **Check memory usage too** - Not just time654. **Profile realistic usage patterns** - Not isolated calls6667See [profiling-best-practices.md](references/profiling-best-practices.md) for examples.6869## Performance Anti-Patterns to Avoid7071- **Don't optimize without measuring** - Profile first72- **Don't over-engineer** - Complex optimizations for 1% gains73- **Don't assume** - "for loops are always slow" is a myth74- **Don't ignore readability costs** - Readable code with targeted optimizations7576See [performance-anti-patterns.md](references/performance-anti-patterns.md) for examples.7778## Modern purrr Patterns7980### Data Frame Binding (purrr 1.0+)8182| Superseded | Modern Replacement |83|------------|-------------------|84| `map_dfr(x, f)` | `map(x, f) \|> list_rbind()` |85| `map_dfc(x, f)` | `map(x, f) \|> list_cbind()` |86| `map2_dfr(x, y, f)` | `map2(x, y, f) \|> list_rbind()` |8788### Side Effects with `walk()`8990Use `walk()` and `walk2()` for side effects (file writing, plotting).9192### Parallel Processing (purrr 1.1.0+)9394Use `in_parallel()` with mirai for scaling across cores.9596See [purrr-patterns.md](references/purrr-patterns.md) for all patterns.9798## Backend Tools for Performance99100When speed is critical, consider:101- **vctrs** - Type-stable vector operations102- **rlang** - Metaprogramming103- **data.table** - Large data operations104105Profile to identify whether these tools will help your specific bottleneck.106107source: Sarah Johnson's gist https://gist.github.com/sj-io/3828d64d0969f2a0f05297e59e6c15ad
jeremy-allen