Deevsdeevs Agent System Polars Expertise

Polars

High-performance DataFrame library built on Apache Arrow. Supports Python and Rust with expression-based API, lazy evaluation, and automatic parallelization.

Quick Start

Python

uv pip install polars
# GPU support: uv pip install polars[gpu]

import polars as pl

# Eager: immediate execution
df = pl.DataFrame({"symbol": ["AAPL", "GOOG"], "price": [150.0, 140.0]})
df.filter(pl.col("price") > 145).select("symbol", "price")

# Lazy: optimized execution (preferred for large data)
lf = pl.scan_parquet("trades.parquet")
result = lf.filter(pl.col("volume") > 1000).group_by("symbol").agg(
    pl.col("price").mean().alias("avg_price")
).collect()

Rust

# Cargo.toml - select features you need
[dependencies]
polars = { version = "0.46", features = ["lazy", "parquet", "temporal"] }

use polars::prelude::*;

fn main() -> PolarsResult<()> {
    // Eager
    let df = df![
        "symbol" => ["AAPL", "GOOG"],
        "price" => [150.0, 140.0]
    ]?;

    // Lazy (preferred)
    let lf = LazyFrame::scan_parquet("trades.parquet", Default::default())?;
    let result = lf
        .filter(col("volume").gt(lit(1000)))
        .group_by([col("symbol")])
        .agg([col("price").mean().alias("avg_price")])
        .collect()?;
    Ok(())
}

Core Pattern: Expressions

Everything in Polars is an expression. Expressions are composable, lazy, and parallelized.

# Expression building blocks
pl.col("price")                      # column reference
pl.col("price") * pl.col("volume")   # arithmetic
pl.col("price").mean().over("symbol") # window function
pl.when(cond).then(a).otherwise(b)   # conditional

Expressions execute in contexts: select(), with_columns(), filter(), group_by().agg()

When to Use Lazy

Lazy benefits: predicate pushdown, projection pushdown, parallel execution, streaming.

Style: Use .alias() for Column Naming

Always use .alias("name") instead of name=expr kwargs:

# GOOD: Explicit .alias() - works everywhere, composable
df.with_columns(
    (pl.col("price") * pl.col("volume")).alias("value"),
    pl.col("price").mean().over("symbol").alias("avg_price")
)

# AVOID: Kwarg style - less flexible, doesn't chain
df.with_columns(
    value=pl.col("price") * pl.col("volume"),  # avoid
    avg_price=pl.col("price").mean().over("symbol")  # avoid
)

.alias() is explicit, chains with other methods, and works consistently in all contexts.

Anti-Patterns - AVOID

# BAD: Python functions kill parallelization
df.with_columns(pl.col("x").map_elements(lambda x: x * 2))  # SLOW

# GOOD: Native expressions are parallel
df.with_columns((pl.col("x") * 2).alias("x"))  # FAST

# BAD: Row iteration
for row in df.iter_rows():  # SLOW
    process(row)

# GOOD: Columnar operations
df.with_columns(process_expr)  # FAST

# BAD: Late projection
lf.filter(...).collect().select("a", "b")  # reads all columns

# GOOD: Early projection
lf.select("a", "b").filter(...).collect()  # reads only needed columns

Performance Checklist

• Using scan_* (lazy) for large files?
• Projecting columns early in pipeline?
• Using native expressions (no map_elements)?
• Categorical dtype for low-cardinality strings?
• Appropriate integer sizes (i32 vs i64)?
• Streaming for out-of-memory data? (collect(engine="streaming"))

Reference Navigator

Python References

Rust References

Shared References

Migration Guides

Time Series / Financial Data Quick Patterns

# OHLCV resampling
df.group_by_dynamic("timestamp", every="1m").agg(
    pl.col("price").first().alias("open"),
    pl.col("price").max().alias("high"),
    pl.col("price").min().alias("low"),
    pl.col("price").last().alias("close"),
    pl.col("volume").sum()
)

# Rolling statistics
df.with_columns(
    pl.col("price").rolling_mean(window_size=20).alias("sma_20"),
    pl.col("price").rolling_std(window_size=20).alias("volatility")
)

# As-of join for market data alignment
trades.join_asof(quotes, on="timestamp", by="symbol", strategy="backward")

Load python/best_practices.md for comprehensive time series patterns.

Runnable Examples

Development Tips

Use LSP for navigating Polars code:

• Python: Pyright/Pylance provides excellent type inference for Polars expressions
• Rust: rust-analyzer understands Polars types and expression chains

LSP operations like goToDefinition and hover help explore Polars API without leaving the editor.

Converted and distributed by TomeVault — claim your Tome and manage your conversions.