Pandera for Polars
Create executable Polars dataframe contracts whose backend, validation depth,
coercion, failure aggregation, and pipeline boundary are explicit.
Boundary
Use this skill only for Pandera's Polars backend. Import it as
pandera.polars; pandas, Ibis, PySpark, and Narwhals-backed behavior differs.
Use the Polars skill for transformation semantics and this skill for runtime
dataframe contracts. Do not replace ordinary Python object validation with a
one-row dataframe schema.
Know the objects and overloaded words
| Object |
Runtime meaning |
Use it for |
DataFrameSchema |
An executable schema object containing Polars column and dataframe checks. |
Dynamic/programmatic schemas and schema composition. |
Column |
A named column contract: dtype, nullability, requirement, uniqueness, coercion, and checks. |
Per-column structural and value rules. |
Check |
A predicate contract evaluated by the backend. |
Domain constraints not captured by dtype/nullability. |
DataFrameModel |
A class-declared schema compiled from annotations, Fields, checks, and config. |
Reusable named contracts with type-checker-friendly declarations. |
Field |
Declarative column constraints inside a DataFrameModel. |
Built-in comparisons, membership, aliases, nullable/unique behavior. |
PolarsData |
Custom-check input holding a LazyFrame and optional column key. |
Native vectorized Polars checks. |
SchemaError / SchemaErrors |
One validation failure or an aggregate of failures. |
Machine-readable failure handling and diagnostics. |
Two kinds of “lazy” must remain separate:
pl.LazyFrame is a deferred Polars query. Pandera's native Polars validation
checks schema-level properties by default and does not automatically execute
all data-level checks on an uncollected plan.
schema.validate(..., lazy=True) requests accumulation of multiple validation
failures before raising; it does not make eager validation computationally
lazy.
Read the schema and validation model before
choosing a schema style or claiming that values were checked.
Ordered workflow
- Recover the data contract: accepted frame type, ordered/required columns,
exact dtypes, nullable fields, uniqueness, extra-column policy, allowed
coercions, cross-column invariants, and failure interface.
- Import
pandera.polars as pa. Confirm the installed Pandera and Polars
versions before copying a backend feature or signature.
- Choose
DataFrameSchema for dynamic composition or DataFrameModel for a
stable named contract. Do not maintain both as independent sources of truth.
- Encode structural rules first, then built-in vectorized checks, then the
smallest native custom check that remains.
- Choose validation depth from the input object. If data-level checks are
required for a
LazyFrame, collect or otherwise establish a supported
execution boundary; do not report schema-only validation as full validation.
- Choose
coerce, strict, and lazy independently. Each changes a different
contract.
- Validate at ingress, after an untrusted/shape-changing stage, or before
egress—not after every expression by habit.
- Test one failure for every rule and inspect
failure_cases/error categories,
not exception prose alone.
Decision table
| Need |
Use |
Do not confuse it with |
| Programmatic/reusable schema value |
pa.DataFrameSchema |
A Python class instance model. |
| Declarative named dataframe contract |
pa.DataFrameModel |
Pydantic BaseModel. |
| Reject unspecified columns |
strict=True |
required=True, which concerns declared columns. |
| Drop unspecified columns intentionally |
Installed strict="filter" support |
Silent schema drift; test passthrough loss. |
| Convert compatible inputs |
coerce=True at the chosen scope |
Validation-only behavior; coercion mutates the returned representation. |
| Gather all failures |
validate(..., lazy=True) |
A Polars LazyFrame. |
| Vectorized custom column rule |
Check receiving PolarsData and returning Boolean LazyFrame output |
Python element callbacks. |
| Cross-column invariant |
Dataframe-level native check |
A column check that cannot see the needed peer fields. |
Read the operation map for schemas, models, checks,
decorators, and pipeline placement.
Canonical strict contract
import pandera.polars as pa
import polars as pl
ORDERS = pa.DataFrameSchema(
{
"order_id": pa.Column(
pl.Int64,
checks=pa.Check.ge(1),
nullable=False,
unique=True,
),
"country": pa.Column(
pl.String,
checks=pa.Check.isin(["DE", "FR", "NL"]),
nullable=False,
),
"amount": pa.Column(
pl.Float64,
checks=pa.Check.ge(0),
nullable=True,
),
},
strict=True,
coerce=False,
)
def validate_orders(frame: pl.DataFrame) -> pl.DataFrame:
return ORDERS.validate(frame, lazy=True)
This rejects extra columns, does not silently coerce identifiers or amounts,
checks all data-level rules on an eager frame, and aggregates failures. If the
boundary intentionally normalizes compatible types, turn on coercion and test
the returned dtypes and failed dirty values; do not use coercion to make an
unknown schema “pass.”
Custom checks without Python row paths
import pandera.polars as pa
import polars as pl
from pandera.polars import PolarsData
def start_before_end(data: PolarsData) -> pl.LazyFrame:
return data.lazyframe.select(pl.col("start") <= pl.col("end"))
INTERVALS = pa.DataFrameSchema(
{
"start": pa.Column(pl.Datetime),
"end": pa.Column(pl.Datetime),
},
checks=pa.Check(start_before_end),
)
A native custom check returns a Boolean LazyFrame shape accepted by the
backend. Avoid element_wise=True for a vectorizable condition; the Polars
backend implements elementwise Python callbacks through a slower row path.
High-risk rules
- Backend imports are part of correctness. Do not use top-level or
pandera.pandas examples in Polars code.
nullable, required, unique, strict, and coerce are independent.
State each required policy rather than relying on defaults.
- A schema validates its declared contract, not business completeness. Add
cross-column checks for relational invariants and tests for duplicate/null
combinations where needed.
- DataFrame validation can return a coerced/filtered frame. Use the returned
value; do not validate and then continue with the unvalidated original.
- For a
LazyFrame, establish whether schema-only validation is acceptable. If
row values must be proven, validate an eager boundary or a currently
documented supported full-data path.
- Treat validation errors as structured evidence. Preserve useful failure cases
without leaking sensitive source values in logs or API responses.
- Feature support differs by backend. Do not copy pandas-only index, groupby
check, parser, hypothesis, synthesis, or custom-registration patterns into a
Polars schema without current evidence.
Run the backend inspector, read version and backend grounding, and use the
validation test matrix.
Completion gate
Do not declare completion until the import selects the Polars backend; the
schema style has one source of truth; exact dtypes, required/null/unique/extra
and coercion policies are asserted; eager versus lazy-frame validation depth is
tested; vectorizable rules stay native; the returned validated frame is used;
every rule has a failing example; aggregated failures are asserted structurally;
and installed-version checks cover backend-specific syntax. Report any
schema-only LazyFrame validation or skipped dependency execution explicitly.
References
- Schema and validation model
- Operation map
- Version and backend grounding
- Testing Pandera Polars contracts
1---2name: pandera-polars3description: Use for writing, reviewing, debugging, or testing Pandera schemas and runtime validation for Polars DataFrame or LazyFrame pipelines installed with pandera[polars]. Trigger on pandera.polars DataFrameSchema, DataFrameModel, Column, Field, Check, PolarsData, decorators, coercion, strictness, lazy error collection, and validation-depth decisions. Do not use for pandas-backed Pandera, Pydantic object models, Polars transformations without Pandera, static dataframe typing alone, or generic data-quality platforms.4argument-hint: "[Pandera Polars schema, validation task, code, or failure]"5metadata:6 source: https://skillmd.com/skills/skillmds/pandera-polars7---8910# Pandera for Polars1112Create executable Polars dataframe contracts whose backend, validation depth,13coercion, failure aggregation, and pipeline boundary are explicit.1415## Boundary1617Use this skill only for Pandera's Polars backend. Import it as18`pandera.polars`; pandas, Ibis, PySpark, and Narwhals-backed behavior differs.19Use the Polars skill for transformation semantics and this skill for runtime20dataframe contracts. Do not replace ordinary Python object validation with a21one-row dataframe schema.2223## Know the objects and overloaded words2425| Object | Runtime meaning | Use it for |26|---|---|---|27| `DataFrameSchema` | An executable schema object containing Polars column and dataframe checks. | Dynamic/programmatic schemas and schema composition. |28| `Column` | A named column contract: dtype, nullability, requirement, uniqueness, coercion, and checks. | Per-column structural and value rules. |29| `Check` | A predicate contract evaluated by the backend. | Domain constraints not captured by dtype/nullability. |30| `DataFrameModel` | A class-declared schema compiled from annotations, `Field`s, checks, and config. | Reusable named contracts with type-checker-friendly declarations. |31| `Field` | Declarative column constraints inside a `DataFrameModel`. | Built-in comparisons, membership, aliases, nullable/unique behavior. |32| `PolarsData` | Custom-check input holding a `LazyFrame` and optional column key. | Native vectorized Polars checks. |33| `SchemaError` / `SchemaErrors` | One validation failure or an aggregate of failures. | Machine-readable failure handling and diagnostics. |3435Two kinds of “lazy” must remain separate:3637- `pl.LazyFrame` is a deferred Polars query. Pandera's native Polars validation38 checks schema-level properties by default and does not automatically execute39 all data-level checks on an uncollected plan.40- `schema.validate(..., lazy=True)` requests accumulation of multiple validation41 failures before raising; it does not make eager validation computationally42 lazy.4344Read [the schema and validation model](references/object-model.md) before45choosing a schema style or claiming that values were checked.4647## Ordered workflow48491. Recover the data contract: accepted frame type, ordered/required columns,50 exact dtypes, nullable fields, uniqueness, extra-column policy, allowed51 coercions, cross-column invariants, and failure interface.522. Import `pandera.polars as pa`. Confirm the installed Pandera and Polars53 versions before copying a backend feature or signature.543. Choose `DataFrameSchema` for dynamic composition or `DataFrameModel` for a55 stable named contract. Do not maintain both as independent sources of truth.564. Encode structural rules first, then built-in vectorized checks, then the57 smallest native custom check that remains.585. Choose validation depth from the input object. If data-level checks are59 required for a `LazyFrame`, collect or otherwise establish a supported60 execution boundary; do not report schema-only validation as full validation.616. Choose `coerce`, `strict`, and `lazy` independently. Each changes a different62 contract.637. Validate at ingress, after an untrusted/shape-changing stage, or before64 egress—not after every expression by habit.658. Test one failure for every rule and inspect `failure_cases`/error categories,66 not exception prose alone.6768## Decision table6970| Need | Use | Do not confuse it with |71|---|---|---|72| Programmatic/reusable schema value | `pa.DataFrameSchema` | A Python class instance model. |73| Declarative named dataframe contract | `pa.DataFrameModel` | Pydantic `BaseModel`. |74| Reject unspecified columns | `strict=True` | `required=True`, which concerns declared columns. |75| Drop unspecified columns intentionally | Installed `strict="filter"` support | Silent schema drift; test passthrough loss. |76| Convert compatible inputs | `coerce=True` at the chosen scope | Validation-only behavior; coercion mutates the returned representation. |77| Gather all failures | `validate(..., lazy=True)` | A Polars `LazyFrame`. |78| Vectorized custom column rule | `Check` receiving `PolarsData` and returning Boolean `LazyFrame` output | Python element callbacks. |79| Cross-column invariant | Dataframe-level native check | A column check that cannot see the needed peer fields. |8081Read [the operation map](references/operations.md) for schemas, models, checks,82decorators, and pipeline placement.8384## Canonical strict contract8586```python87import pandera.polars as pa88import polars as pl899091ORDERS = pa.DataFrameSchema(92 {93 "order_id": pa.Column(94 pl.Int64,95 checks=pa.Check.ge(1),96 nullable=False,97 unique=True,98 ),99 "country": pa.Column(100 pl.String,101 checks=pa.Check.isin(["DE", "FR", "NL"]),102 nullable=False,103 ),104 "amount": pa.Column(105 pl.Float64,106 checks=pa.Check.ge(0),107 nullable=True,108 ),109 },110 strict=True,111 coerce=False,112)113114115def validate_orders(frame: pl.DataFrame) -> pl.DataFrame:116 return ORDERS.validate(frame, lazy=True)117```118119This rejects extra columns, does not silently coerce identifiers or amounts,120checks all data-level rules on an eager frame, and aggregates failures. If the121boundary intentionally normalizes compatible types, turn on coercion and test122the returned dtypes and failed dirty values; do not use coercion to make an123unknown schema “pass.”124125## Custom checks without Python row paths126127```python128import pandera.polars as pa129import polars as pl130from pandera.polars import PolarsData131132133def start_before_end(data: PolarsData) -> pl.LazyFrame:134 return data.lazyframe.select(pl.col("start") <= pl.col("end"))135136137INTERVALS = pa.DataFrameSchema(138 {139 "start": pa.Column(pl.Datetime),140 "end": pa.Column(pl.Datetime),141 },142 checks=pa.Check(start_before_end),143)144```145146A native custom check returns a Boolean `LazyFrame` shape accepted by the147backend. Avoid `element_wise=True` for a vectorizable condition; the Polars148backend implements elementwise Python callbacks through a slower row path.149150## High-risk rules151152- Backend imports are part of correctness. Do not use top-level or153 `pandera.pandas` examples in Polars code.154- `nullable`, `required`, `unique`, `strict`, and `coerce` are independent.155 State each required policy rather than relying on defaults.156- A schema validates its declared contract, not business completeness. Add157 cross-column checks for relational invariants and tests for duplicate/null158 combinations where needed.159- DataFrame validation can return a coerced/filtered frame. Use the returned160 value; do not validate and then continue with the unvalidated original.161- For a `LazyFrame`, establish whether schema-only validation is acceptable. If162 row values must be proven, validate an eager boundary or a currently163 documented supported full-data path.164- Treat validation errors as structured evidence. Preserve useful failure cases165 without leaking sensitive source values in logs or API responses.166- Feature support differs by backend. Do not copy pandas-only index, groupby167 check, parser, hypothesis, synthesis, or custom-registration patterns into a168 Polars schema without current evidence.169170Run [the backend inspector](scripts/inspect_pandera_polars.py), read [version and backend grounding](references/version-grounding.md), and use [the171validation test matrix](references/testing.md).172173## Completion gate174175Do not declare completion until the import selects the Polars backend; the176schema style has one source of truth; exact dtypes, required/null/unique/extra177and coercion policies are asserted; eager versus lazy-frame validation depth is178tested; vectorizable rules stay native; the returned validated frame is used;179every rule has a failing example; aggregated failures are asserted structurally;180and installed-version checks cover backend-specific syntax. Report any181schema-only LazyFrame validation or skipped dependency execution explicitly.182183## References184185- [Schema and validation model](references/object-model.md)186- [Operation map](references/operations.md)187- [Version and backend grounding](references/version-grounding.md)188- [Testing Pandera Polars contracts](references/testing.md)