Pointblank

Pointblank

A data-validation library for Python. Define validation steps against a table, execute them with interrogate(), then inspect results as a rich tabular report or programmatically via pass/fail counts and data extracts.

Quick start

import pointblank as pb

validation = (
    pb.Validate(data=pb.load_dataset("small_table"))
    .col_vals_gt(columns="d", value=100)
    .col_vals_not_null(columns="date_time")
    .col_vals_in_set(columns="f", set=["low", "mid", "high"])
    .interrogate()
)

validation  # displays the HTML validation report

Skill directory structure

This skill ships with companion files for agent consumption:

skills/pointblank/
+-- SKILL.md                    <- This file
+-- references/
    +-- validation-methods.md   <- All col_vals_* / row / schema methods
    +-- column-selectors.md     <- col(), starts_with(), matches(), etc.
    +-- data-backends.md        <- Polars, Pandas, DuckDB, Ibis, files

When to use what

Core concepts

The Validate workflow

Every validation follows three phases:

1. Plan -- Create a Validate object with a data source and chain validation methods to define steps.
2. Interrogate -- Call .interrogate() to execute all steps against the data.
3. Report -- View results with the built-in HTML report (just evaluate the object), or extract results programmatically.

import pointblank as pb

validation = (
    pb.Validate(data=df, tbl_name="orders", label="Daily order check")
    .col_vals_gt(columns="amount", value=0)
    .col_vals_not_null(columns="customer_id")
    .col_vals_between(columns="quantity", left=1, right=1000)
    .interrogate()
)

Data backends

Pointblank works with multiple table types through the same API:

tbl = pb.connect_to_table("duckdb:///warehouse.db::sales")
validation = pb.Validate(data=tbl).col_vals_gt(columns="revenue", value=0).interrogate()

Column selectors

Instead of naming columns one by one, use selectors to target groups:

from pointblank import col, starts_with, ends_with, contains, matches, everything

# All columns starting with "price"
.col_vals_gt(columns=starts_with("price"), value=0)

# Combine selectors with operators
.col_vals_not_null(columns=starts_with("id") | ends_with("_key"))

# Exclude columns
.col_vals_not_null(columns=everything() - matches("_tmp$"))

Selectors: col(), starts_with(), ends_with(), contains(), matches(), everything(), first_n(), last_n().

Operators: & (and), | (or), - (difference), ~ (not).

Thresholds and actions

Set failure thresholds at three severity levels:

validation = (
    pb.Validate(
        data=df,
        thresholds=pb.Thresholds(warning=0.05, error=0.10, critical=0.25),
        actions=pb.Actions(
            warning="Step {step}: {col} has warnings",
            critical=pb.send_slack_notification(webhook_url="..."),
        ),
    )
    .col_vals_gt(columns="amount", value=0)
    .interrogate()
)

Threshold values: <1 = fraction of failing rows, >=1 = absolute count, True = any failure (equivalent to 1).

Per-step thresholds override the global setting.

Extracting results

validation.all_passed()          # True if every step passed
validation.n_passed(i=1)         # count of passing units in step 1
validation.f_failed(i=2)         # fraction of failing units in step 2

# Get the rows that failed step 1
extracts = validation.get_data_extracts(i=1, frame=True)

# Split data into passing and failing subsets
pass_df = validation.get_sundered_data(type="pass")
fail_df = validation.get_sundered_data(type="fail")

# JSON report for downstream systems
json_str = validation.get_json_report()

Reporting

# Full HTML report (default display)
validation.get_tabular_report()

# Per-step detail report
validation.get_step_report(i=1, limit=20)

# Customize report sections
validation.get_tabular_report(
    title="Nightly Checks",
    incl_header=True,
    incl_footer=True,
    incl_footer_timings=True,
)

Serialization

Save and reload validation objects for auditing or scheduling:

pb.write_file(validation, filename="nightly_check.pb")
restored = pb.read_file("nightly_check.pb")

Related skills

Gotchas

1. Call .interrogate() last. Validation methods only define steps; nothing executes until interrogate() is called.
2. Column selectors are case-insensitive by default. Pass case_sensitive=True to starts_with(), contains(), etc. if needed.
3. Threshold fractions vs counts. A threshold of 0.05 means 5% of rows may fail; a threshold of 5 means at most 5 rows may fail.
4. na_pass=False is the default. Null values count as failures unless you set na_pass=True.
5. Database tables require Ibis. Use pb.connect_to_table() with a connection string to get an Ibis table object.
6. File paths work directly. Pass "data.csv" or "data.parquet" as data= and Pointblank reads it automatically.
7. Reports render as HTML. In notebooks, just evaluate the Validate object. In scripts, call get_tabular_report() explicitly.