Portolan CLI

Portolan CLI

Publish and manage cloud-native geospatial data catalogs.

Portolan is a CLI for publishing and managing cloud-native geospatial data catalogs. It orchestrates format conversion (GeoParquet, COG), versioning, and sync to object storage (S3, GCS, Azure) — no running servers, just static files.

Specification: The Portolan Spec defines catalog structure, format requirements, and versioning rules. When in doubt about requirements, read the spec files directly (core.md, structure.md, versions.md, formats/vector.md, formats/raster.md, formats/pointcloud.md, best-practices.md) for the most up-to-date information.

Key concepts:

• STAC (SpatioTemporal Asset Catalog) — The catalog metadata spec. Portolan is a STAC profile, not a competing spec.
• GeoParquet — Cloud-optimized vector data (columnar, spatial indexing)
• COG (Cloud-Optimized GeoTIFF) — Cloud-optimized raster data (HTTP range requests)
• COPC (Cloud-Optimized Point Cloud) — Cloud-optimized point cloud data
• PMTiles — Cloud-optimized vector/raster tiles for web map rendering
• versions.json — Single source of truth for version history, sync state, and checksums

Installation

pipx install portolan-cli    # Isolated (recommended)
pip install portolan-cli     # Or with pip
uv pip install portolan-cli  # Or with uv

If portolan is not installed, guide the user through installation before proceeding.

Catalog Structure

Portolan catalogs use a flat hierarchy with STAC metadata:

project/
├── .portolan/
│   ├── config.yaml              # Internal config (sentinel file)
│   └── state.json               # Local sync state
├── catalog.json                 # STAC Catalog (root metadata)
├── versions.json                # Catalog-level version tracking
└── {collection_id}/
    ├── collection.json          # STAC Collection metadata
    ├── versions.json            # Collection-level versioning
    ├── {data}.parquet           # Single-file collection asset
    ├── {data}.pmtiles           # Visualization derivative
    └── thumbnail.png            # Preview image

Single-file collections (one GeoParquet/COG) use collection-level assets directly — no item directory needed.

Partitioned collections (>2GB) use item subdirectories per partition.

All STAC links use relative paths (SELF_CONTAINED) — catalogs are portable across hosting locations.

CLI Commands

portolan init

Initialize a new Portolan catalog.

portolan init                       # Initialize in current directory
portolan init --auto                # Skip prompts, use defaults
portolan init --title "My Catalog"  # Set title
portolan init /path/to/data --auto  # Initialize in specific directory

portolan scan

Scan a directory for geospatial files and potential issues.

portolan scan                         # Scan current directory
portolan scan --json                  # JSON output
portolan scan /data/geospatial
portolan scan /large/tree --max-depth=2

portolan check

Validate a Portolan catalog or check files for cloud-native status.

portolan check                        # Validate all (metadata + geo-assets)
portolan check --metadata             # Validate metadata only
portolan check --geo-assets           # Check geo-assets only
portolan check --fix                  # Fix both metadata and geo-assets

portolan add

Track files in the catalog.

portolan add demographics/census.parquet
portolan add file1.geojson file2.geojson   # Add multiple files
portolan add imagery/                      # Add all files in directory
portolan add .                             # Add all files in catalog

portolan push

Push local catalog changes to cloud object storage.

portolan push s3://mybucket/catalog --collection demographics
portolan push gs://mybucket/catalog -c imagery --dry-run
portolan push s3://mybucket/catalog
portolan push --dry-run  # Uses configured remote

portolan pull

Pull updates from a remote catalog.

portolan pull s3://mybucket/my-catalog --collection demographics
portolan pull s3://mybucket/catalog -c imagery --dry-run
portolan pull s3://mybucket/catalog
portolan pull s3://mybucket/catalog --workers 4

portolan sync

Sync local catalog with remote storage (pull + push).

portolan sync s3://mybucket/catalog --collection demographics
portolan sync s3://mybucket/catalog -c imagery --dry-run
portolan sync s3://mybucket/catalog -c data --fix --force

portolan clone

Clone a remote catalog to a local directory.

portolan clone s3://mybucket/my-catalog
portolan clone s3://mybucket/my-catalog .
portolan clone s3://mybucket/catalog -c demographics

portolan status

Show local vs remote version state for collections.

portolan status                    # Status for all collections
portolan status -c demographics    # Status for one collection
portolan status --offline          # Skip remote check
portolan status --json             # JSON output for agents

portolan list

List all files in the catalog with tracking status.

portolan list                           # List all files with status
portolan list --collection demographics # Filter by collection
portolan list --tracked-only            # Show only tracked files

portolan info

Show information about a file, collection, or catalog.

portolan info demographics/census.parquet  # File info
portolan info demographics/                # Collection info
portolan info                              # Catalog info
portolan info demographics/census.parquet --json

portolan rm

Remove files from tracking.

portolan rm --keep imagery/old_data.tif     # Safe: untrack only
portolan rm --dry-run vectors/              # Preview
portolan rm -f demographics/census.parquet  # Force delete and untrack

portolan partition

Partition a large GeoParquet file for better query performance.

portolan partition buildings.parquet --preview
portolan partition buildings.parquet output/
portolan partition buildings.parquet output/ --target-rows 50000

portolan extract

Extract data from external sources into Portolan catalogs.

portolan extract arcgis https://services.arcgis.com/.../FeatureServer ./output
portolan extract arcgis URL --layers "Census*" --dry-run
portolan extract arcgis URL --filter "sdn_*" --resume

portolan metadata

Manage catalog metadata for README generation.

portolan metadata init                # Create template at catalog root
portolan metadata init demographics   # Create template for collection
portolan metadata validate            # Validate metadata.yaml

portolan readme

Generate README.md from STAC metadata and metadata.yaml.

portolan readme                    # Generate at catalog root
portolan readme demographics       # Generate for collection
portolan readme --stdout           # Print without writing
portolan readme --check            # CI mode: exit 1 if stale

portolan stac-geoparquet

Generate items.parquet for efficient STAC queries.

portolan stac-geoparquet                       # Generate for ALL collections
portolan stac-geoparquet -c landsat            # Generate for one collection
portolan stac-geoparquet -c imagery --dry-run  # Preview

portolan config

Manage catalog configuration.

portolan config set backend iceberg
portolan config get remote
portolan config list

portolan clean

Remove all Portolan metadata while preserving data files.

portolan clean           # Remove all metadata
portolan clean --dry-run # Preview what would be removed

Common Workflows

Publishing a New Catalog

portolan init --title "My Geospatial Data"
portolan scan /data/geospatial
portolan check --geo-assets --fix    # Convert to cloud-native formats
portolan add demographics/
portolan push s3://mybucket/my-catalog --collection demographics

Full Sync (Recommended)

Single command: pull -> init -> scan -> check -> push:

portolan sync s3://mybucket/my-catalog --collection demographics
portolan sync s3://mybucket/my-catalog -c demographics --fix

Versioning

Versions follow Semantic Versioning per collection:

• Major: Breaking changes (schema changes, column removals, CRS changes)
• Minor: New features (new columns, new items)
• Patch: Data updates (same schema, new data)

Each collection's versions.json tracks version history with SHA-256 checksums per asset.

JSON Output

All commands support --json or --format json:

portolan scan . --json
portolan check --format json
portolan --format json init --auto

Consistent envelope: {"success": true, "command": "...", "data": {...}, "errors": []}.

Python API

from portolan_cli import Catalog, FormatType, detect_format

catalog = Catalog("/path/to/data")
format_type = detect_format("data.parquet")  # Returns FormatType.GEOPARQUET

Troubleshooting