Ndp Github Workflow

NDP GitHub Workflow

What This Skill Does

Defines Trunk-Based Development (TBD) conventions for the Neural Data Platform. Everyone commits directly to main with small, frequent commits.

Always use this skill for:

• Writing commit messages
• Committing to main
• Tagging releases

Trunk-Based Development

Core Principles

1. Commit directly to main - No feature branches
2. Small, frequent commits - Multiple times per day
3. Always deployable - Every commit should build and pass tests
4. Feature tracking via commit scope - Use {type}(feature-id): format

Why TBD for NDP?

Commit Convention

Format

{type}({scope}): {description}

{optional body}

{optional footer}

Types

Scope

The scope identifies what is affected:

Rules

1. Type and scope are lowercase
2. Description is lowercase, no period at end
3. Description is imperative mood ("add" not "added")
4. Scope matches feature ID when working on a feature
5. Max 72 characters for first line
6. Every commit must build - cargo build must pass

Examples

# Feature work - commit directly to main
feat(dp-001): add duckdb silver layer views
feat(dp-001): implement pivot from long to wide format
fix(dp-001): correct timestamp conversion in bronze data
test(dp-001): add partition key regression tests

# Work in progress (builds but incomplete)
wip(dp-002): start timescaledb integration scaffold

# Component work (not feature-specific)
fix(storage): partition by stream_id instead of location_id
refactor(router): simplify validation logic
perf(parquet): reduce memory allocation in batch writes

# Infrastructure
chore(deploy): update docker-compose resource limits
docs(architecture): add silver layer design doc

Multi-line Commits

For complex changes, add a body:

git commit -m "feat(dp-001): route mqtt through ingestion router

- MQTT now goes through SourceManager like HTTP
- IngestionRouter adds stream_id tag to points
- ParquetStore uses stream_id for partition path

Fixes indoor air data writing to device MAC instead of stream name"

Feature Tracking

Features are tracked via commit scope and product/features/ directories, not branches.

Feature ID Format

Feature IDs follow {phase}-{NNN} pattern:

Starting a New Feature

# 1. Create feature directory (for SPARC docs)
mkdir -p product/features/dp-002/{specification,pseudocode,architecture,refinement,completion,bugs,reports}

# 2. Commit directly to main
git add product/features/dp-002/
git commit -m "chore(dp-002): initialize feature directory structure"
git push

# 3. Continue working, committing frequently
git commit -m "feat(dp-002): add timescaledb schema"
git push

git commit -m "feat(dp-002): implement etl from parquet"
git push

Feature Progress

Track progress via:

• Commit history: git log --oneline --grep="dp-001"
• STATUS.md: product/features/dp-001/STATUS.md
• Commit frequency: Multiple small commits per day

Daily Workflow

Before Starting Work

git pull origin main
cargo build
cargo test

During Work

# Make changes, then commit frequently
git add .
git commit -m "feat(dp-001): add silver layer view for indoor air"
git push

# More changes
git add .
git commit -m "fix(dp-001): correct column names in pivot"
git push

End of Day

# Ensure everything is pushed
git status
git push

# If work is incomplete but builds
git commit -m "wip(dp-001): partial grafana integration"
git push

Release Tagging

Version Format

v{major}.{minor}.{patch}

• Major: Breaking changes, architectural shifts
• Minor: New features, phase completions
• Patch: Bug fixes, small improvements

Tagging Process

# Ensure on main with latest
git pull origin main

# Verify build
cargo build
cargo test

# Create annotated tag
git tag -a v1.3.0 -m "Release v1.3.0: DP-001 DuckDB Analytics Layer

Features:
- DuckDB Silver layer with PIVOT views
- MQTT routing through IngestionRouter
- Grafana integration with SQLite export

Bug Fixes:
- Indoor air data now writes to air-quality directory"

# Push tag
git push origin v1.3.0

When to Use Branches (Exceptions)

PRs are optional and only for specific cases:

If You Do Use a Branch

# Create short-lived branch
git checkout -b experiment/try-new-approach

# Work on it
git commit -m "wip: experimental approach"

# Either merge quickly or abandon
git checkout main
git merge experiment/try-new-approach  # or git branch -D to abandon
git branch -d experiment/try-new-approach
git push

Quick Reference Card

Commit Prefixes

feat(scope):     # New feature
fix(scope):      # Bug fix
docs(scope):     # Documentation
test(scope):     # Tests
chore(scope):    # Maintenance
refactor(scope): # Code improvement
perf(scope):     # Performance
wip(scope):      # Work in progress

Daily Commands

git pull                    # Start of day
git add . && git commit     # After each logical change
git push                    # After each commit
git log --oneline -10       # Check recent history

Feature Commands

git log --oneline --grep="dp-001"    # See feature commits
git diff HEAD~5                       # Review recent changes
git tag -a v1.x.x -m "..."           # Release

Example Session

# Morning
git pull origin main
cargo build && cargo test

# Work
git commit -m "feat(dp-002): add timescaledb connection pool"
git push

git commit -m "feat(dp-002): implement hypertable creation"
git push

git commit -m "test(dp-002): add integration tests for etl"
git push

# End of day
git commit -m "wip(dp-002): partial continuous aggregate setup"
git push

Enforcement

All NDP agents MUST follow TBD:

1. Commit directly to main - No feature branches unless explicitly requested
2. Use conventional commits - {type}({scope}): {description}
3. Push frequently - After each logical change
4. Ensure builds pass - cargo build before commit
5. Track features via scope - Use feature ID in commit scope

If you see branch creation for normal work, suggest committing to main instead.