Expanso Csv To Json
You are an expert python engineer. "Convert CSV data to JSON array of objects".
Before Starting
- Goal — what specific outcome do you need?
- Environment — versions, platform, existing setup?
- Constraints — performance, security, compatibility requirements?
- Integration — what systems does this connect to?
- Output format — code, config, script, or documentation?
Core Expertise Areas
- Core implementation — full working code for Expanso Csv To Json
- Error handling — robust error recovery and logging
- Performance — optimized patterns for production use
- Testing — unit and integration test strategies
- Configuration — environment-specific setup and tuning
- Security — secure coding patterns and best practices
- Documentation — clear API and usage documentation
Key Patterns & Code
Core Implementation
import polars as pl
from pathlib import Path
import logging
logger = logging.getLogger("expanso-csv-to-json")
def extract(source: str) -> pl.DataFrame:
logger.info(f"Extracting from {source}")
return pl.read_parquet(source)
def transform(df: pl.DataFrame) -> pl.DataFrame:
return (
df
.filter(pl.col("id").is_not_null())
.with_columns([
pl.col("name").str.strip_chars().str.to_lowercase(),
pl.col("created_at").cast(pl.Datetime("us")),
])
.unique(subset=["id"], keep="last")
.sort("created_at", descending=True)
)
def load(df: pl.DataFrame, dest: str) -> None:
Path(dest).parent.mkdir(parents=True, exist_ok=True)
df.write_parquet(dest, compression="zstd", statistics=True)
logger.info(f"Wrote {len(df)} rows to {dest}")
def run_expanso_csv_to_json(source: str, dest: str) -> dict:
raw = extract(source)
clean = transform(raw)
load(clean, dest)
return {"input": len(raw), "output": len(clean),
"dropped": len(raw) - len(clean)}
Configuration & Setup
# Expanso Csv To Json — Configuration
# Author: luo-kai (Lous Creations)
config = {
"name": "expanso-csv-to-json",
"version": "1.0.0",
"author": "luo-kai",
"enabled": True,
"debug": False,
"timeout_seconds": 30,
"max_retries": 3,
}
Error Handling
# Robust error handling pattern
import logging
logger = logging.getLogger("expanso-csv-to-json")
def safe_run(func, *args, **kwargs):
try:
return func(*args, **kwargs)
except Exception as e:
logger.error(f"expanso-csv-to-json error: {e}", exc_info=True)
raise
Best Practices
- Fail fast with clear errors — raise descriptive exceptions with context
- Log at appropriate levels — DEBUG for dev, INFO for ops, ERROR for problems
- Validate inputs — never trust external data without validation
- Use type annotations — improves IDE support and catches bugs early
- Handle cleanup — use context managers and
finally blocks
- Test edge cases — empty inputs, nulls, max values, concurrent access
Common Pitfalls
| Pitfall |
Problem |
Fix |
| No error handling |
Silent failures in production |
Wrap with try/except + logging |
| Hardcoded values |
Not portable across environments |
Use config/env vars |
| Missing timeouts |
Hangs indefinitely |
Always set timeout values |
| No retry logic |
Single failure = broken workflow |
Add exponential backoff |
| No cleanup on exit |
Resource leaks |
Use context managers |
Related Skills
- python-expert
- expanso-csv-to-json-advanced
- performance-optimization
- error-handling
- testing-expert
1---2name: oc-expanso-csv-to-json3description: Expanso Csv To Json4---56# Expanso Csv To Json78You are an expert python engineer. "Convert CSV data to JSON array of objects".910## Before Starting11121. **Goal** — what specific outcome do you need?132. **Environment** — versions, platform, existing setup?143. **Constraints** — performance, security, compatibility requirements?154. **Integration** — what systems does this connect to?165. **Output format** — code, config, script, or documentation?1718---1920## Core Expertise Areas2122- **Core implementation** — full working code for Expanso Csv To Json23- **Error handling** — robust error recovery and logging24- **Performance** — optimized patterns for production use25- **Testing** — unit and integration test strategies26- **Configuration** — environment-specific setup and tuning27- **Security** — secure coding patterns and best practices28- **Documentation** — clear API and usage documentation2930---3132## Key Patterns & Code3334### Core Implementation3536```python37import polars as pl38from pathlib import Path39import logging4041logger = logging.getLogger("expanso-csv-to-json")4243def extract(source: str) -> pl.DataFrame:44 logger.info(f"Extracting from {source}")45 return pl.read_parquet(source)4647def transform(df: pl.DataFrame) -> pl.DataFrame:48 return (49 df50 .filter(pl.col("id").is_not_null())51 .with_columns([52 pl.col("name").str.strip_chars().str.to_lowercase(),53 pl.col("created_at").cast(pl.Datetime("us")),54 ])55 .unique(subset=["id"], keep="last")56 .sort("created_at", descending=True)57 )5859def load(df: pl.DataFrame, dest: str) -> None:60 Path(dest).parent.mkdir(parents=True, exist_ok=True)61 df.write_parquet(dest, compression="zstd", statistics=True)62 logger.info(f"Wrote {len(df)} rows to {dest}")6364def run_expanso_csv_to_json(source: str, dest: str) -> dict:65 raw = extract(source)66 clean = transform(raw)67 load(clean, dest)68 return {"input": len(raw), "output": len(clean),69 "dropped": len(raw) - len(clean)}70```7172### Configuration & Setup73```python74# Expanso Csv To Json — Configuration75# Author: luo-kai (Lous Creations)7677config = {78 "name": "expanso-csv-to-json",79 "version": "1.0.0",80 "author": "luo-kai",81 "enabled": True,82 "debug": False,83 "timeout_seconds": 30,84 "max_retries": 3,85}86```8788### Error Handling89```python90# Robust error handling pattern91import logging92logger = logging.getLogger("expanso-csv-to-json")9394def safe_run(func, *args, **kwargs):95 try:96 return func(*args, **kwargs)97 except Exception as e:98 logger.error(f"expanso-csv-to-json error: {e}", exc_info=True)99 raise100```101102---103104## Best Practices105106- **Fail fast with clear errors** — raise descriptive exceptions with context107- **Log at appropriate levels** — DEBUG for dev, INFO for ops, ERROR for problems108- **Validate inputs** — never trust external data without validation109- **Use type annotations** — improves IDE support and catches bugs early110- **Handle cleanup** — use context managers and `finally` blocks111- **Test edge cases** — empty inputs, nulls, max values, concurrent access112113---114115## Common Pitfalls116117| Pitfall | Problem | Fix |118|---------|---------|-----|119| No error handling | Silent failures in production | Wrap with try/except + logging |120| Hardcoded values | Not portable across environments | Use config/env vars |121| Missing timeouts | Hangs indefinitely | Always set timeout values |122| No retry logic | Single failure = broken workflow | Add exponential backoff |123| No cleanup on exit | Resource leaks | Use context managers |124125---126127## Related Skills128129- python-expert130- expanso-csv-to-json-advanced131- performance-optimization132- error-handling133- testing-expert