Results for “dataset-preparation”

Generate realistic dummy datasets for testing with customizable columns, constraints, and output formats (CSV, JSON, SQL, Python script).

Profiles datasets automatically to assess data quality, structure, and completeness, generating reports with ydata-profiling, pandera, or manual pandas methods.

Cleans credit risk data and screens variables for pre-loan modeling through an 11-step pipeline covering missing rate calculation, IV/PSI filtering, null importance denoising, and correlation removal.

Guides database design, implementation, optimization, migrations, data pipelines, and operations for SQL, NoSQL, and modern data platforms.

Designs and implements big data architectures, processes large-scale datasets with distributed systems, and optimizes data pipelines for throughput using Hadoop, Spark, and cloud platforms.

Guide through a structured data analysis workflow: define the question, validate data quality, select the appropriate analytical method, and produce decision-ready findings with caveats.