General Data Analysis Skills Index

General Data Analysis Skills

Cross-cutting skills for workspace hygiene, environment setup, and computational performance.

Available Skills

Workspace File Organization

Give every analysis task its own descriptively-named folder (notebook, figures, data, reports inside it) — never scatter files at the workspace root or dump figures into internal dirs.

Skill file: file_organization.md

Environment Management

Detect and configure the best available environment manager (Conda/Mamba/venv) for reproducible Python environments.

Skill file: environment_management.md

Parallel Computing & Performance

Multi-core CPU, GPU acceleration, and memory optimization strategies for large-scale data analysis.

Skill file: parallel_computing.md

HPC Data Transfer via SSH

Transfer data to/from HPC clusters with interactive authentication (password + Duo/MFA) using pexpect and SSH ControlMaster.

Skill file: hpc_data_transfer.md

When to use:

• Downloading data from Stanford Sherlock or similar HPC clusters
• Need to handle Duo two-factor authentication programmatically
• Bulk data transfer with rsync over persistent SSH connections