Binary Path Detection And Validation

binary-path-detection-and-validation

Summary

A configuration-driven mechanism for detecting and validating required binary tool installations (bowtie2, samtools, R, Python) by querying the system PATH and version constraints, then generating an environment-specific locked configuration file. This skill ensures reproducibility by recording exact tool locations and validating minimum version requirements before pipeline execution.

When to use

When setting up a bioinformatics pipeline (particularly Hi-C data processing) that depends on multiple external binaries with version constraints, and you need to configure the environment in a way that is both portable across systems and reproducible across runs. Specifically: (1) users provide a configuration template with optional explicit paths, (2) missing paths must be auto-detected from $PATH, (3) detected tools must meet minimum version thresholds (e.g., samtools ≥1.9, Python >3.7), and (4) the final configuration must be locked to prevent accidental runtime modification.

When NOT to use

• When all tool paths are already hardcoded in pipeline scripts or environment modules—this skill is unnecessary if binaries are already resolved.
• If the pipeline does not have version constraints or does not require specific binaries—the overhead of path detection and validation adds complexity without benefit.
• When deploying via containerized images (Docker, Singularity) where dependencies are pre-installed and environment is fixed—detection is redundant in a sealed container.

Inputs

• config-install.txt template file (plaintext with placeholder entries for tool paths and cluster scheduler)
• System environment variables ($PATH)
• User-specified tool paths (optional, in config-install.txt)

Outputs

• config-system.txt (read-only locked configuration file with resolved paths for all dependencies)
• Installed or verified binary tools (bowtie2, samtools, R, Python) at specified locations
• Installation logs and version validation records

How to apply

First, users edit a config-install.txt template to optionally specify explicit paths for PREFIX, BOWTIE2_PATH, SAMTOOLS_PATH, R_PATH, PYTHON_PATH, and CLUSTER_SYS; any unset entries trigger automated detection. For each undetected tool, query the system PATH using the 'which' command to locate the binary; if not found, attempt automatic installation (bowtie2 and samtools ≥1.9 are auto-installable). Once located (whether user-specified or auto-detected), validate that the installed version meets minimum requirements—samtools ≥1.9 and Python >3.7 are mandatory. Compile all validated paths and system parameters into a structured config-system.txt file with locked read-only permissions to prevent user modification during pipeline execution. Run 'make configure' followed by 'make CONFIG_SYS=config-install.txt install' to execute the full setup workflow.

Related tools

• bowtie2 (Short-read DNA sequence aligner; auto-installed if not detected during binary-path validation) — http://bowtie-bio.sourceforge.net/bowtie2/index.shtml
• samtools (SAM/BAM file manipulation and coordinate sorting; version ≥1.9 required and auto-installable if missing)
• Python (Runtime for iced ICE normalization module and Hi-C data processing scripts; version >3.7 required)
• R (Statistical computing environment for ggplot2 and RColorBrewer visualization packages) — http://www.r-project.org/
• iced (Python module implementing iterative correction and eigenvector decomposition (ICE) normalization of Hi-C contact matrices; must be independently installed) — https://github.com/hiclib/iced
• pysam (Python wrapper for samtools C-API to read and manipulate SAM/BAM alignments; version ≥0.15.4 required) — https://github.com/pysam-developers/pysam

Examples

tar -zxvf HiC-Pro-master.tar.gz && cd HiC-Pro-master && make configure && make CONFIG_SYS=config-install.txt install

Evaluation signals

• config-system.txt file exists, is readable, and contains non-empty valid paths for all mandatory dependencies (BOWTIE2_PATH, SAMTOOLS_PATH, R_PATH, PYTHON_PATH)
• Version validation passes for all detected binaries: samtools --version reports ≥1.9; python --version reports >3.7
• config-system.txt has read-only permissions (mode 444 or equivalent) to prevent accidental user modification
• No 'tool not found' or 'version mismatch' errors appear in installation logs when running 'make install'
• Subsequent pipeline invocations use the locked config-system.txt without re-detecting or re-validating paths, ensuring reproducibility

Limitations

• Automatic installation is limited to bowtie2 and samtools ≥1.9; other dependencies (R, Python, iced) must be pre-installed or manually specified in config-install.txt
• The 'which' command used for PATH queries may behave differently across Unix shells and OS distributions, potentially causing false negatives on non-standard systems
• Version detection relies on tool-specific version flag semantics (e.g., 'samtools --version', 'python --version'); tools with non-standard version output may fail validation
• Once config-system.txt is locked as read-only, correcting a path error requires manual unlock or reinstall—no in-place recovery mechanism is documented
• The skill does not validate that detected tools are functional beyond version checking (e.g., corrupt binaries, missing shared libraries, or insufficient file permissions are not caught)

Evidence

• [methods] Edit the config-install.txt file and set the paths. If not set, the dependencies will be sought in the $PATH: "Edit the config-install.txt file and set the paths. If not set, the dependencies will be sought in the $PATH"
• [methods] A couple of tools such as bowtie2 and samtools (>=1.9) can be automatically installed if not detected.: "A couple of tools such as bowtie2 and samtools (>=1.9) can be automatically installed if not detected."
• [methods] make CONFIG_SYS=config-install.txt install: "make CONFIG_SYS=config-install.txt install"
• [readme] Python (>3.7) with pysam (>=0.15.4), bx-python(>=0.8.8), numpy(>=1.18.1), and scipy(>=1.4.1) libraries: "Python (>3.7) with pysam (>=0.15.4), bx-python(>=0.8.8), numpy(>=1.18.1), and scipy(>=1.4.1) libraries"
• [readme] samtools (>1.9). Unix sort (which support -V option) is required!: "samtools (>1.9). Unix sort (which support -V option) is required!"