LatchBio Integration
Overview
Latch is a Python framework for building and deploying bioinformatics workflows as serverless pipelines. Built on Flyte, create workflows with @workflow/@task decorators, manage cloud data with LatchFile/LatchDir, configure resources, and integrate Nextflow/Snakemake pipelines.
Core Capabilities
The Latch platform provides four main areas of functionality:
1. Workflow Creation and Deployment
- Define serverless workflows using Python decorators
- Support for native Python, Nextflow, and Snakemake pipelines
- Automatic containerization with Docker
- Auto-generated no-code user interfaces
- Version control and reproducibility
2. Data Management
- Cloud storage abstractions (LatchFile, LatchDir)
- Structured data organization with Registry (Projects → Tables → Records)
- Type-safe data operations with links and enums
- Automatic file transfer between local and cloud
- Glob pattern matching for file selection
3. Resource Configuration
- Pre-configured task decorators (@small_task, @large_task, @small_gpu_task, @large_gpu_task)
- Custom resource specifications (CPU, memory, GPU, storage)
- GPU support (K80, V100, A100)
- Timeout and storage configuration
- Cost optimization strategies
4. Verified Workflows
- Production-ready pre-built pipelines
- Bulk RNA-seq, DESeq2, pathway analysis
- AlphaFold and ColabFold for protein structure prediction
- Single-cell tools (ArchR, scVelo, emptyDropsR)
- CRISPR analysis, phylogenetics, and more
Quick Start
Installation and Setup
# Install Latch SDK
python3 -m uv pip install latch
# Login to Latch
latch login
# Initialize a new workflow
latch init my-workflow
# Register workflow to platform
latch register my-workflow
Prerequisites:
- Docker installed and running
- Latch account credentials
- Python 3.8+
Basic Workflow Example
from latch import workflow, small_task
from latch.types import LatchFile
@small_task
def process_file(input_file: LatchFile) -> LatchFile:
"""Process a single file"""
# Processing logic
return output_file
@workflow
def my_workflow(input_file: LatchFile) -> LatchFile:
"""
My bioinformatics workflow
Args:
input_file: Input data file
"""
return process_file(input_file=input_file)
When to Use This Skill
This skill should be used when encountering any of the following scenarios:
Workflow Development:
- "Create a Latch workflow for RNA-seq analysis"
- "Deploy my pipeline to Latch"
- "Convert my Nextflow pipeline to Latch"
- "Add GPU support to my workflow"
- Working with
@workflow, @task decorators
Data Management:
- "Organize my sequencing data in Latch Registry"
- "How do I use LatchFile and LatchDir?"
- "Set up sample tracking in Latch"
- Working with
latch:/// paths
Resource Configuration:
- "Configure GPU for AlphaFold on Latch"
- "My task is running out of memory"
- "How do I optimize workflow costs?"
- Working with task decorators
Verified Workflows:
- "Run AlphaFold on Latch"
- "Use DESeq2 for differential expression"
- "Available pre-built workflows"
- Using
latch.verified module
Detailed Documentation
This skill includes comprehensive reference documentation organized by capability:
references/workflow-creation.md
Read this for:
- Creating and registering workflows
- Task definition and decorators
- Supporting Python, Nextflow, Snakemake
- Launch plans and conditional sections
- Workflow execution (CLI and programmatic)
- Multi-step and parallel pipelines
- Troubleshooting registration issues
Key topics:
latch init and latch register commands
@workflow and @task decorators
- LatchFile and LatchDir basics
- Type annotations and docstrings
- Launch plans with preset parameters
- Conditional UI sections
references/data-management.md
Read this for:
- Cloud storage with LatchFile and LatchDir
- Registry system (Projects, Tables, Records)
- Linked records and relationships
- Enum and typed columns
- Bulk operations and transactions
- Integration with workflows
- Account and workspace management
Key topics:
latch:/// path format
- File transfer and glob patterns
- Creating and querying Registry tables
- Column types (string, number, file, link, enum)
- Record CRUD operations
- Workflow-Registry integration
references/resource-configuration.md
Read this for:
- Task resource decorators
- Custom CPU, memory, GPU configuration
- GPU types (K80, V100, A100)
- Timeout and storage settings
- Resource optimization strategies
- Cost-effective workflow design
- Monitoring and debugging
Key topics:
@small_task, @large_task, @small_gpu_task, @large_gpu_task
@custom_task with precise specifications
- Multi-GPU configuration
- Resource selection by workload type
- Platform limits and quotas
references/verified-workflows.md
Read this for:
- Pre-built production workflows
- Bulk RNA-seq and DESeq2
- AlphaFold and ColabFold
- Single-cell analysis (ArchR, scVelo)
- CRISPR editing analysis
- Pathway enrichment
- Integration with custom workflows
Key topics:
latch.verified module imports
- Available verified workflows
- Workflow parameters and options
- Combining verified and custom steps
- Version management
Common Workflow Patterns
Complete RNA-seq Pipeline
from latch import workflow, small_task, large_task
from latch.types import LatchFile, LatchDir
@small_task
def quality_control(fastq: LatchFile) -> LatchFile:
"""Run FastQC"""
return qc_output
@large_task
def alignment(fastq: LatchFile, genome: str) -> LatchFile:
"""STAR alignment"""
return bam_output
@small_task
def quantification(bam: LatchFile) -> LatchFile:
"""featureCounts"""
return counts
@workflow
def rnaseq_pipeline(
input_fastq: LatchFile,
genome: str,
output_dir: LatchDir
) -> LatchFile:
"""RNA-seq analysis pipeline"""
qc = quality_control(fastq=input_fastq)
aligned = alignment(fastq=qc, genome=genome)
return quantification(bam=aligned)
GPU-Accelerated Workflow
from latch import workflow, small_task, large_gpu_task
from latch.types import LatchFile
@small_task
def preprocess(input_file: LatchFile) -> LatchFile:
"""Prepare data"""
return processed
@large_gpu_task
def gpu_computation(data: LatchFile) -> LatchFile:
"""GPU-accelerated analysis"""
return results
@workflow
def gpu_pipeline(input_file: LatchFile) -> LatchFile:
"""Pipeline with GPU tasks"""
preprocessed = preprocess(input_file=input_file)
return gpu_computation(data=preprocessed)
Registry-Integrated Workflow
from latch import workflow, small_task
from latch.registry.table import Table
from latch.registry.record import Record
from latch.types import LatchFile
@small_task
def process_and_track(sample_id: str, table_id: str) -> str:
"""Process sample and update Registry"""
# Get sample from registry
table = Table.get(table_id=table_id)
records = Record.list(table_id=table_id, filter={"sample_id": sample_id})
sample = records[0]
# Process
input_file = sample.values["fastq_file"]
output = process(input_file)
# Update registry
sample.update(values={"status": "completed", "result": output})
return "Success"
@workflow
def registry_workflow(sample_id: str, table_id: str):
"""Workflow integrated with Registry"""
return process_and_track(sample_id=sample_id, table_id=table_id)
Best Practices
Workflow Design
- Use type annotations for all parameters
- Write clear docstrings (appear in UI)
- Start with standard task decorators, scale up if needed
- Break complex workflows into modular tasks
- Implement proper error handling
Data Management
- Use consistent folder structures
- Define Registry schemas before bulk entry
- Use linked records for relationships
- Store metadata in Registry for traceability
Resource Configuration
- Right-size resources (don't over-allocate)
- Use GPU only when algorithms support it
- Monitor execution metrics and optimize
- Design for parallel execution when possible
Development Workflow
- Test locally with Docker before registration
- Use version control for workflow code
- Document resource requirements
- Profile workflows to determine actual needs
Troubleshooting
Common Issues
Registration Failures:
- Ensure Docker is running
- Check authentication with
latch login
- Verify all dependencies in Dockerfile
- Use
--verbose flag for detailed logs
Resource Problems:
- Out of memory: Increase memory in task decorator
- Timeouts: Increase timeout parameter
- Storage issues: Increase ephemeral storage_gib
Data Access:
- Use correct
latch:/// path format
- Verify file exists in workspace
- Check permissions for shared workspaces
Type Errors:
- Add type annotations to all parameters
- Use LatchFile/LatchDir for file/directory parameters
- Ensure workflow return type matches actual return
Additional Resources
Support
For issues or questions:
- Check documentation links above
- Search GitHub issues
- Ask in Slack community
- Contact support@latch.bio
1---2name: latchbio-integration3description: Latch platform for bioinformatics workflows. Build pipelines with Latch SDK, @workflow/@task decorators, deploy serverless workflows, LatchFile/LatchDir, Nextflow/Snakemake integration.4category: biology5license: Unknown6metadata:7 source: https://skillmd.com/skills/synthetic-sciences/latchbio-integration8 upstream: K-Dense-AI/scientific-agent-skills9 upstream-url: https://github.com/K-Dense-AI/scientific-agent-skills10 upstream-path: scientific-skills/latchbio-integration11 upstream-license: MIT12 upstream-relationship: derived13 adapted-by: Synthetic Sciences14 skill-author: K-Dense Inc.15---1617# LatchBio Integration1819## Overview2021Latch is a Python framework for building and deploying bioinformatics workflows as serverless pipelines. Built on Flyte, create workflows with @workflow/@task decorators, manage cloud data with LatchFile/LatchDir, configure resources, and integrate Nextflow/Snakemake pipelines.2223## Core Capabilities2425The Latch platform provides four main areas of functionality:2627### 1. Workflow Creation and Deployment28- Define serverless workflows using Python decorators29- Support for native Python, Nextflow, and Snakemake pipelines30- Automatic containerization with Docker31- Auto-generated no-code user interfaces32- Version control and reproducibility3334### 2. Data Management35- Cloud storage abstractions (LatchFile, LatchDir)36- Structured data organization with Registry (Projects → Tables → Records)37- Type-safe data operations with links and enums38- Automatic file transfer between local and cloud39- Glob pattern matching for file selection4041### 3. Resource Configuration42- Pre-configured task decorators (@small_task, @large_task, @small_gpu_task, @large_gpu_task)43- Custom resource specifications (CPU, memory, GPU, storage)44- GPU support (K80, V100, A100)45- Timeout and storage configuration46- Cost optimization strategies4748### 4. Verified Workflows49- Production-ready pre-built pipelines50- Bulk RNA-seq, DESeq2, pathway analysis51- AlphaFold and ColabFold for protein structure prediction52- Single-cell tools (ArchR, scVelo, emptyDropsR)53- CRISPR analysis, phylogenetics, and more5455## Quick Start5657### Installation and Setup5859```bash60# Install Latch SDK61python3 -m uv pip install latch6263# Login to Latch64latch login6566# Initialize a new workflow67latch init my-workflow6869# Register workflow to platform70latch register my-workflow71```7273**Prerequisites:**74- Docker installed and running75- Latch account credentials76- Python 3.8+7778### Basic Workflow Example7980```python81from latch import workflow, small_task82from latch.types import LatchFile8384@small_task85def process_file(input_file: LatchFile) -> LatchFile:86 """Process a single file"""87 # Processing logic88 return output_file8990@workflow91def my_workflow(input_file: LatchFile) -> LatchFile:92 """93 My bioinformatics workflow9495 Args:96 input_file: Input data file97 """98 return process_file(input_file=input_file)99```100101## When to Use This Skill102103This skill should be used when encountering any of the following scenarios:104105**Workflow Development:**106- "Create a Latch workflow for RNA-seq analysis"107- "Deploy my pipeline to Latch"108- "Convert my Nextflow pipeline to Latch"109- "Add GPU support to my workflow"110- Working with `@workflow`, `@task` decorators111112**Data Management:**113- "Organize my sequencing data in Latch Registry"114- "How do I use LatchFile and LatchDir?"115- "Set up sample tracking in Latch"116- Working with `latch:///` paths117118**Resource Configuration:**119- "Configure GPU for AlphaFold on Latch"120- "My task is running out of memory"121- "How do I optimize workflow costs?"122- Working with task decorators123124**Verified Workflows:**125- "Run AlphaFold on Latch"126- "Use DESeq2 for differential expression"127- "Available pre-built workflows"128- Using `latch.verified` module129130## Detailed Documentation131132This skill includes comprehensive reference documentation organized by capability:133134### references/workflow-creation.md135**Read this for:**136- Creating and registering workflows137- Task definition and decorators138- Supporting Python, Nextflow, Snakemake139- Launch plans and conditional sections140- Workflow execution (CLI and programmatic)141- Multi-step and parallel pipelines142- Troubleshooting registration issues143144**Key topics:**145- `latch init` and `latch register` commands146- `@workflow` and `@task` decorators147- LatchFile and LatchDir basics148- Type annotations and docstrings149- Launch plans with preset parameters150- Conditional UI sections151152### references/data-management.md153**Read this for:**154- Cloud storage with LatchFile and LatchDir155- Registry system (Projects, Tables, Records)156- Linked records and relationships157- Enum and typed columns158- Bulk operations and transactions159- Integration with workflows160- Account and workspace management161162**Key topics:**163- `latch:///` path format164- File transfer and glob patterns165- Creating and querying Registry tables166- Column types (string, number, file, link, enum)167- Record CRUD operations168- Workflow-Registry integration169170### references/resource-configuration.md171**Read this for:**172- Task resource decorators173- Custom CPU, memory, GPU configuration174- GPU types (K80, V100, A100)175- Timeout and storage settings176- Resource optimization strategies177- Cost-effective workflow design178- Monitoring and debugging179180**Key topics:**181- `@small_task`, `@large_task`, `@small_gpu_task`, `@large_gpu_task`182- `@custom_task` with precise specifications183- Multi-GPU configuration184- Resource selection by workload type185- Platform limits and quotas186187### references/verified-workflows.md188**Read this for:**189- Pre-built production workflows190- Bulk RNA-seq and DESeq2191- AlphaFold and ColabFold192- Single-cell analysis (ArchR, scVelo)193- CRISPR editing analysis194- Pathway enrichment195- Integration with custom workflows196197**Key topics:**198- `latch.verified` module imports199- Available verified workflows200- Workflow parameters and options201- Combining verified and custom steps202- Version management203204## Common Workflow Patterns205206### Complete RNA-seq Pipeline207208```python209from latch import workflow, small_task, large_task210from latch.types import LatchFile, LatchDir211212@small_task213def quality_control(fastq: LatchFile) -> LatchFile:214 """Run FastQC"""215 return qc_output216217@large_task218def alignment(fastq: LatchFile, genome: str) -> LatchFile:219 """STAR alignment"""220 return bam_output221222@small_task223def quantification(bam: LatchFile) -> LatchFile:224 """featureCounts"""225 return counts226227@workflow228def rnaseq_pipeline(229 input_fastq: LatchFile,230 genome: str,231 output_dir: LatchDir232) -> LatchFile:233 """RNA-seq analysis pipeline"""234 qc = quality_control(fastq=input_fastq)235 aligned = alignment(fastq=qc, genome=genome)236 return quantification(bam=aligned)237```238239### GPU-Accelerated Workflow240241```python242from latch import workflow, small_task, large_gpu_task243from latch.types import LatchFile244245@small_task246def preprocess(input_file: LatchFile) -> LatchFile:247 """Prepare data"""248 return processed249250@large_gpu_task251def gpu_computation(data: LatchFile) -> LatchFile:252 """GPU-accelerated analysis"""253 return results254255@workflow256def gpu_pipeline(input_file: LatchFile) -> LatchFile:257 """Pipeline with GPU tasks"""258 preprocessed = preprocess(input_file=input_file)259 return gpu_computation(data=preprocessed)260```261262### Registry-Integrated Workflow263264```python265from latch import workflow, small_task266from latch.registry.table import Table267from latch.registry.record import Record268from latch.types import LatchFile269270@small_task271def process_and_track(sample_id: str, table_id: str) -> str:272 """Process sample and update Registry"""273 # Get sample from registry274 table = Table.get(table_id=table_id)275 records = Record.list(table_id=table_id, filter={"sample_id": sample_id})276 sample = records[0]277278 # Process279 input_file = sample.values["fastq_file"]280 output = process(input_file)281282 # Update registry283 sample.update(values={"status": "completed", "result": output})284 return "Success"285286@workflow287def registry_workflow(sample_id: str, table_id: str):288 """Workflow integrated with Registry"""289 return process_and_track(sample_id=sample_id, table_id=table_id)290```291292## Best Practices293294### Workflow Design2951. Use type annotations for all parameters2962. Write clear docstrings (appear in UI)2973. Start with standard task decorators, scale up if needed2984. Break complex workflows into modular tasks2995. Implement proper error handling300301### Data Management3026. Use consistent folder structures3037. Define Registry schemas before bulk entry3048. Use linked records for relationships3059. Store metadata in Registry for traceability306307### Resource Configuration30810. Right-size resources (don't over-allocate)30911. Use GPU only when algorithms support it31012. Monitor execution metrics and optimize31113. Design for parallel execution when possible312313### Development Workflow31414. Test locally with Docker before registration31515. Use version control for workflow code31616. Document resource requirements31717. Profile workflows to determine actual needs318319## Troubleshooting320321### Common Issues322323**Registration Failures:**324- Ensure Docker is running325- Check authentication with `latch login`326- Verify all dependencies in Dockerfile327- Use `--verbose` flag for detailed logs328329**Resource Problems:**330- Out of memory: Increase memory in task decorator331- Timeouts: Increase timeout parameter332- Storage issues: Increase ephemeral storage_gib333334**Data Access:**335- Use correct `latch:///` path format336- Verify file exists in workspace337- Check permissions for shared workspaces338339**Type Errors:**340- Add type annotations to all parameters341- Use LatchFile/LatchDir for file/directory parameters342- Ensure workflow return type matches actual return343344## Additional Resources345346- **Official Documentation**: https://docs.latch.bio347- **GitHub Repository**: https://github.com/latchbio/latch348- **Slack Community**: Join Latch SDK workspace349- **API Reference**: https://docs.latch.bio/api/latch.html350- **Blog**: https://blog.latch.bio351352## Support353354For issues or questions:3551. Check documentation links above3562. Search GitHub issues3573. Ask in Slack community3584. Contact support@latch.bio359