LatchBio Integration
Overview
Latch is a Python framework for building and deploying bioinformatics workflows as serverless pipelines. Built on Flyte, create workflows with @workflow/@task decorators, manage cloud data with LatchFile/LatchDir, configure resources, and integrate Nextflow/Snakemake pipelines.
Core Capabilities
The Latch platform provides four main areas of functionality:
1. Workflow Creation and Deployment
- Define serverless workflows using Python decorators
- Support for native Python, Nextflow, and Snakemake pipelines
- Automatic containerization with Docker
- Auto-generated no-code user interfaces
- Version control and reproducibility
2. Data Management
- Cloud storage abstractions (LatchFile, LatchDir)
- Structured data organization with Registry (Projects → Tables → Records)
- Type-safe data operations with links and enums
- Automatic file transfer between local and cloud
- Glob pattern matching for file selection
3. Resource Configuration
- Pre-configured task decorators (@small_task, @large_task, @small_gpu_task, @large_gpu_task)
- Custom resource specifications (CPU, memory, GPU, storage)
- GPU support (K80, V100, A100)
- Timeout and storage configuration
- Cost optimization strategies
4. Verified Workflows
- Production-ready pre-built pipelines
- Bulk RNA-seq, DESeq2, pathway analysis
- AlphaFold and ColabFold for protein structure prediction
- Single-cell tools (ArchR, scVelo, emptyDropsR)
- CRISPR analysis, phylogenetics, and more
Quick Start
Installation and Setup
# Install Latch SDK
uv pip install latch
# Login to Latch
latch login
# Initialize a new workflow
latch init my-workflow
# Register workflow to platform
latch register my-workflow
Prerequisites:
- Docker installed and running
- Latch account credentials
- Python 3.8+
Basic Workflow Example
from latch import workflow, small_task
from latch.types import LatchFile
@small_task
def process_file(input_file: LatchFile) -> LatchFile:
"""Process a single file"""
# Processing logic
return output_file
@workflow
def my_workflow(input_file: LatchFile) -> LatchFile:
"""
My bioinformatics workflow
Args:
input_file: Input data file
"""
return process_file(input_file=input_file)
When to Use This Skill
This skill should be used when encountering any of the following scenarios:
Workflow Development:
- "Create a Latch workflow for RNA-seq analysis"
- "Deploy my pipeline to Latch"
- "Convert my Nextflow pipeline to Latch"
- "Add GPU support to my workflow"
- Working with
@workflow, @task decorators
Data Management:
- "Organize my sequencing data in Latch Registry"
- "How do I use LatchFile and LatchDir?"
- "Set up sample tracking in Latch"
- Working with
latch:/// paths
Resource Configuration:
- "Configure GPU for AlphaFold on Latch"
- "My task is running out of memory"
- "How do I optimize workflow costs?"
- Working with task decorators
Verified Workflows:
- "Run AlphaFold on Latch"
- "Use DESeq2 for differential expression"
- "Available pre-built workflows"
- Using
latch.verified module
Detailed Documentation
This skill includes comprehensive reference documentation organized by capability:
references/workflow-creation.md
Read this for:
- Creating and registering workflows
- Task definition and decorators
- Supporting Python, Nextflow, Snakemake
- Launch plans and conditional sections
- Workflow execution (CLI and programmatic)
- Multi-step and parallel pipelines
- Troubleshooting registration issues
Key topics:
latch init and latch register commands
@workflow and @task decorators
- LatchFile and LatchDir basics
- Type annotations and docstrings
- Launch plans with preset parameters
- Conditional UI sections
references/data-management.md
Read this for:
- Cloud storage with LatchFile and LatchDir
- Registry system (Projects, Tables, Records)
- Linked records and relationships
- Enum and typed columns
- Bulk operations and transactions
- Integration with workflows
- Account and workspace management
Key topics:
latch:/// path format
- File transfer and glob patterns
- Creating and querying Registry tables
- Column types (string, number, file, link, enum)
- Record CRUD operations
- Workflow-Registry integration
references/resource-configuration.md
Read this for:
- Task resource decorators
- Custom CPU, memory, GPU configuration
- GPU types (K80, V100, A100)
- Timeout and storage settings
- Resource optimization strategies
- Cost-effective workflow design
- Monitoring and debugging
Key topics:
@small_task, @large_task, @small_gpu_task, @large_gpu_task
@custom_task with precise specifications
- Multi-GPU configuration
- Resource selection by workload type
- Platform limits and quotas
references/verified-workflows.md
Read this for:
- Pre-built production workflows
- Bulk RNA-seq and DESeq2
- AlphaFold and ColabFold
- Single-cell analysis (ArchR, scVelo)
- CRISPR editing analysis
- Pathway enrichment
- Integration with custom workflows
Key topics:
latch.verified module imports
- Available verified workflows
- Workflow parameters and options
- Combining verified and custom steps
- Version management
Common Workflow Patterns
Complete RNA-seq Pipeline
from latch import workflow, small_task, large_task
from latch.types import LatchFile, LatchDir
@small_task
def quality_control(fastq: LatchFile) -> LatchFile:
"""Run FastQC"""
return qc_output
@large_task
def alignment(fastq: LatchFile, genome: str) -> LatchFile:
"""STAR alignment"""
return bam_output
@small_task
def quantification(bam: LatchFile) -> LatchFile:
"""featureCounts"""
return counts
@workflow
def rnaseq_pipeline(
input_fastq: LatchFile,
genome: str,
output_dir: LatchDir
) -> LatchFile:
"""RNA-seq analysis pipeline"""
qc = quality_control(fastq=input_fastq)
aligned = alignment(fastq=qc, genome=genome)
return quantification(bam=aligned)
GPU-Accelerated Workflow
from latch import workflow, small_task, large_gpu_task
from latch.types import LatchFile
@small_task
def preprocess(input_file: LatchFile) -> LatchFile:
"""Prepare data"""
return processed
@large_gpu_task
def gpu_computation(data: LatchFile) -> LatchFile:
"""GPU-accelerated analysis"""
return results
@workflow
def gpu_pipeline(input_file: LatchFile) -> LatchFile:
"""Pipeline with GPU tasks"""
preprocessed = preprocess(input_file=input_file)
return gpu_computation(data=preprocessed)
Registry-Integrated Workflow
from latch import workflow, small_task
from latch.registry.table import Table
from latch.registry.record import Record
from latch.types import LatchFile
@small_task
def process_and_track(sample_id: str, table_id: str) -> str:
"""Process sample and update Registry"""
# Get sample from registry
table = Table.get(table_id=table_id)
records = Record.list(table_id=table_id, filter={"sample_id": sample_id})
sample = records[0]
# Process
input_file = sample.values["fastq_file"]
output = process(input_file)
# Update registry
sample.update(values={"status": "completed", "result": output})
return "Success"
@workflow
def registry_workflow(sample_id: str, table_id: str):
"""Workflow integrated with Registry"""
return process_and_track(sample_id=sample_id, table_id=table_id)
Best Practices
Workflow Design
- Use type annotations for all parameters
- Write clear docstrings (appear in UI)
- Start with standard task decorators, scale up if needed
- Break complex workflows into modular tasks
- Implement proper error handling
Data Management
- Use consistent folder structures
- Define Registry schemas before bulk entry
- Use linked records for relationships
- Store metadata in Registry for traceability
Resource Configuration
- Right-size resources (don't over-allocate)
- Use GPU only when algorithms support it
- Monitor execution metrics and optimize
- Design for parallel execution when possible
Development Workflow
- Test locally with Docker before registration
- Use version control for workflow code
- Document resource requirements
- Profile workflows to determine actual needs
Troubleshooting
Common Issues
Registration Failures:
- Ensure Docker is running
- Check authentication with
latch login
- Verify all dependencies in Dockerfile
- Use
--verbose flag for detailed logs
Resource Problems:
- Out of memory: Increase memory in task decorator
- Timeouts: Increase timeout parameter
- Storage issues: Increase ephemeral storage_gib
Data Access:
- Use correct
latch:/// path format
- Verify file exists in workspace
- Check permissions for shared workspaces
Type Errors:
- Add type annotations to all parameters
- Use LatchFile/LatchDir for file/directory parameters
- Ensure workflow return type matches actual return
Additional Resources
Support
For issues or questions:
- Check documentation links above
- Search GitHub issues
- Ask in Slack community
- Contact support@latch.bio
1---2name: latchbio-integration3description: Latch platform for bioinformatics workflows. Build pipelines with Latch SDK, @workflow/@task decorators, deploy serverless workflows, LatchFile/LatchDir, Nextflow/Snakemake integration.4license: MIT5metadata:6 source: https://skillmd.com/skills/stanfish06/latchbio-integration7 version: "1.0"8 skill-author: K-Dense Inc.9---1011# LatchBio Integration1213## Overview1415Latch is a Python framework for building and deploying bioinformatics workflows as serverless pipelines. Built on Flyte, create workflows with @workflow/@task decorators, manage cloud data with LatchFile/LatchDir, configure resources, and integrate Nextflow/Snakemake pipelines.1617## Core Capabilities1819The Latch platform provides four main areas of functionality:2021### 1. Workflow Creation and Deployment22- Define serverless workflows using Python decorators23- Support for native Python, Nextflow, and Snakemake pipelines24- Automatic containerization with Docker25- Auto-generated no-code user interfaces26- Version control and reproducibility2728### 2. Data Management29- Cloud storage abstractions (LatchFile, LatchDir)30- Structured data organization with Registry (Projects → Tables → Records)31- Type-safe data operations with links and enums32- Automatic file transfer between local and cloud33- Glob pattern matching for file selection3435### 3. Resource Configuration36- Pre-configured task decorators (@small_task, @large_task, @small_gpu_task, @large_gpu_task)37- Custom resource specifications (CPU, memory, GPU, storage)38- GPU support (K80, V100, A100)39- Timeout and storage configuration40- Cost optimization strategies4142### 4. Verified Workflows43- Production-ready pre-built pipelines44- Bulk RNA-seq, DESeq2, pathway analysis45- AlphaFold and ColabFold for protein structure prediction46- Single-cell tools (ArchR, scVelo, emptyDropsR)47- CRISPR analysis, phylogenetics, and more4849## Quick Start5051### Installation and Setup5253```bash54# Install Latch SDK55uv pip install latch5657# Login to Latch58latch login5960# Initialize a new workflow61latch init my-workflow6263# Register workflow to platform64latch register my-workflow65```6667**Prerequisites:**68- Docker installed and running69- Latch account credentials70- Python 3.8+7172### Basic Workflow Example7374```python75from latch import workflow, small_task76from latch.types import LatchFile7778@small_task79def process_file(input_file: LatchFile) -> LatchFile:80 """Process a single file"""81 # Processing logic82 return output_file8384@workflow85def my_workflow(input_file: LatchFile) -> LatchFile:86 """87 My bioinformatics workflow8889 Args:90 input_file: Input data file91 """92 return process_file(input_file=input_file)93```9495## When to Use This Skill9697This skill should be used when encountering any of the following scenarios:9899**Workflow Development:**100- "Create a Latch workflow for RNA-seq analysis"101- "Deploy my pipeline to Latch"102- "Convert my Nextflow pipeline to Latch"103- "Add GPU support to my workflow"104- Working with `@workflow`, `@task` decorators105106**Data Management:**107- "Organize my sequencing data in Latch Registry"108- "How do I use LatchFile and LatchDir?"109- "Set up sample tracking in Latch"110- Working with `latch:///` paths111112**Resource Configuration:**113- "Configure GPU for AlphaFold on Latch"114- "My task is running out of memory"115- "How do I optimize workflow costs?"116- Working with task decorators117118**Verified Workflows:**119- "Run AlphaFold on Latch"120- "Use DESeq2 for differential expression"121- "Available pre-built workflows"122- Using `latch.verified` module123124## Detailed Documentation125126This skill includes comprehensive reference documentation organized by capability:127128### references/workflow-creation.md129**Read this for:**130- Creating and registering workflows131- Task definition and decorators132- Supporting Python, Nextflow, Snakemake133- Launch plans and conditional sections134- Workflow execution (CLI and programmatic)135- Multi-step and parallel pipelines136- Troubleshooting registration issues137138**Key topics:**139- `latch init` and `latch register` commands140- `@workflow` and `@task` decorators141- LatchFile and LatchDir basics142- Type annotations and docstrings143- Launch plans with preset parameters144- Conditional UI sections145146### references/data-management.md147**Read this for:**148- Cloud storage with LatchFile and LatchDir149- Registry system (Projects, Tables, Records)150- Linked records and relationships151- Enum and typed columns152- Bulk operations and transactions153- Integration with workflows154- Account and workspace management155156**Key topics:**157- `latch:///` path format158- File transfer and glob patterns159- Creating and querying Registry tables160- Column types (string, number, file, link, enum)161- Record CRUD operations162- Workflow-Registry integration163164### references/resource-configuration.md165**Read this for:**166- Task resource decorators167- Custom CPU, memory, GPU configuration168- GPU types (K80, V100, A100)169- Timeout and storage settings170- Resource optimization strategies171- Cost-effective workflow design172- Monitoring and debugging173174**Key topics:**175- `@small_task`, `@large_task`, `@small_gpu_task`, `@large_gpu_task`176- `@custom_task` with precise specifications177- Multi-GPU configuration178- Resource selection by workload type179- Platform limits and quotas180181### references/verified-workflows.md182**Read this for:**183- Pre-built production workflows184- Bulk RNA-seq and DESeq2185- AlphaFold and ColabFold186- Single-cell analysis (ArchR, scVelo)187- CRISPR editing analysis188- Pathway enrichment189- Integration with custom workflows190191**Key topics:**192- `latch.verified` module imports193- Available verified workflows194- Workflow parameters and options195- Combining verified and custom steps196- Version management197198## Common Workflow Patterns199200### Complete RNA-seq Pipeline201202```python203from latch import workflow, small_task, large_task204from latch.types import LatchFile, LatchDir205206@small_task207def quality_control(fastq: LatchFile) -> LatchFile:208 """Run FastQC"""209 return qc_output210211@large_task212def alignment(fastq: LatchFile, genome: str) -> LatchFile:213 """STAR alignment"""214 return bam_output215216@small_task217def quantification(bam: LatchFile) -> LatchFile:218 """featureCounts"""219 return counts220221@workflow222def rnaseq_pipeline(223 input_fastq: LatchFile,224 genome: str,225 output_dir: LatchDir226) -> LatchFile:227 """RNA-seq analysis pipeline"""228 qc = quality_control(fastq=input_fastq)229 aligned = alignment(fastq=qc, genome=genome)230 return quantification(bam=aligned)231```232233### GPU-Accelerated Workflow234235```python236from latch import workflow, small_task, large_gpu_task237from latch.types import LatchFile238239@small_task240def preprocess(input_file: LatchFile) -> LatchFile:241 """Prepare data"""242 return processed243244@large_gpu_task245def gpu_computation(data: LatchFile) -> LatchFile:246 """GPU-accelerated analysis"""247 return results248249@workflow250def gpu_pipeline(input_file: LatchFile) -> LatchFile:251 """Pipeline with GPU tasks"""252 preprocessed = preprocess(input_file=input_file)253 return gpu_computation(data=preprocessed)254```255256### Registry-Integrated Workflow257258```python259from latch import workflow, small_task260from latch.registry.table import Table261from latch.registry.record import Record262from latch.types import LatchFile263264@small_task265def process_and_track(sample_id: str, table_id: str) -> str:266 """Process sample and update Registry"""267 # Get sample from registry268 table = Table.get(table_id=table_id)269 records = Record.list(table_id=table_id, filter={"sample_id": sample_id})270 sample = records[0]271272 # Process273 input_file = sample.values["fastq_file"]274 output = process(input_file)275276 # Update registry277 sample.update(values={"status": "completed", "result": output})278 return "Success"279280@workflow281def registry_workflow(sample_id: str, table_id: str):282 """Workflow integrated with Registry"""283 return process_and_track(sample_id=sample_id, table_id=table_id)284```285286## Best Practices287288### Workflow Design2891. Use type annotations for all parameters2902. Write clear docstrings (appear in UI)2913. Start with standard task decorators, scale up if needed2924. Break complex workflows into modular tasks2935. Implement proper error handling294295### Data Management2966. Use consistent folder structures2977. Define Registry schemas before bulk entry2988. Use linked records for relationships2999. Store metadata in Registry for traceability300301### Resource Configuration30210. Right-size resources (don't over-allocate)30311. Use GPU only when algorithms support it30412. Monitor execution metrics and optimize30513. Design for parallel execution when possible306307### Development Workflow30814. Test locally with Docker before registration30915. Use version control for workflow code31016. Document resource requirements31117. Profile workflows to determine actual needs312313## Troubleshooting314315### Common Issues316317**Registration Failures:**318- Ensure Docker is running319- Check authentication with `latch login`320- Verify all dependencies in Dockerfile321- Use `--verbose` flag for detailed logs322323**Resource Problems:**324- Out of memory: Increase memory in task decorator325- Timeouts: Increase timeout parameter326- Storage issues: Increase ephemeral storage_gib327328**Data Access:**329- Use correct `latch:///` path format330- Verify file exists in workspace331- Check permissions for shared workspaces332333**Type Errors:**334- Add type annotations to all parameters335- Use LatchFile/LatchDir for file/directory parameters336- Ensure workflow return type matches actual return337338## Additional Resources339340- **Official Documentation**: https://docs.latch.bio341- **GitHub Repository**: https://github.com/latchbio/latch342- **Slack Community**: Join Latch SDK workspace343- **API Reference**: https://docs.latch.bio/api/latch.html344- **Blog**: https://blog.latch.bio345346## Support347348For issues or questions:3491. Check documentation links above3502. Search GitHub issues3513. Ask in Slack community3524. Contact support@latch.bio353