LatchBio Integration
Overview
Latch is a Python framework for building and deploying bioinformatics workflows as serverless pipelines. Built on Flyte, create workflows with @workflow/@task decorators, manage cloud data with LatchFile/LatchDir, configure resources, and integrate Nextflow/Snakemake pipelines.
Core Capabilities
The Latch platform provides four main areas of functionality:
1. Workflow Creation and Deployment
- Define serverless workflows using Python decorators
- Support for native Python, Nextflow, and Snakemake pipelines
- Automatic containerization with Docker
- Auto-generated no-code user interfaces
- Version control and reproducibility
2. Data Management
- Cloud storage abstractions (LatchFile, LatchDir)
- Structured data organization with Registry (Projects → Tables → Records)
- Type-safe data operations with links and enums
- Automatic file transfer between local and cloud
- Glob pattern matching for file selection
3. Resource Configuration
- Pre-configured task decorators (@small_task, @large_task, @small_gpu_task, @large_gpu_task)
- Custom resource specifications (CPU, memory, GPU, storage)
- GPU support (K80, V100, A100)
- Timeout and storage configuration
- Cost optimization strategies
4. Verified Workflows
- Production-ready pre-built pipelines
- Bulk RNA-seq, DESeq2, pathway analysis
- AlphaFold and ColabFold for protein structure prediction
- Single-cell tools (ArchR, scVelo, emptyDropsR)
- CRISPR analysis, phylogenetics, and more
Quick Start
Installation and Setup
# Install Latch SDK
python3 -m uv pip install latch
# Login to Latch
latch login
# Initialize a new workflow
latch init my-workflow
# Register workflow to platform
latch register my-workflow
Prerequisites:
- Docker installed and running
- Latch account credentials
- Python 3.8+
Basic Workflow Example
from latch import workflow, small_task
from latch.types import LatchFile
@small_task
def process_file(input_file: LatchFile) -> LatchFile:
"""Process a single file"""
# Processing logic
return output_file
@workflow
def my_workflow(input_file: LatchFile) -> LatchFile:
"""
My bioinformatics workflow
Args:
input_file: Input data file
"""
return process_file(input_file=input_file)
When to Use This Skill
This skill should be used when encountering any of the following scenarios:
Workflow Development:
- "Create a Latch workflow for RNA-seq analysis"
- "Deploy my pipeline to Latch"
- "Convert my Nextflow pipeline to Latch"
- "Add GPU support to my workflow"
- Working with
@workflow, @task decorators
Data Management:
- "Organize my sequencing data in Latch Registry"
- "How do I use LatchFile and LatchDir?"
- "Set up sample tracking in Latch"
- Working with
latch:/// paths
Resource Configuration:
- "Configure GPU for AlphaFold on Latch"
- "My task is running out of memory"
- "How do I optimize workflow costs?"
- Working with task decorators
Verified Workflows:
- "Run AlphaFold on Latch"
- "Use DESeq2 for differential expression"
- "Available pre-built workflows"
- Using
latch.verified module
Detailed Documentation
This skill includes comprehensive reference documentation organized by capability:
references/workflow-creation.md
Read this for:
- Creating and registering workflows
- Task definition and decorators
- Supporting Python, Nextflow, Snakemake
- Launch plans and conditional sections
- Workflow execution (CLI and programmatic)
- Multi-step and parallel pipelines
- Troubleshooting registration issues
Key topics:
latch init and latch register commands
@workflow and @task decorators
- LatchFile and LatchDir basics
- Type annotations and docstrings
- Launch plans with preset parameters
- Conditional UI sections
references/data-management.md
Read this for:
- Cloud storage with LatchFile and LatchDir
- Registry system (Projects, Tables, Records)
- Linked records and relationships
- Enum and typed columns
- Bulk operations and transactions
- Integration with workflows
- Account and workspace management
Key topics:
latch:/// path format
- File transfer and glob patterns
- Creating and querying Registry tables
- Column types (string, number, file, link, enum)
- Record CRUD operations
- Workflow-Registry integration
references/resource-configuration.md
Read this for:
- Task resource decorators
- Custom CPU, memory, GPU configuration
- GPU types (K80, V100, A100)
- Timeout and storage settings
- Resource optimization strategies
- Cost-effective workflow design
- Monitoring and debugging
Key topics:
@small_task, @large_task, @small_gpu_task, @large_gpu_task
@custom_task with precise specifications
- Multi-GPU configuration
- Resource selection by workload type
- Platform limits and quotas
references/verified-workflows.md
Read this for:
- Pre-built production workflows
- Bulk RNA-seq and DESeq2
- AlphaFold and ColabFold
- Single-cell analysis (ArchR, scVelo)
- CRISPR editing analysis
- Pathway enrichment
- Integration with custom workflows
Key topics:
latch.verified module imports
- Available verified workflows
- Workflow parameters and options
- Combining verified and custom steps
- Version management
Common Workflow Patterns
Complete RNA-seq Pipeline
from latch import workflow, small_task, large_task
from latch.types import LatchFile, LatchDir
@small_task
def quality_control(fastq: LatchFile) -> LatchFile:
"""Run FastQC"""
return qc_output
@large_task
def alignment(fastq: LatchFile, genome: str) -> LatchFile:
"""STAR alignment"""
return bam_output
@small_task
def quantification(bam: LatchFile) -> LatchFile:
"""featureCounts"""
return counts
@workflow
def rnaseq_pipeline(
input_fastq: LatchFile,
genome: str,
output_dir: LatchDir
) -> LatchFile:
"""RNA-seq analysis pipeline"""
qc = quality_control(fastq=input_fastq)
aligned = alignment(fastq=qc, genome=genome)
return quantification(bam=aligned)
GPU-Accelerated Workflow
from latch import workflow, small_task, large_gpu_task
from latch.types import LatchFile
@small_task
def preprocess(input_file: LatchFile) -> LatchFile:
"""Prepare data"""
return processed
@large_gpu_task
def gpu_computation(data: LatchFile) -> LatchFile:
"""GPU-accelerated analysis"""
return results
@workflow
def gpu_pipeline(input_file: LatchFile) -> LatchFile:
"""Pipeline with GPU tasks"""
preprocessed = preprocess(input_file=input_file)
return gpu_computation(data=preprocessed)
Registry-Integrated Workflow
from latch import workflow, small_task
from latch.registry.table import Table
from latch.registry.record import Record
from latch.types import LatchFile
@small_task
def process_and_track(sample_id: str, table_id: str) -> str:
"""Process sample and update Registry"""
# Get sample from registry
table = Table.get(table_id=table_id)
records = Record.list(table_id=table_id, filter={"sample_id": sample_id})
sample = records[0]
# Process
input_file = sample.values["fastq_file"]
output = process(input_file)
# Update registry
sample.update(values={"status": "completed", "result": output})
return "Success"
@workflow
def registry_workflow(sample_id: str, table_id: str):
"""Workflow integrated with Registry"""
return process_and_track(sample_id=sample_id, table_id=table_id)
Best Practices
Workflow Design
- Use type annotations for all parameters
- Write clear docstrings (appear in UI)
- Start with standard task decorators, scale up if needed
- Break complex workflows into modular tasks
- Implement proper error handling
Data Management
- Use consistent folder structures
- Define Registry schemas before bulk entry
- Use linked records for relationships
- Store metadata in Registry for traceability
Resource Configuration
- Right-size resources (don't over-allocate)
- Use GPU only when algorithms support it
- Monitor execution metrics and optimize
- Design for parallel execution when possible
Development Workflow
- Test locally with Docker before registration
- Use version control for workflow code
- Document resource requirements
- Profile workflows to determine actual needs
Troubleshooting
Common Issues
Registration Failures:
- Ensure Docker is running
- Check authentication with
latch login
- Verify all dependencies in Dockerfile
- Use
--verbose flag for detailed logs
Resource Problems:
- Out of memory: Increase memory in task decorator
- Timeouts: Increase timeout parameter
- Storage issues: Increase ephemeral storage_gib
Data Access:
- Use correct
latch:/// path format
- Verify file exists in workspace
- Check permissions for shared workspaces
Type Errors:
- Add type annotations to all parameters
- Use LatchFile/LatchDir for file/directory parameters
- Ensure workflow return type matches actual return
Additional Resources
Support
For issues or questions:
- Check documentation links above
- Search GitHub issues
- Ask in Slack community
- Contact support@latch.bio
1---2name: "latchbio-integration"3description: "Latch platform for bioinformatics workflows. Build pipelines with Latch SDK, @workflow/@task decorators, deploy serverless workflows, LatchFile/LatchDir, Nextflow/Snakemake integration."4metadata:5 source: https://skillmd.com/skills/viggyv/latchbio-integration6---78# LatchBio Integration910## Overview1112Latch is a Python framework for building and deploying bioinformatics workflows as serverless pipelines. Built on Flyte, create workflows with @workflow/@task decorators, manage cloud data with LatchFile/LatchDir, configure resources, and integrate Nextflow/Snakemake pipelines.1314## Core Capabilities1516The Latch platform provides four main areas of functionality:1718### 1. Workflow Creation and Deployment19- Define serverless workflows using Python decorators20- Support for native Python, Nextflow, and Snakemake pipelines21- Automatic containerization with Docker22- Auto-generated no-code user interfaces23- Version control and reproducibility2425### 2. Data Management26- Cloud storage abstractions (LatchFile, LatchDir)27- Structured data organization with Registry (Projects → Tables → Records)28- Type-safe data operations with links and enums29- Automatic file transfer between local and cloud30- Glob pattern matching for file selection3132### 3. Resource Configuration33- Pre-configured task decorators (@small_task, @large_task, @small_gpu_task, @large_gpu_task)34- Custom resource specifications (CPU, memory, GPU, storage)35- GPU support (K80, V100, A100)36- Timeout and storage configuration37- Cost optimization strategies3839### 4. Verified Workflows40- Production-ready pre-built pipelines41- Bulk RNA-seq, DESeq2, pathway analysis42- AlphaFold and ColabFold for protein structure prediction43- Single-cell tools (ArchR, scVelo, emptyDropsR)44- CRISPR analysis, phylogenetics, and more4546## Quick Start4748### Installation and Setup4950```bash51# Install Latch SDK52python3 -m uv pip install latch5354# Login to Latch55latch login5657# Initialize a new workflow58latch init my-workflow5960# Register workflow to platform61latch register my-workflow62```6364**Prerequisites:**65- Docker installed and running66- Latch account credentials67- Python 3.8+6869### Basic Workflow Example7071```python72from latch import workflow, small_task73from latch.types import LatchFile7475@small_task76def process_file(input_file: LatchFile) -> LatchFile:77 """Process a single file"""78 # Processing logic79 return output_file8081@workflow82def my_workflow(input_file: LatchFile) -> LatchFile:83 """84 My bioinformatics workflow8586 Args:87 input_file: Input data file88 """89 return process_file(input_file=input_file)90```9192## When to Use This Skill9394This skill should be used when encountering any of the following scenarios:9596**Workflow Development:**97- "Create a Latch workflow for RNA-seq analysis"98- "Deploy my pipeline to Latch"99- "Convert my Nextflow pipeline to Latch"100- "Add GPU support to my workflow"101- Working with `@workflow`, `@task` decorators102103**Data Management:**104- "Organize my sequencing data in Latch Registry"105- "How do I use LatchFile and LatchDir?"106- "Set up sample tracking in Latch"107- Working with `latch:///` paths108109**Resource Configuration:**110- "Configure GPU for AlphaFold on Latch"111- "My task is running out of memory"112- "How do I optimize workflow costs?"113- Working with task decorators114115**Verified Workflows:**116- "Run AlphaFold on Latch"117- "Use DESeq2 for differential expression"118- "Available pre-built workflows"119- Using `latch.verified` module120121## Detailed Documentation122123This skill includes comprehensive reference documentation organized by capability:124125### references/workflow-creation.md126**Read this for:**127- Creating and registering workflows128- Task definition and decorators129- Supporting Python, Nextflow, Snakemake130- Launch plans and conditional sections131- Workflow execution (CLI and programmatic)132- Multi-step and parallel pipelines133- Troubleshooting registration issues134135**Key topics:**136- `latch init` and `latch register` commands137- `@workflow` and `@task` decorators138- LatchFile and LatchDir basics139- Type annotations and docstrings140- Launch plans with preset parameters141- Conditional UI sections142143### references/data-management.md144**Read this for:**145- Cloud storage with LatchFile and LatchDir146- Registry system (Projects, Tables, Records)147- Linked records and relationships148- Enum and typed columns149- Bulk operations and transactions150- Integration with workflows151- Account and workspace management152153**Key topics:**154- `latch:///` path format155- File transfer and glob patterns156- Creating and querying Registry tables157- Column types (string, number, file, link, enum)158- Record CRUD operations159- Workflow-Registry integration160161### references/resource-configuration.md162**Read this for:**163- Task resource decorators164- Custom CPU, memory, GPU configuration165- GPU types (K80, V100, A100)166- Timeout and storage settings167- Resource optimization strategies168- Cost-effective workflow design169- Monitoring and debugging170171**Key topics:**172- `@small_task`, `@large_task`, `@small_gpu_task`, `@large_gpu_task`173- `@custom_task` with precise specifications174- Multi-GPU configuration175- Resource selection by workload type176- Platform limits and quotas177178### references/verified-workflows.md179**Read this for:**180- Pre-built production workflows181- Bulk RNA-seq and DESeq2182- AlphaFold and ColabFold183- Single-cell analysis (ArchR, scVelo)184- CRISPR editing analysis185- Pathway enrichment186- Integration with custom workflows187188**Key topics:**189- `latch.verified` module imports190- Available verified workflows191- Workflow parameters and options192- Combining verified and custom steps193- Version management194195## Common Workflow Patterns196197### Complete RNA-seq Pipeline198199```python200from latch import workflow, small_task, large_task201from latch.types import LatchFile, LatchDir202203@small_task204def quality_control(fastq: LatchFile) -> LatchFile:205 """Run FastQC"""206 return qc_output207208@large_task209def alignment(fastq: LatchFile, genome: str) -> LatchFile:210 """STAR alignment"""211 return bam_output212213@small_task214def quantification(bam: LatchFile) -> LatchFile:215 """featureCounts"""216 return counts217218@workflow219def rnaseq_pipeline(220 input_fastq: LatchFile,221 genome: str,222 output_dir: LatchDir223) -> LatchFile:224 """RNA-seq analysis pipeline"""225 qc = quality_control(fastq=input_fastq)226 aligned = alignment(fastq=qc, genome=genome)227 return quantification(bam=aligned)228```229230### GPU-Accelerated Workflow231232```python233from latch import workflow, small_task, large_gpu_task234from latch.types import LatchFile235236@small_task237def preprocess(input_file: LatchFile) -> LatchFile:238 """Prepare data"""239 return processed240241@large_gpu_task242def gpu_computation(data: LatchFile) -> LatchFile:243 """GPU-accelerated analysis"""244 return results245246@workflow247def gpu_pipeline(input_file: LatchFile) -> LatchFile:248 """Pipeline with GPU tasks"""249 preprocessed = preprocess(input_file=input_file)250 return gpu_computation(data=preprocessed)251```252253### Registry-Integrated Workflow254255```python256from latch import workflow, small_task257from latch.registry.table import Table258from latch.registry.record import Record259from latch.types import LatchFile260261@small_task262def process_and_track(sample_id: str, table_id: str) -> str:263 """Process sample and update Registry"""264 # Get sample from registry265 table = Table.get(table_id=table_id)266 records = Record.list(table_id=table_id, filter={"sample_id": sample_id})267 sample = records[0]268269 # Process270 input_file = sample.values["fastq_file"]271 output = process(input_file)272273 # Update registry274 sample.update(values={"status": "completed", "result": output})275 return "Success"276277@workflow278def registry_workflow(sample_id: str, table_id: str):279 """Workflow integrated with Registry"""280 return process_and_track(sample_id=sample_id, table_id=table_id)281```282283## Best Practices284285### Workflow Design2861. Use type annotations for all parameters2872. Write clear docstrings (appear in UI)2883. Start with standard task decorators, scale up if needed2894. Break complex workflows into modular tasks2905. Implement proper error handling291292### Data Management2936. Use consistent folder structures2947. Define Registry schemas before bulk entry2958. Use linked records for relationships2969. Store metadata in Registry for traceability297298### Resource Configuration29910. Right-size resources (don't over-allocate)30011. Use GPU only when algorithms support it30112. Monitor execution metrics and optimize30213. Design for parallel execution when possible303304### Development Workflow30514. Test locally with Docker before registration30615. Use version control for workflow code30716. Document resource requirements30817. Profile workflows to determine actual needs309310## Troubleshooting311312### Common Issues313314**Registration Failures:**315- Ensure Docker is running316- Check authentication with `latch login`317- Verify all dependencies in Dockerfile318- Use `--verbose` flag for detailed logs319320**Resource Problems:**321- Out of memory: Increase memory in task decorator322- Timeouts: Increase timeout parameter323- Storage issues: Increase ephemeral storage_gib324325**Data Access:**326- Use correct `latch:///` path format327- Verify file exists in workspace328- Check permissions for shared workspaces329330**Type Errors:**331- Add type annotations to all parameters332- Use LatchFile/LatchDir for file/directory parameters333- Ensure workflow return type matches actual return334335## Additional Resources336337- **Official Documentation**: https://docs.latch.bio338- **GitHub Repository**: https://github.com/latchbio/latch339- **Slack Community**: Join Latch SDK workspace340- **API Reference**: https://docs.latch.bio/api/latch.html341- **Blog**: https://blog.latch.bio342343## Support344345For issues or questions:3461. Check documentation links above3472. Search GitHub issues3483. Ask in Slack community3494. Contact support@latch.bio