Vista Rnaseq De

VISTA RNA-seq DE

Use this skill to create analysis-directory QMDs for bulk RNA-seq count generation and VISTA-native DE analysis.

Default Workflow

1. Inspect the project tree before editing: sample names, STAR BAM/log layout, existing analysis directory, existing QMD numbering, metadata/sample sheets, and organism annotation.
2. Put deliverables in the user-requested analysis directory, not beside raw FASTQs or alignment outputs, unless the user explicitly asks otherwise.
3. Use self-contained Quarto reports by default: embed-resources: true, code-fold: true, code-tools: true, readable wide figures, and all code embedded in the QMD.
4. Do not render QMDs unless the user explicitly asks to run/render. Static-check them when feasible.
5. Use numbered names that match the local workflow, usually 01_featurecounts_count_matrix.qmd and 02_vista_de_analysis.qmd.
6. When adding helper shell scripts, write Bash scripts explicitly and tell the user to run them with bash or execute permissions, not sh, if they use Bash-only options like pipefail.
7. If sample metadata or biological comparisons are missing, create the reusable QMD structure only with clearly marked metadata requirements, or ask for the design before final DE code. Do not infer biology from sample numbers alone.

Required Inputs

Before finalizing a DE QMD, identify or request:

• count source: STAR BAMs for featureCounts, raw count matrix, featureCounts/STAR/HTSeq output, tximport-like counts, RSEM gene results, or a SummarizedExperiment
• sample metadata with sample IDs matching count columns; use derive_vista_metadata() only for starter metadata when biological fields are unknown
• grouping column, numerator and denominator levels, optional covariates, optional design_formula, and desired backend (deseq2, edger, limma, or both)
• organism, annotation GTF, OrgDb, key type, and MSigDB species
• strandedness for featureCounts; default to strandSpecific = 0 only when unknown

FeatureCounts Conventions

• Discover STAR BAMs as 02_alignment/<sample>/<sample>_star_alignAligned.sortedByCoord.out.bam when that layout exists.
• For mouse projects, default the GTF to /spectrum/GSCT/REF/mouse/Mouse-GRCm38-release86/transcripts.gtf unless local files or the user specify a better annotation.
• Write outputs under 01_featurecounts: gene count matrix TSV, sample metadata TSV, featureCounts annotation TSV, assignment stats TSV, and an RDS object.
• Include package checks for Rsubread, dplyr, tidyr, readr, ggplot2, and knitr as needed.

VISTA Conventions

Use VISTA-native helpers rather than replacing the workflow with generic DESeq2 code:

• read_vista_counts()
• read_vista_metadata()
• match_vista_inputs()
• create_vista(method = "deseq2")
• set_rowdata()
• get_deg_count_barplot()
• get_volcano_plot() when available
• get_pca_plot()
• get_umap_plot()
• get_corr_heatmap()
• get_expression_heatmap()
• get_foldchange_heatmap()
• get_expression_lineplot()
• get_foldchange_lineplot()
• get_pathway_heatmap()
• get_deg_venn_diagram()
• get_go_enrichment()
• get_msigdb_enrichment()
• get_gsea()
• get_enrichment_plot()
• save_vista_plot()

Default VISTA cutoffs from the established workflow are min_counts = 10, min_replicates = 1, log2fc_cutoff = 0.6, pval_cutoff = 0.05, and p_value_type = "padj", unless the user or local workflow specifies otherwise.

Input and Metadata Handling

• VISTA DE models should start from raw integer-like counts, not TPM/CPM/log-normalized expression. Use normalized values only for visualization inputs if explicitly appropriate.
• Prefer VISTA import helpers for supported formats: matrix/data frame, featureCounts, STAR gene counts, HTSeq-count, tximport-like inputs, and RSEM gene results.
• Use match_vista_inputs() before create_vista(); never trust sample metadata row order without matching count columns.
• If metadata is missing, use derive_vista_metadata() only to create a starter sheet and clearly mark biological fields that still need user confirmation.
• If the user already has a SummarizedExperiment, consider as_vista() and validate_vista() instead of rebuilding text files.

Advanced VISTA Design

• Default to method = "deseq2" for continuity unless the user or project specifies "edger", "limma", or "both".
• Use covariates = c(...) for additive adjustment when metadata columns such as batch, donor, sex, or cell line should be modeled.
• Use design_formula = ~ covariate + group only when the design is explicit and compatible with the requested comparison.
• For consensus analysis, use method = "both", set result_source intentionally, and use set_de_source() when switching active DE tables.
• Do not infer covariates or interaction terms from sample names alone; ask when the design is biologically ambiguous.

Native Export and Report Options

• For project-local QMDs, continue writing numbered notebooks in the analysis directory.
• When the user asks for VISTA-native export bundles or quick report automation, consider export_vista_assets(), save_vista_data(), save_vista_plot(), and run_vista_report() with VISTA's YAML report template.
• Prefer custom QMDs when the user asks for project-specific sections, special comparisons, or exact plot layouts.

Expanded VISTA Plot Catalog

Use VISTA-native plots when requested rather than recreating them manually: get_mds_plot(), get_pairwise_corr_plot(), DEG count donut/pie/alluvial plots, expression box/violin/bar/lollipop/scatter/density/joy/raincloud plots, fold-change scatter/bar/lollipop/box/raincloud plots, get_kegg_enrichment(), get_enrichment_chord(), and cell-type/deconvolution plots such as get_celltype_heatmap().

Reference Routing

Read references/vista-qmd-patterns.md before writing or updating featureCounts or VISTA QMDs. Load the focused references only when the request needs that depth:

• references/vista-design-patterns.md: consensus DE, covariate-adjusted designs, explicit design formulas, and active DE source switching.
• references/vista-plot-catalog.md: expanded VISTA-native plotting, KEGG, enrichment chord, pathway heatmaps, and special expression/fold-change plot patterns.
• references/vista-export-reporting.md: native VISTA asset export, save_vista_data(), save_vista_plot(), and run_vista_report() workflows.

VISTA Color Policy

Use VISTA's internal color state before introducing ad hoc plot palettes:

• Set sample/group colors once on the VISTA object with set_vista_group_colors(vista, color_map). The named vector must cover all active groups in the VISTA grouping column. Extra names warn and are ignored.
• Set comparison colors once with set_vista_comparison_colors(vista, color_map). The named vector must cover all active comparisons from names(comparisons(vista)). Extra names warn and are ignored.
• Retrieve stored group colors with group_colors(vista) and stored group palette information with group_palette(vista) rather than redefining group palettes in every chunk.
• For PCA and UMAP, use color_by = <group_column> and use_group_colors = TRUE. Do not use deprecated use_vista_colors.
• For expression heatmap annotations, pass stored group colors when annotation colors must match sample-level plots, e.g. column_anno_colors = list(condition = VISTA::group_colors(vista)).
• For expression line plots, use VISTA group colors only when coloring by group; otherwise let VISTA manage gene colors.
• For fold-change and comparison-level plots, prefer set_vista_comparison_colors() over manual comparison palettes.
• Regulation colors are separate from VISTA group/comparison colors. Volcano, MA, and DEG count plots should use named vectors with Up, Down, and Other where applicable.

Known VISTA Pitfalls

• comparisons(vista)[[comp]] may already include gene_id; check before adding row names to avoid duplicate gene_id columns.
• row_data(vista) should contain SYMBOL, GENENAME, and ENTREZID after set_rowdata(). Join these onto DE results and relocate SYMBOL immediately after gene_id.
• get_gsea() returns a wrapper list; use $enrich for export and plotting.
• Raw counts are required for VISTA DE modeling; do not feed TPM/CPM/log-normalized values into create_vista().
• VISTA sample_group arguments expect group labels, not sample IDs.
• Heatmap annotation colors do not automatically reuse VISTA group colors; pass column_anno_colors from group_colors(vista) when consistency matters.
• Guard Venn diagrams: VISTA expects 2-4 comparisons.
• Prefer BiocManager::install("VISTA", ask = FALSE, update = FALSE) for quick installs. If package install was interrupted, verify no R install process is still running before removing stale 00LOCK* directories.

Standard Outputs

For VISTA reports, include tables and plots for:

• VISTA object RDS
• DEG summary CSV
• annotated DE result CSV
• DEG count bar plot
• volcano plot
• PCA
• UMAP and MDS when requested
• sample correlation and pairwise correlation plots when requested
• z-score expression heatmaps and fold-change heatmaps when requested
• expression and fold-change line plots when requested
• Hallmark ORA Up, Down, and Both
• Hallmark ORA pathway-gene heatmaps with get_pathway_heatmap() when requested
• GO BP, MF, and CC
• KEGG enrichment and enrichment chord diagrams when requested
• Hallmark GSEA tables, VISTA enrichment plots, NES bar plots, and top-pathway gseaplot2() when requested
• Venn diagram only when 2-4 comparisons are available

Reference Pattern

Start with references/vista-qmd-patterns.md; it contains compact implementation snippets and output naming conventions. Use the focused reference files listed under Reference Routing for advanced designs, plot-heavy reports, or native VISTA export/report requests.