Uncertainty-aware significance assessment
Summary
Assess statistical significance of differential splicing events by comparing observed changes (ΔPSI) against the distribution of within-replicate variation, accounting for transcript expression uncertainty. This method provides calibrated p-values that reflect biological noise rather than assuming constant variance across expression levels.
When to use
You have PSI matrices for two or more conditions with replicates per condition, and you need to determine which alternative splicing events show significant changes between conditions while accounting for measurement uncertainty that scales with transcript expression levels.
When NOT to use
- Samples are unpaired or replicate structure is unclear — the method requires at least 2 replicates per condition to estimate within-condition variance
- Input PSI values are already aggregated across replicates — the method requires individual per-sample PSI and expression values
- You only have a single biological condition — differential analysis requires two or more distinct conditions
Inputs
- PSI matrix files (one per condition, rows = events, columns = samples)
- Transcript expression quantification files (TPM or equivalent units, one per sample)
- Sample-to-condition mapping and replicate structure
Outputs
- Differential splicing table with event identifiers, ΔPSI values, p-values, and optional adjusted significance thresholds
How to apply
Load PSI matrices and transcript expression quantification for each replicate within each condition. Compute the observed ΔPSI (difference in percent-spliced-in) between conditions for each event. Generate a null distribution of ΔPSI values by computing pairwise ΔPSI across replicates within each condition, stratified by expression level of transcripts defining the event. Compare the observed ΔPSI against this empirical null distribution to derive a p-value that accounts for expression-dependent measurement noise. For large replicate counts (>10 per condition), alternative classical tests such as Wilcoxon rank-sum can be applied instead of the empirical distribution method.
Related tools
- SUPPA2 (Computes differential splicing statistics (ΔPSI and uncertainty-aware p-values) from PSI matrices and expression data via the diffSplice subcommand) — https://github.com/comprna/SUPPA
Examples
python3 suppa.py diffSplice -m psiPerEvent -i condition1.psi condition2.psi -e condition1.exp condition2.exp -o diff_splicing_results
Evaluation signals
- Output p-values should span a range from near-zero to 1.0 and show calibration (not uniform or heavily left-skewed) when plotted as a histogram or Q-Q plot against expected null distribution
- Events with small ΔPSI but high expression should yield higher p-values than events with large ΔPSI and similar expression, demonstrating expression-dependent filtering
- Replicates within a condition should show small mean ΔPSI (close to zero) with p-values predominantly > 0.05, validating that within-condition variation is properly captured
- Results should be reproducible across independent runs with the same input data and parameters
- When using classical Wilcoxon test on high-replicate datasets (>10 per condition), p-values should correlate strongly with empirical distribution-based p-values, showing method consistency
Limitations
- Requires minimum 2 replicates per condition; with fewer replicates, the empirical null distribution becomes sparse and p-values unreliable
- Assumes transcript expression measurements are reasonably accurate; severe quantification bias will inflate or deflate the expression-dependent variance estimate
- PSI values are bounded [0,1], creating distributional challenges near boundaries; the method may be less sensitive for events with mean PSI < 0.1 or > 0.9
- Computational cost scales with number of replicates and events; large studies (>500 samples, >100k events) may require memory optimization
- Does not account for isoform-level quantification uncertainty if using summary-level expression proxies (e.g., gene-level counts) rather than transcript-level abundances
Evidence
- [readme] Statistical significance is calculated by comparing the observed ΔPSI between conditions with the distribution of the ΔPSI between replicates as a function of the expression of the transcripts defining the events: "Statistical significance is calculated by comparing the observed ΔPSI between conditions with the distribution of the ΔPSI between replicates as a function of the expression of the transcripts"
- [readme] SUPPA2: fast, accurate, and uncertainty-aware differential splicing analysis across multiple conditions: "SUPPA2: fast, accurate, and uncertainty-aware differential splicing analysis across multiple conditions"
- [readme] When there is a large (>10) number of replicates per condition, you can also run SUPPA with a classical statistical test (Wilcoxon) per local event or per transcript: "When there is a large (>10) number of replicates per condition, you can also run SUPPA with a classical statistical test (Wilcoxon) per local event or per transcript"
- [other] Calculate p-values for each event using the uncertainty-aware method implemented in SUPPA2: "Calculate p-values for each event using the uncertainty-aware method implemented in SUPPA2"
- [other] Apply the SUPPA2 diffSplice statistical test to compute per-event ΔPSI (difference in percent-spliced-in) between conditions: "Apply the SUPPA2 diffSplice statistical test to compute per-event ΔPSI (difference in percent-spliced-in) between conditions"
1---2name: uncertainty-aware-significance-assessment3description: Use when you have PSI matrices for two or more conditions with replicates per condition, and you need to determine which alternative splicing events show significant changes between conditions while accounting for measurement uncertainty that scales with transcript expression levels.4license: CC-BY-4.05---67# Uncertainty-aware significance assessment89## Summary1011Assess statistical significance of differential splicing events by comparing observed changes (ΔPSI) against the distribution of within-replicate variation, accounting for transcript expression uncertainty. This method provides calibrated p-values that reflect biological noise rather than assuming constant variance across expression levels.1213## When to use1415You have PSI matrices for two or more conditions with replicates per condition, and you need to determine which alternative splicing events show significant changes between conditions while accounting for measurement uncertainty that scales with transcript expression levels.1617## When NOT to use1819- Samples are unpaired or replicate structure is unclear — the method requires at least 2 replicates per condition to estimate within-condition variance20- Input PSI values are already aggregated across replicates — the method requires individual per-sample PSI and expression values21- You only have a single biological condition — differential analysis requires two or more distinct conditions2223## Inputs2425- PSI matrix files (one per condition, rows = events, columns = samples)26- Transcript expression quantification files (TPM or equivalent units, one per sample)27- Sample-to-condition mapping and replicate structure2829## Outputs3031- Differential splicing table with event identifiers, ΔPSI values, p-values, and optional adjusted significance thresholds3233## How to apply3435Load PSI matrices and transcript expression quantification for each replicate within each condition. Compute the observed ΔPSI (difference in percent-spliced-in) between conditions for each event. Generate a null distribution of ΔPSI values by computing pairwise ΔPSI across replicates within each condition, stratified by expression level of transcripts defining the event. Compare the observed ΔPSI against this empirical null distribution to derive a p-value that accounts for expression-dependent measurement noise. For large replicate counts (>10 per condition), alternative classical tests such as Wilcoxon rank-sum can be applied instead of the empirical distribution method.3637## Related tools3839- **SUPPA2** (Computes differential splicing statistics (ΔPSI and uncertainty-aware p-values) from PSI matrices and expression data via the diffSplice subcommand) — https://github.com/comprna/SUPPA4041## Examples4243```44python3 suppa.py diffSplice -m psiPerEvent -i condition1.psi condition2.psi -e condition1.exp condition2.exp -o diff_splicing_results45```4647## Evaluation signals4849- Output p-values should span a range from near-zero to 1.0 and show calibration (not uniform or heavily left-skewed) when plotted as a histogram or Q-Q plot against expected null distribution50- Events with small ΔPSI but high expression should yield higher p-values than events with large ΔPSI and similar expression, demonstrating expression-dependent filtering51- Replicates within a condition should show small mean ΔPSI (close to zero) with p-values predominantly > 0.05, validating that within-condition variation is properly captured52- Results should be reproducible across independent runs with the same input data and parameters53- When using classical Wilcoxon test on high-replicate datasets (>10 per condition), p-values should correlate strongly with empirical distribution-based p-values, showing method consistency5455## Limitations5657- Requires minimum 2 replicates per condition; with fewer replicates, the empirical null distribution becomes sparse and p-values unreliable58- Assumes transcript expression measurements are reasonably accurate; severe quantification bias will inflate or deflate the expression-dependent variance estimate59- PSI values are bounded [0,1], creating distributional challenges near boundaries; the method may be less sensitive for events with mean PSI < 0.1 or > 0.960- Computational cost scales with number of replicates and events; large studies (>500 samples, >100k events) may require memory optimization61- Does not account for isoform-level quantification uncertainty if using summary-level expression proxies (e.g., gene-level counts) rather than transcript-level abundances6263## Evidence6465- [readme] Statistical significance is calculated by comparing the observed ΔPSI between conditions with the distribution of the ΔPSI between replicates as a function of the expression of the transcripts defining the events: "Statistical significance is calculated by comparing the observed ΔPSI between conditions with the distribution of the ΔPSI between replicates as a function of the expression of the transcripts"66- [readme] SUPPA2: fast, accurate, and uncertainty-aware differential splicing analysis across multiple conditions: "SUPPA2: fast, accurate, and uncertainty-aware differential splicing analysis across multiple conditions"67- [readme] When there is a large (>10) number of replicates per condition, you can also run SUPPA with a classical statistical test (Wilcoxon) per local event or per transcript: "When there is a large (>10) number of replicates per condition, you can also run SUPPA with a classical statistical test (Wilcoxon) per local event or per transcript"68- [other] Calculate p-values for each event using the uncertainty-aware method implemented in SUPPA2: "Calculate p-values for each event using the uncertainty-aware method implemented in SUPPA2"69- [other] Apply the SUPPA2 diffSplice statistical test to compute per-event ΔPSI (difference in percent-spliced-in) between conditions: "Apply the SUPPA2 diffSplice statistical test to compute per-event ΔPSI (difference in percent-spliced-in) between conditions"
