Liberty91LTD Confidence Levels
Every analytical judgment produced by this platform MUST carry a confidence level. Confidence reflects the quality and quantity of evidence supporting the judgment, NOT the probability that the event will occur (that's likelihood — see likelihood-language skill).
Primary Scale: Named Bands
| Band | Score Range | Meaning |
|---|---|---|
| Very High | 90-100 | Based on high-quality information from multiple independent sources. Analyst has no significant concerns about the validity of the sources. Well-corroborated assessment. |
| High | 75-89 | Based on high-quality information, or from multiple sources with minor inconsistencies. Key assumptions are well-supported. |
| Moderate | 50-74 | Based on credible information that is not sufficient to warrant higher confidence. Key assumptions are reasonable but not fully validated. Alternative interpretations exist. |
| Low | 25-49 | Based on limited or fragmentary information. Key assumptions have significant uncertainty. Multiple plausible alternative interpretations. |
| Very Low | 0-24 | Based on sparse, unreliable, or largely circumstantial information. Assessment is essentially speculative. |
What Determines Confidence
Confidence is determined by three factors:
1. Quality of Sources
- How reliable are the sources? (Cross-reference with Admiralty Scale)
- Are sources independent or derivative?
- Is there potential for deception or disinformation?
2. Quantity and Corroboration
- How many independent sources support the assessment?
- Do sources confirm each other or contradict?
- Is the evidence diverse (technical + human + open source)?
3. Analytical Logic
- How strong is the analytical reasoning?
- Have key assumptions been tested?
- Have alternative hypotheses been considered and evaluated?
How to Express Confidence
In text
"We assess with high confidence that APT28 is responsible for this campaign, based on infrastructure overlaps with three previously attributed operations and consistent TTP alignment documented by two independent vendors."
In frontmatter
confidence: high
confidence_score: 82
confidence_rationale: "Based on corroboration from CrowdStrike and Microsoft reporting, plus internal telemetry matching known APT28 C2 patterns."
In tables/structured output
| Assessment | Confidence | Rationale |
|---|---|---|
| APT28 attribution | High (82) | 3 independent source corroboration + TTP match |
Confidence vs Likelihood
These are DIFFERENT concepts:
- Confidence: How sure are we about our assessment? (evidence quality)
- Likelihood: How probable is a future event? (predictive judgment)
Example:
"We assess with high confidence that Group X has the capability to target financial sector organisations. It is likely (50-75%) that they will conduct such operations in the next 12 months."
The confidence in the capability assessment is high (strong evidence). The likelihood of future targeting is a separate judgment.
Calibrating Confidence
Upgrade confidence when:
- New independent source corroborates the assessment
- Technical evidence directly supports the judgment
- Key assumptions are validated through testing
Downgrade confidence when:
- A previously reliable source is found to have errors
- New information contradicts a key assumption
- Evidence turns out to be derivative (multiple reports trace back to one source)
Common Mistakes
- Conflating confidence with likelihood ("high confidence it will happen" — this is two concepts mashed together)
- Not explaining WHY confidence is at a given level (always include rationale)
- Defaulting to "moderate" to avoid commitment — if the evidence is strong, say so
- Changing confidence based on desired outcome rather than evidence
- Treating confidence as static — reassess when new evidence emerges