Information Geometry

Information Geometry Skill: Fisher-Rao Metric on Statistical Manifolds

Status: Production Ready Trit: 0 (ERGODIC) Color: #D8E826 (Chartreuse) Principle: Natural gradient is parameterization-invariant optimization Frame: Statistical manifold with Fisher metric and dual connections

Overview

Information Geometry treats probability distributions as points on a Riemannian manifold equipped with the Fisher-Rao metric. Implements:

1. Fisher information matrix: g_{ij} = E[d log p / d theta_i * d log p / d theta_j]
2. Divergences: KL, Fisher-Rao, Hellinger, alpha-divergence, Renyi
3. Geodesics: m-geodesic (mixture) and e-geodesic (exponential)
4. Natural gradient: F^{-1} * grad (parameterization-invariant)
5. Dually flat structure: m-connection / e-connection pair
6. Manifold curvature: Scalar curvature, Amari-Chentsov tensor

Correct by construction: Fisher-Rao is the unique Riemannian metric invariant under sufficient statistics (Chentsov's theorem).

Core Formulae

Fisher information matrix:
  g_{ij}(theta) = E_theta[d log p(x;theta)/d theta_i * d log p(x;theta)/d theta_j]

For categorical:  g_{ij} = delta_{ij} / p_i  (diagonal)
For Gaussian:     g = diag(1/sigma^2, 2/sigma^2)

Fisher-Rao distance:
  d_FR(p,q) = 2 * arccos(sum_i sqrt(p_i * q_i))

KL divergence:
  KL(p||q) = sum_i p_i * log(p_i/q_i)

Natural gradient:
  theta_new = theta - lr * F(theta)^{-1} * nabla L(theta)

Dually flat structure:
  m-geodesic: gamma(t) = (1-t)*p + t*q  (flat in mixture coords)
  e-geodesic: gamma(t) ~ p^{1-t} * q^t  (flat in natural coords)

Scalar curvature (simplex S^{n-1}):
  R = (n-1)(n-2)/4

Gadgets

1. FisherInformation

Compute Fisher information for various models:

(defn fisher-information-categorical [p]
  ;; g_{ij} = delta_{ij}/p_i
  (vec (for [i (range (count p))]
    (vec (for [j (range (count p))]
      (if (= i j) (/ 1.0 (max 1e-10 (nth p i))) 0.0))))))

(defn fisher-information-gaussian [mu sigma]
  [[(/ 1.0 (* sigma sigma)) 0.0]
   [0.0 (/ 2.0 (* sigma sigma))]])

2. DivergenceSuite

Complete family of statistical divergences:

(kl-divergence p q)         ;; asymmetric
(fisher-rao-distance p q)   ;; true geodesic metric
(hellinger-distance p q)    ;; symmetric, bounded
(alpha-divergence p q alpha) ;; parametric family
(renyi-divergence p q alpha) ;; order-alpha generalization

3. NaturalGradient

Parameterization-invariant optimization:

(defn natural-gradient-step [params grad fisher learning-rate]
  ;; theta_new = theta - lr * F^{-1} * grad
  (let [F-inv (matrix-inverse fisher)
        nat-grad (mat-vec-mul F-inv grad)]
    (vec-sub params (vec-scale learning-rate nat-grad))))

4. GeodesicTracer

Trace paths on statistical manifold:

(defn mixture-connection [p q t]
  (mapv #(+ (* (- 1.0 t) %1) (* t %2)) p q))

(defn exponential-connection [p q t]
  (normalize (mapv #(* (Math/pow %1 (- 1.0 t)) (Math/pow %2 t)) p q)))

BCI Integration (Layer 18)

Part of the 18-layer BCI orchestration pipeline:

Cross-Layer Connections

• L7 Active Inference: Free energy F = KL(Q||P) is a divergence; natural gradient minimizes it
• L17 de Rham Cohomology: Fisher metric defines Hodge star; alpha-connections are affine connections
• L16 Spectral Methods: Laplacian on statistical manifold via Fisher metric
• L15 Stochastic Resonance: Fisher information maximized at resonance; SNR relates to mutual info
• L5 Riemannian Manifolds: Fisher-Rao is a specific Riemannian metric on distribution space

Geometry Chain: L5 -> L17 -> L18

L5 (Riemannian): General curvature on signal manifold
L17 (de Rham): Differential forms, Hodge theory
L18 (Info Geometry): Fisher metric on probability distributions

Skill Name: information-geometry Type: Statistical Manifold / Fisher-Rao Metric / Natural Gradient Trit: 0 (ERGODIC) Color: #D8E826 (Chartreuse) GF(3): Forms valid triads with PLUS + MINUS skills

Integration with GF(3) Triads

stochastic-resonance (+1) ⊗ information-geometry (0) ⊗ derham-cohomology (-1) = 0 ✓
gay-mcp (+1) ⊗ information-geometry (0) ⊗ persistent-homology (-1) = 0 ✓