plurigrid affective-taxis
Models affective valence as the directional derivative of an interoceptive energy landscape (Sennesh & Ramstead 2025), providing alignment validation through structural conservation laws.
Use When
- Building alignment-aware RL agents that respect structural invariants
- Validating GF(3) conservation across reward trajectories
- Training Langevin-based policies as an alternative to PPO (which breaks conservation)
- Implementing fold-change detection reward signals in POMDP environments
- Bridging affective neuroscience models with RL training loops
Workflow
- Define energy landscape: set attractant/repellent positions, sigmas, and amplitudes
- Compute fold-change detection signal:
r(t) = nabla_z log gamma(z; beta) . v - Run Langevin dynamics:
dz/dt = nabla_z log gamma(z; beta) + sqrt(2) dW(t) - Classify trit per timestep: positive derivative → +1, orthogonal → 0, negative → -1
- Verify conservation:
sum(trits) === 0 (mod 3)across trajectory
Core Equations
| Equation | Formula | Meaning |
|---|---|---|
| Fold-change detection | r(t) = nabla_z log gamma(z; beta) . v |
Reward = directional derivative of log-concentration |
| Langevin dynamics | dz/dt = nabla_z log gamma(z; beta) + sqrt(2) dW(t) |
Navigation = gradient ascent + stochastic exploration |
| Trit classification | sign(r(t)) mapped to {-1, 0, +1} |
GF(3) valence of each step |
Implementation Paths
| Path | File | LOC | Language | Domain |
|---|---|---|---|---|
| 0 | affective-taxis.jl |
1700 | Julia | Core theory (16 sections) |
| 1 | affective_taxis_env.py |
576 | Python | Gymnasium POMDP |
| 2 | bridge_9_affective_taxis.py |
1172 | Python | BCI bridge |
| 3 | taxis_landscape_acset.jl |
1453 | Julia | ACSet sheaf |
| 4 | taxis_persistent_homology.py |
1500 | Python | Ripser topology |
| 5 | taxis_clearing.py |
1419 | Python | Market clearing |
| 6 | aella/taxis.el |
400 | Elisp | Circuit taxis |
| 7 | taxis_functorial_persistence.jl |
500 | Julia | Functor bridge |
| RL | train_aligned_agent.py |
500 | Python | PPO vs Langevin |
RL Alignment Results (dt=0.1)
| Policy | GradAlign | MeanConc | GF3 Balance | Mean Reward |
|---|---|---|---|---|
| Oracle | +0.415 | 0.226 | no | +0.503 |
| PPO | +0.239 | 0.526 | no | -0.041 |
| Langevin | -0.084 | 0.448 | YES | +0.089 |
| Random | -0.469 | 0.032 | no | -0.958 |
Key finding: PPO has higher gradient alignment but breaks GF(3) conservation. Langevin is the ONLY policy that conserves the tripartite structure. This is Goodhart's Law: optimizing the reward metric doesn't preserve structural invariants.
Concomitant Skills
| Skill | Trit | Interface |
|---|---|---|
langevin-dynamics |
0 | SDE analysis of taxis navigation |
fokker-planck-analyzer |
+1 | Stationary distribution of energy landscape |
modelica |
0 | Circuit/DAE formulation of taxis landscape |
open-games |
+1 | Multi-agent clearing = compositional game |
persistent-homology |
-1 | Topological taxis signal |
gf3-tripartite |
0 | Conservation law verification |
Modelica Formulation
The affective-taxis POMDP maps naturally to Modelica's acausal equation framework:
model AffectiveTaxis
// State variables
Real z[2](start={0,0}) "Position in chemical landscape";
Real v[2](start={0,0}) "Velocity";
Real beta(start=1.0) "Internal allostatic parameter";
// Landscape: gamma(z) = sum A_i * exp(-|z - mu_i|^2 / (2*sigma_i^2))
parameter Real mu[2,2] = {{3,3},{-3,-3}};
parameter Real sigma[2] = {1.5, 1.5};
parameter Real A[2] = {1.0, -0.4};
// Langevin parameters
parameter Real kappa = 0.5 "Concentration-to-setpoint gain";
parameter Real tau = 1.0 "Relaxation timescale";
parameter Real noise_amp = 0.1 "Langevin noise amplitude";
// Derived quantities
Real gamma "Concentration at z";
Real grad_log_gamma[2] "Gradient of log concentration";
Real fcd "Fold-change detection signal (= reward)";
Integer trit "GF(3) classification of fcd";
equation
gamma = sum(A[i] * exp(-sum((z[j]-mu[i,j])^2 for j in 1:2) / (2*sigma[i]^2)) for i in 1:2);
// ... (see affective_taxis.mo for full implementation)
end AffectiveTaxis;
See affective_taxis.mo for the complete Modelica model.
Quick Start
Julia (core theory)
julia affective-taxis.jl
Python (RL training)
env -u PYTHONPATH /path/to/.venv/bin/python3 train_aligned_agent.py
Modelica (circuit analogy)
# Requires OpenModelica or Wolfram SystemModeler
omc affective_taxis.mo
Key References
- Sennesh & Ramstead 2025: arXiv:2505.17024
- Karin & Alon 2022: PLoS Comp Bio (dopamine reward-taxis)
- Karin & Alon 2021: iScience (gradient tempering)
- Shenhav 2024: Trends Cogn Sci (affective gradient hypothesis)
- Ma et al 2015: NeurIPS (Langevin = Bayesian inference)