curiositech Hierarchical Skills and Affordance-Based Representation
When to Use This Skill
Load this skill when facing problems involving autonomous skill acquisition, sensorimotor grounding, hierarchical control composition, or bridging symbolic reasoning with continuous control. Particularly effective when agents must learn domain-general capabilities without task supervision.
Decision Points
Schema Selection for Ambiguous Objects
IF object_uncertainty > convergence_threshold:
IF low-cost_visual_exploration available:
→ Execute visual inspection from multiple angles
→ Update Bayesian belief over object affordances
ELSE IF tactile_exploration safe:
→ Execute gentle contact with surface normals
→ Track force convergence patterns
ELSE:
→ Default to most probable schema based on priors
IF object_uncertainty ≤ convergence_threshold:
IF goal_affordance_confidence > action_threshold:
→ Execute goal schema (grasp, manipulate)
ELSE:
→ Select schema maximizing I(action; goal_affordance)
Curriculum Progression Logic
IF schema_convergence_rate < stability_threshold:
→ Check prerequisite schemas are stable
→ Reduce DOF constraints further
→ Increase practice iterations before advancement
IF schema_convergence_rate ≥ stability_threshold:
IF subordinate_schemas available AND superior_schema stable:
→ Attempt nullspace composition: subordinate ⊳ superior
ELSE IF next_complexity_level unlocked:
→ Add sensorimotor resource (additional DOF, sensor modality)
→ Initialize new schema learning
Hierarchical Control Composition
IF multiple_control_objectives active:
Rank by criticality:
IF safety_constraint violated:
→ All controllers ⊳ collision_avoidance
ELSE IF visual_track required for task:
→ force_control ⊳ visual_track
→ orientation_adjust ⊳ (force_control ⊳ visual_track)
ELSE:
→ Apply standard priority hierarchy from training
Calculate: a_combined = a_superior + (I - J_superior†J_superior) * a_subordinate
Failure Modes
Schema Interference Cascade
Detection Rule: If subordinate controller performance degrades when superior controller activates, or if combined error increases monotonically
- Symptom: Tracking error increases when force control added to visual servoing
- Diagnosis: Nullspace projection incorrectly computed or superior Jacobian ill-conditioned
- Fix: Recalculate J_superior†, add damping term, or reduce subordinate controller gains
Premature Convergence Lock
Detection Rule: If schema reports "converged" but goal affordance uncertainty remains high (H(g) > threshold)
- Symptom: Robot attempts grasp after single visual glimpse, fails due to pose uncertainty
- Diagnosis: Convergence threshold too loose or information gain computation disabled
- Fix: Tighten convergence criteria, enable exploratory action selection, validate affordance confidence before commitment
Resource Scaffolding Collapse
Detection Rule: If complex schema learning fails repeatedly and prerequisite schemas show instability
- Symptom: Force-based manipulation never converges despite extended training
- Diagnosis: Visual tracking (prerequisite) insufficiently stable to support force control learning
- Fix: Return to simpler resource restrictions, retrain prerequisite schemas, validate tracking error bounds
Affordance Model Drift
Detection Rule: If object recognition confidence decreases over time despite consistent sensory input
- Symptom: System increasingly uncertain about object identity during repeated interactions
- Diagnosis: Generative affordance model not updating correctly or accumulating prediction errors
- Fix: Recalibrate Bayesian priors, validate sensor calibration, reset affordance statistics
Exploration-Exploitation Deadlock
Detection Rule: If mutual information I(a; g) never decreases below action threshold despite multiple exploratory actions
- Symptom: Robot endlessly circles object without attempting goal action
- Diagnosis: Information gain computation stuck or action threshold unreachable
- Fix: Implement exploration timeout, reduce information threshold, validate sensor noise models
Worked Examples
Example 1: Learning Cup Grasping with Resource Progression
Initial State: Robot with 7-DOF arm, RGB camera, force sensors. No prior cup knowledge.
Phase 1 - Visual Tracking (L1)
- Resource restriction: Vision only, 3 DOF (pan/tilt/zoom)
- Schema: SEARCHTRACK_visual on cup rim
- Curriculum: Random cup poses, reward convergence (tracking error < 5px for 2 seconds)
- Novice miss: Would try to learn full 7-DOF grasping immediately
- Expert insight: Visual stability is prerequisite for force-based schemas
Phase 2 - Reach Coordination (L2)
- Resource addition: Arm motion (3 DOF translation)
- Schema: REACH ⊳ SEARCHTRACK_visual (reaching in nullspace of visual tracking)
- Decision point: If visual tracking error > threshold → pause reach, stabilize vision first
- Failure recovery: Visual tracking lost during reach → return to search phase, reacquire target
Phase 3 - Force Integration (L3)
- Full resources: 7-DOF + vision + force
- Schema: SEARCHTRACK_force ⊳ (REACH ⊳ SEARCHTRACK_visual)
- Complex decision: Force contact detected → switch superior objective from visual to force while maintaining visual in nullspace
- Expert insight: Hierarchical composition prevents control conflicts
Final affordance model: Cup = {rim_visual_tracking: [x,y,θ] distribution, surface_force_tracking: normal directions, grasp_points: force + visual intersection}
Example 2: Object Recognition Under Uncertainty
Scenario: Ambiguous cylindrical object (cup vs. can vs. bottle) partially occluded.
Decision trace:
- Initial observation → P(cup)=0.4, P(can)=0.3, P(bottle)=0.3, H(object)=1.58 bits
- Goal: grasp handle (only cups have handles)
- Handle uncertainty: H(handle_affordance)=1.2 bits > threshold=0.5 bits
- Information gain calculation:
- I(rotate_view; handle) = 0.8 bits (high - could reveal handle)
- I(gentle_contact; handle) = 0.2 bits (low - tactile doesn't detect handles well)
- Execute: rotate view 45°
- Updated belief: P(cup)=0.8, handle visible with confidence 0.9
- H(handle_affordance)=0.3 bits < threshold → execute grasp schema
- Expert insight: Information-theoretic action selection prevented failed grasp attempt
Quality Gates
- Schema convergence rate >80% for target complexity level before advancing
- Tracking error remains <10% of signal range during stable convergence
- Nullspace projection reduces subordinate/superior control conflict to <5% performance degradation
- Object affordance models achieve >90% recognition accuracy across pose variations
- Information gain computations reduce goal uncertainty below action threshold within 5 exploratory actions
- Prerequisite schema stability maintained when learning dependent schemas
- Bayesian belief updates converge within 3 sigma bounds of predicted distributions
- Control hierarchy maintains real-time performance (update rate >10Hz for dynamic tasks)
- Schema composition produces emergent behaviors not present in individual components
- Failure recovery mechanisms restore stable operation within 2 seconds of detected breakdown
NOT-FOR Boundaries
Do NOT use for:
- Pure symbolic reasoning without sensorimotor grounding → Use [logical-reasoning] instead
- Tasks requiring millisecond response times → Use [reactive-control] instead
- Environments with no convergent feedback signals → Use [open-loop-planning] instead
- Single-shot recognition without action consequences → Use [pattern-classification] instead
Delegate when:
- High-level task planning needed → Use [hierarchical-planning] for goal decomposition
- Natural language understanding required → Use [semantic-parsing] for command interpretation
- Social interaction dynamics → Use [multi-agent-coordination] for collaborative behaviors
- Abstract mathematical reasoning → Use [symbolic-computation] for formula manipulation
Resource requirements:
- Minimum: Closed-loop control capability, multiple DOF, reliable sensor feedback
- Scales poorly: With discrete-only state spaces, purely reactive tasks, human-speed interactions