Quadrupedal Locomotion Eval

quadrupedal-locomotion-eval

A Real-World Quadrupedal Locomotion Benchmark for Offline Reinforcement Learning — Zhang et al. (2023) (arXiv:2309.16718, 2023)

What this evaluates

This benchmark evaluates offline reinforcement learning algorithms on real-world quadrupedal locomotion tasks. It probes the policy's ability to accurately track locomotion commands, maintain energy efficiency, and exhibit stability under real-world environmental stochasticity and terrain variations.

Datasets

• Real-World Quadrupedal Locomotion Dataset — total ?; splits: train (-1), test (-1)

Metrics

• Return (primary) — range: other
  • Cumulative undiscounted reward over T real-world interactions: M1 = sum_{t=1}^{T} r_t. Reflects task response accuracy and serves as the primary performance measure.
• COT — range: other
  • Cost of Transportation: M2 = (1/T) * sum_{t=1}^{T} [ (|tau_t * q_dot_t|) / (mg * ||v_t||_2) ]. Represents positive mechanical power applied by actuators per unit weight and unit locomotion speed.
• COV — range: other
  • Coefficient of Variation: M3 = sigma(M1^i) / mu(M1^i). Measures the stability of an algorithm by quantifying the dispersion/fluctuations of Return across runs for a given task.

Input / output format

Input: 44-dimensional continuous state vector comprising COM linear velocity (2D), attitude angle (3D), angular velocity (3D), joint angles (12D), joint angular velocities (12D), and the action at the last time step (12D).

Output: 12-dimensional vector of desired joint angles.

Scoring recipe

def compute_metrics(rewards, actions, states, T, m, g):
    # Return
    ret = sum(rewards[:T])
    # COT
    cot_sum = 0.0
    for t in range(T):
        tau = actions[t]
        q_dot = states[t][10:22]  # joint angular velocities
        v = states[t][0:2]        # COM linear velocity
        cot_sum += abs(tau * q_dot).sum() / (m * g * np.linalg.norm(v))
    cot = cot_sum / T
    # COV (computed across multiple evaluation runs/tasks)
    cov = np.std(ret_across_runs) / np.mean(ret_across_runs)
    return ret, cot, cov

Common pitfalls

• Real-world environmental stochasticity (friction, slope, motor damping) causes high performance variance; policies must act conservatively to avoid hardware damage, unlike in simulation.
• The dataset is collected via MPC, making it 'Near-Expert' rather than truly expert, and lacks the diversity of simulated benchmarks due to safety constraints during data collection.
• Proprioception-only state space limits the robot's ability to handle complex tasks like obstacle avoidance, which is a known limitation of this benchmark.

Evidence (verbatim from paper)

For online policy evaluation, we deploy the trained ORL algorithms to the realistic robot, and calculate the cumulative undiscounted reward (Return) M1=sum_{t}^{T}r_{t}, where T is the number of real-world interactions. This metric mainly reflects the robot’s task response accuracy. It will directly serve as a measure of the final performance of the algorithm, which is a common metric in the DRL community. Furthermore, the dimensionless cost of transportation (COT) is a metric in the legged locomotion research, and is defined as M2=sum_{t}^{T}[(|tau_{t}dot{q_{t}}|)/(mg||v_{t}||_{2})]/T... To quantitatively reflect the instability of ORL algorithms in realistic robot, the coefficient of variation (COV) is utilized as a novel evaluation metric, which is a dimensionless statistic to measure the degree of data dispersion M3=sigma(M1^{i})/mu(M1^{i}).

Citation

@misc{zhang2023quadrupedal,
  title={A Real-World Quadrupedal Locomotion Benchmark for Offline Reinforcement Learning},
  author={Zhang et al. (2023)},
  year={2023},
  note={arXiv:2309.16718}
}

• arXiv: 2309.16718