# Grasping With Planner

> Top-down grasping via a fast axis-locked linear descend with a collision-aware cuRobo fallback. A single `grasp_descend_linear` node rises, translates over the target, and descends straight down (Z-locked, orientation held) onto the object; if the straight-line solve is infeasible (far-edge reach, no IK for the fixed wrist) it falls back to the cuRobo planner over the candidate fan. Use when the curobo tool bundle is installed and a perceived object (OBB) must be grasped — the default grasping skill whenever cuRobo is deployed, in clean and cluttered scenes alike.

- Skill: `graph-robots/grasping-with-planner` (Agent Skill, multi-file: 10 files)
- Install (CLI): `npx skillmds@latest add graph-robots/grasping-with-planner`
- Raw SKILL.md: https://api.skillmd.com/api/skills/graph-robots/grasping-with-planner/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: graph-robots (https://skillmd.com/u/graph-robots)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/graph-robots/grasping-with-planner

---


# grasping-with-planner

Top-down grasping with a fast **axis-locked linear descend** and a
**collision-aware cuRobo fallback**. The primary path rises to a hover height,
translates over the target while rotating to the grasp yaw, and descends
straight down (Z-only, orientation locked) onto the object — which, unlike a
goalset planner, happily grips a *flat* object by simply lowering onto it. If
the straight-line solve is infeasible, the same node hands off to the cuRobo
planner, which builds a per-observation collision world and searches the whole
candidate fan for a reachable, collision-free wrist. This is the
`grocery_packing` grasp motion, distilled to the general single-object case.

## Install

This skill depends on the **curobo tool bundle** (`curobo.plan_directed_linear`,
`curobo.plan_to_grasp_poses`). cuRobo JIT-compiles CUDA extensions at install
time — build isolation must be off and `CUDA_HOME` must point at a toolkit
matching your torch build:

```bash
export CUDA_HOME=/usr/local/cuda
uv sync --extra curobo   # (pip: pip install -e "open-robot-skills[curobo]" --no-build-isolation)
```

See `tools/curobo/SKILL.md` for the full recipe and gotchas.

## When to use

- Default grasping skill whenever `curobo` is deployed — clean or cluttered.
- Flat / low-profile objects (a butter box, cream cheese) where a goalset
  planner struggles but a straight lower-on-top succeeds.

## When NOT to use

- `curobo` not deployed. Use `grasping-direct-ik` instead.
- The graspable region is NOT the OBB centroid — bowl rim, mug / moka-pot /
  frying-pan handle, or any off-center grasp. The OBB top-down candidates from
  `geometry.top_down_grasp_candidates` are centered on the OBB XY, so they slip
  on hollow centers and miss handles. Use `grasping-short-axis` for elongated
  handles, where the centroid is graspable but orientation is what matters.

## Recommended subgraph state flow

6 states, in order:

```text
open → compute_grasp → goto_grasp → observe → close → grasped
```

(`grasped` is the success-marker `noop` from `sg.add_exit("grasped")`,
with an edge to `END`.)

State details:

1. **`open`** — `type: tool`, `tool: "robot.open_gripper"`, `inputs: { settle_steps: 40 }`.
2. **`compute_grasp`** — `type: tool`, `tool: "geometry.top_down_grasp_candidates"`,
   `inputs: { obb: Ref("in.target_obb") }`. Returns
   `candidates: {poses: list[Se3Pose]}` — a yaw fan of top-down grasps.
3. **`goto_grasp`** — `type: script`, file `scripts/<sg>/grasp_descend_linear.py`
   from this bundle's canonical_scripts. Inputs:
   `grasp_pose = Ref("compute_grasp.candidates.poses.0")`,
   `candidate_poses = Ref("compute_grasp.candidates.poses")`,
   `target_obb = Ref("in.target_obb")`, `hover_z = 0.2`. Rises to `hover_z`,
   translates over the target at the grasp yaw, then Z-locked linear-descends
   onto it (shallow grip near the perceived top, floored a hair above the
   object base so the fingers never ram the table). On an infeasible cartesian
   solve it falls back to `curobo.plan_to_grasp_poses` over `candidate_poses`.
4. **`observe`** — `type: tool`, `tool: "robot.get_observation"`,
   `inputs: {}`. Captures the arm state AT the grasp (post-descend, pre-close)
   so `ee_pose_at_grasp` reflects the real TCP pose the object was gripped at.
5. **`close`** — `type: tool`, `tool: "robot.close_gripper"`, `inputs: { settle_steps: 60 }`.
   Edge directly from `close` to the `grasped` success marker; the subgraph's
   `on_error: "failed"` catches any raise from `goto_grasp` (both paths failed).
   Whether the gripper actually closed on the object is checked by the
   `target_held` postcondition checkpoint (see `## Checkpoints`), NOT by a
   re-check-and-raise node (none such exists).

   The subgraph publishes **two** cross-subgraph outputs:

   - `ee_pose_at_grasp` — the live TCP pose captured at the `observe` step
     (which sits between the descend and `close`). Downstream
     `transporting-objects` uses it to compute a drop height that accounts for
     the panda hand-to-tcp offset and the held object's geometry.
   - `grasp_pose` — the **computed** grasp pose emitted by `compute_grasp`,
     i.e. what the descend is *targeting*. Distinct from `ee_pose_at_grasp`
     (the *actual* EE pose at grasp time). Exposing `grasp_pose` lets the
     checkpoint author write an output-anchored verifier like
     `predicate=lambda w, o: o["grasp_pose"]["position"]["z"] > 0.01` to catch
     sub-table grasp poses *before* they cascade into a `target_held=False`
     failure.

   **Hard rule on the output binding:** the `robot.get_observation` response
   is an `Observation { cameras: list[CameraFrame]; arms: list[ArmState] }`.
   There is **no** flat `ee_pose` field — the EE pose lives at
   `arms[0].ee_pose`. The binding must therefore be exactly:

   ```python
   sg.set_outputs(
       ee_pose_at_grasp=Ref("observe.arms.0.ee_pose"),
       grasp_pose=Ref("compute_grasp.candidates.poses.0"),
   )
   ```

   Do **not** write `Ref("observe.ee_pose")` — that path does not exist and the
   cross-subgraph binding will silently resolve to None, sending the downstream
   `compute_drop_pose.py` into its inferior no-`ee_pose_at_grasp` fallback
   (drop height too high, placement misses).

   Cross-subgraph data flow is by name, so any downstream subgraph that
   declares `ee_pose_at_grasp` as an input automatically receives it.

   ```json
   "edges": [ ..., ["close", "grasped"], ["grasped", "END"] ],
   "conditional_edges": {},
   "exit": { "router_field": null, "success_values": ["grasped"] },
   "on_error": "failed"
   ```

   The lift onto a safe carry height is handled by the next
   `transporting-objects` subgraph (its `waypoint_move` script lifts before
   lateral motion); do NOT add a lift step here.

## Optional candidate-reordering state

For a thin/elongated target (frypan handle, screwdriver, spoon, rod) insert ONE
`type: script` state `select_short_axis` (`scripts/<sg>/select_short_axis.py`)
between `compute_grasp` and `goto_grasp`. Inputs:
`target_obb = Ref("in.target_obb")`,
`candidate_poses = Ref("compute_grasp.candidates.poses")`. It reorders the
candidate fan so poses whose finger-opening axis aligns with the OBB's short
horizontal axis come first (count and pose values preserved — only the ordering
changes), then wire `goto_grasp`'s `grasp_pose`/`candidate_poses` against
`Ref("select_short_axis.poses...")` instead. For a deterministic,
geometry-locked single short-axis pose use the dedicated `grasping-short-axis`
skill instead.

## Required end states

| End state | Meaning |
|---|---|
| `grasped` | Gripper has closed on the object after the descend. Route to next subgraph (typically `transporting-objects`). |
| `failed` | Grasp-attempt failure: the cartesian descend AND the cuRobo planner fallback both failed (a raise to `on_error`). Coordinator routes to abort. Lives only in `on_error` — never declare a `failed` node. |


## See also

- `references/design_grasp_curobo.md` — the Z-locked (fingertip-frame,
  orientation-LOCK) linear descend and why it beats a blended rotate+descend.
- `references/gripper_settle_constants.md` — settle-step tunings.
- `scripts/{grasp_descend_linear,select_short_axis}.py` — canonical scripts.

