Transporting Objects

transporting-objects

The pick has happened; the gripper is holding the target. This subgraph moves the held object above the destination container and releases it.

When to use

• After a successful grasp (grasped end state of any grasping-* skill).
• When the destination is a container with a known OBB (cup, bin, basket).

When NOT to use

• Cluttered transport paths where the lifted object risks colliding with other scene objects in transit — but see the waypoint_move_carve variant below, which routes the lift/translate through curobo.plan_with_grasped_object against a rebuilt collision world.

Recommended subgraph state flow

Choose the flow by WHERE the object must land.

Default — a walled container (basket / bin / box / tote): 2 states

HARD RULE — pick this path whenever the destination is a container with walls (a basket, bin, tote, box, or cup) — i.e. essentially every "put/place X in the " task. Use the 2-state transport_move → release flow below and do NOT use the compute_drop → move_above → release drop-pose path (that path releases at/above the rim for a walled container and the object misses). The drop-pose path in the next section is ONLY for placing onto a surface or into a named sub-region.

The object must end up INSIDE the walls, so descend straight down into it. This is the reference VAB packing recipe and needs only container_obb:

transport_move → release

• transport_move — type: script, scripts/<sg>/transport_descend_linear.py. Use the canonical script as-is (materialized automatically; do not re-emit it). Inputs: container_obb = Ref("in.container_obb"), place_offset = -0.06 (the TCP descends to just above the rim so the held object clears the walls and drops in). It lifts, moves over the container, and does an axis-locked straight-Z descend (curobo.plan_directed_linear, fingertip-frame). Lift and transport heights derive from the rim and the current wrist height (leave lift_z / transport_z at their -1.0 sentinels; pin them only for a workspace you have measured), the wrist keeps the rotation it arrived with, every cartesian leg is checked against robot.get_ee_pose (a leg that stops >3 cm short raises), and the lateral leg searches the container's reachable interior when its centre is out of reach — so place_position XY may differ from the container centre. Returns place_position.
• release — type: script, scripts/<sg>/place_release.py. Input place_position = Ref("transport_move.place_position"). Opens the gripper with a settle so the object lands, then a linear straight-up retract to 10 cm above the placement (hover_z left at -1.0), keeping the wrist rotation the transport arrived with (robot.get_ee_pose). On success → exit placed; on failure → blocked.

Emit examples/canonical_subgraph.json verbatim for this path. Do not add a compute_drop/move_above here — the descend already places into the walls.

Placement ONTO a surface, or a described sub-region/zone, or a subpart grasp: the drop-pose path

Here the target is a surface top or a localized zone (not "inside walls"), so compute an explicit drop pose. Either 3 states or 4 states (when a sub-region must be localized at drop time). Do not add more.

3 states:

compute_drop → move_above → release

4 states (when the task description names a placement sub-region):

perceive_zone → compute_drop → move_above → release

For this path prefer the descend_release_linear release variant (TCP-aware straight Cartesian line) over descend_release — the latter goes through robot.go_to_pose, which on some backends targets the hand link and drops the hand-to-TCP offset, a common source of vertical placement misses.

Hard rule — no re-perception of the container. Do NOT add states named re_perceive_container, reobserve_container, re_observe_*, re_filter_obb, or any equivalent that re-detect the container. The container's OrientedBoundingBox and Mask flow in via in.container_obb / in.container_mask from the upstream perception subgraph and are reliable for the drop.

Carve-out — placement-zone perception is allowed. When the task description names a sub-region (e.g. "the left compartment of the caddy", "the inside of the top drawer of the cabinet", "on top of the cabinet shelf"), insert ONE state named perceive_zone of type: script running the canonical scripts/<sg>/perceive_placement_zone.py. The script calls robot.get_observation once and grounds the zone with a two-path pipeline that mirrors perceiving-objects's picking pattern: (A) grounding-dino.detect + vlm.query letter-pick over labeled boxes + sam3.segment_box; (B) sam3.segment_text fallback. The VLM only ever returns a single letter (or none) — never pixel coordinates. Its output flows into compute_drop as container_interior_obb and is the ONLY downstream consumer. When the placement is unambiguously the bare container (e.g. "put X in the basket"), omit the perceive_zone state entirely (3-state flow). Do not add any other perception/observation states inside this subgraph.

State details:

0. perceive_zone (optional — include only when the task names a placement sub-region) — type: script, file scripts/<sg>/perceive_placement_zone.py. Use the canonical script as-is; do not re-emit a python: block for this path. Returns {placement_zone_obb: OrientedBoundingBox | None}.

   Hard rule on placement_description: like object_name in perceiving-objects, this is a literal Python string — the natural noun phrase describing where the held object should land, drawn from this subgraph's description. It is a constant per subgraph instance, NOT a binding. DO NOT write Ref("in.placement_description") or any $ref; the coordinator does not declare placement_description as a subgraph input. Write the string directly.

   The phrase should describe the placement REGION, not the named reference object. Examples:

   • Task: "Pick up the book and place it in the left compartment of the caddy." → "the left compartment of the caddy".
   • Task: "Put the chocolate pudding to the left of the plate." → "the area to the left of the plate".
   • Task: "Put the ketchup in the top drawer of the cabinet." → "the inside of the top drawer of the cabinet".
   • Task: "Pick up the book and place it on top of the shelf." → "on top of the cabinet shelf".

   "perceive_zone": {
     "type": "script",
     "script": "scripts/<sg>/perceive_placement_zone.py",
     "inputs": {
       "placement_description": "the left compartment of the caddy",
       "container_obb":  Ref("in.container_obb"),
       "container_mask": Ref("in.container_mask")
     }
   }
   

   Both container_obb and container_mask are OPTIONAL but strongly recommended:

   • container_mask filters DINO detections to those whose box center falls inside the container (caddy/drawer/shelf), which removes noise boxes elsewhere in the image and frees the limited labeled-box slots for actual sub-region candidates.
   • container_obb lets the script reject candidate zones that drift too far in XY from the container center (a common failure mode when the VLM's chosen mask projects to background through bad depth). It's also used to choose between the DINO and SAM3-text paths — the candidate closer to the container center wins.

   When the script returns None (zone not visible, low confidence, sanity-check failure), the downstream compute_drop falls back to the bare-container path automatically — no extra graph wiring needed.

1. compute_drop — type: script, file scripts/<sg>/compute_drop_pose.py. Use the canonical script as-is; do not emit a python:scripts//compute_drop_pose.py block — the bundle's canonical script is materialized into the workflow directory automatically and re-emitting it overrides the correct implementation with an LLM reimplementation.

   Hard rule on parameter names: the canonical script's def run signature is (ctx, container_obb, container_interior_obb=None, ee_pose_at_grasp=None, drop_clearance=0.05, approach_height=0.20, held_obb=None, wrist_to_tcp=-1.0). Bind the held object as held_obb, not target_obb. Renaming held_obb → target_obb causes the runtime to silently drop the value (extra kwargs are warned and discarded); the script then falls into the no-held-geometry branch and the drop pose is wrong by the held object's full height.

   When the optional perceive_zone state is present, bind container_interior_obb to its placement_zone_obb output so the drop targets the named sub-region instead of the bare container center:

   "compute_drop": {
     "type": "script",
     "script": "scripts/<sg>/compute_drop_pose.py",
     "inputs": {
       "container_obb":           Ref("in.container_obb"),
       "held_obb":                Ref("in.target_obb"),
       "ee_pose_at_grasp":        Ref("in.ee_pose_at_grasp"),
       "container_interior_obb":  Ref("perceive_zone.placement_zone_obb")
     }
   }
   

   When perceive_zone is omitted (3-state flow), drop the container_interior_obb line:

   "compute_drop": {
     "type": "script",
     "script": "scripts/<sg>/compute_drop_pose.py",
     "inputs": {
       "container_obb":    Ref("in.container_obb"),
       "held_obb":         Ref("in.target_obb"),
       "ee_pose_at_grasp": Ref("in.ee_pose_at_grasp")
     }
   }
   

   ee_pose_at_grasp is required for the LIBERO In(obj, region) predicate to fire after release — the script uses it to convert the desired held-object Z into a TCP target accounting for the hand's wrist-to-TCP offset (read off robot.describe_arm / robot.describe_gripper unless wrist_to_tcp pins it; the drop yaw is composed by robot.grasp_frame). The upstream grasping-with-planner subgraph publishes it as a cross-subgraph output (produces_outputs.ee_pose_at_grasp); this subgraph declares ee_pose_at_grasp in its required_inputs so the coordinator wires the binding by name.

   Returns drop_position (Vec3), drop_pose (Se3Pose), approach_pose (Se3Pose).

1a. drop_offset — type: script, file scripts/<sg>/drop_offset_pose.py. REQUIRED whenever the upstream grasp was on a SUBPART (e.g. frypan handle, kettle spout, bottle neck, tool grip). Omit ONLY when the grasp was on the object's geometric centroid (parent_obb == held_obb case).

**HOW TO TELL: the upstream perception subgraph is
`perceiving-object-parts`**, OR the task description names the grasp
location explicitly ("grasp the pan by its handle", "lift it by
the spout"). In both cases the gripper closes on a subpart and
the parent body hangs off-axis — without `drop_offset` the
`compute_drop` script puts the *grasp point* at the placement
zone, which means the *body* lands off the support. This is the
dominant failure mode for handle-grasp-then-place-ON tasks: the
goal predicate reports `pan.bottom_z << burner.top_z` (pan
centroid below the support) and `xy_coverage_over` < 0.5 even
though the workflow itself returned `placed`.

**Repair trigger.** When repairing a transport subgraph that
failed with the above signature AND the upstream is
`perceiving-object-parts`, **insert `drop_offset` even if a prior
iteration omitted it**. Do not assume the absence of `drop_offset`
in the existing workflow is intentional; it is the single most
common omission.

Inputs:
`drop_pose = Ref("compute_drop.drop_pose")`,
`ee_pose_at_grasp = Ref("in.ee_pose_at_grasp")`,
`held_obb = Ref("in.target_obb")` (the grasped subpart OBB —
same as compute_drop.held_obb),
`parent_obb = Ref("in.parent_obb")` (the full object OBB — the
coordinator must declare `parent_obb` in this subgraph's `inputs`
and wire it from the perception subgraph's parent output, e.g.
`perception_sg.parent_obb`).
Returns `drop_position`, `drop_pose`, `approach_pose` — all shifted
in XY so the parent centroid lands at the original drop XY.
Downstream `move_above` / `release` then reference
`drop_offset.drop_position` instead of `compute_drop.drop_position`.

2. move_above — type: script, file scripts/<sg>/waypoint_move.py. Inputs: drop_x = Ref("compute_drop.drop_position.x"), drop_y = Ref("compute_drop.drop_position.y") (or Ref("drop_offset.drop_position.x") / .y when the optional drop_offset node is present). Lifts to a safe height at the current XY, then moves laterally to above the drop XY. The safe height is robot.describe_workspace's transport_z unless safe_height pins it (pin it when a held tool hangs below the fingertips and a container rim sits above the work surface); when the lateral leg is out of reach it retries up to three rungs 3 cm lower — never below the surface plus align_clearance_m — and reports flown_z / descents beside done.

   Variant — collision-aware lift/translate (waypoint_move_carve). Same inputs (drop_x, drop_y) and same return shape, but the node rebuilds the world from a fresh observation and routes through curobo.plan_with_grasped_object instead of the connector's robot.go_to_pose_cartesian (which is TCP-aware but not world-aware). Use when a known obstacle sits on the transport path between the grasp pose and the drop XY (an oven door, a shelf above the table, a tall bottle the lift would clip). A repair pass may flip the transport subgraph to this variant when the transport stage pass-rate drops below the configured threshold — its hypothesis is "free-space transport plowed through a perceived obstacle".

3. release — type: script, file scripts/<sg>/descend_release.py. Inputs: drop_position = Ref("compute_drop.drop_position") (or Ref("drop_offset.drop_position") when drop_offset is present). Descends, opens the gripper, retracts home (robot.go_home on the work arm named by robot.describe_arm; the default rotation is this hand's robot.grasp_frame). On success → exit placed; on failure → exit blocked.

   Variant — linear descent (descend_release_linear). Same node-level contract, but the descent goes through the connector's TCP-aware robot.go_to_pose_cartesian (cuRobo linear plan with a plan_to_pose fallback built into the backend), so the held object descends on a straight Cartesian line with the orientation held — the cleanest release dynamics for subpart-grasp + place-ON tasks (frypan handle → stove). After the open it takes a 5 cm straight-up retreat at the same wrist rotation and robot.wait_steps before robot.go_home, so the fingers cannot drag the just-released object.

Required end states

See also

• references/clearance_constants.md — the magic numbers.
• references/design_transport.md — why no planner is needed for the default scope.
• scripts/{compute_drop_pose,drop_offset_pose,waypoint_move,waypoint_move_carve,descend_release,descend_release_linear,approach_above,lift_grasped,perceive_placement_zone}.py — canonical scripts.
• prompts/vlm_select_zone.md — VLM prompt template for the optional perceive_zone state.

Placing an object rather than a tool centre

transport_descend_linear plans, by default, for the point between the pads: the carry height clears the rim by a fixed amount and the interior margin is a single symmetric inset. That is right when what is held is small next to the container and wrong when it is not -- a thick object hangs below the tool centre and is dragged over the dividers the thin one flies across, and a long object put down crosswise lands on a divider rather than in a cell.

Four opt-in parameters close that gap, each defaulting to the tool-centre rule so no existing caller moves (checked in tests/test_promotion_is_behaviour_preserving.py):

surface_inset_m tells target_obb how far below the object's top surface the grasp put the tool centre; it must match whatever the grasp node used, and it is what turns thickness into hang.