Registering Held Objects

registering-held-objects

cameras is a list of CameraFrame -- an Observation's cameras field. It is stated here rather than under required_inputs because the type registry names no bare list of frames.

Use immediately after grasping or again at a pre-contact pose. For pre-contact realignment, pass the prior feature-in-TCP and attachment: wrist views localize the functional feature directly, fit a loop's geometric center and plane rather than the centroid of its visible arc, and retain the collision model. Grasp-time geometry remains an explicit low-confidence fallback (registration_confidence == 0.25); the script does not raise for it, so a graph that must stop on a weak registration routes on the confidence.

How the feature is measured

• A loop is segmented from functional_feature.description on every pass and its plane and circle centre are fitted. Only the minimum rotation that aligns the prior normal with the fitted normal is applied, so the in-plane roll of the prior (or grasp-time) frame is preserved: a circle has no observable roll and the fit's roll changes between cameras. The coordinate along the normal is the robust midpoint of the observed ring thickness.
• A tip is segmented from object_description. With direction_marker_description set (a cap, a coloured band, a head) and both masks visible in the same wrist view, the tip is re-derived as the endpoint of the object's principal axis directed toward the marker; with a prior, the prior roll about that axis is preserved. Without a marker the cloud median is compared with the predicted feature centre.
• With a prior, an observed centre more than 4 cm from the prediction is a mask on the gripper or background and is rejected (confidence 0.25).
• On the first pass for non-loop features, an observed cloud whose 3D extent is outside 0.55--1.80 of the reference cloud's extent is rejected before it becomes an enormous attachment or clearance waypoint (confidence 0.25).

Grasp transform and attachment

• grasp_pose (optional) is the commanded grasp TCP pose. When given, the reference cloud and the pre-grasp feature pose are carried by the rigid transform current-TCP · inv(grasp_pose), so the fallback feature-in-hand is inv(grasp_pose) · feature_world rather than inv(current TCP) · feature_world. Pass it whenever registration happens after a lift or a verification move.
• attachment_source selects the cloud that bounds the collision model when no prior_attached_object is given: "reference" (default) fits the complete pre-grasp cloud, carried as above; "observed" fits the accepted wrist cloud, falling back to the carried reference cloud when the wrist saw nothing usable. object_in_tcp is centred on the same cloud.
• The attachment is curobo.cloud_to_attachment(surface_radius=0.002, margin=0.002, max_spheres=64): 64 MORPHIT spheres on a watertight convex hull, radii contracted by 2 mm, fitted where cuRobo lives. An attachment_fit_type of surface or voxel goes to geometry.cloud_to_attachment instead, for explicit fitting experiments only.
• camera_name_filter (default "eye_in_hand") is a substring the camera name must contain; set it to another camera name when a held object is better observed from an overview camera.

Object-specific refinement (CAD registration, per-object landmarks) is not part of this skill; a graph performs it in its own node and passes the result as prior_feature_in_tcp / prior_object_in_tcp.