Executor And Planner

Executor & Planner — operational

Companion docs: knowledge/architecture/executor.md, knowledge/architecture/planner.md, knowledge/architecture/query-lifecycle.md.

The executor and planner are coupled by the Path-then-Plan handoff and by the 1:1 correspondence between plan-node types and executor node types. If you add a new way to execute something, you almost always touch both subsystems.

Part A — Adding a new executor node

A node type has four trees to keep in sync: Plan (plannodes.h), PlanState (execnodes.h), the Path that builds it (pathnodes.h, optional if your node is only emitted by a special-case planner path), and the dispatch switches in execProcnode.c. Skipping any one of these compiles cleanly and fails at run time with unrecognized node type.

A.1 The five-or-six functions to implement

For a new node Foo, in src/backend/executor/nodeFoo.c plus header src/include/executor/nodeFoo.h (see parser-and-nodes skill for the NodeTag / copy / equal / out / read machinery you'll need before any of this compiles — also recapped in A.6):

The ExecProcNode field of PlanState is set inside ExecInitFoo by assigning scanstate->ps.ExecProcNode = ExecFoo; (nodeSeqscan.c:281 is the canonical example — seqscan even picks one of four specialised variants depending on qual/projection presence). ExecInitNode then wraps it via ExecSetExecProcNode so stack-depth and instrumentation hooks layer on transparently (execProcnode.c:391, 430).

A.2 Wiring into the central dispatch

execProcnode.c has three switches on nodeTag(node) that must learn about the new type:

1. ExecInitNode (execProcnode.c:142) — add case T_Foo: calling ExecInitFoo.
2. ExecEndNode (execProcnode.c:543) — add case T_FooState: calling ExecEndFoo.
3. ExecReScan in execAmi.c — add case T_FooState: calling ExecReScanFoo.
4. MultiExecProcNode (execProcnode.c:488) — only if the node returns something other than tuples (Hash builds a hashtable, Bitmap nodes build a TIDBitmap). Tuple-returning scans and joins do NOT need a case here.

Tag asymmetry — the #1 cause of "unrecognized node type" after adding a new node: ExecInitNode switches on the Plan tag (T_Foo) because its input is a Plan *. ExecEndNode and ExecReScan switch on the PlanState tag (T_FooState) because their input is a PlanState *. MultiExecProcNode also switches on the PlanState tag. Mixing them compiles cleanly and dies at run time.

And include your new header in execProcnode.c (alphabetical block at execProcnode.c:77-119). Also add the .c file to src/backend/executor/meson.build and Makefile.

A.3 The Plan ↔ PlanState shape rule

The executor builds a PlanState tree mirroring the Plan tree (executor README §Plan Trees and State Trees, lines 47-79). The Plan tree is read-only; all mutable runtime data lives on PlanState. Concretely:

• Embed Plan plan; as the first field of Foo (plannodes.h) so the lefttree/righttree/qual/targetlist links work generically.
• Embed ScanState ss; (which embeds PlanState ps;) in FooState if it's a scan; otherwise embed PlanState ps; directly. ScanState adds a scan slot and current-relation pointer that the generic scan helpers expect (execnodes.h:1657).

A.4 Memory & ExprContext discipline

ExecInitFoo runs in the per-query memory context (set up by standard_ExecutorStart at execMain.c:180). Call ExecAssignExprContext to get a per-tuple context for evaluating quals and projections — see memory-contexts skill. The per-tuple context is reset before each call to ExecFoo produces a tuple (typically by the parent node calling ResetExprContext — nestloop does this at nodeNestloop.c:92).

A.5 EXPLAIN integration

src/backend/commands/explain.c is the dispatcher. For a new node you need to:

1. Add a label in ExplainNode (the giant switch (nodeTag(plan)) near the top) — usually just a string like "Foo Scan".
2. If the node has per-node detail to print (sort keys, hash buckets, …), add a helper like show_foo_info(FooState *, ExplainState *) and call it from ExplainNode after the generic prefix.
3. If the node has VERBOSE-only output (output columns, etc.), use ExplainPropertyText / ExplainPropertyInteger so JSON/XML/YAML format variants get it for free.

A.6 NodeTag, copy/equal/out/read

Both Foo (in plannodes.h or execnodes.h) and FooState (execnodes.h) need NodeTags. The gen_node_support.pl machinery generates them automatically as long as the header is in @all_input_files. PlanState nodes are typically declared pg_node_attr(no_copy_equal, no_read, no_query_jumble, nodetag_only) — see how every *State struct in execnodes.h is annotated. See the parser-and-nodes skill for the full node-type checklist.

Part B — Working in the planner

B.1 The five-stage pipeline

planner / standard_planner    planner.c:333 / 351
  └─ subquery_planner         planner.c:775         (recurses per subquery)
       ├─ pull_up_subqueries, preprocess_expression, … (prep/)
       ├─ deconstruct_jointree            (initsplan.c)
       └─ grouping_planner                planner.c:1704
            ├─ query_planner              (planmain.c) — builds base RelOptInfos
            │     └─ make_one_rel
            │           ├─ set_base_rel_pathlists → set_rel_pathlist (allpaths.c:516)
            │           └─ make_rel_from_joinlist
            │                 └─ standard_join_search (allpaths.c:3948) OR geqo
            ├─ apply_scanjoin_target_to_paths   (upper rels)
            ├─ create_grouping_paths, create_window_paths, create_distinct_paths, …
            └─ create_ordered_paths, create_limit_path
  └─ create_plan                createplan.c:339    (Path → Plan)
  └─ set_plan_references         setrefs.c          (final Var renumbering)

Trigger condition for GEQO: enable_geqo && levels_needed >= geqo_threshold (allpaths.c:3911). Default geqo_threshold = 12, so up to 11 base rels you get the exhaustive DP search; at 12+ you get a genetic algorithm. Per-pair path generation still goes through the same per-rel routines in path/ either way.

B.2 The Path → Plan handoff

The planner explores alternatives as Paths (lightweight, share structure) and only the chosen Path is converted to a Plan by create_plan_recurse (createplan.c:390). The dispatch is a switch (best_path->pathtype) — adding a new Path-type means adding a create_foo_plan function alongside it.

Rule of thumb:

• Path = "could we do it this way? at what cost?"
• Plan = "this is what the executor will run."
• Path types omit fields the executor doesn't need; Plan types omit fields the planner used only for cost/ordering reasoning.

B.3 RelOptInfo lifecycle

For each base rel and each considered joinrel, exactly one RelOptInfo exists in the PlannerInfo (optimizer README lines 31-42). The DP algorithm guarantees that when we finish populating rel.pathlist for a rel, no higher-level rel has cached a reference yet — so add_path is free to pfree dominated paths (README lines 174-182).

Key fields you'll touch (pathnodes.h:1009):

• pathlist — candidate Paths, sorted by (disabled_nodes, total_cost).
• cheapest_startup_path / cheapest_total_path — set after path generation is complete by set_cheapest. Higher levels read these.
• partial_pathlist — Paths usable by parallel workers.
• rows, reltarget — size and output column estimates.
• consider_startup, consider_param_startup — gates that let add_path ignore startup-cost-only wins for upper rels that don't care.

B.4 Cost units

costsize.c top comment (lines 1-72) is the spec. Costs are in arbitrary units anchored on:

Every Path stores two costs (pathnodes.h:2007):

• startup_cost — work done before the first tuple is available (sort fill, hash build, parameter binding).
• total_cost — work to fetch all tuples.

Interpolation actual = startup + (total-startup) * fetched/rows is what upper plan nodes assume when a LIMIT or EXISTS truncates fetching (costsize.c lines 40-50).

Additionally each Path tracks disabled_nodes — the number of disabled-by- GUC nodes at or below it. add_path compares (disabled_nodes, total_cost) lexicographically, so enable_seqscan=off is respected as far down the tree as possible without distorting raw costs (costsize.c lines 53-62).

B.5 Adding a Path candidate

In set_rel_pathlist (or a join-search routine), after computing the cost, call add_path(rel, (Path *) new_path). Skeleton:

Path *path = create_foo_path(root, rel, ...);
/* cost_foo() fills startup_cost, total_cost, rows */
add_path(rel, path);

add_path (pathnode.c:459) keeps pathlist sorted and prunes any path dominated on all four dimensions: cost, sort order (pathkeys), parameterization (PATH_REQ_OUTER), and parallel-safety (pathnode.c:407-412). Conversely, a new path is rejected only if some existing path dominates it on all four. This is why a path that's slightly more expensive but produces sorted output, or that requires a different outer relid set, will still be kept — they may win later when a merge join or a nestloop-with-param wants them.

For an early-bail check before doing the full cost computation, use add_path_precheck (pathnode.c:686) — it walks the already-sorted pathlist and returns false if no path could possibly survive.

For partial paths (parallel workers), use add_partial_path instead.

B.6 Adding a new scan path for a base rel

If your work introduces a new way to scan a single relation (e.g., a new table-AM-aware scan):

1. Add a case to set_rel_pathlist (allpaths.c:516) — for plain relations that's set_plain_rel_pathlist. Or hook it in set_plain_rel_pathlist directly.
2. Implement create_foo_path in pathnode.c (mirror create_seqscan_path at line 1026).
3. Implement cost_foo in costsize.c (mirror cost_seqscan at line 270).
4. Implement create_foo_plan in createplan.c (mirror create_seqscan_plan at line 2755) and wire it into the switch (best_path->pathtype) in create_plan_recurse (createplan.c:390).
5. Then the executor side: see Part A.

B.7 Adding a new join path

Mirror the structure of one of nodeNestloop/nodeHashjoin/nodeMergejoin. Join path generation happens in joinpath.c (add_paths_to_joinrel); add a helper there that calls your create_foo_join_path and add_paths the result. Costing lives in costsize.c under initial_cost_* / final_cost_* pairs (the two-stage costing lets add_path_precheck short-circuit early).

B.8 What subquery_planner recursing actually means

subquery_planner is called once per Query node — that's once at the top level, plus once per RTE_SUBQUERY that wasn't pulled up, plus once per CTE that wasn't inlined, plus once per SubLink that survived SS_process_sublinks. Each call gets its own PlannerInfo (root) but shares the single PlannerGlobal (glob) so things like paramExecTypes, finalrtable, relationOids accumulate across the whole plan tree (planner.c:775-836).

Common mistakes

• Forgetting set_plan_references. After create_plan, the planner runs set_plan_references (setrefs.c) which renumbers Vars from per-subquery numbering to executor-global slot numbering (INNER_VAR / OUTER_VAR / scan attno). A new plan node that holds expressions must be added to fix_*_references_walker cases or your Vars will not resolve at run time. Concretely: see set_plan_refs (setrefs.c:642, the big switch on nodeTag) and the fix_scan_expr (setrefs.c:160) / fix_join_expr (setrefs.c:186) / fix_upper_expr (setrefs.c:196) walkers below it. New scan-like nodes plug into fix_scan_expr; new join-like nodes plug into fix_join_expr; new nodes operating above joins (Agg/Window/Sort-like) use fix_upper_expr. set_plan_references itself (setrefs.c:291) is the entry point that also assembles finalrtable, accumulates relationOids, and records invalItems for plan-cache invalidation.

• Allocating with malloc in ExecInit*. Use palloc so the per-query context reclaims it on FreeExecutorState. See memory-contexts skill.

• Forgetting to free buffer pins / close relations in ExecEnd*. The memory context will reclaim memory but not relation refs, buffer pins, or open scans. Executor README lines 324-327.

• Touching the Plan tree at run time. Plan trees are shared across executions (plan caching) and must stay read-only. Mutable state goes on PlanState. README lines 56-59.

• Adding a Path without add_path. Always go through add_path (or add_partial_path) — directly lappending to rel->pathlist skips the dominance pruning and inflates planning memory.

• Setting cost manually instead of calling cost_foo. The cost helpers encode the model assumptions; bypassing them desynchronises your path's cost from comparable paths and produces nonsense EXPLAIN plans.

Files-examined

• source/src/backend/executor/README (1-450) — definitive on Plan/PlanState split, memory contexts, control flow, EvalPlanQual.
• source/src/backend/executor/execMain.c:120-275, 308-475 — standard_ExecutorStart / _Run / _End.
• source/src/backend/executor/execProcnode.c:1-590 — dispatch switches.
• source/src/backend/executor/nodeSeqscan.c:1-360 — simple scan template.
• source/src/backend/executor/nodeNestloop.c:1-400 — simple join template.
• source/src/include/nodes/execnodes.h:281-1295 — ExprContext, EState, PlanState, ScanState.
• source/src/backend/optimizer/README (1-200) — Paths, RelOptInfo, DP join search.
• source/src/backend/optimizer/plan/planner.c:333-855, 1704+ — standard_planner, subquery_planner, grouping_planner.
• source/src/backend/optimizer/path/allpaths.c:80-3950 — set_rel_pathlist, make_rel_from_joinlist, standard_join_search.
• source/src/backend/optimizer/path/costsize.c:1-300 — cost model anchors, cost_seqscan.
• source/src/backend/optimizer/util/pathnode.c:400-690, 1020-1060 — add_path, add_path_precheck, create_seqscan_path.
• source/src/backend/optimizer/plan/createplan.c:339-410, 2755+ — create_plan, create_seqscan_plan.
• source/src/include/nodes/pathnodes.h:1009-2015 — RelOptInfo, Path.

Cross-references

• .claude/skills/parser-and-nodes/SKILL.md — upstream of this skill: Query tree → Plan tree pipeline; node-tag conventions; mutators / walkers.
• .claude/skills/access-method-apis/SKILL.md — amcostestimate interaction with the planner; IndexAmRoutine/TableAmRoutine callback shape for new AMs.
• .claude/skills/parallel-query/SKILL.md — per-node parallel hooks (ExecXXXInitializeDSM / ExecXXXInitializeWorker); Gather / GatherMerge.
• .claude/skills/memory-contexts/SKILL.md — per-query / per-tuple contexts; es_query_cxt; ExprContext.
• .claude/skills/locking/SKILL.md — buffer-pin / content-lock rules for executor scan nodes.
• .claude/skills/testing/SKILL.md — EXPLAIN (COSTS OFF) for portable test output; isolation specs for concurrency.
• knowledge/architecture/executor.md, knowledge/architecture/planner.md — long-form architecture.
• knowledge/subsystems/optimizer.md, knowledge/subsystems/executor.md — deep-dive corpus docs.