Bump Endstone to a new BDS version
Every bump is the same two jobs:
- Regenerate the symbol offset tables - which hook resolves to which
address (
src/bedrock/symbols/{windows,linux}.h).
- Port
src/bedrock/ to the new ABI - fix the signatures, vtable orders
and member layouts that changed, so those offsets land on the right code and
memory is read at the right offsets.
How you discover what changed (and the new signatures job 1 needs) depends on
your reference material. Pick the scenario - the rest of the skill is split
along it:
- Scenario A - full (you have
bedrock-headers). The dwarf2cpp header diff
tells you exactly what changed and why. The canonical path; use it whenever
headers for the target version exist. -> Scenario A below.
- Scenario B - limited (no headers; only IDA databases). You have a Linux
BDS database (RTTI present) and a Windows BDS database, and maybe a stale
PDB - but no header diff. You reverse-engineer each ABI change directly from
the binaries, driven by symbol misses (build) and runtime crashes.
-> Scenario B below.
Both scenarios share The symbol pipeline, Editing src/bedrock correctly,
Finish, and most Gotchas. A real bump is often mostly A with a few B
spot-checks (confirm a vtable against the binary), or runs as B until headers
land and then finishes as A.
NDA boundary (read first)
This workflow may use two private Mojang-derived artifacts:
bedrock-headers - C++ headers reconstructed from BDS binaries. Required for
Scenario A; absent by definition in Scenario B.
bedrock_server.pdb - useful for Windows symbol resolution when available and
current, but not published for every release (and a stale one is a trap -
see Scenario B).
Both artifacts are NDA-protected. Never copy header bodies, class
definitions, full member layouts, PDB dumps, symbol listings, or other private
artifact contents into the public endstone repo, its commits, PRs, issues,
logs, or this skill. Endstone's src/bedrock/ is a hand-written, minimal
reimplementation - only what Endstone needs, in Endstone's own naming - which is
the DMCA-safe form. Treat headers, PDBs, generated dumps, decompiler output, and
diffs as private working references only.
The symbol pipeline (shared)
How it works
scripts/configs/{windows,linux}.toml signature configs, hand-maintained
| scripts/dump_symbols.py
v
src/bedrock/symbols/{windows,linux}.h std::array of name -> offset (committed)
src/bedrock/symbol.h get_symbol() looks a symbol up by __FUNCDNAME__ - the
mangled name of Endstone's own declaration in src/bedrock/. The symbol-table
key IS the signature of Endstone's reimplementation. Unresolved symbols are
written as 0 and dropped by the dumper (that hook is disabled; the build still
succeeds, unless any TU actually consumes the missing name - then consteval
get_symbol() throws at compile time).
Prerequisites
uv - runs dump_symbols.py (PEP 723 inline deps, no manual install).
pdbtool - cargo install pdbtool (Microsoft pdb-rs). Reads a Windows PDB
when one is available and current.
- The target version published in
EndstoneMC/bedrock-server-data (the Linux
path downloads the binary from it - check its versions.json).
- Scenario A only:
bedrock-headers for the target version (must remain
private).
- Optional: the Windows BDS PDB (
bedrock_server.pdb) for the target version.
Confirm its version matches the exe before trusting --pdb (a stale PDB
silently mis-resolves moved symbols - see Scenario B).
Procedure
- Branch off the current release branch (e.g.
v0.11):
git checkout -b feat/<NN.NN>-support (naming follows feat/26.10-support).
- Bump the config versions - set
version = "<X.Y.Z>" in both
scripts/configs/windows.toml and scripts/configs/linux.toml, using the
3-component release string from bedrock-server-data versions.json.
- Regenerate (run in the background, 25 s to a few minutes):
- Windows:
uv run --script scripts/dump_symbols.py scripts/configs/windows.toml --pdb <path>/bedrock_server.pdb
- Linux:
uv run --script scripts/dump_symbols.py scripts/configs/linux.toml
- No (current) Windows PDB? Drop
--pdb and rely on the byte-pattern
fallback per entry; resolve the gaps the Scenario-B way.
- Triage the failures - this tells you which symbols broke (the input to
the porting work). How you find the fix is per-scenario.
- Windows (PDB by name, then byte pattern): each entry is looked up by
mangled
name in the PDB; entries the PDB has no public record for
(lambdas, function-local statics) fall back to scanning the entry's
pattern. A miss means both failed - the mangled name is gone (MSVC
encodes the full signature incl. return type, const-ness and access) and
the byte pattern no longer matches. A PDB hit is name-verified; a fallback
hit ("Found signature (fallback)") is only pattern-verified, like Linux.
- Linux (byte-pattern scan): a miss = the
pattern in
configs/linux.toml no longer matches. The function usually still exists -
the pattern went stale. A Linux hit is a pattern match labelled with the
config name; it is not name-verified.
- Failed on both -> real signature/API change.
- Windows only -> the mangled name changed: signature, return type,
const-ness or access. Itanium omits the return type, so a pure return-type
change leaves the Linux name intact. A const/access change is fixed in
Endstone's
src/bedrock/ declaration (__FUNCDNAME__ derives from it),
not the config alone.
- Linux only -> stale byte pattern; re-extract it (see Gotchas).
Editing src/bedrock correctly (shared)
Whatever told you what changed, the edit obeys the same rules. ABI edits are
easy to get subtly wrong - a wrong vtable slot or member offset corrupts memory
silently, caught by neither a compile nor a PR review. Build and test
iteratively; never batch many unverified ABI edits.
- Function signatures (especially hooked /
ENDSTONE_HOOK) - parameter
types, const/ref, return type must match BDS exactly, or __FUNCDNAME__ stops
matching the symbol.
- Virtual functions - the vtable order must match BDS. An added / removed /
reordered virtual shifts every slot below it; mirror the new order (use
= 0
placeholders for virtuals Endstone does not implement). Only the slot count
matters for ABI - one virtual void <name>() = 0; is one slot whatever its
signature.
- Members - type, order and size must match for layout; member names
stay Endstone's own (
lower_case_), never Mojang's. Width-ambiguous integers:
bedrock-headers/Linux build unsigned long is 64-bit, Windows (LLP64) 32-bit -
port unsigned long as std::uint64_t (64-bit on both targets).
- The first member after a base is per-ABI. Itanium allocates derived
members from
dsize(base), MSVC from sizeof(base), so a first member with
alignment < 8 lands at 44 on Linux and 48 on Windows under Packet (48/44).
Mirror whatever BDS's own class starts with - an inline scalar shifts the same
way, an 8-aligned sub-object does not. clang++ --target=x86_64-pc-linux-gnu -Xclang -fdump-record-layouts on a self-contained repro prints dsize and
every offset; run it for both targets rather than reasoning about it.
- Template arguments - a class template's default arguments are part of
its declaration: copy them verbatim, never guess (e.g.
brstd::bitset's
word-type defaults to unsigned int). Never drop an explicit argument to
lean on a default; spell every argument the actual instantiation spells
(apply the int-width rule to those too).
- One type per corresponding file - a needed BDS type Endstone lacks goes in
its own
src/bedrock/ header mirroring the BDS file (snake_case path), then
#included - do not paste a foreign definition inline. A forward declaration
used across many headers goes in src/bedrock/forward.h (alphabetical); for a
heavy include chain, forward-declare and use the type incomplete (fine for
pointers, references, and container value types).
- Every header must be self-contained. A sweep that adds one
#include to
an events/shard header can re-order the whole chain and expose headers that
were silently borrowing a transitive include - the symptom is no template named 'X' plus a cascade of static_assert size failures in a file the sweep
never touched. Include what you use, in the file that uses it. Verify with a
one-line TU (#include "<the header>") compiled /Zs (-fsyntax-only) using
flags lifted from the build log, not from the repo-root
compile_commands.json, which goes stale and can miss defines (-DNOMINMAX,
-DWIN32_LEAN_AND_MEAN). Sweep the whole sibling directory at once - latent
cases cluster.
- Structural refactors - when BDS introduces a base class, mirror it (add
the base header, re-parent, move shared members down). When BDS removes a
class,
git rm once grep confirms nothing references it. Follow BDS
structure; only the file name differs (snake_case). Keep it minimal.
- Knowing the type vs placeholdering it. Scenario A: declare the real type -
never a same-size stand-in. Scenario B: when you cannot name a type/signature
precisely, use a documented placeholder (see Scenario B - Placeholders) -
but the size / order / slot-count must still be exact.
After the edit: update the mangled name in scripts/configs/{windows,linux}.toml,
re-run the dumper, and update any affected hook in
src/endstone/runtime/bedrock_hooks/.
Scenario A - full port with bedrock-headers
The header diff is the source of truth: it lists every signature, vtable and
member change. Work it stage by stage, then apply each via Editing src/bedrock
correctly.
Source: the header diff
dwarf2cpp reconstructs C++ headers from a DWARF-bearing BDS build (the Android
build libminecraftpe.so carries DWARF; the Windows/Linux server binaries are
stripped). Output lands in bedrock-headers, one branch per BDS release
(android/r26_u1, android/r26_u2, ...).
dwarf2cpp <libminecraftpe.so> --base-dir <build-root> -o <out> (or uvx dwarf2cpp).
- In
bedrock-headers: git checkout -b android/r<NN>_u<N>, place the output, commit.
git diff android/r<prev> android/r<new> is the change set.
The actionable set
src/bedrock/ is ~655 hand-maintained headers - a small subset of BDS. Most of
a release diff (5000+ files) touches nothing Endstone declares. So:
actionable work = (changed headers) intersect (the 655 src/bedrock headers)
Match by normalized basename (lowercase, strip _ and -): bedrock-headers
Mob.h <-> Endstone mob.h; BlockSource.h <-> block_source.h.
Staged review order
Review the diff in stages - foundational types first, so later stages do not
rework. Scope: the handheld/ tree and the top-level src/base/ tree;
skip handheld/src-client/ (game client) and the other top-level src/
subtrees (account, external, gui - client / Xbox / third-party). Within
each stage, deep-dive only the intersection. (The first attempt used 3 coarse
stages; "handheld/src non-world" alone was 467 files / 44 intersecting - too
big. Use this finer split:)
src/base (top-level, not under handheld/) - the shared Core library:
foundational utilities and low-level types (BinaryStream, ...). Easy to
miss because every staged path below lives under handheld/ while this tree
is separate; a missed change here (e.g. a new BinaryStream virtual)
silently shifts a vtable that Phase 1 can never flag.
src-deps/SharedTypes - shared types and enums
src/common/network - packets, network types, packet-id / disconnect enums
src/common/server (incl. server/commands) - server and command system
src/common/entity - ECS components
src/common/{certificates,resources,scripting,platform,locale,gameplayhandlers,...} - remaining non-world
src/common/world/actor
src/common/world/item
src/common/world/level/block
src/common/world/level/{dimension,biome} and remaining src/common/world/level/* (chunk, material, storage, level core)
src/common/world/* - remaining world (attribute, effect, events, inventory, response, ...)
src-deps other than SharedTypes (Certificates, VanillaComponents, ...), then anything else
- Cross-validate - once every ABI change is in, re-review the whole
src/bedrock/ diff against the bedrock-headers diff. Every edited function
signature, vtable slot, member type/order, and structural change must trace
to a concrete change in git diff android/r<prev> android/r<new>. Reject
anything not backed by the diff: no invented types, no guessed members, no
hallucinated signatures, no "looks-right" edits. A change that cannot be
matched to the header diff is wrong - revert or fix it. This stage exists
because the porting stages, especially when parallelised across agents, can
introduce plausible but unfounded edits - they must all be matched up.
Reading the diff: noise to skip
dwarf2cpp churn that is not a real BDS change:
- Versioned-namespace churn -
SharedTypes/v1_26_10/... becomes
v1_26_20/...; most of that subtree's diff is just the version bump.
- Template-instantiation churn -
SharedPtr.h / SharedCounter and similar
enumerate concrete instantiations (CopperBlock<ThinFenceBlock>, ...). The
set churns every release; Endstone uses its own templates - ignore.
- File regrouping - dwarf2cpp regroups types into different generated files.
A file shown as fully deleted (e.g.
CommonTypes.h) often just means its
types moved. Confirm a type is genuinely gone, not relocated.
- Lambda source-location churn -
match<(lambda at .../Foo.cpp:47:3)> -
line/column numbers shift every build. Pure noise.
- Declaration reordering - declarations reordered within a file; the diff
shows -/+ pairs of identical content moved.
Scenario B - limited port from the binaries (Linux RTTI + Windows DB)
No header diff. You have:
- a Linux BDS database - stripped of function names but RTTI is intact
(
_ZTV<len><Class> vtables, _ZTI typeinfo), so polymorphic classes,
vtables, and Itanium-mangled names are recoverable;
- a Windows BDS database - what Endstone actually hooks (and may carry
partial symbols: some methods demangled even though ctors/vtables are not);
- a previous, named reference DB for both platforms (the last version, with
PDB symbols) to diff against;
- possibly a stale PDB - treat with suspicion.
Run everything through the ida-pro py_eval (see [[reference_idalib_mcp_quirks]]);
note that in py_eval two top-level defs cannot call each other (exec scope) -
nest helpers in one function. find_bytes + py_eval xrefs stay responsive
when search_text / xrefs_to / make_signature time out on the busy DB.
The loop
Without a diff, work is driven by two signals, fixed one at a time (build/test
between each - see Editing src/bedrock correctly):
- Symbol misses from the dumper (Phase 1 triage) -> Finding a new symbol /
offset below.
- Runtime crashes / misbehaviour once it runs -> a vtable shift
(Detecting vtable changes) or a member-offset shift (Detecting data-member
layout changes). An AV in an accessor/
_get/_setControlBlock/unique_ptr
deref means a field is read at the wrong offset; clean misbehaviour with no
fault (e.g. a hook whose argument is garbage) often means a hook landed on the
wrong function. A std::_Throw_bad_variant_access thrown from a
Script<...>GameplayHandler::handleEvent* (event.visit(...)) is an
event-variant drift (Detecting event-variant changes).
Finding a new symbol / offset without a header diff
- Navigate by string anchor, not symbol. To locate an unnamed function:
take a string literal it references (an error/i18n key like
commands.setmaxplayers.success.lowerbound), find_bytes the ASCII hex of
the string, xref to the referencing function, and read it. Diff it against
the previous DB's named equivalent (e.g. SetMaxPlayersCommand::execute) to
read off the new offsets/signature. Always lookup_funcs the name first - the
Windows DB's partial symbols may already have it.
- Re-cut a stale / wrong byte pattern. Prefer a prologue pattern (the
push sequence + sub rsp) over a call-site one; the match offset is then the
function start. For a virtual, re-cut from the vtable, not a raw scan: find
the class vtable (Linux RTTI _ZTV<len><Class>; Windows via the documented
string -> ctor -> __vftable store route), take the exact slot (mind the
dtor-slot difference: Itanium 2 dtor slots, MSVC 1), read the prologue there,
and wildcard only displacements/immediates.
- Verify a pattern-resolved offset two ways, not one. (1) It must be a
function start -
ida_funcs.get_func(ea).start_ea == ea; an offset that
lands mid-function is conclusively wrong. (2) Decompile it and confirm it
is the intended function ([[feedback_decompile_to_confirm]]) - same-named
overloads (sendPacket(string&, Reliability, Compressibility) vs
sendPacket(string&, Packet&, ...)) have different bodies; match the body to
what your hook expects. A stale prologue pattern does not just miss - it can
silently match a different function with the old shape (this bit
BatchedNetworkPeer::sendPacket at 1.26.32: its codegen added push r12..r15,
so the old 55 56 57 53 ... pattern collided with a packet-trace overload).
- Sweeping/verifying the whole table: compare the committed offset's body
against the previous version's named function - never against a name. Two
traps that each produce a false verdict (both bit a real 1.26.32 sweep):
- Same RVA != same function across versions. Do not identify the new
function by reading what name sits at that RVA in the old DB - code
relocates every release, so the old DB's
0x8e8a00 (changeToValueType)
says nothing about the new DB's 0x8e8a00 (which was the correct
RepositorySources::initializePackSource). Decompile the new offset's body
and match its behaviour to the old named function: distinctive callees,
member-offset writes, or constants (the FNV 0x100000001B3; literal
factory-call args like 6/4). A near-match in line count / arg count is
expected to drift with inlining - judge by behaviour, never by size.
- Don't trust the target DB's auto-names or hexrays' inferred prototype.
The fresh DB mislabelled a 123 KB function as
ItemInstance::fromTag while
the correct small one was an unnamed sub_; and the real 2-arg
initializePackSource(this, PackSourceFactory&) decompiled as a 4-arg
(__int64*, const char*, __int64, __int64). The body is ground truth; the
label and the prototype are guesses.
- Cheap pre-filter for a 60+ entry table: for each entry, confirm the offset
is a function start and that its referenced string set is a superset of
the old named function's strings (strings are version-stable). That clears
the string-bearing majority; decompile-and-compare only the string-less
residue. (Callee-name overlap does not work - the target DB's callees
are almost all unnamed
sub_.)
- Beware a stale PDB overriding your fix.
--pdb resolves by name first,
so a PDB older than the exe returns the old RVA for any moved symbol,
ignoring your re-cut pattern. When the PDB version can't be trusted, do not
blanket-regenerate (it can clobber currently-correct offsets with stale ones).
Instead fix the one verified entry in src/bedrock/symbols/<platform>.h
directly (hand-patch the offset) and update the pattern for the next
clean regen. Cross-check the other platform - the same function on Linux
(_ZN...) often resolved fine (different codegen), confirming a Windows-only
change.
Detecting vtable changes (Linux RTTI)
Confirm directly against the binary whether a virtual was added / removed /
reordered, and at exactly which slot - name-free. Every polymorphic class has
_ZTV<len><Class> + _ZTI<len><Class> even in a fresh, PDB-less DB; the slot
targets are sub_ in both DBs (even the named reference only has RTTI symbols,
not the virtuals), so you diff layout without any virtual-function names.
- Walk the vtable from its address point. Resolve
_ZTV<len><Class>, skip
each (offset_to_top, _ZTI<len><Class>) header pair, then collect qwords
while the target sits in an executable segment
(ida_segment.getseg(q).perm & 1) or is __cxa_pure_virtual; stop at the
first non-pointer / next typeinfo. Do not gate on ida_funcs.get_func -
the fresh DB has not defined most vtable targets as functions yet, so it stops
the walk early; exec-segment membership works regardless of analysis.
- Length per class brackets the change. A primary vtable lays out
[base virtuals][derived adds] in order, so each class's own _ZTV length
localises a net add/remove to one class. For a hierarchy, the most-derived
concrete class (e.g. ServerPlayer covers Actor -> Mob -> Player -> ServerPlayer) gives the whole chain in one read. Equal length on every level
= no net change (still verify order).
- Structural fingerprint confirms no same-count shuffle, name-free: tag each
slot
P = __cxa_pure_virtual, T = this-adjusting thunk (48 83 ef /
48 81 ef = sub rdi), R = repeats previous target (shared-stub runs),
. = normal. Identical tag strings across DBs => no reshuffle.
- Localise the exact slot by signature alignment. Per slot, build a
recompilation-robust signature - decode the first ~6 instructions
(
ida_ua.decode_insn) keeping mnemonic + operand register classes but
dropping immediates and displacements. Bridge the two IDA processes via a
temp file: dump the old DB's per-slot signatures to JSON, switch DBs,
recompute, difflib.SequenceMatcher the two lists. insert/delete opcodes
are the real structural change; replace opcodes are functions whose body
changed at the same slot - ignore them.
- Confirm by decompile, never by the heuristic alone
([[feedback_decompile_to_confirm]]). Decompile the boundary in both DBs: the
shifted neighbour (
NEW[slot+1]) must match OLD[slot], and the inserted
NEW[slot] is often referenced by its shifted neighbour (e.g. a new
per-position helper the shifted loop calls via vtbl+offset) - the tightest
possible confirmation.
- Map the slot to the Endstone declaration, then validate locally. Count
the header's virtuals (dtor = 2 slots; each overload = 1; skip commented-out;
honour
#ifdef __linux__) plus any base's slots. Anchor the count on a
virtual whose address you know in both versions (a hooked one from the toml) -
its slot must equal your predicted index. Beware: an Endstone header may omit
Linux-only virtuals (the #blameMojang ones), so the cumulative count can be
short of the real vtable - do not trust it globally. Decompile the few slots
around the change and match them to neighbouring declarations (a const/
non-const overload pair returning the same this+N subobject is an
unmistakable anchor). If the local sequence lines up, the insert point is
pinned regardless of any global gap.
- Add the placeholder. A single non-dtor
virtual void <name>() = 0;
occupies exactly one slot. Name it a clearly-marked placeholder (don't invent
a Mojang name) and comment the observed signature/behaviour. If you cannot
confirm the change is on both platforms, match the existing #ifdef __linux__
pattern rather than risk shifting the Windows vtable.
For one class you do not need IDA at all: lief + capstone on the shipped
binaries resolve _ZTS<len><Class> -> typeinfo -> vtable -> per-slot disassembly
in seconds, on both the previous and the new ELF. (lief's Binary.relocations
comes back empty on the stripped BDS ELF - parse .rela.dyn yourself as 24-byte
(offset, info, addend) records and keep info & 0xffffffff == 8.)
Locating a vtable on Windows (no RTTI)
/GR- leaves no type descriptors, but a small interface is still findable
name-free - and this is the only way to confirm a Linux-derived vtable verdict on
the platform Endstone actually hooks.
- Scan
.rdata for a run of consecutive pointers to lea rax, [rcx+d]; ret
stubs. Trivial base-subobject getters are ICF-folded to one stub per
displacement binary-wide (~100 in a 160 MB .text), so a class with N of
them is a run of N adjacent stubs with distinct increasing displacements, and
the displacements read off the member layout directly. These stubs are 16-byte
aligned and cc-padded but have no .pdata record (leaf, no unwind) - a
.pdata function-start filter silently drops every one of them and the scan
returns nothing.
- Identify the class from the dtor slot, not a name. The slot before the run
is the scalar deleting dtor; its teardown pins both the class and its
sizeof
(a virtual-deleting unique_ptr at +N = the last member). Comparing that
body against the last release that shipped a PDB is the identification.
- Never read the table's END from "the next qword is not code". MSVC packs
vftables back to back in
.rdata, so the next table's slot 0 is a code
pointer - the same class reads as 4 slots in one release and 8 in the next
purely by what the linker put after it. The terminator is: the next qword's own
address is the target of a lea + mov [reg], rax vfptr store.
- A displacement proves the slot order, never the semantics. Which member a
getter returns comes from the last PDB-bearing release (
??_7<Class>@@6B@ plus
the named getters); carry that mapping forward version by version and diff
displacements. Equal displacements at equal slots across the chain is what
makes an "unchanged" verdict conclusive rather than merely shape-compatible.
- To check ONE virtual's slot index in a 400-slot table, don't walk the
table - find the function (byte fingerprint of its body, member displacement
wildcarded), locate the
.rdata qword holding it, and walk backwards while
the qwords are .text pointers; the distance is the index. Do it in the
PDB-bearing release first to learn the constant offset between the binary index
and Endstone's declaration count (a base contributing only a virtual dtor is
+1), then apply the same offset to the new release. Cross-anchor on a
const/non-const overload pair - ICF folds them to one address, so they show up
as two adjacent slots sharing a target, which is unmistakable.
Tracing a Bedrock::PubSub notification path
When an Endstone event fed by a Connector/Publisher stops firing, clear or
convict the BDS side before touching src/bedrock/. Four checks settle it.
- Resolve a slot-numbered lead to a NAME first. Itanium spends 2 slots on
the dtor, MSVC 1, so the same virtual is Itanium slot N and MSVC slot N-1 -
an off-by-two between platforms. Name the slots from the last PDB-bearing
release (the proxy/manager virtuals are usually public even when the vftable
is not) and confirm by
.text address order, which follows declaration
order. Acting on "slot 5 was restructured" without this reads a void
helper as the notification gate.
- A
dispatch<...> instantiation is per-signature and normally has exactly
ONE caller. Xref it in both binaries: equal caller sets prove there is a
single publish site and it did not move. This is far stronger than diffing
the publisher, and it is two E8 rel32 scans.
- Read the notifier itself, not the publisher.
Level::onChunkLoaded-style
notifiers are thin: a chain of proxy vcalls (a read-only early-out, a
fire-once latch returning bool, some side effects, an argument getter) then
the dispatch. Diff it instruction-by-instruction across versions - a stable
one stays byte-identical apart from relocated displacements.
- Ordering against a state field needs the notifier's caller, not the
notifier. Find it by the literal argument pair at the state-transition
call (
tryChangeState(expected, desired)); that pair is version-stable and
usually unique. Beware: BDS discards the CAS result and often publishes
outside the lock, so an Endstone-side state >= X gate is unsound by
construction even when the ordering is unchanged. Prefer the guarantees BDS
already provides (the fire-once latch) over re-deriving them from a field.
Detecting data-member layout changes (ctor/dtor RE)
A struct's member layout - a member inserted, removed, resized, or moved - is
recoverable directly from its constructor and destructor, no headers needed.
Usually crash-driven.
- The crash points at the member. Map the crashing read (an accessor /
_get / _setControlBlock / unique_ptr deref) back to the Endstone member,
then diff that struct's ctor/dtor: new (stripped target) DB vs the previous
named-reference DB.
- Two-DB diff. The previous DB has PDB symbols (named ctor
??0Class@@,
dtor ??1Class@@); the new one usually does not. Extract each member's offset
from both and align top-down. The first offset that differs localises the
change; the delta is the size inserted/removed before it. Confirm a later
anchor member shifted by the same delta (a non-uniform delta means more than
one change - keep going).
- Extract offsets with a
this-relative store tracker. Decompiled ctors are
noisy. Run a small register-tracker over the ctor: seed this(rcx)=0,
propagate this-derived values through mov/lea reg copies and stack
spills, and log every mov [reg+disp], ... whose reg is this-relative plus
every call whose rcx is this-relative (member sub-object ctors). Sorted
offsets = layout in construction (= declaration) order. Same engine on the
dtor gives teardown order.
- Identify a member's TYPE by its ctor/dtor fingerprint (MSVC sizes):
std::string (32): ctor writes capacity =15 at +24; SSO test is the
0x80000000000000 bit on the capacity word.
std::vector (24): ctor zeroes 3 pointers; dtor
if (begin) operator delete(begin, end - begin) - one sized free off the
stored last/end pointers.
std::unordered_map/unordered_set (always 64, any K/V): ctor sets load
factor 1.0f (0x3f800000), mask =7, bucket count =8, a 32-byte
sentinel list node; dtor frees the bucket vector then walks the list. The
per-node operator delete(node, N) reveals the value type via node size.
shared_ptr/weak_ptr (16) and Bedrock::NonOwnerPointer<T> (24 =
shared_ptr 16 + T* 8): dtor is an atomic refcount release -
if (rep) { atomic_dec(rep->uses); vcall rep->__on_zero(vtbl); ... }. This
tells a 24-byte NonOwnerPointer from a 24-byte vector:
refcount-release-with-virtual-call vs operator delete(begin, end-begin).
unique_ptr<T> (8): dtor if (p) { ~T...; operator delete(p, sizeof(T)) }.
Polymorphic T -> virtual deleting dtor (**p)(p, 1); concrete T ->
fixed-size operator delete(p, N). Distinguishes which of two adjacent
unique_ptrs moved, and tells unique_ptr from a raw pointer (not destroyed).
- Other fixed sizes:
std::function 64 (dtor calls a manager via a stored
vtable), BaseGameVersion 32, Core::Cache 72, AABB 24, HashedString
48, mce::Color 16.
- No symbols and no locatable ctor/dtor -> Linux RTTI.
_ZTV<len><Class>
exists in a stripped Linux DB; the dtor is at vtable+16 (Itanium D1
complete) / +24 (D0 deleting). Decompile it and read the teardown the same
way. Caveat: libc++ layouts are not the MSVC offsets, but member order and
member kind are the same, and vector(24)/shared_ptr(16)/NonOwnerPointer(24)
match Windows sizes - enough to confirm what kind of member changed and
where in the order. Diff the new Linux dtor vs the old to see the
extra/missing teardown (the new member appears as an extra teardown adjacent
to its neighbour).
- Caveats.
- A member the dtor destroys is owned (vector/string/smart-ptr/unique_ptr);
a reference or raw pointer member never appears in the dtor, so absence
there is not absence in layout - cross-check the ctor store list.
- Check what Endstone actually accesses before padding the unused middle
(grep the
*Ref/wrapper that exposes the type) - if it reads a tail member,
the whole tail must be byte-accurate, not just padded to size.
- Confirm by decompile, never by size/heuristic alone
([[feedback_decompile_to_confirm]]); the tightest confirmation is a shifted
neighbour whose new offset equals the old member's offset.
Start from the crash, not from a sweep
A layout bug almost always surfaces as an access violation inside a trivial
accessor that just returns a member - getX() { return x_; } - or inside
NonOwnerPointer::_setControlBlock / a shared_ptr copy, because those touch a
control block and fault on garbage. When that happens:
- Read the displacement out of BDS's own accessor in both versions. That one
number is the whole answer and takes minutes; a full ctor/dtor walk takes
hours. Do it first and only escalate if the accessor cannot be located.
- Fix, rebuild, re-run. Each fix moves the crash one step further in, and the
next trace names the next class for free. Iterating the crash is dramatically
faster than trying to statically clear every class up front.
- Rule out your own recent edits before blaming BDS. If a hook's declared
return type or parameters changed, a corrupted
this produces the identical
symptom. Discriminate by where it survives: if the hook already called the
real function through ENDSTONE_HOOK_CALL_ORIGINAL with that this and got
back, this is fine and it is a member offset.
- Beware the accessor that appears to work: a
shared_ptr's pointer is its
first 8 bytes, so a getter returning .get() keeps working while every member
after it is 8 bytes out. Silent, and it hides the real breakage.
The change BDS actually makes most often
A member changing KIND at an unchanged offset, growing 8 -> 16 and shifting
everything after it - overwhelmingly std::unique_ptr -> std::shared_ptr, and
it tends to arrive in clusters across ownership-holding classes in one release.
When you find one, go looking for its siblings in related classes before the
next crash finds them for you.
Judge it by kind, not size: a shared_ptr teardown is an atomic refcount
decrement plus a virtual __on_zero call; a unique_ptr reset is an inline
delete. Same 8-byte delta, completely different fingerprint. Other shapes seen:
a unique_ptr<T> replaced by an inline std::optional<T> (the destructor stops
running a deleter and starts testing an engaged flag over the value's own body),
and a member relocated within the struct with sizeof unchanged - which no size
check can ever detect.
Proof techniques that settle it quickly
- A single instruction changing WIDTH at an unchanged offset proves an
append. A ctor's
mov qword [this+N], 0 becoming movups xmmword means a
new 8-byte member now sits at N+8 and is zero-initialised with its
neighbour. Identical on both platforms, and hard to misread.
- Uniform-delta check over the whole object. If every store from the first
divergence to the end moved by exactly the same delta, there is exactly ONE
change and nothing before it moved. A mixed band (some +8, some 0, some +16)
means multiple changes - keep going.
- Index-align the two dtors' displacement LISTS, don't set-difference them.
Collect the distinct
this-relative displacements each version's D1 touches,
sort both, and pair them up by index. When the two lists are the same length
the pairing is exact and every shift boundary falls out in one read - a run of
d -> d, then a run of d -> d-8, then d -> d-16 says there are two
independent 8-byte shrinks and tells you the offset each one starts at. A set
difference of the same two lists just yields two unaligned piles that look
like seven unrelated changes. ResourcePackManager @ 1.26.44: unchanged
through 144, -8 from 160, -16 from 360, which located both shrinks without
decompiling anything. Cross-check the count first - if the lists differ in
length, a member was added or removed and index pairing is invalid.
- Base-class removal is visible in the typeinfo kind. Itanium
__vmi_class_type_info (multiple bases, with the secondary vtable groups) ->
__si_class_type_info (single base) is a removed base, and the removed base's
own _ZTS name disappears from the whole binary. Every member then shifts by
that base's size.
- RTTI
offset_to_top doubles as a size oracle for a base subobject. A
secondary base's offset_to_top moving -32 -> -40 says the primary subobject
grew 8 bytes, without decompiling anything.
make_shared's allocation size is a whole-object oracle - subtract the right
control block. Its operator new immediate is control block + sizeof(T):
16 on MSVC (_Ref_count_obj2 = vptr + two uint32; the add reg, 0x10
that derives the object confirms it) and 24 on libc++ (its counters are
long, not int). Subtract the wrong one and every size is 8 bytes out. This
is the only size oracle Windows has, /GR- leaving no typeinfo. For a packet
the route needs no symbols beyond one the table already holds:
MinecraftPackets::createPacket is a jump table indexed directly by
MinecraftPacketIds, so table[id] -> make_packet<T> -> operator new is
two hops. Cross-check on Linux, where the D0 dtor's sized
operator delete(this, N) gives sizeof independently.
- A factory's ordered
operator new immediates fingerprint it across versions -
the name-free way to carry a PDB-named sizeof forward. The function that
builds a big object allocates dozens of sub-objects, and that ordered list of
immediates is effectively unique and survives a release nearly unchanged. Match
the list in the new binary to re-identify the same factory, and the one entry
that moved is the new sizeof. ServerLevel (33 allocations, only slot 2
changing 0x998 -> 0x9a0) and ServerScriptManager (0x4e8 -> 0x500) were
both settled this way on Windows with no RTTI and no PDB. Read the immediate
from the operator new argument, or from the mov qword [rsp+0x28], N the
allocation-failure assert spills - the latter is greppable as a byte pattern.
- When no
operator new site exists, MSVC's scalar deleting destructor has
the size. A class that is only ever stack-constructed (most packets -
StartGamePacket has no operator new immediate anywhere in the image) still
gets operator delete(this, sizeof(T)) emitted in vftable slot 0, so one
mov edx, N settles it. Reach slot 0 name-free: a string literal the class
owns -> the stub referencing it -> the .rdata qword holding that stub ->
walk back while the qwords are .text. The walk over-runs into the previous
vftable (MSVC packs them back to back), so take slot 0 to be the
scalar-deleting-dtor body (`mov [rcx], vfta
…(truncated)
1---2name: bump-bds3description: Update Endstone to support a new Bedrock Dedicated Server (BDS) version - regenerate the symbol offset tables and port src/bedrock to the new ABI. Use when bumping the supported BDS version (e.g. "add support for BDS 1.26.x", "bump the BDS version").4---56# Bump Endstone to a new BDS version78Every bump is the same two jobs:9101. **Regenerate the symbol offset tables** - which hook resolves to which11 address (`src/bedrock/symbols/{windows,linux}.h`).122. **Port `src/bedrock/` to the new ABI** - fix the signatures, vtable orders13 and member layouts that changed, so those offsets land on the right code and14 memory is read at the right offsets.1516*How you discover what changed* (and the new signatures job 1 needs) depends on17your reference material. Pick the scenario - the rest of the skill is split18along it:1920- **Scenario A - full (you have `bedrock-headers`).** The dwarf2cpp header diff21 tells you exactly what changed and why. The canonical path; use it whenever22 headers for the target version exist. -> *Scenario A* below.23- **Scenario B - limited (no headers; only IDA databases).** You have a Linux24 BDS database (RTTI present) and a Windows BDS database, and maybe a *stale*25 PDB - but no header diff. You reverse-engineer each ABI change directly from26 the binaries, driven by **symbol misses** (build) and **runtime crashes**.27 -> *Scenario B* below.2829Both scenarios share **The symbol pipeline**, **Editing src/bedrock correctly**,30**Finish**, and most **Gotchas**. A real bump is often mostly A with a few B31spot-checks (confirm a vtable against the binary), or runs as B until headers32land and then finishes as A.3334## NDA boundary (read first)3536This workflow may use two private Mojang-derived artifacts:3738- `bedrock-headers` - C++ headers reconstructed from BDS binaries. Required for39 Scenario A; **absent by definition in Scenario B**.40- `bedrock_server.pdb` - useful for Windows symbol resolution when available and41 *current*, but not published for every release (and a stale one is a trap -42 see Scenario B).4344**Both artifacts are NDA-protected.** Never copy header bodies, class45definitions, full member layouts, PDB dumps, symbol listings, or other private46artifact contents into the public `endstone` repo, its commits, PRs, issues,47logs, or this skill. Endstone's `src/bedrock/` is a hand-written, minimal48reimplementation - only what Endstone needs, in Endstone's own naming - which is49the DMCA-safe form. Treat headers, PDBs, generated dumps, decompiler output, and50diffs as private working references only.5152---5354# The symbol pipeline (shared)5556## How it works5758```59scripts/configs/{windows,linux}.toml signature configs, hand-maintained60 | scripts/dump_symbols.py61 v62src/bedrock/symbols/{windows,linux}.h std::array of name -> offset (committed)63```6465`src/bedrock/symbol.h` `get_symbol()` looks a symbol up by `__FUNCDNAME__` - the66mangled name of Endstone's own declaration in `src/bedrock/`. The symbol-table67key IS the signature of Endstone's reimplementation. Unresolved symbols are68written as `0` and dropped by the dumper (that hook is disabled; the build still69succeeds, unless any TU actually consumes the missing name - then `consteval`70`get_symbol()` throws at compile time).7172## Prerequisites7374- `uv` - runs `dump_symbols.py` (PEP 723 inline deps, no manual install).75- `pdbtool` - `cargo install pdbtool` (Microsoft pdb-rs). Reads a Windows PDB76 when one is available *and current*.77- The target version published in `EndstoneMC/bedrock-server-data` (the Linux78 path downloads the binary from it - check its `versions.json`).79- **Scenario A only:** `bedrock-headers` for the target version (must remain80 private).81- Optional: the Windows BDS PDB (`bedrock_server.pdb`) for the target version.82 Confirm its version matches the exe before trusting `--pdb` (a stale PDB83 silently mis-resolves moved symbols - see Scenario B).8485## Procedure86871. **Branch** off the current release branch (e.g. `v0.11`):88 `git checkout -b feat/<NN.NN>-support` (naming follows `feat/26.10-support`).892. **Bump the config versions** - set `version = "<X.Y.Z>"` in both90 `scripts/configs/windows.toml` and `scripts/configs/linux.toml`, using the91 3-component release string from bedrock-server-data `versions.json`.923. **Regenerate** (run in the background, 25 s to a few minutes):93 - Windows: `uv run --script scripts/dump_symbols.py scripts/configs/windows.toml --pdb <path>/bedrock_server.pdb`94 - Linux: `uv run --script scripts/dump_symbols.py scripts/configs/linux.toml`95 - No (current) Windows PDB? Drop `--pdb` and rely on the byte-`pattern`96 fallback per entry; resolve the gaps the Scenario-B way.974. **Triage the failures** - this tells you *which* symbols broke (the input to98 the porting work). *How* you find the fix is per-scenario.99 - **Windows (PDB by name, then byte pattern):** each entry is looked up by100 mangled `name` in the PDB; entries the PDB has no public record for101 (lambdas, function-local statics) fall back to scanning the entry's102 `pattern`. A miss means *both* failed - the mangled name is gone (MSVC103 encodes the full signature incl. return type, const-ness and access) *and*104 the byte pattern no longer matches. A PDB hit is name-verified; a fallback105 hit ("Found signature (fallback)") is only pattern-verified, like Linux.106 - **Linux (byte-pattern scan):** a miss = the `pattern` in107 `configs/linux.toml` no longer matches. The function usually still exists -108 the pattern went stale. A Linux hit is a pattern match *labelled* with the109 config name; it is not name-verified.110 - Failed on **both** -> real signature/API change.111 - **Windows only** -> the mangled name changed: signature, return type,112 const-ness or access. Itanium omits the return type, so a pure return-type113 change leaves the Linux name intact. A const/access change is fixed in114 Endstone's `src/bedrock/` *declaration* (`__FUNCDNAME__` derives from it),115 not the config alone.116 - **Linux only** -> stale byte pattern; re-extract it (see Gotchas).117118---119120# Editing src/bedrock correctly (shared)121122Whatever told you *what* changed, the edit obeys the same rules. ABI edits are123easy to get subtly wrong - a wrong vtable slot or member offset corrupts memory124silently, caught by neither a compile nor a PR review. **Build and test125iteratively; never batch many unverified ABI edits.**126127- **Function signatures** (especially hooked / `ENDSTONE_HOOK`) - parameter128 types, const/ref, return type must match BDS exactly, or `__FUNCDNAME__` stops129 matching the symbol.130- **Virtual functions** - the vtable order must match BDS. An added / removed /131 reordered virtual shifts every slot below it; mirror the new order (use `= 0`132 placeholders for virtuals Endstone does not implement). Only the slot *count*133 matters for ABI - one `virtual void <name>() = 0;` is one slot whatever its134 signature.135- **Members** - **type, order and size must match for layout**; member *names*136 stay Endstone's own (`lower_case_`), never Mojang's. Width-ambiguous integers:137 bedrock-headers/Linux build `unsigned long` is 64-bit, Windows (LLP64) 32-bit -138 port `unsigned long` as `std::uint64_t` (64-bit on both targets).139- **The first member after a base is per-ABI.** Itanium allocates derived140 members from `dsize(base)`, MSVC from `sizeof(base)`, so a first member with141 alignment < 8 lands at 44 on Linux and 48 on Windows under `Packet` (48/44).142 Mirror whatever BDS's own class starts with - an inline scalar shifts the same143 way, an 8-aligned sub-object does not. `clang++ --target=x86_64-pc-linux-gnu144 -Xclang -fdump-record-layouts` on a self-contained repro prints `dsize` and145 every offset; run it for both targets rather than reasoning about it.146- **Template arguments** - a class template's *default* arguments are part of147 its declaration: copy them verbatim, never guess (e.g. `brstd::bitset`'s148 word-type defaults to `unsigned int`). Never drop an *explicit* argument to149 lean on a default; spell every argument the actual instantiation spells150 (apply the int-width rule to those too).151- **One type per corresponding file** - a needed BDS type Endstone lacks goes in152 its *own* `src/bedrock/` header mirroring the BDS file (snake_case path), then153 `#include`d - do not paste a foreign definition inline. A forward declaration154 used across many headers goes in `src/bedrock/forward.h` (alphabetical); for a155 heavy include chain, forward-declare and use the type incomplete (fine for156 pointers, references, and container value types).157- **Every header must be self-contained.** A sweep that adds one `#include` to158 an events/shard header can re-order the whole chain and expose headers that159 were silently borrowing a transitive include - the symptom is `no template160 named 'X'` plus a cascade of `static_assert` size failures in a file the sweep161 never touched. Include what you use, in the file that uses it. Verify with a162 one-line TU (`#include "<the header>"`) compiled `/Zs` (`-fsyntax-only`) using163 flags lifted from the **build log**, not from the repo-root164 `compile_commands.json`, which goes stale and can miss defines (`-DNOMINMAX`,165 `-DWIN32_LEAN_AND_MEAN`). Sweep the whole sibling directory at once - latent166 cases cluster.167- **Structural refactors** - when BDS introduces a base class, mirror it (add168 the base header, re-parent, move shared members down). When BDS removes a169 class, `git rm` once `grep` confirms nothing references it. Follow BDS170 structure; only the file name differs (snake_case). Keep it minimal.171- **Knowing the type vs placeholdering it.** Scenario A: declare the real type -172 *never* a same-size stand-in. Scenario B: when you cannot name a type/signature173 precisely, use a documented placeholder (see *Scenario B - Placeholders*) -174 but the **size / order / slot-count must still be exact**.175176After the edit: update the mangled `name` in `scripts/configs/{windows,linux}.toml`,177re-run the dumper, and update any affected hook in178`src/endstone/runtime/bedrock_hooks/`.179180---181182# Scenario A - full port with bedrock-headers183184The header diff is the source of truth: it lists every signature, vtable and185member change. Work it stage by stage, then apply each via *Editing src/bedrock186correctly*.187188## Source: the header diff189190`dwarf2cpp` reconstructs C++ headers from a DWARF-bearing BDS build (the Android191build `libminecraftpe.so` carries DWARF; the Windows/Linux server binaries are192stripped). Output lands in `bedrock-headers`, one branch per BDS release193(`android/r26_u1`, `android/r26_u2`, ...).1941951. `dwarf2cpp <libminecraftpe.so> --base-dir <build-root> -o <out>` (or `uvx dwarf2cpp`).1962. In `bedrock-headers`: `git checkout -b android/r<NN>_u<N>`, place the output, commit.1973. `git diff android/r<prev> android/r<new>` is the change set.198199## The actionable set200201`src/bedrock/` is ~655 hand-maintained headers - a small subset of BDS. Most of202a release diff (5000+ files) touches nothing Endstone declares. So:203204> **actionable work = (changed headers) intersect (the 655 src/bedrock headers)**205206Match by normalized basename (lowercase, strip `_` and `-`): bedrock-headers207`Mob.h` <-> Endstone `mob.h`; `BlockSource.h` <-> `block_source.h`.208209## Staged review order210211Review the diff in stages - foundational types first, so later stages do not212rework. Scope: the `handheld/` tree **and** the top-level `src/base/` tree;213**skip `handheld/src-client/`** (game client) and the other top-level `src/`214subtrees (`account`, `external`, `gui` - client / Xbox / third-party). Within215each stage, deep-dive only the intersection. (The first attempt used 3 coarse216stages; "handheld/src non-world" alone was 467 files / 44 intersecting - too217big. Use this finer split:)2182191. `src/base` (top-level, *not* under `handheld/`) - the shared `Core` library:220 foundational utilities and low-level types (`BinaryStream`, ...). Easy to221 miss because every staged path below lives under `handheld/` while this tree222 is separate; a missed change here (e.g. a new `BinaryStream` virtual)223 silently shifts a vtable that Phase 1 can never flag.2242. `src-deps/SharedTypes` - shared types and enums2253. `src/common/network` - packets, network types, packet-id / disconnect enums2264. `src/common/server` (incl. `server/commands`) - server and command system2275. `src/common/entity` - ECS components2286. `src/common/{certificates,resources,scripting,platform,locale,gameplayhandlers,...}` - remaining non-world2297. `src/common/world/actor`2308. `src/common/world/item`2319. `src/common/world/level/block`23210. `src/common/world/level/{dimension,biome}` and remaining `src/common/world/level/*` (chunk, material, storage, level core)23311. `src/common/world/*` - remaining world (`attribute`, `effect`, `events`, `inventory`, `response`, ...)23412. `src-deps` other than SharedTypes (Certificates, VanillaComponents, ...), then anything else23513. **Cross-validate** - once every ABI change is in, re-review the whole236 `src/bedrock/` diff against the bedrock-headers diff. Every edited function237 signature, vtable slot, member type/order, and structural change must trace238 to a concrete change in `git diff android/r<prev> android/r<new>`. Reject239 anything not backed by the diff: no invented types, no guessed members, no240 hallucinated signatures, no "looks-right" edits. A change that cannot be241 matched to the header diff is wrong - revert or fix it. This stage exists242 because the porting stages, especially when parallelised across agents, can243 introduce plausible but unfounded edits - they must all be matched up.244245## Reading the diff: noise to skip246247dwarf2cpp churn that is *not* a real BDS change:248249- **Versioned-namespace churn** - `SharedTypes/v1_26_10/...` becomes250 `v1_26_20/...`; most of that subtree's diff is just the version bump.251- **Template-instantiation churn** - `SharedPtr.h` / `SharedCounter` and similar252 enumerate concrete instantiations (`CopperBlock<ThinFenceBlock>`, ...). The253 set churns every release; Endstone uses its own templates - ignore.254- **File regrouping** - dwarf2cpp regroups types into different generated files.255 A file shown as fully deleted (e.g. `CommonTypes.h`) often just means its256 types moved. Confirm a type is genuinely gone, not relocated.257- **Lambda source-location churn** - `match<(lambda at .../Foo.cpp:47:3)>` -258 line/column numbers shift every build. Pure noise.259- **Declaration reordering** - declarations reordered within a file; the diff260 shows -/+ pairs of identical content moved.261262---263264# Scenario B - limited port from the binaries (Linux RTTI + Windows DB)265266No header diff. You have:267268- a **Linux** BDS database - stripped of function names but **RTTI is intact**269 (`_ZTV<len><Class>` vtables, `_ZTI` typeinfo), so polymorphic classes,270 vtables, and Itanium-mangled names are recoverable;271- a **Windows** BDS database - what Endstone actually hooks (and may carry272 *partial* symbols: some methods demangled even though ctors/vtables are not);273- a **previous, named reference DB** for both platforms (the last version, with274 PDB symbols) to diff against;275- possibly a **stale PDB** - treat with suspicion.276277Run everything through the ida-pro `py_eval` (see [[reference_idalib_mcp_quirks]]);278note that in `py_eval` two top-level `def`s cannot call each other (exec scope) -279nest helpers in one function. `find_bytes` + `py_eval` xrefs stay responsive280when `search_text` / `xrefs_to` / `make_signature` time out on the busy DB.281282## The loop283284Without a diff, work is driven by two signals, fixed one at a time (build/test285between each - see *Editing src/bedrock correctly*):2862871. **Symbol misses** from the dumper (Phase 1 triage) -> *Finding a new symbol /288 offset* below.2892. **Runtime crashes / misbehaviour** once it runs -> a vtable shift290 (*Detecting vtable changes*) or a member-offset shift (*Detecting data-member291 layout changes*). An AV in an accessor/`_get`/`_setControlBlock`/`unique_ptr`292 deref means a field is read at the wrong offset; clean misbehaviour with no293 fault (e.g. a hook whose argument is garbage) often means a hook landed on the294 wrong function. A `std::_Throw_bad_variant_access` thrown from a295 `Script<...>GameplayHandler::handleEvent*` (`event.visit(...)`) is an296 event-variant drift (*Detecting event-variant changes*).297298## Finding a new symbol / offset without a header diff299300- **Navigate by string anchor, not symbol.** To locate an unnamed function:301 take a string literal it references (an error/i18n key like302 `commands.setmaxplayers.success.lowerbound`), `find_bytes` the *ASCII hex* of303 the string, `xref` to the referencing function, and read it. Diff it against304 the previous DB's *named* equivalent (e.g. `SetMaxPlayersCommand::execute`) to305 read off the new offsets/signature. Always `lookup_funcs` the name first - the306 Windows DB's partial symbols may already have it.307- **Re-cut a stale / wrong byte pattern.** Prefer a **prologue** pattern (the308 `push` sequence + `sub rsp`) over a call-site one; the match offset is then the309 function start. For a virtual, re-cut **from the vtable**, not a raw scan: find310 the class vtable (Linux RTTI `_ZTV<len><Class>`; Windows via the documented311 string -> ctor -> `__vftable` store route), take the exact slot (mind the312 dtor-slot difference: Itanium 2 dtor slots, MSVC 1), read the prologue there,313 and wildcard only displacements/immediates.314- **Verify a pattern-resolved offset two ways, not one.** (1) It must be a315 **function start** - `ida_funcs.get_func(ea).start_ea == ea`; an offset that316 lands mid-function is conclusively wrong. (2) **Decompile it** and confirm it317 is the *intended* function ([[feedback_decompile_to_confirm]]) - same-named318 overloads (`sendPacket(string&, Reliability, Compressibility)` vs319 `sendPacket(string&, Packet&, ...)`) have different bodies; match the body to320 what your hook expects. A stale prologue pattern does not just *miss* - it can321 silently match a *different* function with the old shape (this bit322 `BatchedNetworkPeer::sendPacket` at 1.26.32: its codegen added `push r12..r15`,323 so the old `55 56 57 53 ...` pattern collided with a packet-trace overload).324- **Sweeping/verifying the whole table: compare the committed offset's *body*325 against the previous version's *named* function - never against a name.** Two326 traps that each produce a false verdict (both bit a real 1.26.32 sweep):327 - **Same RVA != same function across versions.** Do *not* identify the new328 function by reading what name sits at that RVA in the *old* DB - code329 relocates every release, so the old DB's `0x8e8a00` (`changeToValueType`)330 says nothing about the new DB's `0x8e8a00` (which was the correct331 `RepositorySources::initializePackSource`). Decompile the *new* offset's body332 and match its behaviour to the *old named* function: distinctive callees,333 member-offset writes, or constants (the FNV `0x100000001B3`; literal334 factory-call args like `6`/`4`). A near-match in line count / arg count is335 expected to drift with inlining - judge by behaviour, never by size.336 - **Don't trust the target DB's auto-names or hexrays' inferred prototype.**337 The fresh DB mislabelled a 123 KB function as `ItemInstance::fromTag` while338 the *correct* small one was an unnamed `sub_`; and the real 2-arg339 `initializePackSource(this, PackSourceFactory&)` decompiled as a 4-arg340 `(__int64*, const char*, __int64, __int64)`. The body is ground truth; the341 label and the prototype are guesses.342 - Cheap pre-filter for a 60+ entry table: for each entry, confirm the offset343 is a function start and that its referenced **string set** is a superset of344 the old named function's strings (strings are version-stable). That clears345 the string-bearing majority; decompile-and-compare only the string-less346 residue. (Callee-*name* overlap does **not** work - the target DB's callees347 are almost all unnamed `sub_`.)348- **Beware a stale PDB overriding your fix.** `--pdb` resolves by *name* first,349 so a PDB older than the exe returns the *old* RVA for any moved symbol,350 ignoring your re-cut pattern. When the PDB version can't be trusted, do **not**351 blanket-regenerate (it can clobber currently-correct offsets with stale ones).352 Instead fix the one verified entry in `src/bedrock/symbols/<platform>.h`353 **directly** (hand-patch the offset) and update the `pattern` for the next354 clean regen. Cross-check the other platform - the same function on Linux355 (`_ZN...`) often resolved fine (different codegen), confirming a Windows-only356 change.357358## Detecting vtable changes (Linux RTTI)359360Confirm directly against the binary whether a virtual was **added / removed /361reordered**, and at exactly which slot - name-free. Every polymorphic class has362`_ZTV<len><Class>` + `_ZTI<len><Class>` even in a fresh, PDB-less DB; the slot363targets are `sub_` in *both* DBs (even the named reference only has RTTI symbols,364not the virtuals), so you diff layout without any virtual-function names.3653661. **Walk the vtable from its address point.** Resolve `_ZTV<len><Class>`, skip367 each `(offset_to_top, _ZTI<len><Class>)` header pair, then collect qwords368 while the target sits in an executable segment369 (`ida_segment.getseg(q).perm & 1`) or is `__cxa_pure_virtual`; stop at the370 first non-pointer / next typeinfo. **Do not gate on `ida_funcs.get_func`** -371 the fresh DB has not defined most vtable targets as functions yet, so it stops372 the walk early; exec-segment membership works regardless of analysis.3732. **Length per class brackets the change.** A primary vtable lays out374 `[base virtuals][derived adds]` in order, so each class's *own* `_ZTV` length375 localises a net add/remove to one class. For a hierarchy, the most-derived376 concrete class (e.g. `ServerPlayer` covers `Actor -> Mob -> Player ->377 ServerPlayer`) gives the whole chain in one read. Equal length on every level378 = no net change (still verify order).3793. **Structural fingerprint** confirms no same-count shuffle, name-free: tag each380 slot `P` = `__cxa_pure_virtual`, `T` = this-adjusting thunk (`48 83 ef` /381 `48 81 ef` = `sub rdi`), `R` = repeats previous target (shared-stub runs),382 `.` = normal. Identical tag strings across DBs => no reshuffle.3834. **Localise the exact slot by signature alignment.** Per slot, build a384 recompilation-robust signature - decode the first ~6 instructions385 (`ida_ua.decode_insn`) keeping mnemonic + operand register classes but386 **dropping immediates and displacements**. Bridge the two IDA processes via a387 temp file: dump the old DB's per-slot signatures to JSON, switch DBs,388 recompute, `difflib.SequenceMatcher` the two lists. `insert`/`delete` opcodes389 are the real structural change; `replace` opcodes are functions whose body390 changed at the same slot - ignore them.3915. **Confirm by decompile, never by the heuristic alone**392 ([[feedback_decompile_to_confirm]]). Decompile the boundary in both DBs: the393 shifted neighbour (`NEW[slot+1]`) must match `OLD[slot]`, and the inserted394 `NEW[slot]` is often *referenced by* its shifted neighbour (e.g. a new395 per-position helper the shifted loop calls via `vtbl+offset`) - the tightest396 possible confirmation.3976. **Map the slot to the Endstone declaration, then validate locally.** Count398 the header's virtuals (dtor = 2 slots; each overload = 1; skip commented-out;399 honour `#ifdef __linux__`) plus any base's slots. **Anchor the count** on a400 virtual whose address you know in both versions (a hooked one from the toml) -401 its slot must equal your predicted index. Beware: an Endstone header may omit402 Linux-only virtuals (the `#blameMojang` ones), so the cumulative count can be403 short of the real vtable - do not trust it globally. Decompile the few slots404 *around* the change and match them to neighbouring declarations (a const/405 non-const overload pair returning the same `this+N` subobject is an406 unmistakable anchor). If the local sequence lines up, the insert point is407 pinned regardless of any global gap.4087. **Add the placeholder.** A single non-dtor `virtual void <name>() = 0;`409 occupies exactly one slot. Name it a clearly-marked placeholder (don't invent410 a Mojang name) and comment the observed signature/behaviour. If you cannot411 confirm the change is on *both* platforms, match the existing `#ifdef __linux__`412 pattern rather than risk shifting the Windows vtable.413414For one class you do not need IDA at all: `lief` + `capstone` on the shipped415binaries resolve `_ZTS<len><Class>` -> typeinfo -> vtable -> per-slot disassembly416in seconds, on both the previous and the new ELF. (`lief`'s `Binary.relocations`417comes back empty on the stripped BDS ELF - parse `.rela.dyn` yourself as 24-byte418`(offset, info, addend)` records and keep `info & 0xffffffff == 8`.)419420## Locating a vtable on Windows (no RTTI)421422`/GR-` leaves no type descriptors, but a small interface is still findable423name-free - and this is the only way to confirm a Linux-derived vtable verdict on424the platform Endstone actually hooks.4254261. **Scan `.rdata` for a run of consecutive pointers to `lea rax, [rcx+d]; ret`427 stubs.** Trivial base-subobject getters are ICF-folded to **one stub per428 displacement** binary-wide (~100 in a 160 MB `.text`), so a class with N of429 them is a run of N adjacent stubs with distinct increasing displacements, and430 the displacements read off the member layout directly. These stubs are 16-byte431 aligned and `cc`-padded but have **no `.pdata` record** (leaf, no unwind) - a432 `.pdata` function-start filter silently drops every one of them and the scan433 returns nothing.4342. **Identify the class from the dtor slot, not a name.** The slot before the run435 is the scalar deleting dtor; its teardown pins both the class and its `sizeof`436 (a virtual-deleting `unique_ptr` at `+N` = the last member). Comparing that437 body against the last release that shipped a PDB is the identification.4383. **Never read the table's END from "the next qword is not code".** MSVC packs439 vftables back to back in `.rdata`, so the next table's slot 0 is a code440 pointer - the same class reads as 4 slots in one release and 8 in the next441 purely by what the linker put after it. The terminator is: the next qword's own442 *address* is the target of a `lea` + `mov [reg], rax` vfptr store.4434. **A displacement proves the slot order, never the semantics.** Which member a444 getter returns comes from the last PDB-bearing release (`??_7<Class>@@6B@` plus445 the named getters); carry that mapping forward version by version and diff446 displacements. Equal displacements at equal slots across the chain is what447 makes an "unchanged" verdict conclusive rather than merely shape-compatible.4485. **To check ONE virtual's slot index in a 400-slot table**, don't walk the449 table - find the function (byte fingerprint of its body, member displacement450 wildcarded), locate the `.rdata` qword holding it, and walk *backwards* while451 the qwords are `.text` pointers; the distance is the index. Do it in the452 PDB-bearing release first to learn the constant offset between the binary index453 and Endstone's declaration count (a base contributing only a virtual dtor is454 `+1`), then apply the same offset to the new release. Cross-anchor on a455 const/non-const overload pair - ICF folds them to one address, so they show up456 as two adjacent slots sharing a target, which is unmistakable.457458## Tracing a Bedrock::PubSub notification path459460When an Endstone event fed by a `Connector`/`Publisher` stops firing, clear or461convict the BDS side before touching `src/bedrock/`. Four checks settle it.4624631. **Resolve a slot-numbered lead to a NAME first.** Itanium spends 2 slots on464 the dtor, MSVC 1, so the same virtual is Itanium slot N and MSVC slot N-1 -465 an off-by-two between platforms. Name the slots from the last PDB-bearing466 release (the proxy/manager virtuals are usually public even when the vftable467 is not) and confirm by `.text` address order, which follows declaration468 order. Acting on "slot 5 was restructured" without this reads a `void`469 helper as the notification gate.4702. **A `dispatch<...>` instantiation is per-signature and normally has exactly471 ONE caller.** Xref it in both binaries: equal caller sets prove there is a472 single publish site and it did not move. This is far stronger than diffing473 the publisher, and it is two `E8` rel32 scans.4743. **Read the notifier itself, not the publisher.** `Level::onChunkLoaded`-style475 notifiers are thin: a chain of proxy vcalls (a read-only early-out, a476 fire-once latch returning `bool`, some side effects, an argument getter) then477 the dispatch. Diff it instruction-by-instruction across versions - a stable478 one stays byte-identical apart from relocated displacements.4794. **Ordering against a state field needs the notifier's caller, not the480 notifier.** Find it by the *literal* argument pair at the state-transition481 call (`tryChangeState(expected, desired)`); that pair is version-stable and482 usually unique. Beware: BDS discards the CAS result and often publishes483 *outside* the lock, so an Endstone-side `state >= X` gate is unsound by484 construction even when the ordering is unchanged. Prefer the guarantees BDS485 already provides (the fire-once latch) over re-deriving them from a field.486487## Detecting data-member layout changes (ctor/dtor RE)488489A struct's member layout - a member inserted, removed, resized, or moved - is490recoverable directly from its **constructor and destructor**, no headers needed.491Usually crash-driven.4924931. **The crash points at the member.** Map the crashing read (an accessor /494 `_get` / `_setControlBlock` / `unique_ptr` deref) back to the Endstone member,495 then diff that struct's ctor/dtor: new (stripped target) DB vs the previous496 named-reference DB.4972. **Two-DB diff.** The previous DB has PDB symbols (named ctor `??0Class@@`,498 dtor `??1Class@@`); the new one usually does not. Extract each member's offset499 from both and align top-down. The first offset that differs localises the500 change; the delta is the size inserted/removed before it. Confirm a later501 anchor member shifted by the *same* delta (a non-uniform delta means more than502 one change - keep going).5033. **Extract offsets with a `this`-relative store tracker.** Decompiled ctors are504 noisy. Run a small register-tracker over the ctor: seed `this`(rcx)=0,505 propagate `this`-derived values through `mov`/`lea` reg copies and stack506 spills, and log every `mov [reg+disp], ...` whose reg is `this`-relative plus507 every `call` whose rcx is `this`-relative (member sub-object ctors). Sorted508 offsets = layout in construction (= declaration) order. Same engine on the509 dtor gives teardown order.5104. **Identify a member's TYPE by its ctor/dtor fingerprint** (MSVC sizes):511 - `std::string` (32): ctor writes capacity `=15` at `+24`; SSO test is the512 `0x80000000000000` bit on the capacity word.513 - `std::vector` (24): ctor zeroes 3 pointers; dtor514 `if (begin) operator delete(begin, end - begin)` - one sized free off the515 stored last/end pointers.516 - `std::unordered_map`/`unordered_set` (always 64, any K/V): ctor sets load517 factor `1.0f` (`0x3f800000`), mask `=7`, bucket count `=8`, a 32-byte518 sentinel list node; dtor frees the bucket vector then walks the list. The519 per-node `operator delete(node, N)` reveals the value type via node size.520 - `shared_ptr`/`weak_ptr` (16) and `Bedrock::NonOwnerPointer<T>` (24 =521 shared_ptr 16 + `T*` 8): dtor is an **atomic refcount release** -522 `if (rep) { atomic_dec(rep->uses); vcall rep->__on_zero(vtbl); ... }`. This523 tells a 24-byte `NonOwnerPointer` from a 24-byte `vector`:524 refcount-release-with-virtual-call vs `operator delete(begin, end-begin)`.525 - `unique_ptr<T>` (8): dtor `if (p) { ~T...; operator delete(p, sizeof(T)) }`.526 Polymorphic `T` -> **virtual deleting dtor** `(**p)(p, 1)`; concrete `T` ->527 fixed-size `operator delete(p, N)`. Distinguishes which of two adjacent528 unique_ptrs moved, and tells `unique_ptr` from a raw pointer (not destroyed).529 - Other fixed sizes: `std::function` 64 (dtor calls a manager via a stored530 vtable), `BaseGameVersion` 32, `Core::Cache` 72, `AABB` 24, `HashedString`531 48, `mce::Color` 16.5325. **No symbols *and* no locatable ctor/dtor -> Linux RTTI.** `_ZTV<len><Class>`533 exists in a stripped Linux DB; the dtor is at `vtable+16` (Itanium D1534 complete) / `+24` (D0 deleting). Decompile it and read the teardown the same535 way. Caveat: libc++ layouts are *not* the MSVC offsets, but **member order and536 member kind are the same**, and `vector`(24)/`shared_ptr`(16)/`NonOwnerPointer`(24)537 match Windows sizes - enough to confirm *what kind* of member changed and538 *where* in the order. Diff the new Linux dtor vs the old to see the539 extra/missing teardown (the new member appears as an extra teardown adjacent540 to its neighbour).5416. **Caveats.**542 - A member the dtor *destroys* is owned (vector/string/smart-ptr/unique_ptr);543 a **reference or raw pointer member never appears in the dtor**, so absence544 there is not absence in layout - cross-check the ctor store list.545 - Check what Endstone **actually accesses** before padding the unused middle546 (grep the `*Ref`/wrapper that exposes the type) - if it reads a tail member,547 the whole tail must be byte-accurate, not just padded to size.548 - Confirm by decompile, never by size/heuristic alone549 ([[feedback_decompile_to_confirm]]); the tightest confirmation is a shifted550 neighbour whose new offset equals the old member's offset.551552### Start from the crash, not from a sweep553554A layout bug almost always surfaces as an **access violation inside a trivial555accessor that just returns a member** - `getX() { return x_; }` - or inside556`NonOwnerPointer::_setControlBlock` / a `shared_ptr` copy, because those touch a557control block and fault on garbage. When that happens:558559- **Read the displacement out of BDS's own accessor in both versions. That one560 number is the whole answer** and takes minutes; a full ctor/dtor walk takes561 hours. Do it first and only escalate if the accessor cannot be located.562- **Fix, rebuild, re-run.** Each fix moves the crash one step further in, and the563 next trace names the next class for free. Iterating the crash is dramatically564 faster than trying to statically clear every class up front.565- **Rule out your own recent edits before blaming BDS.** If a hook's declared566 return type or parameters changed, a corrupted `this` produces the identical567 symptom. Discriminate by *where* it survives: if the hook already called the568 real function through `ENDSTONE_HOOK_CALL_ORIGINAL` with that `this` and got569 back, `this` is fine and it is a member offset.570- Beware the accessor that appears to work: a `shared_ptr`'s pointer is its571 first 8 bytes, so a getter returning `.get()` keeps working while every member572 *after* it is 8 bytes out. Silent, and it hides the real breakage.573574### The change BDS actually makes most often575576**A member changing KIND at an unchanged offset**, growing 8 -> 16 and shifting577everything after it - overwhelmingly `std::unique_ptr` -> `std::shared_ptr`, and578it tends to arrive in clusters across ownership-holding classes in one release.579When you find one, **go looking for its siblings** in related classes before the580next crash finds them for you.581582Judge it by **kind, not size**: a `shared_ptr` teardown is an atomic refcount583decrement plus a virtual `__on_zero` call; a `unique_ptr` reset is an inline584delete. Same 8-byte delta, completely different fingerprint. Other shapes seen:585a `unique_ptr<T>` replaced by an inline `std::optional<T>` (the destructor stops586running a deleter and starts testing an engaged flag over the value's own body),587and a member relocated within the struct with `sizeof` unchanged - which no size588check can ever detect.589590### Proof techniques that settle it quickly591592- **A single instruction changing WIDTH at an unchanged offset proves an593 append.** A ctor's `mov qword [this+N], 0` becoming `movups xmmword` means a594 new 8-byte member now sits at `N+8` and is zero-initialised with its595 neighbour. Identical on both platforms, and hard to misread.596- **Uniform-delta check over the whole object.** If *every* store from the first597 divergence to the end moved by exactly the same delta, there is exactly ONE598 change and nothing before it moved. A mixed band (some +8, some 0, some +16)599 means multiple changes - keep going.600- **Index-align the two dtors' displacement LISTS, don't set-difference them.**601 Collect the distinct `this`-relative displacements each version's D1 touches,602 sort both, and pair them up by index. When the two lists are the same length603 the pairing is exact and every shift boundary falls out in one read - a run of604 `d -> d`, then a run of `d -> d-8`, then `d -> d-16` says there are two605 independent 8-byte shrinks and tells you the offset each one starts at. A set606 difference of the same two lists just yields two unaligned piles that look607 like seven unrelated changes. `ResourcePackManager @ 1.26.44`: unchanged608 through 144, -8 from 160, -16 from 360, which located both shrinks without609 decompiling anything. Cross-check the count first - if the lists differ in610 length, a member was added or removed and index pairing is invalid.611- **Base-class removal is visible in the typeinfo kind.** Itanium612 `__vmi_class_type_info` (multiple bases, with the secondary vtable groups) ->613 `__si_class_type_info` (single base) is a removed base, and the removed base's614 own `_ZTS` name disappears from the whole binary. Every member then shifts by615 that base's size.616- **RTTI `offset_to_top` doubles as a size oracle for a base subobject.** A617 secondary base's `offset_to_top` moving -32 -> -40 says the primary subobject618 grew 8 bytes, without decompiling anything.619- **`make_shared`'s allocation size is a whole-object oracle - subtract the right620 control block.** Its `operator new` immediate is `control block + sizeof(T)`:621 **16** on MSVC (`_Ref_count_obj2` = vptr + two `uint32`; the `add reg, 0x10`622 that derives the object confirms it) and **24** on libc++ (its counters are623 `long`, not `int`). Subtract the wrong one and every size is 8 bytes out. This624 is the only size oracle Windows has, `/GR-` leaving no typeinfo. For a packet625 the route needs no symbols beyond one the table already holds:626 `MinecraftPackets::createPacket` is a jump table indexed *directly* by627 `MinecraftPacketIds`, so `table[id]` -> `make_packet<T>` -> `operator new` is628 two hops. Cross-check on Linux, where the D0 dtor's sized629 `operator delete(this, N)` gives `sizeof` independently.630- **A factory's ordered `operator new` immediates fingerprint it across versions -631 the name-free way to carry a PDB-named `sizeof` forward.** The function that632 builds a big object allocates dozens of sub-objects, and that ordered list of633 immediates is effectively unique and survives a release nearly unchanged. Match634 the list in the new binary to re-identify the same factory, and the one entry635 that moved is the new `sizeof`. `ServerLevel` (33 allocations, only slot 2636 changing `0x998` -> `0x9a0`) and `ServerScriptManager` (`0x4e8` -> `0x500`) were637 both settled this way on Windows with no RTTI and no PDB. Read the immediate638 from the `operator new` argument, or from the `mov qword [rsp+0x28], N` the639 allocation-failure assert spills - the latter is greppable as a byte pattern.640- **When no `operator new` site exists, MSVC's scalar deleting destructor has641 the size.** A class that is only ever stack-constructed (most packets -642 `StartGamePacket` has no `operator new` immediate anywhere in the image) still643 gets `operator delete(this, sizeof(T))` emitted in **vftable slot 0**, so one644 `mov edx, N` settles it. Reach slot 0 name-free: a string literal the class645 owns -> the stub referencing it -> the `.rdata` qword holding that stub ->646 walk back while the qwords are `.text`. The walk over-runs into the *previous*647 vftable (MSVC packs them back to back), so take slot 0 to be the648 scalar-deleting-dtor body (`mov [rcx], vfta649650…(truncated)