Fvtt Data Migrations

Foundry VTT Data Migrations

Implement safe, version-controlled data migrations for Foundry VTT modules when changing data structures or storage locations.

When to Use This Skill

Invoke this skill when implementing changes that require migrating existing user data:

✅ ALWAYS Create Migration For:

1. Moving data between storage locations:

   • actor.system.field → actor.flags.myModule.field
   • actor.system.field → embedded item
   • Flag namespace changes

2. Changing data structures:

   • Array → Object/Map
   • Object → Array
   • Flat structure → nested structure
   • Renaming properties

3. Changing data types:

   • String → Number
   • Boolean → String (enum)
   • Single value → Array
   • Null semantics changes

4. Removing deprecated data:

   • Cleaning up orphaned flags
   • Removing obsolete system fields
   • Purging invalid data

5. Foundry version compatibility:

   • API changes between Foundry versions
   • Deprecation of core fields

❌ NO Migration Needed For:

1. Additive changes only:

   • Adding NEW optional fields (with defaults)
   • Adding NEW features that don't touch existing data
   • UI-only changes (CSS, templates without data changes)

2. Non-breaking changes:

   • Adding new flags alongside existing ones
   • Extending data without modifying existing structure
   • Backwards-compatible additions

Foundry Migration System Overview

Schema Version Pattern

Foundry modules track data structure versions using a schemaVersion setting:

// In settings.js - register the version tracker
game.settings.register("my-module", "schemaVersion", {
  name: "Schema Version",
  scope: "world",        // World-level (not per-client)
  config: false,         // Hidden from UI
  type: Number,
  default: 0,            // 0 = never migrated
});

How it works:

1. New worlds start at version 0
2. Each migration increments the version (1, 2, 3...)
3. Migration runs ONCE per world, on first ready hook
4. If currentVersion >= targetVersion, migration is skipped

Migration Lifecycle

World Created (v0)
    ↓
First Load → ready hook
    ↓
Migration.migrate() called
    ↓
Check: currentVersion (0) < targetVersion (1)?
    ↓ YES
Run migration steps
    ↓
Set schemaVersion = 1
    ↓
Show completion notification
    ↓
Future loads: currentVersion (1) >= targetVersion (1) → SKIP

Step-by-Step Migration Workflow

Step 1: Identify the Breaking Change

Ask yourself:

• What data structure is changing?
• Where is the old data stored? (system field? flag? embedded doc?)
• Where will the new data be stored?
• What transformation is needed?

Example scenarios:

// Scenario A: Moving system field to flag
OLD: actor.system["background-details"]
NEW: actor.flags["bitd-alternate-sheets"].background_details

// Scenario B: Changing structure
OLD: actor.flags.myModule["equipped-items"] = [{ id: "abc" }, { id: "def" }]
NEW: actor.flags.myModule["equipped-items"] = { "abc": {...}, "def": {...} }

// Scenario C: Cleaning up orphaned data
OLD: actor.flags.myModule.abilityProgress = { "Reflexes": 2, "abc123": 1 }
NEW: actor.flags.myModule.abilityProgress = { "abc123": 1 }  // Remove name-based keys

Step 2: Increment Target Schema Version

In scripts/migration.js:

export class Migration {
    static async migrate() {
        const currentVersion = game.settings.get(MODULE_ID, "schemaVersion") || 0;
        const targetVersion = 2;  // ← INCREMENT THIS (was 1, now 2)

        if (currentVersion < targetVersion) {
            // ... migration logic
        }
    }
}

Version numbering:

• Start: targetVersion = 1 (first migration)
• Each new migration: increment by 1 (2, 3, 4...)
• Never skip numbers
• Never decrement

Step 3: Create Migration Method

Add a new static method to Migration class:

static async migrateFieldName(actor) {
    const oldValue = foundry.utils.getProperty(actor, "system.old-field");
    const newValue = actor.getFlag(MODULE_ID, "new_field");

    // Check if migration needed
    if (!oldValue) return;  // No old data to migrate
    if (newValue) {
        // New data already exists - just clean up old
        await actor.update({ "system.-=old-field": null });
        return;
    }

    // Perform migration
    const updates = {};
    updates[`flags.${MODULE_ID}.new_field`] = oldValue;
    updates["system.-=old-field"] = null;

    await actor.update(updates);
    console.log(`[MyModule] Migrated old-field for ${actor.name}`);
}

Migration method naming:

• migrate{FeatureName} - Descriptive of what's being migrated
• Examples: migrateEquippedItems, migrateAbilityProgress, migrateLegacyFields

Step 4: Call Migration Method in migrate()

Add to the migration sequence:

static async migrate() {
    const currentVersion = game.settings.get(MODULE_ID, "schemaVersion") || 0;
    const targetVersion = 2;

    if (currentVersion < targetVersion) {
        ui.notifications.info(
            "My Module: Migrating data, please wait..."
        );

        for (const actor of game.actors) {
            if (actor.type !== "character") continue;

            // Step 1: Existing migration
            await this.migrateEquippedItems(actor);

            // Step 2: NEW migration (add here)
            await this.migrateFieldName(actor);
        }

        await game.settings.set(MODULE_ID, "schemaVersion", targetVersion);
        ui.notifications.info("My Module: Migration complete.");
    }
}

Order considerations:

• Dependencies first (if migration B relies on migration A, run A first)
• Independent migrations can be in any order
• Comment the purpose of each step

Step 5: Test Migration

Testing checklist:

1. Create test world with OLD data structure:

   • Use console to create old-format data: game.actors.getName("Test").update({ "system.old-field": "test" })
   • Verify old data exists

2. Trigger migration:

   • Reload world (triggers ready hook)
   • Watch console for migration logs
   • Check for errors

3. Verify migration results:

   • Check new data exists: game.actors.getName("Test").getFlag("my-module", "new_field")
   • Check old data removed: game.actors.getName("Test").system["old-field"] (should be undefined)
   • Verify notification appeared

4. Verify migration runs ONCE:

   • Reload world again
   • Check console - should NOT see migration logs (already at target version)

5. Test with multiple actors:

   • Create several test actors with old data
   • Verify all migrate correctly

Safe Migration Patterns

Pattern 1: System Field → Flag

static async migrateLegacyFields(actor) {
    const updates = {};
    let changed = false;

    // Get old and new locations
    const oldValue = foundry.utils.getProperty(actor, "system.old-field");
    const newValue = actor.getFlag(MODULE_ID, "new_field");

    if (oldValue && !newValue) {
        // Old exists, new doesn't - migrate
        updates[`flags.${MODULE_ID}.new_field`] = oldValue;
        updates["system.-=old-field"] = null;
        changed = true;
    } else if (oldValue && newValue) {
        // Both exist - favor new, clean up old
        updates["system.-=old-field"] = null;
        changed = true;
    }

    if (changed) {
        await actor.update(updates);
        console.log(`[MyModule] Migrated old-field for ${actor.name}`);
    }
}

Key points:

• Check both old and new locations
• Handle case where both exist (favor new)
• Handle case where only old exists (migrate)
• Batch updates into single actor.update() call

Pattern 2: Array → Object/Map

static async migrateEquippedItems(actor) {
    const equipped = actor.getFlag(MODULE_ID, "equipped-items");

    // Check if still in old format (array)
    if (Array.isArray(equipped)) {
        const newMap = {};
        for (const item of equipped) {
            if (item && item.id) {
                newMap[item.id] = item;
            }
        }

        await actor.setFlag(MODULE_ID, "equipped-items", newMap);
        console.log(`[MyModule] Migrated equipped items for ${actor.name}`);
    }
}

Key points:

• Use Array.isArray() to detect old format
• Transform data structure
• Handle null/undefined items gracefully
• Use setFlag() to replace entire structure

Pattern 3: Clean Up Orphaned Data

static async migrateAbilityProgress(actor) {
    const progressMap = actor.getFlag(MODULE_ID, "abilityProgress") || {};
    if (foundry.utils.isEmpty(progressMap)) return;

    let changed = false;
    const updates = {};

    for (const [key, value] of Object.entries(progressMap)) {
        // Foundry IDs are always 16 characters
        if (key.length !== 16) {
            // This is likely a name-based key (orphaned) - remove it
            updates[`flags.${MODULE_ID}.abilityProgress.-=${key}`] = null;
            changed = true;
        }
    }

    if (changed) {
        await actor.update(updates);
        console.log(`[MyModule] Cleaned orphaned flags for ${actor.name}`);
    }
}

Key points:

• Use heuristics to identify orphaned data (length, format, etc.)
• Use -= syntax to remove specific keys: "flags.module.field.-=keyToRemove"
• Log what was cleaned for debugging

Pattern 4: Batch Multiple Field Migrations

static async migrateLegacyFields(actor) {
    const updates = {};
    let changed = false;

    // Migration 1: background-details
    const oldDetails = foundry.utils.getProperty(actor, "system.background-details");
    const newDetails = actor.getFlag(MODULE_ID, "background_details");
    if (oldDetails && !newDetails) {
        updates[`flags.${MODULE_ID}.background_details`] = oldDetails;
        updates["system.-=background-details"] = null;
        changed = true;
    }

    // Migration 2: vice-purveyor
    const oldPurveyor = foundry.utils.getProperty(actor, "system.vice-purveyor");
    const newPurveyor = actor.getFlag(MODULE_ID, "vice_purveyor");
    if (oldPurveyor && !newPurveyor) {
        updates[`flags.${MODULE_ID}.vice_purveyor`] = oldPurveyor;
        updates["system.-=vice-purveyor"] = null;
        changed = true;
    }

    // Single batched update for all changes
    if (changed) {
        await actor.update(updates);
        console.log(`[MyModule] Migrated legacy fields for ${actor.name}`);
    }
}

Key points:

• Accumulate all updates in single updates object
• Single actor.update() call at the end
• More efficient than multiple sequential updates

Foundry Document Update Syntax

Setting Values

updates["system.field"] = newValue;                 // Set system field
updates[`flags.${MODULE_ID}.field`] = newValue;     // Set flag
updates["system.nested.deep.field"] = newValue;     // Set nested field

Removing Fields (Unset)

updates["system.-=oldField"] = null;                // Remove system field
updates[`flags.${MODULE_ID}.-=oldFlag`] = null;     // Remove flag
updates["flags.module.map.-=keyName"] = null;       // Remove map key

The -= syntax:

• Foundry's special unset operator
• Completely removes the field from the document
• Different from setting to null or undefined (which leaves the key)

Using foundry.utils.getProperty

// Safe nested property access (returns undefined if any level missing)
const value = foundry.utils.getProperty(actor, "system.deep.nested.field");

// vs direct access (throws if intermediate property missing)
const value = actor.system.deep.nested.field;  // Error if 'deep' undefined!

Actor Iteration Patterns

By Actor Type

for (const actor of game.actors) {
    if (actor.type !== "character") continue;  // Only migrate characters
    await this.migrateCharacterFields(actor);
}

All Actor Types

for (const actor of game.actors) {
    // Migrate all actors regardless of type
    await this.migrateCommonFields(actor);
}

Specific Types Only

for (const actor of game.actors) {
    if (actor.type === "character") {
        await this.migrateCharacterFields(actor);
    } else if (actor.type === "crew") {
        await this.migrateCrewFields(actor);
    }
}

User Notifications

// Start notification
ui.notifications.info("My Module: Migrating data, please wait...");

// After migration completes
ui.notifications.info("My Module: Migration complete.");

// Error notification (if migration fails)
ui.notifications.error("My Module: Migration failed. See console for details.");

When to notify:

• Always show start notification (sets user expectation)
• Always show completion notification
• Show error notification if any migration fails
• Keep messages concise and actionable

Console Logging

// Per-actor migration success
console.log(`[MyModule] Migrated equipped items for ${actor.name}`);

// Overall migration start
console.log("[MyModule] Starting migration to schema v2");

// Debug information
console.log(`[MyModule] Found ${Object.keys(progressMap).length} progress entries`);

Logging best practices:

• Use module prefix: [MyModule]
• Include actor name for per-actor logs
• Use console.log for success (not console.error)
• Log meaningful progress (not every check, just actions taken)

Common Migration Scenarios

Scenario: Renaming a Flag

static async renameFlag(actor) {
    const oldValue = actor.getFlag(MODULE_ID, "oldName");
    const newValue = actor.getFlag(MODULE_ID, "newName");

    if (oldValue && !newValue) {
        await actor.setFlag(MODULE_ID, "newName", oldValue);
        await actor.unsetFlag(MODULE_ID, "oldName");
        console.log(`[MyModule] Renamed flag for ${actor.name}`);
    }
}

Scenario: Combining Multiple Flags

static async combineFlags(actor) {
    const flag1 = actor.getFlag(MODULE_ID, "flag1");
    const flag2 = actor.getFlag(MODULE_ID, "flag2");
    const combined = actor.getFlag(MODULE_ID, "combined");

    if ((flag1 || flag2) && !combined) {
        const newValue = {
            part1: flag1 || {},
            part2: flag2 || {}
        };

        const updates = {};
        updates[`flags.${MODULE_ID}.combined`] = newValue;
        updates[`flags.${MODULE_ID}.-=flag1`] = null;
        updates[`flags.${MODULE_ID}.-=flag2`] = null;

        await actor.update(updates);
        console.log(`[MyModule] Combined flags for ${actor.name}`);
    }
}

Scenario: Migrating Embedded Documents

static async migrateItemData(actor) {
    const updates = [];

    for (const item of actor.items) {
        if (item.type === "ability") {
            const oldField = item.system.oldField;
            if (oldField) {
                updates.push({
                    _id: item.id,
                    "system.newField": oldField,
                    "system.-=oldField": null
                });
            }
        }
    }

    if (updates.length > 0) {
        await actor.updateEmbeddedDocuments("Item", updates);
        console.log(`[MyModule] Migrated ${updates.length} items for ${actor.name}`);
    }
}

Error Handling

static async migrate() {
    const currentVersion = game.settings.get(MODULE_ID, "schemaVersion") || 0;
    const targetVersion = 2;

    if (currentVersion < targetVersion) {
        ui.notifications.info("My Module: Migrating data, please wait...");

        try {
            for (const actor of game.actors) {
                if (actor.type !== "character") continue;

                // Wrap each migration in try-catch to prevent one failure from blocking others
                try {
                    await this.migrateEquippedItems(actor);
                    await this.migrateAbilityProgress(actor);
                } catch (err) {
                    console.error(`[MyModule] Migration failed for ${actor.name}:`, err);
                    ui.notifications.warn(`Migration failed for ${actor.name} - see console`);
                }
            }

            // Update version even if some actors failed
            await game.settings.set(MODULE_ID, "schemaVersion", targetVersion);
            ui.notifications.info("My Module: Migration complete.");

        } catch (err) {
            console.error("[MyModule] Critical migration error:", err);
            ui.notifications.error("My Module: Migration failed. See console for details.");
        }
    }
}

Error handling strategies:

• Wrap entire migration in try-catch
• Optionally wrap per-actor migrations to continue on individual failures
• Always log errors with context (actor name, migration step)
• Set schemaVersion even if some migrations fail (prevents infinite retry loop)

Testing Migration with Console

Create Test Data (Old Format)

// In browser console
const actor = game.actors.getName("Test Character");

// Set old system field
await actor.update({ "system.old-field": "test value" });

// Set old array-based flag
await actor.setFlag("my-module", "equipped-items", [{ id: "abc123" }]);

// Verify old data
console.log(actor.system["old-field"]);  // "test value"
console.log(actor.getFlag("my-module", "equipped-items"));  // [{ id: "abc123" }]

Trigger Migration Manually

// Force schemaVersion back to 0
await game.settings.set("my-module", "schemaVersion", 0);

// Run migration
await Migration.migrate();

// Check results
const actor = game.actors.getName("Test Character");
console.log(actor.getFlag("my-module", "new_field"));  // "test value"
console.log(actor.system["old-field"]);  // undefined (removed)

Quick Checklist

Before committing migration code:

• Incremented targetVersion in migrate() method
• Created migration method with descriptive name (migrate{FeatureName})
• Checked both old and new data locations (handle both-exist case)
• Batched updates into single actor.update() call
• Used -= syntax to remove old fields
• Added console logging with module prefix and actor name
• Called migration method in migrate() sequence
• Added user notifications (start + complete)
• Tested migration with test world containing old data
• Verified migration runs ONCE (doesn't re-run on reload)
• Tested with multiple actors
• Documented migration in PR description

References

• Migration implementation: scripts/migration.js
• Schema version registration: scripts/settings.js
• Migration trigger: scripts/module.js (ready hook)
• Testing guidelines: CONTRIBUTING.md line 56

For BitD Alternate Sheets module:

• Schema version registered in scripts/settings.js line 59
• Migration triggered in scripts/module.js line 43 (ready hook)
• Current target version: Check scripts/migration.js line 6
• Typical actor types: character, crew, npc