Cocos AI Automation Development

Cocos AI Automation Development

Purpose

Use this skill to drive Cocos Creator as an AI-assisted development target: install and connect MCP, operate the editor through MCP and UI automation, create scenes/assets/scripts, tune behavior, and validate results with small runnable gates.

This skill complements ai-mini-game-development: use that skill for game-phase discipline, and use this skill for Cocos-specific tooling and editor-operation details.

Core Workflow

1. Establish the project and editor state.

   • Identify the Cocos project root and Cocos Creator version.
   • Check whether the editor is open, whether the target scene is loaded, and whether a preview server is running.
   • Preserve user changes. Do not reset or overwrite scenes unless explicitly requested.

2. Connect MCP before modifying the scene.

   • Prefer the maintained Tuya fork when available: https://github.com/hzq510473048/cocos_mcp.
   • Use the original upstream only when the Tuya fork is unavailable or when comparing upstream changes: https://github.com/RomaRogov/cocos-mcp.
   • Install it under the project extension folder when the user wants project-local automation.
   • Ask the user to start the MCP service from Cocos when needed; Codex can then call tools against http://localhost:3000/mcp.
   • Confirm available tools with client.listTools() before assuming a capability exists.

3. Make a minimal testable change first.

   • Create a visible object, material, light, camera, and simple behavior before adding polish.
   • For physics, verify the basic loop before adding realistic textures, rails, controls, or camera work.
   • Save the scene and read properties back through MCP.

4. Use the right operation channel.

   • Use MCP for nodes, components, assets, scene save/load, and property queries.
   • Use filesystem edits for TypeScript scripts and generated assets.

• Use Computer Use for Cocos UI actions that MCP cannot do reliably, such as opening panels, clicking refresh, or visually inspecting the editor.
• Use browser/preview only after the editor has compiled scripts and the scene has been saved.

5. Validate after every meaningful step.
   • Query components and node hierarchy through MCP.
   • Inspect Cocos logs with operate_current_scene/get-last-logs.
   • Use the editor view or preview to verify visual behavior.
   • Stop and ask for user feedback when the next judgment is about feel, visual quality, or gameplay intent.

MCP Client Pattern

Run MCP calls from the installed extension directory so the SDK dependency resolves:

cd /path/to/CocosProject/extensions/cocos-mcp
node --input-type=module - <<'NODE'
import { Client } from './node_modules/@modelcontextprotocol/sdk/dist/esm/client/index.js';
import { StreamableHTTPClientTransport } from './node_modules/@modelcontextprotocol/sdk/dist/esm/client/streamableHttp.js';

const client = new Client({ name: 'codex', version: '0.1.0' });
await client.connect(new StreamableHTTPClientTransport(new URL('http://localhost:3000/mcp')));

const tools = await client.listTools();
console.log(tools.tools.map((tool) => tool.name).join('\n'));

await client.close();
NODE

Prefer short scripts that:

• connect once,
• call several MCP tools,
• print compact JSON/text,
• save the scene,
• close the client.

MCP Installation Sources

Use one of these installation paths for new Cocos projects.

Preferred: Install From The Tuya Fork

Use this when network and GitHub access are available.

mkdir -p <CocosProject>/extensions
cd <CocosProject>/extensions
git clone https://github.com/hzq510473048/cocos_mcp.git cocos-mcp
cd cocos-mcp
npm install
npm run build

Then restart Cocos Creator, open the cocos-mcp panel, start the server, and verify script_hot_reload appears in client.listTools().

Offline Or Transfer Install: Use The Standard Archive

Use this when moving between computers or when GitHub access is unavailable. The expected archive name is:

cocos-mcp-tuya-1.0.2-script-hot-reload.zip

Install it like this:

mkdir -p <CocosProject>/extensions
cd <CocosProject>/extensions
unzip /path/to/cocos-mcp-tuya-1.0.2-script-hot-reload.zip
cd cocos-mcp
npm install
npm run build

If the archive already includes dist, it may load immediately, but still run npm install && npm run build after transfer to normalize dependencies for the local machine.

Updating The Fork From Upstream

When the original author changes RomaRogov/cocos-mcp, keep local custom work in a branch and rebase or merge upstream:

cd <CocosProject>/extensions/cocos-mcp
git remote add upstream https://github.com/RomaRogov/cocos-mcp.git 2>/dev/null || true
git fetch upstream
git checkout main
git rebase upstream/main
npm install
npm run build

After resolving conflicts, restart Cocos Creator/MCP and confirm script_hot_reload still appears.

Practical Rules

• Read references/quick-workflow.md before starting a new Cocos automation task.
• Read references/pitfalls.md when working with physics, script hot reload, editor UI refresh, scene preview, or visual polish.
• Use generated bitmap assets for quick material validation, then import them into assets/ and refresh Cocos so .meta files are created.
• When adding TypeScript components, edit the .ts file first, wait for Cocos to compile it, confirm it appears in get_available_component_types, then add it with modify_nodes.
• When a component schema changes, use the fixed hot-reload recovery flow instead of ad hoc debugging:
  0. if the local cocos-mcp exposes script_hot_reload, call it first with the script path, expected component type, and one or more code snippets that must appear in generated editor/preview chunks;
  1. trigger Cocos import/compile by focusing the editor, clicking asset refresh, or reloading the editor window if needed;
  2. verify the generated script chunk or editor status reflects the new source, not only that the .ts file changed;
  3. query get_available_component_types and query_components;
  4. if new @property fields are still absent, remove and re-add the component, then serialize properties again;
  5. save the scene and verify the scene file/MCP query agree. Do not trust stale component data or ask the user to diagnose this pattern unless the recovery flow fails.
• For this project-local cocos-mcp fork, script_hot_reload is available after restarting Cocos Creator/MCP. Example:

{
  "scriptAssetPath": "assets/JuggleGameController.ts",
  "componentType": "JuggleGameController",
  "expectedRuntimeSnippets": [
    "Math.min(this._lastDownwardSpeed * this.naturalBounceRestitution"
  ],
  "targets": ["editor", "preview"],
  "touchBeforeRefresh": true,
  "waitMs": 1200,
  "retryCount": 2
}

Treat a failed script_hot_reload result as a real stale-runtime warning. Do not proceed to gameplay tuning until either generated chunks match the source or the editor/MCP service has been restarted and checked again.

• For physics demos, favor stable observable behavior over pure engine restitution when the backend is inconsistent. Document any scripted correction as a validation aid.
• When tuning rigid-body scenes, treat visual symptoms such as "sticking", "micro bouncing", "sliding instead of rolling", or "teleport-like state jumps" as coupling problems first. Inspect material restitution/friction, rigid-body sleep/damping, kinematic transform sync, scripted surface correction, scripted bounce assist, and visual roll helpers together before changing one parameter repeatedly.
• Keep tuning values serialized as component properties so later agents can adjust them through MCP.

Gameplay-Specific Skills

Keep this skill focused on Cocos Creator automation. For specific game genres or mechanics, layer a narrower gameplay skill underneath it.

• For Cocos ball, rebound, rolling, flipper, seesaw, paddle, launch-arc, sticking, or micro-bounce gameplay, also use cocos-ball-physics-gameplay.
• For genre-specific tuning, keep detailed physics, input, and gameplay rules in the gameplay skill, not here.
• This skill should still own Cocos-specific execution concerns: opening the correct scene, MCP tool availability, component serialization, asset import, script hot reload, generated chunk verification, and editor/preview validation.

Done Criteria

A Cocos automation task is not done until:

• the scene is saved,
• modified nodes/components/assets are queried back successfully,
• Cocos logs show no relevant script or import errors,
• the user can run or preview the scene,
• any remaining subjective tuning points are clearly named.