# UI Navigation Trajectory Retrieval

> Use this skill when the user asks questions about how to find specific features or perform multi-step tasks within a software application. Trigger it for requests like 'where is the settings menu?', 'guide me to the logout button', 'how do I view the privacy policy in this app?', or 'show me the path to buy a stock'. This skill is specifically for retrieving structured navigation sequences from specialized knowledge bases that map user intents to application UI transition graphs.

- Skill: `dingxingdi/ui-navigation-trajectory-retrieval` (Agent Skill, multi-file: 5 files)
- Install (CLI): `npx skillmds@latest add dingxingdi/ui-navigation-trajectory-retrieval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/dingxingdi/ui-navigation-trajectory-retrieval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: dingxingdi (https://skillmd.com/u/dingxingdi)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/dingxingdi/ui-navigation-trajectory-retrieval

---


# Skill: ui-navigation-trajectory-retrieval

## 1. Capability Definition & Real Case
* **Professional Definition**: The ability to retrieve and synthesize actionable multi-step navigation trajectories from a structured vector database derived from UI Transition Graphs (UTGs). This involves mapping natural language user intents to pre-computed sequences of UI actions (clicks, swipes, inputs) within a bounded application environment, enabling the agent to provide precise navigational guidance even for deeply nested or unconventional interface structures.
* **Dimension Hierarchy**: Specialized Knowledge Navigation->Corpus-Specific Retrieval & Reasoning->ui-navigation-trajectory-retrieval

### Real Case
**[Case 1]**
* **Initial Environment**: A RAG-ready structured knowledge base containing the UI Transition Graph (UTG) for a reading application. The graph contains nodes representing screens (Home, Me, Settings, About) and edges representing click actions on specific widgets.
* **Real Question**: View Pomodoro Reading App’s Privacy Policy.
* **Real Trajectory**: [{'step': 1, 'observation': 'Home Screen with various icons.', 'action': "Click the 'Me' button to enter the user profile section."}, {'step': 2, 'observation': 'Profile Screen with account options.', 'action': "Click the 'Settings' gear icon."}, {'step': 3, 'observation': 'Settings List.', 'action': "Click 'About Pomodoro Reading App'."}, {'step': 4, 'observation': 'About page with legal links.', 'action': "Click 'Privacy Policy' to fulfill the request."}]
* **Real Answer**: To view the Privacy Policy, first click on the 'Me' icon, then go to 'Settings', select 'About Pomodoro Reading App', and finally click on 'Privacy Policy'.
* **Why this demonstrates the capability**: The question requires navigating a deeply nested UI structure where the target ('Privacy Policy') is not visible from the start. The agent demonstrates the capability by retrieving a specific four-step trajectory from the structured knowledge base that successfully links the initial 'Home' state to the final 'Privacy Policy' state through intermediate milestones.
---
**[Case 2]**
* **Initial Environment**: A vector database representing a financial application's navigational flow. The database indexes high-level intents like 'Invest', 'Support', and 'Account Management' alongside their required interaction sequences.
* **Real Question**: Find online financial customer support.
* **Real Trajectory**: [{'step': 1, 'observation': 'App Landing Page.', 'action': "Click the 'Financial Management' menu button."}, {'step': 2, 'observation': 'Financial main interface.', 'action': 'Click the search bar at the top.'}, {'step': 3, 'observation': 'Search input activated.', 'action': "Input 'online financial customer support'."}, {'step': 4, 'observation': 'Search results list.', 'action': "Click the 'Online Support' button to enter the chat."}]
* **Real Answer**: You can find support by clicking 'Financial Management', using the search bar to type 'online financial customer support', and then selecting the 'Online Support' button.
* **Why this demonstrates the capability**: This case shows intent-guided retrieval where a complex search-and-click sequence is recovered from its vector embedding. Success depends on the RAG agent's ability to match the user's abstract intent to a concrete, multi-modal action sequence (text input + buttons) found in the specialized application corpus.

## Pipeline Execution Instructions
To synthesize data for this capability, you must strictly follow a 3-phase pipeline. **Do not hallucinate steps.** Read the corresponding reference file for each phase sequentially:

1. **Phase 1: Environment Exploration**
   Read the exploration guidelines to discover raw knowledge seeds:
   `references/EXPLORATION.md`

2. **Phase 2: Trajectory Selection**
   Once Phase 1 is complete, read the selection criteria to evaluate the trajectory:
   `references/SELECTION.md`

3. **Phase 3: Data Synthesis**
   Once a trajectory passes Phase 2, read the synthesis instructions to generate the final data:
   `references/SYNTHESIS.md`