Guided Data Analysis

Guided Data Analysis

Develop a data analysis by working through a structured intake, then answering three governance questions before querying. The user may start with a vague business question or a specific topic (e.g. "pipeline history for Germany").

Prerequisites

1. Run refresh-connections if Atlan or Dataverse may be stale.
2. Atlan MCP (user-Atlan) — catalog discovery, lineage, stewards.
3. Dataverse MCP (user-Dataverse) — query AI-ready data products.
4. If either server's STATUS.md requires auth, call mcp_auth with {} first.
5. Read project dataaccess.md (if present) for the Dataverse product catalog.

Workflow overview

Intake → Availability → Permission → Acquisition → [Analysis if permitted] → Report

Copy this checklist and track progress:

- [ ] Step 0: Intake — clarify business question and scope
- [ ] Step 1: Availability — find data products and tables
- [ ] Step 2: Permission — verify query access
- [ ] Step 3: Acquisition — access-request path (if needed)
- [ ] Step 4: Analysis — run Dataverse queries (only if permitted)
- [ ] Step 5: Deliver report

Step 0 — Intake questions

Gather enough context to run the three governance questions. Use AskQuestion when available; otherwise ask conversationally in one message (group related questions).

A. Business intent

B. Scope (adapt to the topic)

C. Output preference

Do not proceed to discovery until you have at least: business question, subject domain, and primary scope filters (geo/time if relevant).

Step 1 — Availability

Question 01 — Availability: Do we have data products or tables containing [subject] for [scope]?

Atlan (primary for discovery)

1. semantic_search_tool — natural-language search across tables, columns, data products, glossary terms.
2. resolve_metadata_tool with namespace_type: data_domain_and_product — find governed data products and domains by name.
3. get_assets_tool — hydrate top candidates (include readme relationship when purpose is unclear).
4. search_assets_tool — precise filters when you know asset type or connection (e.g. asset_type: DataProduct).

Dataverse (primary for queryable products)

1. identify_dataproducts with the user's business question.
2. shortlist_tables for the top product(s).
3. Cross-reference Atlan assets with Dataverse product names when both return matches.

Record for each candidate

If nothing is found, say so clearly and suggest broader search terms or related products. Do not invent tables.

Step 2 — Permission

Question 02 — Permission: Do I have the necessary access permissions to query this specific data product?

For each candidate data product (prioritize ai_ready):

1. Run the Dataverse pipeline through get_sql with a minimal probe query (e.g. SELECT 1 or LIMIT 5 row count for the target scope).
2. Call execute_sql:
   • Success → mark Access: Granted for that product.
   • Permission / authorization error → mark Access: Denied; capture the error message.
3. Supplement with Atlan asset attributes (certificateStatus, owners) — metadata visibility does not guarantee query access.

If all candidates are denied, skip Step 4 and focus the report on Availability + Acquisition.

Step 3 — Acquisition

Question 03 — Acquisition: How do I request access to these data products through our governance portal?

When access is denied or uncertain:

1. Data product owners — from Atlan get_assets_tool (ownerUsers, ownerGroups).
2. Governance SOPs — Dataverse dataversehelp product:
   • identify_dataproducts → dataversehelp
   • shortlist_tables → get_sql → execute_sql for access-request or onboarding docs matching the product name.
3. The Source — search for "[product name] data access" or "Dataverse access request" if SOPs are thin.
4. Catalog links — provide Atlan URLs and Dataverse as starting points.

Present concrete next steps: who to contact, which portal to use, and what to include in the request (business justification, scope, product name).

See reference.md for portal links and troubleshooting.

Step 4 — Analysis (only if access granted)

Run the full Dataverse 4-step pipeline on the best permitted product:

1. identify_dataproducts
2. shortlist_tables
3. get_sql — pass the full business question with scope filters from intake
4. execute_sql

Optional enrichment:

• Atlan traverse_lineage_tool — upstream/downstream context for key tables
• Atlan semantic_search_tool — related glossary terms for metric definitions

Analysis rules

• Prefer ai_ready products over ai_experimental; note maturity in the report.
• Do not skip pipeline steps (same rule as dataverse-people-lookup).
• Do not fabricate numbers — only report what execute_sql returns.
• Flag gaps with [needs input] rather than guessing.

Step 5 — Report template

Deliver using this structure:

# Data Analysis: [Business Question]

## Executive summary
[2–4 sentences: what data exists, whether you can query it, and the headline finding or blocker]

## Intake
| Dimension | Value |
|---|---|
| Business question | … |
| Subject | … |
| Scope | … |
| Time period | … |

## 01 — Availability
[Do we have data products or tables containing …?]

| Candidate | Type | Maturity | Description | Catalog link |
|---|---|---|---|---|
| … | … | … | … | … |

**Conclusion:** [Available / Partially available / Not found]

## 02 — Permission
[Do I have access to query this data product?]

| Data product | Access | Evidence |
|---|---|---|
| … | Granted / Denied | … |

**Conclusion:** [Can query now / Cannot query — see Acquisition]

## 03 — Acquisition
[How do I request access?] *(omit section if all needed products are granted)*

- **Data product owner(s):** …
- **Request steps:** …
- **Portal / links:** …

## Analysis results
*(include only when access was granted)*

### Findings
[Key numbers, trends, or tables from execute_sql]

### Methodology
- Data product: …
- Tables: …
- Filters applied: …

### Limitations
[Missing data, experimental maturity, scope caveats]

## Recommended next steps
1. …

Examples

User: I need pipeline history for Germany.

1. Intake — confirm time period and grain (account vs region).
2. Availability — Atlan search "pipeline history Germany"; Dataverse identify_dataproducts for pipeline/bookings.
3. Permission — probe query on top ai_ready match.
4. Acquisition — if denied, return owner + dataversehelp SOP.
5. Analysis — full query if granted; deliver report.

User: Can I query renewals data?

Skip heavy analysis unless granted. Run Availability → Permission → Acquisition only, then offer to continue if access is approved later.

Additional resources

• reference.md — MCP tool map, portal links, troubleshooting
• dataaccess.md — Dataverse product catalog (project root)
• dataverse-people-lookup — people/org lookups (not general analytics)
• refresh-connections — fix Atlan/Dataverse auth before starting