Gds Explorer

GDS Explorer

A GDS explorer orients in unfamiliar spheres — understanding what entity lines exist, which patterns capture meaningful geometry, how the population is distributed, and where to focus deeper investigation.

The goal is efficient orientation: 3 calls to understand the landscape, then targeted profiling to find the signal, then hand off to the right specialist skill. Depth beats breadth — 20 calls on one promising pattern outperform 5 calls across 4 patterns.

For concrete tool output examples, see references/examples.md.

Orientation

Every exploration starts here, regardless of the question:

sphere_overview()        -> patterns, anomaly rates, calibration health, profiling_alerts
get_sphere_info()        -> lines, columns, aliases, FTS availability
anomaly_summary(top_pattern) -> what drives anomalies

Performance: Always use detail="summary" (default) for interactive work. detail="full" runs event-rate-divergence scans that can take minutes on large spheres (>100K entities). Reserve "full" for deep-dive diagnostics only.

After these 3 calls, check for signals that guide next steps:

• dimension_kinds in sphere_overview — shows distribution-kind tags per dimension (e.g. "bernoulli x4, poisson x2, gaussian x8"). Use this as a quick orientation to understand which patterns have count-heavy vs magnitude-heavy vs binary geometries. Patterns with many bernoulli dims perform best with metric="Linf" in find_anomalies. Patterns with mixed kinds (poisson + gaussian + bernoulli) benefit from metric="bregman".
• confidence_distribution in sphere_overview — when present, shows the spread of bootstrap anomaly confidence across the flagged population. A distribution skewed toward low confidence suggests the anomaly rate is inflated by unstable detections. In that case, filter downstream calls: find_anomalies(pattern_id, min_confidence=0.7) to focus on high-confidence detections before investing in full investigation.
• profiling_alerts — run each alert's suggested call (find_anomalies(rank_by_property=<dim>)) on every pattern including composites.
• dim_quality_warnings — five silent-failure auditors fire per pattern: dead_dim (zero-variance dim; z-score undefined), sparse_dim (mostly-zero with rare nonzero tail; gaussian assumption wrong), dominant_dim_mass (pattern-level: one dim ≥ 70 % of tail variance — sphere is a one-dim detector on this pattern, multi-dim composition adds little), negative_space (gaussian-declared dim sitting at point-mass-at-zero — z-score treats zero as typical when it is the absence value), and heteroscedasticity (pattern-level: Brown-Forsythe Levene p < 0.01 on delta_norm partitioned by the pattern's group_by_property — the global θ assumption is statistically violated; dim_label carries the grouping variable name, not a δ-dim). If dominant_dim_mass, negative_space, or heteroscedasticity fires, raise a calibration ticket rather than investigating the flagged entities — they are likely noise. Full triage rules in gds-monitor.
• has_temporal: true — for each temporal pattern, consider running find_drifting_entities(pattern_id, top_n=3) to detect behavioral changes.
• Event patterns with edge tables — run edge_stats(pattern_id) for each event pattern. If has_edge_table: true, note that graph traversal tools (find_geometric_path, discover_chains) and behavioural-vs-graph cross-tab (find_graph_geometry_tension(primary_key, anchor_pattern, line_id=event_pattern)) are available for that pattern. The stats (row_count, unique_from/to, avg_degree) help estimate graph density before committing to path-finding or chain discovery calls.

MANDATORY: always run find_clusters on every anchor pattern during orientation. Population archetypes reveal structural segments (e.g. airport zones vs residential vs dead zones, premium vs budget customers) that contextualize all subsequent findings. Skipping archetypes is a common investigation gap — anomaly rates and drift only make sense relative to the population structure that clustering reveals.

Then profile raw dimension values before diving into anomalies.

Profiling raw dimensions

For each anchor pattern with derived dimensions:

get_line_profile(entity_line, dim_property)

Check 2-3 key dimensions per pattern. What to look for:

• MAX >> P99 — an extreme cluster exists that find_anomalies may rank low (e.g. max =400 when p99=50 means a hidden cluster of extreme entities that delta_norm ranks below the majority)
• MIN << P01 — same in the other direction
• Bimodal split — two populations mixed in one pattern

This catches signals that delta_norm misses. Normalization absorbs outlier clusters into population mu/sigma — raw profiles reveal them.

Matching signals to next steps

Tools by question

Parameter guidance

• sample_size=5000 for find_clusters on populations > 10K
• sample_size=50000 for heavy aggregate on populations > 500K
• goto() then get_polygon() — always sequential, never parallel

Selection modes for anomaly results

find_anomalies, attract_boundary, find_hubs, and find_drifting_entities support two selection modes: the default select="top_norm" returns the most extreme entities by delta norm, while select="diverse" applies submodular facility location to return a geometrically spread set covering different anomaly types. During exploration, top_norm answers "show me the most extreme" and diverse answers "show me all the kinds of anomaly present." An optional fdr_alpha parameter applies Benjamini-Hochberg FDR control to filter statistically insignificant results — useful when handing results to specialist skills, but exploratory orientation can leave it unset to see the full landscape.

find_anomalies also accepts min_confidence (0–1) to filter by bootstrap anomaly confidence. In exploration, this is useful when the population has many borderline detections: min_confidence=0.7 returns only entities where the anomaly is stable across bootstrap resamples. Omit for initial orientation to see the full landscape, then narrow when handing suspects to investigation skills. Not applicable for populations > 50K, group_by_property, or use_mahalanobis patterns (confidence is absent).

Avoiding common exploration traps

Exploring ALL patterns, ALL aliases, ALL tools equally (100+ calls, 20% recall) is tempting but inefficient. Orient in 3 calls, find what matters, go deep on that (60 calls, 95% recall).

Minimal spheres

If sphere_overview returns 0 entities or no patterns, the sphere may be empty or misconfigured. Report this to the user — do not attempt clustering or profiling on empty populations. If population is very small (<100 entities), skip sampling and use full population for all operations.

Report structure

Reference skill phase names in report section headers so reviewers can trace which workflow drove each finding. Example: "## Orientation (gds-explorer)", "## Ground Truth Validation (gds-investigator)", "## Temporal Health (gds-monitor)".

When things don't work

• Tool returns empty results — try different parameters, wider sample, or a different pattern. The sphere may lack data for the requested analysis.
• Tool errors — check pattern_id and version, verify sphere is open. If the error persists, report it rather than retrying in a loop.
• No anomalies found — not every sphere has every anomaly type. Report "no signal detected" as a valid finding.
• sphere_overview returns 0 patterns — the sphere may be empty or misconfigured. Report to the user and stop.
• find_clusters produces a single cluster — the population is too homogeneous for geometric separation. Try increasing n_clusters or check if the pattern has enough derived dimensions.
• get_line_profile returns all zeros — the property may not be populated for this entity line. Try a different property or check get_line_schema for available columns.

Skill delegation

Full exploration examples: references/examples.md