Validation

Overview

This skill is MANDATORY after creating or modifying any Honeydew object.

Validation ensures:

• The object compiles and executes without errors
• Results are returned (not NULL or empty)
• Values make business sense
• Related objects are consistent with each other

Validation by Object Type

Metrics

Step 1: Execute the metric

Call get_data_from_fields with:

• metrics: ["<entity>.<metric_name>"]

If the tool call fails (API error, permission denied, timeout), report the error to the user before proceeding. Do not confuse a tool error with suspicious data.

Step 2: Sanity checks

Step 3: Cross-validation

If the new metric is a filtered subset of an existing metric, or a ratio of existing metrics, query them together to verify consistency:

• For filtered metrics: query the filtered metric alongside the unfiltered total — the filtered value should be less than or equal to the total.
• For ratio/derived metrics: query the numerator and denominator independently to confirm they return sensible values before checking the ratio.

Call get_data_from_fields with both metrics:

• metrics: ["<entity>.<filtered_metric>", "<entity>.<total_metric>"]

Alert user if:

• Metric returns $0 or NULL unexpectedly
• Revenue/count is negative
• Ratio exceeds 100% or is negative (unless expected)
• Magnitude seems off by orders of magnitude

Attributes

Step 1: Execute the attribute

If the attribute references a related entity (multi-entity attribute), first verify the relation exists using get_entity on the source entity. If the relation is missing, report that before attempting to query the attribute.

Sample rows — call get_data_from_fields with:

• attributes: ["<entity>.<attribute_name>"]

For boolean attributes, check distribution — call get_data_from_fields with:

• attributes: ["<entity>.<boolean_attribute>"]
• metrics: ["<entity>.count"]

Step 2: Sanity checks by attribute type

Alert user if:

• All values are NULL or >50% NULLs
• Boolean is 100% one value
• Dates are invalid or in wrong century
• Buckets are mostly "Unknown"

Entities

Step 1: Verify entity was created

Use list_entities and filter results for the new entity name.

Step 2: Verify data flows

Call get_data_from_fields with:

• metrics: ["<entity>.count"]

Also call with a list of attributes to verify they are accessible:

• attributes: ["<entity>.<attribute1>", "<entity>.<attribute2>", "<entity>.<attribute3>"]

Step 3: Sanity checks

Alert user if:

• Entity not found after creation
• Zero rows returned
• Key column has duplicates (for non-fact tables)

Validating Relations (part of entity validation)

Relations are not standalone objects — they are defined within an entity's YAML. Validate them as part of the entity that contains them.

Step 1: Verify relation exists

Use get_entity on the source entity and check its relations list for the new relation.

Step 2: Test the join works

Call get_data_from_fields with a cross-entity query:

• attributes: ["<target_entity>.<attribute>"]
• metrics: ["<source_entity>.<metric>"]

Step 3: Sanity checks

Alert user if:

• Relation not found on the entity
• Cross-entity query fails
• Row count explodes (indicates wrong cardinality)
• All joined values are NULL (bad join condition)

Domains

Step 1: Verify domain exists

Use search_model (with search_mode: EXACT) to find the new domain by name.

Step 2: Test with a scoped query

Call get_data_from_fields with the domain parameter to verify entities are accessible and filters apply:

• metrics: ["<entity>.count"]
• domain: "<domain_name>"

Step 3: Verify filters apply

If the domain has semantic or source filters, compare results with and without the domain to confirm filters reduce the data as expected:

• Query a metric with the domain set — note the result.
• Query the same metric without the domain — note the result.
• The domain-scoped result should be less than or equal to the unscoped result (for filters that restrict rows).

Step 4: Sanity checks

Alert user if:

• Domain not found after creation
• Scoped query returns an error (likely bad filter SQL or missing entity)
• Filters have no effect (scoped count equals unscoped count when a filter is expected to reduce rows)
• Excluded fields are still accessible (field selector not applied correctly)

Validating Updates (update_object)

When modifying an existing object, compare before and after:

1. Before altering, query the object and note the current result.
2. After altering, query again and compare.
3. Report the difference to the user: "The metric revenue changed from $1,234,567 to $1,198,432 after applying the discount logic."

This confirms the change had the intended effect and helps catch unintended regressions.

Error Handling

If a tool call fails, distinguish between:

Alerting Guidelines

When results seem wrong, report to the user with:

1. The actual value returned by the query
2. Why it looks suspicious (e.g., negative revenue, 100% NULL, row count doubled)
3. A suggested next step (e.g., "Should I check the SQL expression?" or "Should I verify the join keys?")

Do not silently fix issues — always surface findings and ask before making changes.

Quick Reference