Alibabacloud Cadt Probe

CADT Alibaba Cloud Resource Probe

Use the CADT probe service (BPStudio API) to discover cloud resources under an Alibaba Cloud account.

Quick Start

Prerequisites

pip install alibabacloud_bpstudio20210931==8.1.1 alibabacloud_credentials==1.0.5

Configure credentials using the Alibaba Cloud default credential chain. Choose one of the following methods:

• Run aliyun configure to set up a credential profile
• Set environment variables ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET (auto-detected by the default chain)
• Use an ECS instance RAM role (when running on ECS)
• Configure ~/.alibabacloud/credentials file

Pre-conditions

The Cloud Config service must be enabled in the Alibaba Cloud console, as CADT probe relies on resource data provided by Cloud Config:

• Enable at: https://config.console.aliyun.com/
• Monitoring scope must be set to "All resources in account"
• After initial activation, wait 30-60 minutes for the initial scan to complete

RAM Policy

This skill requires the following RAM permissions to execute BPStudio probe operations and discover cloud resources across 20+ Alibaba Cloud services.

For the complete permission list including a least-privilege RAM policy JSON, see references/ram-policies.md.

Key permissions:

[MUST] Permission Failure Handling: When any command or API call fails due to permission errors at any point during execution, follow this process:

1. Read references/ram-policies.md to get the full list of permissions required by this SKILL
2. Use ram-permission-diagnose skill to guide the user through requesting the necessary permissions
3. Pause and wait until the user confirms that the required permissions have been granted

Run Probe

python scripts/probe.py

Parameters

Usage Examples

1. Basic Probe

python scripts/probe.py

2. Export JSON Results

python scripts/probe.py --output probe_result.json

3. Skip Probe and Pull Existing Results

If you have previously probed (resource data is cached on the server side), you can directly pull existing results without creating a new task:

python scripts/probe.py --skip-probe
python scripts/probe.py --skip-probe --output result.json
python scripts/probe.py --skip-probe --list-types ecs,vpc

4. Agent Mode (Quiet Mode)

Use the -q flag to suppress all human-readable output, printing only pure JSON to stdout for easy Agent or script parsing. By default, the attributes field of each resource is stripped to reduce output size:

python scripts/probe.py -q
python scripts/probe.py -q --output result.json
python scripts/probe.py -q --regions cn-hangzhou --list-types ecs,vpc

To retain full attributes, use --keep-attributes:

python scripts/probe.py -q --keep-attributes

Output Auto-Degradation (Agent Mode)

When the JSON output is large (e.g., 127 security groups with Permissions arrays + 100 vswitches), the script automatically degrades output to prevent terminal truncation:

When output exceeds 200KB, stdout returns a summary like:

{
  "status": "output_too_large",
  "output_file": "/path/to/file.json",
  "total_resources": 246,
  "by_type": {"security_group": 127, "vswitch": 100, "ecs": 19},
  "hint": "Read the output file for full results"
}

Recommendation: Always use --output to write full results to a file, then read the file for parsing. This avoids any output size issues:

python scripts/probe.py -q --output result.json

5. Summary Mode (Aggregated Statistics Only)

When there are many resources, use --summary to skip the detailed resource list and only output aggregated statistics by region/type, suitable for dashboard analysis:

python scripts/probe.py --summary
python scripts/probe.py -q --summary
python scripts/probe.py --summary --output summary.json

6. Query Related Resources for a Specific Resource

Query all resources related to a specific resource (e.g., ECS instance's security groups, vswitches, etc.):

python scripts/probe.py --related-resource-id i-xxx --related-types security_group,vswitch --output related_items_ecs.json

This calls the GetProbeRelatedItems API, which returns all resources topologically related to the specified resource in the probe results. It does NOT create a new probe task. The attributes field of each related resource is stripped by default (same as probe results), which significantly reduces output size for resources like security groups that contain large Permissions arrays.

Two-step workflow (get a resource ID first, then query its related resources):

# Step 1: Get resource list (use --skip-probe if already probed, -q for machine-readable output)
python scripts/probe.py --skip-probe -q --list-types ecs --regions cn-hangzhou

# Step 2: Pick any instance ID from Step 1 output, then query its related resources
python scripts/probe.py --related-resource-id i-bp1xxxxx --related-types security_group,vswitch --output related_items_ecs.json

Supported --related-types values (same as --list-types resource type keys):

• security_group — Security Group
• vswitch — VSwitch
• vpc — Virtual Private Cloud
• disk — Cloud Disk
• eni — Elastic Network Interface
• eip — Elastic IP Address
• ... (any resource type key listed in the supported resource types)

Leave --related-types empty to return all related resource types:

python scripts/probe.py --related-resource-id i-xxx --output all_related.json

Output format (when using --related-resource-id):

{
  "mode": "related-items",
  "query_time": "2025-04-17 16:00:00",
  "resourceId": "i-bp1xxxxx",
  "serviceTypes": ["security_group", "vswitch"],
  "related_items": { ... }
}

How It Works

The script implements resource probing through a two-phase flow using the BPStudio SDK:

Phase 1: Submit Probe Task

Call ExecuteOperationSync API:

• ServiceType = probe
• Operation = CreateOneClickJob
• Attributes = {"regions": [...]}

The API returns an operation_id (task ID).

Phase 2: Poll for Results

Call ExecuteOperationSync API:

• ServiceType = probe
• Operation = GetOneClickStatus
• Attributes = {"jobId": <jobId>}
• Poll every 10 seconds, timeout after 30 minutes, until the task completes
• When status == "SUCCESS", the arguments field contains the probed resource data (JSON)
• When status == "FAILURE", the message field contains the error information

Execution Modes

• Default (full probe): CreateOneClickJob → poll GetOneClickStatus (every 10s, timeout 30min) until SUCCESS → GetProbeResourceSummary → GetProbeResult → GetLastProbeTime + GetProbeRegions (cache validation) → output
• --skip-probe: Skip CreateOneClickJob and polling, directly pull cached results via GetProbeResourceSummary → GetProbeResult → cache validation (cache_info included in output)
• --related-resource-id: Skip full probe, directly call GetProbeRelatedItems with resourceId and optional serviceTypes filter → output. Requires a prior probe run; empty result if no probe data exists.

Cache Validation (All Modes)

After fetching results, the script always calls GetLastProbeTime and GetProbeRegions to validate the cache state:

• cache_info (only in --skip-probe mode): Contains last_probe_minutes_ago (minutes since last probe, -1 if no valid job) and cached_regions (list of region IDs in cache)
• cache_warnings (all modes): Array of warning strings when:
  • No valid probe job exists (lastTime == -1)
  • --regions includes regions not present in the cache → results for those regions will be empty

Output Description

Console Output

======================================================================
CADT Resource Probe (BPStudio)
======================================================================
  Probe time:   2025-04-17 15:30:00
======================================================================

Submitting probe task...
  Task ID: op-xxxxxxxxxxxxx
  Waiting for probe to complete...
  Current status: Running (elapsed 5s)
  Current status: SUCCESS (elapsed 15s)
  Probe complete!

======================================================================
Probe Results
======================================================================
<Resource list details>
======================================================================

JSON Output

When using the --output flag, a full JSON file is written:

{
  "regions": ["cn-hangzhou", "cn-beijing"],
  "probe_time": "2025-04-17 15:30:00",
  "job_id": "12345",
  "progress": {"status": "SUCCESS"},
  "summary": { ... },
  "probe_result": { ... },
  "probe_result_filter": {
    "regions": "all",
    "serviceType": "all"
  }
}

--skip-probe Output (with cache_info)

When using --skip-probe, the output includes a cache_info field:

{
  "regions": ["cn-hangzhou"],
  "probe_time": "2025-04-17 16:00:00",
  "job_id": "",
  "progress": {},
  "cache_info": {
    "last_probe_minutes_ago": 45,
    "cached_regions": ["cn-hangzhou", "cn-beijing"]
  },
  "summary": { ... },
  "probe_result": { ... },
  "probe_result_filter": { ... }
}

--related-resource-id Output

When using --related-resource-id, the output structure is different from the full probe output:

{
  "mode": "related-items",
  "query_time": "2025-04-17 16:00:00",
  "resourceId": "i-bp1xxxxx",
  "serviceTypes": ["security_group", "vswitch"],
  "related_items": {
    "list": [
      { "resourceId": "sg-xxx", "resourceType": "security_group", ... },
      { "resourceId": "vsw-xxx", "resourceType": "vswitch", ... }
    ]
  }
}

Key fields:

• mode: Always "related-items" for this query type
• resourceId: The resource ID that was queried
• serviceTypes: The type filters applied (empty list means no filter)
• related_items: Raw response from GetProbeRelatedItems, containing a list of related resources

_relationships (Auto-extracted Relationship IDs)

Each resource in probe_result.list and related_items.list automatically includes a _relationships field (when applicable) that extracts cross-resource reference IDs from the nested attributes JSON. This field is added before attributes are stripped, so it is always available even without --keep-attributes.

Example — an ECS instance resource:

{
  "resourceId": "i-bp1xxxxx",
  "resourceType": "ecs",
  "_relationships": {
    "security_group_ids": ["sg-abc", "sg-def"],
    "vswitch_id": "vsw-123",
    "vpc_id": "vpc-456"
  }
}

Supported resource types and extracted relationships:

If a resource type is not in the table or no relationship fields are found, _relationships is omitted.

cache_warnings

When cache issues are detected, a cache_warnings array is added to the output (in all modes):

{
  "cache_warnings": [
    "Requested region(s) cn-shenzhen not in probe cache — results for these regions will be empty. Cached regions: cn-hangzhou, cn-beijing"
  ]
}

Observability

User-Agent

All Alibaba Cloud API calls made by this skill (SDK and CLI) include a custom User-Agent:

--user-agent AlibabaCloud-Agent-Skills/alibabacloud-cadt-probe/{session-id}

Rule: Every aliyun CLI command that calls a cloud API MUST include the --user-agent flag above. Local utility commands (e.g. configure, plugin, version) do not support this flag and should be excluded.

Session ID

• Source: environment variable SKILL_SESSION_ID
• Fallback: if SKILL_SESSION_ID is not set, a random 32-character hex UUID is generated
• Scope: one session ID per invocation (created when the BPStudio client is initialized)

Exit Codes

Error Handling

Credentials Not Configured

Error: Unable to get credential. Please configure the default credential chain.

Probe Task Submission Failed

Check:

• Whether credentials are correct
• Whether the account has BPStudio operation permissions (see references/ram-policies.md for the full permission list)
• Whether the Cloud Config service has been enabled

If the error is a permission error, follow the Permission Failure Handling process described in the RAM Policy section above.

Probe Task Timeout

Default timeout is 30 minutes (1800 seconds). If it times out, the script outputs the current status and task ID, which can be used to query later.

Cache Warnings (stderr)

Cache warnings are printed to stderr and included in the JSON output as cache_warnings:

• No valid probe job found in cache — results may be empty or stale
• Requested region(s) <regions> not in probe cache — results for these regions will be empty. Cached regions: <cached>

Supported Resource Types

CADT probe supports 60+ Alibaba Cloud resource types. See resource_types.md for the full list.

File Structure

alibabacloud-cadt-probe/
├── SKILL.md                    # This file
├── references/
│   ├── ram-policies.md         # Required RAM permissions
│   └── resource_types.md       # Supported resource types reference
└── scripts/
    └── probe.py                # Core probe script

Dependencies

alibabacloud_bpstudio20210931==8.1.1
alibabacloud_credentials==1.0.5