# Adapter Assistant > Complete adapter lifecycle assistant for LimaCharlie. Supports External Adapters (cloud-managed), Cloud Sensors (SaaS/cloud integrations), and On-prem USP adapters. Dynamically researches adapter types from local docs and GitHub usp-adapters repo. Creates, validates, deploys, and troubleshoots adapter configurations. Handles parsing rules (Grok, regex), field mappings, credential setup, and multi-adapter configs. Use when setting up new data sources (Okta, S3, Azure Event Hub, syslog, webhook, etc.), troubleshooting ingestion issues, or managing adapter deployments. - Skill: `refractionpoint/adapter-assistant` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add refractionpoint/adapter-assistant` - Raw SKILL.md: https://api.skillmd.com/api/skills/refractionpoint/adapter-assistant/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: DevOps & Infra - Author: refractionpoint (https://skillmd.com/u/refractionpoint) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/refractionpoint/adapter-assistant --- # Adapter Assistant A comprehensive, dynamic assistant for LimaCharlie adapter lifecycle management. This skill researches adapter configurations from multiple sources and helps you create, validate, deploy, and troubleshoot adapters for any data source. --- ## LimaCharlie Integration > **Prerequisites**: Run `/init-lc` to initialize LimaCharlie context. ### LimaCharlie CLI Access All LimaCharlie operations use the `limacharlie` CLI directly: ```bash limacharlie --oid --output yaml [flags] ``` For command help and discovery: `limacharlie --ai-help` ### Critical Rules | Rule | Wrong | Right | |------|-------|-------| | **CLI Access** | Call MCP tools or spawn api-executor | Use `Bash("limacharlie ...")` directly | | **Output Format** | `--output json` | `--output yaml` (more token-efficient) | | **Filter Output** | Pipe to jq/yq | Use `--filter JMESPATH` to select fields | | **LCQL Queries** | Write query syntax manually | Use `limacharlie ai generate-query` first | | **Timestamps** | Calculate epoch values | Use `date +%s` or `date -d '7 days ago' +%s` | | **OID** | Use org name | Use UUID (call `limacharlie org list` if needed) | --- ## When to Use Use this skill when: - **Setting up new data sources**: Connect syslog, webhooks, cloud services, or any external data to LimaCharlie - **Configuring SaaS integrations**: Okta, CrowdStrike, Microsoft Defender, Office 365, AWS, Azure, etc. - **Troubleshooting adapters**: Data not flowing, parsing issues, credential problems - **Modifying existing adapters**: Update parsing rules, change credentials, add field mappings - **Auditing adapters**: List and review adapters across organizations - **Connecting unknown products**: Research how to integrate any product that sends logs or webhooks Common scenarios: - "I want to ingest Okta system logs" - "Set up a syslog adapter for our firewall" - "My Azure Event Hub adapter isn't receiving data" - "How do I connect this product that sends webhooks?" - "Create a webhook to receive alerts from our monitoring system" - "List all adapters across my organizations" - "Help me parse these custom log formats" ## What This Skill Does This skill is **truly dynamic** - it researches adapter configurations from multiple sources in real-time: 1. **Local LimaCharlie documentation** - Adapter types, configuration options, examples 2. **GitHub usp-adapters repository** - Source code for 50+ adapter implementations 3. **External product documentation** - API docs, webhook formats, authentication requirements The skill then guides you through creating, validating, and deploying adapter configurations with proper parsing rules and credential management. ### Supported Adapter Categories | Category | Description | Examples | |----------|-------------|----------| | **External Adapter** | Cloud-managed syslog, webhook, or API receivers | syslog, webhook, custom API | | **Cloud Sensor** | Cloud-to-cloud SaaS integrations | Okta, CrowdStrike, O365, AWS S3, Azure Event Hub | | **On-prem/USP Adapter** | Binary deployments with local configuration | file, syslog receiver, Kubernetes pods | ## Dynamic Research Strategy **This is the key capability** - the skill researches ANY data source dynamically, not just predefined ones. > **Microsoft sources — STOP and read the reference first.** If the data source is anything Microsoft (Defender, Office 365 / M365, Entra ID / Azure AD, Azure logs, Windows Event Logs, EVTX), do NOT research or guess: `Read("${CLAUDE_PLUGIN_ROOT}/skills/adapter-assistant/MICROSOFT.md")` and use exactly the adapter type + `platform` pairing from its decision matrix. Microsoft exposes the same products through multiple feeds with different formats, and picking the wrong `platform` (e.g. `json` instead of `msdefender`, or `azure_event_hub_namespace` for data merely transiting an Event Hub) silently breaks parsing. ### For ANY Data Source Request: #### Step 1: Check for Native LimaCharlie Adapter For a fast LOCAL view of supported adapter types and per-type config fields, ask the CLI directly (this complements — does not replace — the usp-adapters repo, which remains authoritative for field definitions): ```bash # Supported types limacharlie cloud-adapter list-types --oid --output yaml limacharlie external-adapter list-types --oid --output yaml # Config field listing for one type limacharlie cloud-adapter schema --type --oid --output yaml ``` Search local documentation: ``` Glob("./docs/limacharlie/doc/Sensors/Adapters/Adapter_Types/*{keyword}*.md") ``` Check GitHub usp-adapters repository (use API at root - adapters are NOT in a subdirectory): ``` WebFetch( url="https://api.github.com/repos/refractionPOINT/usp-adapters/contents", prompt="List all available adapter directories from the JSON response" ) ``` #### Step 2: Read LimaCharlie Adapter Documentation If a native adapter exists, first list files in the adapter directory, then fetch the main source: ``` # Step 1: List files to find the main config file WebFetch( url="https://api.github.com/repos/refractionPOINT/usp-adapters/contents/{adapter}", prompt="List all .go files in this adapter directory" ) # Step 2: Fetch the config (usually client.go, but some adapters differ - e.g., sentinelone uses s1.go) WebFetch( url="https://raw.githubusercontent.com/refractionPOINT/usp-adapters/master/{adapter}/client.go", prompt="Extract all configuration fields from the Config struct" ) ``` #### Step 3: Research the External Product **ALWAYS** research the external product to understand its capabilities: ``` WebSearch("{product name} API documentation") WebSearch("{product name} webhook integration") WebSearch("{product name} audit logs export") ``` Extract: - Authentication method (API key, OAuth, service account, etc.) - Available event types or log categories - Webhook payload format (if applicable) - Rate limits and API best practices - Required scopes or permissions #### Step 4: Build Comprehensive Integration Plan If NO native adapter exists, determine how to connect: - Can it send webhooks? → LimaCharlie webhook adapter - Does it write to S3/GCS/Azure Blob? → Cloud storage adapters - Does it push to Kafka/Pub/Sub/SQS? → Queue adapters - Can we pull via API? → May need polling or custom integration Then map out: - External product configuration requirements - Matching LimaCharlie adapter configuration - Parsing requirements for the data format - Sample data format for validation ## Workflow Phases ### Phase 0: Requirements Gathering Use `AskUserQuestion` to understand the user's needs: 1. **Operation type**: Create new adapter, modify existing, troubleshoot, or audit 2. **Data source**: What product/system are they connecting? 3. **Adapter category preference**: Cloud-managed or on-prem deployment ### Phase 1: Dynamic Research Execute the dynamic research strategy above to gather all relevant information about: - LimaCharlie adapter capabilities - External product API/webhook specifications - Configuration requirements on both sides ### Phase 2: Organization and Configuration Discovery **Get organizations:** ```bash limacharlie org list --output yaml ``` **List existing External Adapters:** ```bash limacharlie external-adapter list --oid --output yaml ``` **List existing Cloud Sensors:** ```bash limacharlie cloud-adapter list --oid --output yaml ``` **Get existing configuration (if modifying):** ```bash limacharlie external-adapter get --key --oid --output yaml # or for cloud sensors: limacharlie cloud-adapter get --key --oid --output yaml ``` ### Phase 3: Configuration Generation Build the configuration based on research: #### Core client_options (required for all adapters): ```yaml client_options: identity: oid: "" installation_key: "" # or use hive://secret/... platform: "text|json|carbon_black|gcp|..." sensor_seed_key: "" hostname: "" mapping: parsing_grok: message: '%{PATTERN:field} ...' # For text platform event_type_path: "field/path" event_time_path: "timestamp/path" sensor_hostname_path: "host/path" ``` #### Credential references (recommended): Use Hive secrets for sensitive values: ```yaml apikey: "hive://secret/okta-api-key" client_secret: "hive://secret/azure-client-secret" ``` Check existing secrets: ```bash limacharlie secret list --oid --output yaml ``` #### Indexing configuration (optional): ```yaml indexing: - events_included: ["*"] path: "src_ip" index_type: "ip" - events_included: ["*"] path: "user/email" index_type: "user" ``` Supported index types: `file_hash`, `file_path`, `file_name`, `domain`, `ip`, `user`, `service_name`, `package_name` ### Phase 4: Validation and Testing **Validate parsing rules before deployment:** ```bash # Write adapter config with mapping and sample data to a YAML file: cat > /tmp/usp-test.yaml << 'EOF' parsing_grok: message: "%{TIMESTAMP_ISO8601:timestamp} %{WORD:action} ..." event_type_path: action event_time_path: timestamp sample_data: - "2024-01-15T10:30:00Z LOGIN user@example.com" EOF limacharlie usp validate \ --platform text \ --input-file /tmp/usp-test.yaml \ --oid --output yaml ``` **Review validation results:** - Verify all expected fields are extracted - Check timestamp parsing - Confirm event type assignment - Iterate on pattern if validation fails **For complex parsing**, invoke the parsing-helper skill: ``` Skill("parsing-helper") ``` **For local testing before production deployment:** ``` Skill("test-limacharlie-adapter") ``` ### Phase 5: Deployment **Deploy External Adapter:** ```bash # Write configuration to a temp file first cat > /tmp/adapter-config.yaml << 'EOF' EOF limacharlie external-adapter set --key --input-file /tmp/adapter-config.yaml --oid --output yaml ``` **Deploy Cloud Adapter:** ```bash cat > /tmp/cloud-adapter-config.yaml << 'EOF' EOF limacharlie cloud-adapter set --key --input-file /tmp/cloud-adapter-config.yaml --oid --output yaml ``` **For On-prem Adapters**, generate deployment artifacts: **YAML Configuration:** ```yaml syslog: client_options: identity: installation_key: "" oid: "" platform: text sensor_seed_key: "" hostname: "" mapping: parsing_grok: message: '%{PATTERN:field} ...' event_type_path: "..." port: 514 is_udp: true ``` **CLI Command:** ```bash ./lc_adapter syslog \ client_options.identity.installation_key= \ client_options.identity.oid= \ client_options.platform=text \ client_options.sensor_seed_key= \ "client_options.mapping.parsing_grok.message=%{PATTERN:field} ..." \ port=514 \ is_udp=true ``` **Docker Command:** ```bash docker run -d --rm -p 514:514/udp refractionpoint/lc-adapter syslog \ client_options.identity.installation_key= \ client_options.identity.oid= \ client_options.platform=text \ client_options.sensor_seed_key= \ port=514 \ is_udp=true ``` **Verify deployment:** One step — ask the adapter record for its own live sensor(s). This matches by the adapter's installation-key IID for you and reports clearly when no sensor exists yet (empty result = the adapter has not delivered any events): ```bash limacharlie cloud-adapter sensors --key --oid --output yaml # for an External Adapter record: limacharlie external-adapter sensors --key --oid --output yaml ``` Equivalent manual lookup (matches the same IID by hand) if you only have the IID: ```bash limacharlie sensor list --selector "iid == \`\`" --oid --output yaml ``` ### Phase 6: Post-Deployment **Troubleshoot if data not appearing:** 1. Check adapter last_error field: ```bash limacharlie external-adapter get --key --oid --output yaml ``` 2. Check organization errors for adapter issues: ```bash limacharlie org errors --oid --output yaml ``` Look for errors with component names containing the adapter name. 3. Verify sensor exists in sensor list 4. Query for recent events: ```bash # First calculate timestamps dynamically start=$(date -d '1 hour ago' +%s) && end=$(date +%s) limacharlie event list --sid --start $start --end $end --oid --output yaml ``` 5. Check for unparsed events (`event_type: "unknown_event"` with only `text` field) **Offer D&R rule creation:** ``` Skill("detection-engineering") ``` ## Multi-Organization Adapter Audit For auditing adapters across multiple organizations, spawn parallel sub-agents: ``` Task( subagent_type="lc-essentials:multi-org-adapter-auditor", prompt="Audit adapters for organization: {org_name} ({oid}) Return: - List of external adapters with status - List of cloud sensors with status - Any adapters with errors - Configuration issues detected" ) ``` ## Adapter Type Quick Reference | Adapter Type | Platform | Deployment | Key Configuration | |-------------|----------|------------|-------------------| | syslog | text | External/On-prem | port, is_udp, parsing_grok | | webhook | json | Cloud Sensor | secret, client_options | | okta | json | Cloud Sensor | apikey, url | | s3 | varies | Cloud Sensor/On-prem | bucket_name, access_key, secret_key, prefix | | azure_event_hub | varies — match the feed streamed into the hub (`msdefender`, `azure_ad`, `azure_monitor`, …); see [MICROSOFT.md](./MICROSOFT.md) | Cloud Sensor/On-prem | connection_string (must include EntityPath) | | office365 | office365 | Cloud Sensor | domain, tenant_id, publisher_id, client_id, client_secret, endpoint, content_types | | defender | msdefender | Cloud Sensor | tenant_id, client_id, client_secret (Graph alerts_v2 polling) | | entraid | entraid | Cloud Sensor | tenant_id, client_id, client_secret (Graph riskDetections polling) | | ms_graph | json | Cloud Sensor | tenant_id, client_id, client_secret, url (custom Graph endpoint) | | wel | wel | On-prem (Windows only) | evt_sources | | evtx | wel (no `evtx` platform exists) | On-prem | file_path | | falconcloud | json | Cloud Sensor/On-prem | client_id, client_secret | | pubsub | varies | Cloud Sensor/On-prem | project_id, subscription_id, service_account_creds | | file | varies | On-prem | file_path, backfill | | gmail | gmail | Cloud Sensor/On-prem | client_id, client_secret, refresh_token (single mailbox) or service_account_credentials, subject/subjects/discover_mailboxes (Workspace); collect_* capability flags | | threatlocker | json | Cloud Sensor/On-prem | api_key, instance; optional feeds (defaults: approval_request, unified_audit, system_audit) | | harmony | json | Cloud Sensor/On-prem | client_id, access_key; events.enabled / entities.queries blocks | | cato | cato | Cloud Sensor/On-prem | apikey, accountid | For a quick LOCAL listing of supported types and a given type's config fields, use `limacharlie cloud-adapter list-types` / `external-adapter list-types` and `limacharlie cloud-adapter schema --type `. **IMPORTANT**: For authoritative field definitions, always check the **usp-adapters repo** (the local `schema` output complements but does not replace it): - URL: `https://github.com/refractionPOINT/usp-adapters` - Each adapter has a `client.go` with a `*Config` struct defining all valid fields - All Cloud Sensors require a nested structure with `client_options` containing `identity`, `platform`, and `sensor_seed_key` ## Example Usage ### Example 1: Set Up Okta System Log Adapter **User**: "I want to ingest Okta system logs into LimaCharlie" **Workflow**: 1. **Research Phase**: - Read local docs: `./docs/limacharlie/doc/Sensors/Adapters/Adapter_Types/adapter-types-okta.md` - WebSearch: "Okta System Log API documentation 2024" - Extract: API key requirements, URL format, event types 2. **Configuration**: ```yaml sensor_type: "okta" okta: apikey: "hive://secret/okta-api-key" url: "https://your-company.okta.com" client_options: identity: oid: "" installation_key: "" hostname: "okta-system-logs" platform: "json" sensor_seed_key: "okta-logs-sensor" mapping: sensor_hostname_path: "client/device" event_type_path: "eventType" event_time_path: "published" ``` 3. **External Setup Instructions**: - Create API token in Okta Admin Console - Store token in LimaCharlie secrets: `limacharlie secret set okta-api-key --oid ` 4. **Deploy**: Use `set_cloud_sensor` ### Example 2: Syslog Adapter with Custom Grok Parsing **User**: "Set up a syslog adapter for our firewall. Sample log: `<134>Nov 15 12:30:45 fw01 ACCEPT TCP 192.168.1.100:54321 10.0.0.5:443`" **Workflow**: 1. **Analyze log format**: Priority, syslog timestamp, hostname, action, protocol, src:port dst:port 2. **Generate Grok pattern**: ```yaml parsing_grok: message: '<%{INT:priority}>%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:host} %{WORD:action} %{WORD:protocol} %{IP:src_ip}:%{NUMBER:src_port} %{IP:dst_ip}:%{NUMBER:dst_port}' ``` 3. **Validate** with `validate_usp_mapping` using the sample log 4. **Deploy as External Adapter**: ```bash cat > /tmp/firewall-adapter.yaml << 'EOF' adapter_type: syslog port: 514 is_udp: true client_options: identity: oid: "" installation_key: "" platform: text sensor_seed_key: firewall-logs hostname: firewall-syslog mapping: parsing_grok: message: "<%{INT:priority}>%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:host} %{WORD:action} %{WORD:protocol} %{IP:src_ip}:%{NUMBER:src_port} %{IP:dst_ip}:%{NUMBER:dst_port}" event_type_path: action event_time_path: timestamp sensor_hostname_path: host EOF limacharlie external-adapter set --key firewall-syslog --input-file /tmp/firewall-adapter.yaml --oid --output yaml ``` ### Example 3: Connect Unknown Product via Webhook **User**: "I want to connect this monitoring tool that sends webhooks, I don't know if LimaCharlie supports it" **Workflow**: 1. **Ask clarifying questions**: - What's the product name? - Do you have sample webhook payload? 2. **Research the product**: ``` WebSearch("{product name} webhook documentation") WebSearch("{product name} webhook payload format") ``` 3. **Read LimaCharlie webhook adapter docs**: ``` Read("./docs/limacharlie/doc/Sensors/Adapters/Adapter_Types/adapter-types-webhook.md") ``` 4. **Build integration plan**: - LimaCharlie webhook adapter configuration - Grok/JSON parsing for the webhook payload - Field mappings for proper event normalization 5. **Deploy and provide webhook URL to user** ### Example 4: Troubleshoot Adapter Not Receiving Data **User**: "My Azure Event Hub adapter isn't receiving data" **Workflow**: 1. **Get current configuration**: ```bash limacharlie cloud-adapter get --key azure-event-hub --oid --output yaml ``` 2. **Check for errors** in `last_error` field of `sys_mtd` 3. **Common issues**: - Connection string missing EntityPath - Consumer group not created - Credentials expired - Diagnostic settings not configured in Azure 4. **Research Azure-side setup**: ``` WebSearch("Azure Event Hub diagnostic settings configuration") ``` 5. **Provide fix guidance** based on error analysis ## Common Pitfalls 1. **Installation Key vs IID**: For adapters, use the IID (UUID format like `e9a3bcdf-efa2-47ae-b6df-579a02f3a54d`), not the full base64 installation key 2. **DATESTAMP vs TIMESTAMP_ISO8601**: - `YYYY-MM-DD HH:MM:SS` → Use `%{TIMESTAMP_ISO8601}` - `MM-DD-YY HH:MM:SS` → Use `%{DATESTAMP}` - `Jan 15 12:30:45` → Use `%{SYSLOGTIMESTAMP}` 3. **Grok field name**: Always use `message` as the key in `parsing_grok` for text platform: ```yaml parsing_grok: message: '%{PATTERN:field}' # Correct ``` 4. **Azure Event Hub connection string**: Must include `EntityPath=` for consumer applications 5. **Credential storage**: Always use `hive://secret/` for sensitive values in production 6. **Unparsed events**: If you see `event_type: "unknown_event"` with only a `text` field, parsing is not configured 7. **Timestamp calculations**: Always use Bash to calculate epoch timestamps dynamically: ```bash start=$(date -d '1 hour ago' +%s) && end=$(date +%s) ``` ## Related Skills - `parsing-helper` - For complex Grok pattern generation and validation - `test-limacharlie-adapter` - For local testing before production deployment - `detection-engineering` - For creating D&R rules on adapter data ## Reference For more details: - **Microsoft ecosystem (Defender, M365, Entra ID, Azure, WEL/EVTX)**: [MICROSOFT.md](./MICROSOFT.md) — authoritative adapter/platform decision matrix - Adapter usage: `./docs/limacharlie/doc/Sensors/Adapters/adapter-usage.md` - Adapter deployment: `./docs/limacharlie/doc/Sensors/Adapters/adapter-deployment.md` - Adapter types: `./docs/limacharlie/doc/Sensors/Adapters/Adapter_Types/` - GitHub source: https://github.com/refractionPOINT/usp-adapters