input-configurations
When to use
Load this skill whenever tasks include:
- building, modifying, or reviewing
agent/stream/*.yml.hbs templates for non-CEL input types
- configuring request, response, pagination, cursor, or authentication blocks in HTTPJSON templates
- wiring up cloud storage inputs (AWS S3, GCS, Azure Blob, Azure EventHub)
- setting up network inputs (TCP, UDP, HTTP Endpoint, WebSocket)
- configuring file-based inputs (Filestream, Logfile, Journald, Winlog)
- enabling Federated Identity (Cloud Connectors) on an AWS integration package
When not to use
Do not use this skill as the primary guide for:
- CEL program development (
cel-programs) -- CEL program structure, state model, and mito workflow. Exception: the auth.aws / use_cloud_connectors template block for Federated Identity is owned here via references/federated-identity-aws.md
- ingest pipeline processor design (
ingest-pipelines)
- field mappings and ECS compliance (
ecs-field-mappings)
var_groups / provider_permissions schema and format_version floors alone (package-spec) -- use this skill for the end-to-end federation procedure that applies them
Mandatory first read
Always load references/common-input-patterns.md first. It covers patterns that apply to every input type (tags, processors passthrough, variable conventions, forwarded/publisher_pipeline.disable_host coupling). These patterns are prerequisites for all type-specific guides.
Type routing table
Detect the input type from the filename pattern in agent/stream/ or from the data stream manifest input: field, then load the matching guide.
| Input type |
Filename pattern |
Guide |
| HTTPJSON |
httpjson.yml.hbs |
references/httpjson-guide.md |
| AWS S3 |
aws-s3.yml.hbs |
references/aws-s3-guide.md |
| CloudWatch |
aws-cloudwatch.yml.hbs |
references/aws-cloudwatch-guide.md |
| Azure Blob Storage |
azure-blob-storage.yml.hbs |
references/azure-blob-storage-guide.md |
| Azure Event Hub |
azure-eventhub.yml.hbs |
references/azure-eventhub-guide.md |
| GCS |
gcs.yml.hbs |
references/gcs-guide.md |
| GCP Pub/Sub |
gcp-pubsub.yml.hbs |
references/gcp-pubsub-guide.md |
| TCP |
tcp.yml.hbs |
references/tcp-udp-guide.md |
| UDP |
udp.yml.hbs |
references/tcp-udp-guide.md |
| HTTP Endpoint |
http_endpoint.yml.hbs |
references/http-endpoint-guide.md |
| Filestream |
filestream.yml.hbs |
references/filestream-logfile-guide.md |
| Logfile |
log.yml.hbs |
references/filestream-logfile-guide.md |
| Journald |
journald.yml.hbs |
references/journald-guide.md |
| Winlog |
winlog.yml.hbs |
references/winlog-guide.md |
| WebSocket |
websocket.yml.hbs |
references/websocket-guide.md |
Load only the guide for the detected input type, not all guides.
For Federated Identity tasks (any eligible input type, including cel), load
references/federated-identity-aws.md regardless of which input type is
involved.
Handoff
- For CEL program logic, hand off to the
cel-programs skill. Keep this skill loaded for Federated Identity auth.aws / use_cloud_connectors edits on cel.yml.hbs.
- For manifest schema (
var_groups, provider_permissions, format_version / conditions floors), hand off to the package-spec skill (references/var-groups-and-provider-permissions.md).
- For pipeline issues discovered while reviewing input templates, hand off to the
ingest-pipelines skill.
- For field mapping issues found in template variable wiring, hand off to the
ecs-field-mappings skill.
References
references/common-input-patterns.md -- tags, processors passthrough, variable conventions, review flags (applies to ALL input types)
references/httpjson-guide.md -- HTTPJSON template syntax, structure, validation rules, pagination patterns, authentication, cursor persistence
references/aws-s3-guide.md -- S3 bucket/SQS notification collection
references/aws-cloudwatch-guide.md -- CloudWatch log group collection
references/azure-blob-storage-guide.md -- Azure Blob Storage collection
references/azure-eventhub-guide.md -- Azure Event Hub collection
references/gcs-guide.md -- Google Cloud Storage collection
references/gcp-pubsub-guide.md -- GCP Pub/Sub collection
references/tcp-udp-guide.md -- TCP and UDP (syslog-style) listeners
references/http-endpoint-guide.md -- HTTP Endpoint (webhook receiver)
references/filestream-logfile-guide.md -- Filestream and legacy Logfile inputs
references/journald-guide.md -- Journald collection
references/winlog-guide.md -- Windows Event Log collection
references/websocket-guide.md -- WebSocket streaming (may embed CEL)
references/federated-identity-aws.md -- AWS Federated Identity procedure: input classification, federation vars, auth.aws / use_cloud_connectors, input gating (schema lives in package-spec)
1---
2name: input-configurations
3description: >-
4 Input template configuration for Elastic integrations. Covers agent stream
5 templates (agent/stream/*.yml.hbs) for all non-CEL input types: HTTPJSON,
6 AWS S3, CloudWatch, Azure Blob, Azure EventHub, GCS, GCP Pub/Sub, TCP, UDP,
7 HTTP Endpoint, Filestream, Logfile, Journald, Winlog, and WebSocket. Also
8 covers Federated Identity (Cloud Connectors) for agentless AWS integrations,
9 including the auth.aws / use_cloud_connectors block on CEL stream templates.
10 For CEL program logic, use the cel-programs skill.
11license: Apache-2.0
12metadata:
13 source: https://skillmd.com/skills/elastic/input-configurations
14 author: elastic
15 version: "1.0"
16---
17
18# input-configurations
19
20## When to use
21
22Load this skill whenever tasks include:
23- building, modifying, or reviewing `agent/stream/*.yml.hbs` templates for non-CEL input types
24- configuring request, response, pagination, cursor, or authentication blocks in HTTPJSON templates
25- wiring up cloud storage inputs (AWS S3, GCS, Azure Blob, Azure EventHub)
26- setting up network inputs (TCP, UDP, HTTP Endpoint, WebSocket)
27- configuring file-based inputs (Filestream, Logfile, Journald, Winlog)
28- enabling Federated Identity (Cloud Connectors) on an AWS integration package
29
30## When not to use
31
32Do not use this skill as the primary guide for:
33- CEL *program* development (`cel-programs`) -- CEL program structure, state model, and mito workflow. Exception: the `auth.aws` / `use_cloud_connectors` template block for Federated Identity is owned here via `references/federated-identity-aws.md`
34- ingest pipeline processor design (`ingest-pipelines`)
35- field mappings and ECS compliance (`ecs-field-mappings`)
36- `var_groups` / `provider_permissions` *schema* and `format_version` floors alone (`package-spec`) -- use this skill for the end-to-end federation procedure that applies them
37
38## Mandatory first read
39
40**Always load `references/common-input-patterns.md` first.** It covers patterns that apply to every input type (tags, processors passthrough, variable conventions, `forwarded`/`publisher_pipeline.disable_host` coupling). These patterns are prerequisites for all type-specific guides.
41
42## Type routing table
43
44Detect the input type from the filename pattern in `agent/stream/` or from the data stream manifest `input:` field, then load the matching guide.
45
46| Input type | Filename pattern | Guide |
47|---|---|---|
48| HTTPJSON | `httpjson.yml.hbs` | `references/httpjson-guide.md` |
49| AWS S3 | `aws-s3.yml.hbs` | `references/aws-s3-guide.md` |
50| CloudWatch | `aws-cloudwatch.yml.hbs` | `references/aws-cloudwatch-guide.md` |
51| Azure Blob Storage | `azure-blob-storage.yml.hbs` | `references/azure-blob-storage-guide.md` |
52| Azure Event Hub | `azure-eventhub.yml.hbs` | `references/azure-eventhub-guide.md` |
53| GCS | `gcs.yml.hbs` | `references/gcs-guide.md` |
54| GCP Pub/Sub | `gcp-pubsub.yml.hbs` | `references/gcp-pubsub-guide.md` |
55| TCP | `tcp.yml.hbs` | `references/tcp-udp-guide.md` |
56| UDP | `udp.yml.hbs` | `references/tcp-udp-guide.md` |
57| HTTP Endpoint | `http_endpoint.yml.hbs` | `references/http-endpoint-guide.md` |
58| Filestream | `filestream.yml.hbs` | `references/filestream-logfile-guide.md` |
59| Logfile | `log.yml.hbs` | `references/filestream-logfile-guide.md` |
60| Journald | `journald.yml.hbs` | `references/journald-guide.md` |
61| Winlog | `winlog.yml.hbs` | `references/winlog-guide.md` |
62| WebSocket | `websocket.yml.hbs` | `references/websocket-guide.md` |
63
64Load **only** the guide for the detected input type, not all guides.
65
66For **Federated Identity** tasks (any eligible input type, including `cel`), load
67`references/federated-identity-aws.md` regardless of which input type is
68involved.
69
70## Handoff
71
72- For **CEL program logic**, hand off to the `cel-programs` skill. Keep this skill loaded for Federated Identity `auth.aws` / `use_cloud_connectors` edits on `cel.yml.hbs`.
73- For **manifest schema** (`var_groups`, `provider_permissions`, `format_version` / conditions floors), hand off to the `package-spec` skill (`references/var-groups-and-provider-permissions.md`).
74- For **pipeline issues** discovered while reviewing input templates, hand off to the `ingest-pipelines` skill.
75- For **field mapping issues** found in template variable wiring, hand off to the `ecs-field-mappings` skill.
76
77## References
78
79- `references/common-input-patterns.md` -- tags, processors passthrough, variable conventions, review flags (applies to ALL input types)
80- `references/httpjson-guide.md` -- HTTPJSON template syntax, structure, validation rules, pagination patterns, authentication, cursor persistence
81- `references/aws-s3-guide.md` -- S3 bucket/SQS notification collection
82- `references/aws-cloudwatch-guide.md` -- CloudWatch log group collection
83- `references/azure-blob-storage-guide.md` -- Azure Blob Storage collection
84- `references/azure-eventhub-guide.md` -- Azure Event Hub collection
85- `references/gcs-guide.md` -- Google Cloud Storage collection
86- `references/gcp-pubsub-guide.md` -- GCP Pub/Sub collection
87- `references/tcp-udp-guide.md` -- TCP and UDP (syslog-style) listeners
88- `references/http-endpoint-guide.md` -- HTTP Endpoint (webhook receiver)
89- `references/filestream-logfile-guide.md` -- Filestream and legacy Logfile inputs
90- `references/journald-guide.md` -- Journald collection
91- `references/winlog-guide.md` -- Windows Event Log collection
92- `references/websocket-guide.md` -- WebSocket streaming (may embed CEL)
93- `references/federated-identity-aws.md` -- AWS Federated Identity procedure: input classification, federation vars, `auth.aws` / `use_cloud_connectors`, input gating (schema lives in `package-spec`)