How do I install the Setting Up Log Aggregation skill?

Run npx skillmds@latest add jsgforever/setting-up-log-aggregation in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Setting Up Log Aggregation safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Setting Up Log Aggregation?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Setting Up Log Aggregation free to use?

Yes. Installing skills from SkillMD is free. This skill is licensed under MIT.

Who published Setting Up Log Aggregation?

jsgforever (@jsgforever) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Setting Up Log Aggregation by jsgforever

Name: Setting Up Log Aggregation
Author: jsgforever

Setting Up Log Aggregation

Use when setting up log aggregation solutions using ELK, Loki, or Splunk. Trigger with phrases like "setup log aggregation", "deploy ELK stack", "configure Loki", or "install Splunk". Generates production-ready configurations for data ingestion, processing, storage, and visualization with proper security and scalability.

jsgforever e47fb26 14 files · 42.6 KB Updated Sep 17, 2026

File contents

Prerequisites

Before using this skill, ensure:

Target infrastructure is identified (Kubernetes, Docker, VMs)
Storage requirements are calculated based on log volume
Network connectivity between log sources and aggregation platform
Authentication mechanism is defined (LDAP, OAuth, basic auth)
Resource allocation planned (CPU, memory, disk)

Instructions

Select Platform: Choose ELK, Loki, Grafana Loki, or Splunk
Configure Ingestion: Set up log shippers (Filebeat, Promtail, Fluentd)
Define Storage: Configure retention policies and index lifecycle
Set Up Processing: Create parsing rules and field extractions
Deploy Visualization: Configure Kibana/Grafana dashboards
Implement Security: Enable authentication, encryption, and RBAC
Test Pipeline: Verify logs flow from sources to visualization

Output

ELK Stack (Docker Compose):

# {baseDir}/elk/docker-compose.yml
version: '3.8'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=true
    volumes:
      - es-data:/usr/share/elasticsearch/data
    ports:
      - "9200:9200"

  logstash:
    image: docker.elastic.co/logstash/logstash:8.11.0
    volumes:
      - ./logstash.conf:/usr/share/logstash/pipeline/logstash.conf
    depends_on:
      - elasticsearch

  kibana:
    image: docker.elastic.co/kibana/kibana:8.11.0
    ports:
      - "5601:5601"
    depends_on:
      - elasticsearch

Loki Configuration:

# {baseDir}/loki/loki-config.yaml
auth_enabled: false

server:
  http_listen_port: 3100

ingester:
  lifecycler:
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
  chunk_idle_period: 5m
  chunk_retain_period: 30s

schema_config:
  configs:
    - from: 2024-01-01
      store: boltdb-shipper
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 24h

Error Handling

Out of Memory

Error: "Elasticsearch heap space exhausted"
Solution: Increase heap size in elasticsearch.yml or add more nodes

Connection Refused

Error: "Cannot connect to Elasticsearch"
Solution: Verify network connectivity and firewall rules

Index Creation Failed

Error: "Failed to create index"
Solution: Check disk space and index template configuration

Log Parsing Errors

Error: "Failed to parse log line"
Solution: Review grok patterns or JSON parsing configuration

Resources

ELK Stack guide: https://www.elastic.co/guide/
Loki documentation: https://grafana.com/docs/loki/
Example configurations in {baseDir}/log-aggregation-examples/

---name: setting-up-log-aggregationdescription: Use when setting up log aggregation solutions using ELK, Loki, or Splunk. Trigger with phrases like "setup log aggregation", "deploy ELK stack", "configure Loki", or "install Splunk". Generates production-ready configurations for data ingestion, processing, storage, and visualization with proper security and scalability.license: MIT---## PrerequisitesBefore using this skill, ensure:- Target infrastructure is identified (Kubernetes, Docker, VMs)- Storage requirements are calculated based on log volume- Network connectivity between log sources and aggregation platform- Authentication mechanism is defined (LDAP, OAuth, basic auth)- Resource allocation planned (CPU, memory, disk)## Instructions1. **Select Platform**: Choose ELK, Loki, Grafana Loki, or Splunk2. **Configure Ingestion**: Set up log shippers (Filebeat, Promtail, Fluentd)3. **Define Storage**: Configure retention policies and index lifecycle4. **Set Up Processing**: Create parsing rules and field extractions5. **Deploy Visualization**: Configure Kibana/Grafana dashboards6. **Implement Security**: Enable authentication, encryption, and RBAC7. **Test Pipeline**: Verify logs flow from sources to visualization## Output**ELK Stack (Docker Compose):**```yaml# {baseDir}/elk/docker-compose.ymlversion: '3.8'services: elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0 environment: - discovery.type=single-node - xpack.security.enabled=true volumes: - es-data:/usr/share/elasticsearch/data ports: - "9200:9200" logstash: image: docker.elastic.co/logstash/logstash:8.11.0 volumes: - ./logstash.conf:/usr/share/logstash/pipeline/logstash.conf depends_on: - elasticsearch kibana: image: docker.elastic.co/kibana/kibana:8.11.0 ports: - "5601:5601" depends_on: - elasticsearch```**Loki Configuration:**```yaml# {baseDir}/loki/loki-config.yamlauth_enabled: falseserver: http_listen_port: 3100ingester: lifecycler: ring: kvstore: store: inmemory replication_factor: 1 chunk_idle_period: 5m chunk_retain_period: 30sschema_config: configs: - from: 2024-01-01 store: boltdb-shipper object_store: filesystem schema: v11 index: prefix: index_ period: 24h```## Error Handling**Out of Memory**- Error: "Elasticsearch heap space exhausted"- Solution: Increase heap size in elasticsearch.yml or add more nodes**Connection Refused**- Error: "Cannot connect to Elasticsearch"- Solution: Verify network connectivity and firewall rules**Index Creation Failed**- Error: "Failed to create index"- Solution: Check disk space and index template configuration**Log Parsing Errors**- Error: "Failed to parse log line"- Solution: Review grok patterns or JSON parsing configuration## Resources- ELK Stack guide: https://www.elastic.co/guide/- Loki documentation: https://grafana.com/docs/loki/- Example configurations in {baseDir}/log-aggregation-examples/

Frequently asked questions