LLM Aiops

LLM AIOps Reference

Overview

Curated reference applying Large Language Models to AIOps, based on the awesome-LLM-AIOps research taxonomy. Covers incident management, log analysis, and infrastructure management.

When to Use

• Incidents: triaging, diagnosing, mitigating cloud incidents
• RCA: root cause analysis of system failures
• Logs: parsing, anomaly detection, logging statement generation
• Infrastructure: IaC, performance diagnosis, LLM training platforms
• AIOps QA: domain-specific question answering for operations

Taxonomy & Key Approaches

1. LLM for Incident Management

2. LLM for Log Analysis

3. LLM for Infrastructure Management

Common LLM Techniques Used

• Prompting: ICL (In-Context Learning), CoT (Chain-of-Thought), ToT (Tree-of-Thought)
• Fine-tuning: Instruction tuning, domain-specific adaptation (e.g., OWL for IT ops)
• Agent-based: Tool-augmented LLMs, multi-agent collaboration, SOP-guided agents
• RAG: Retrieval-Augmented Generation for incident resolution recommendations
• Code Synthesis: LLM generates rule code for anomaly detection (CodeAD)

Key Benchmarks

• ITBench (ICML 2025) — Diverse real-world IT automation tasks
• AIOpsLab (MLSys 2025) — Holistic autonomous cloud agent evaluation
• OpsEval — Comprehensive AIOps task benchmark
• OpenRCA (ICLR 2025) — RCA benchmark with fault propagation awareness
• IaC-Eval (NeurIPS 2024) — Infrastructure-as-Code generation benchmark

Common Mistakes

• ❌ Treating log parsing as a solved problem — LLM-based parsers still struggle with unseen log formats; use adaptive caching (LILAC pattern)
• ❌ Applying generic LLMs to ops without domain tuning — Domain-specific models (OWL) or RAG significantly outperform general-purpose prompting for AIOps Q&A
• ❌ Single-agent RCA for complex microservice failures — Multi-agent systems (mABC, Flow-of-Action) with SOP guidance outperform single-agent approaches
• ❌ Ignoring confidence estimation in RCA — Use PACE-LM style confidence calibration before acting on LLM-generated RCA results

Real-World Impact

The field has rapidly evolved from basic prompting (2023) to sophisticated multi-agent systems with tool augmentation (2024-2025), with production-grade systems like Nissist, STRATUS, and iKnow deployed in real cloud environments. Key trend: moving from single LLM calls to autonomous agent loops with verification.