Detecting Shadow IT Cloud Usage
Overview
Shadow IT refers to unauthorized SaaS applications and cloud services used without IT approval. This skill analyzes proxy logs, DNS query logs, and firewall/netflow data to identify unauthorized cloud service usage, classify discovered domains against known SaaS categories, measure data transfer volumes, and flag high-risk services based on security posture and compliance requirements.
Prerequisites
- Python 3.9+ with
pandas, tldextract
- Proxy logs (Squid, Zscaler, or Palo Alto format) or DNS query logs
- SaaS application catalog/blocklist for classification
- Network firewall logs with FQDN resolution (optional)
Steps
- Parse proxy access logs and extract destination domains with traffic volumes
- Parse DNS query logs to identify resolved cloud service domains
- Aggregate traffic by domain using pandas — total bytes, request counts, unique users
- Classify domains against known SaaS categories (storage, email, dev tools, AI)
- Flag unauthorized services not on the approved application list
- Calculate risk scores based on data volume, user count, and service category
- Generate shadow IT discovery report with remediation recommendations
Expected Output
- JSON report listing discovered cloud services with traffic volumes, user counts, risk scores, and approval status
- Top unauthorized services ranked by data exfiltration risk
1---2name: detecting-shadow-it-cloud-usage3description: Detect unauthorized SaaS and cloud service usage (shadow IT) by analyzing proxy logs, DNS query logs, and netflow data using Python pandas for traffic pattern analysis and domain classification.4domain: cybersecurity5subdomain: cloud-security6tags: [shadow-IT, SaaS-discovery, proxy-logs, DNS-analysis, netflow, cloud-security, pandas]7version: "1.0"8author: mahipal9license: Apache-2.010---1112# Detecting Shadow IT Cloud Usage1314## Overview1516Shadow IT refers to unauthorized SaaS applications and cloud services used without IT approval. This skill analyzes proxy logs, DNS query logs, and firewall/netflow data to identify unauthorized cloud service usage, classify discovered domains against known SaaS categories, measure data transfer volumes, and flag high-risk services based on security posture and compliance requirements.1718## Prerequisites1920- Python 3.9+ with `pandas`, `tldextract`21- Proxy logs (Squid, Zscaler, or Palo Alto format) or DNS query logs22- SaaS application catalog/blocklist for classification23- Network firewall logs with FQDN resolution (optional)2425## Steps26271. Parse proxy access logs and extract destination domains with traffic volumes282. Parse DNS query logs to identify resolved cloud service domains293. Aggregate traffic by domain using pandas — total bytes, request counts, unique users304. Classify domains against known SaaS categories (storage, email, dev tools, AI)315. Flag unauthorized services not on the approved application list326. Calculate risk scores based on data volume, user count, and service category337. Generate shadow IT discovery report with remediation recommendations3435## Expected Output3637- JSON report listing discovered cloud services with traffic volumes, user counts, risk scores, and approval status38- Top unauthorized services ranked by data exfiltration risk
autohandai