# Arkime Query

> Query Arkime/OpenSearch for session counts, unique ASes, and traffic statistics. Use when user wants to check NTP/TCP/UDP/ICMP sessions or analyze source AS data.

- Skill: `goodluckz/arkime-query` (Agent Skill)
- Install (CLI): `npx skillmds@latest add goodluckz/arkime-query`
- Raw SKILL.md: https://api.skillmd.com/api/skills/goodluckz/arkime-query/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: goodluckz (https://skillmd.com/u/goodluckz)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/goodluckz/arkime-query

---


# Arkime Query Skill

Query Arkime API directly for session counts, unique source ASes, and traffic analysis on the
`scan-analysis-prod` project.

## Architecture (current)

The old `analyutils` package has been **integrated into** `src/awsntpdagster/defs/arkime_resource.py`.
Use `ArkimeResource` (a Dagster `ConfigurableResource`) — it can be instantiated standalone outside
Dagster and exposes a unified caching layer, auth, and dozens of query helpers.

| Cluster | Port | Description |
|---------|------|-------------|
| ENV-41 (dev) | `8041` | Test cluster, smaller dataset |
| ENV-42 (prod, unified) | `8042` | **Full dataset** — all HoneyNet + AWS NTP experiments |

SSH tunnel must be active: `ssh -N -L 8042:localhost:8042 dns-analy4 &` (or 8041 for dev).
Check with `lsof -i :8042`.

**Redis is optional.** `RedisCacheManager` in `arkime_resource.py` falls back to no-cache mode
automatically (logs `Redis unavailable — running without cache`). Start with `brew services start redis`
only if you want cache hits across runs.

## Canonical query pattern

```bash
uv run python3 -c "
from awsntpdagster.defs.arkime_resource import ArkimeResource

ark = ArkimeResource(port='8042', startTime='0', stopTime='1780297200')  # up to 2026-06-01

# Total sessions to a /64 prefix
total = ark.get_sessions('ip.dst == 2001:200:0:c0b0::/64')

# Per-protocol breakdown — use the ClassVar expressions
ntp  = ark.get_sessions(f'ip.dst == 2001:200:0:c0b0::/64 && ({ark.ntp_protocol_expression})')
tcp  = ark.get_sessions(f'ip.dst == 2001:200:0:c0b0::/64 && ({ark.tcp_protocol_expression})')
udp  = ark.get_sessions(f'ip.dst == 2001:200:0:c0b0::/64 && ({ark.udp_protocol_expression})')
icmp = ark.get_sessions(f'ip.dst == 2001:200:0:c0b0::/64 && ({ark.icmp_protocol_expression})')

print(f'total={total:,} ntp={ntp:,} tcp={tcp:,} udp={udp:,} icmp={icmp:,}')
"
```

## Useful built-in methods on `ArkimeResource`

Most analyses don't need raw expressions — there's a method for it.

| Method | Returns |
|--------|---------|
| `get_sessions(expr)` | int session count |
| `get_sessions_for_time_range(expr, start, stop)` | int (custom time range) |
| `get_sessions_overtime(dst_ip)` | DataFrame, time-bucketed histogram |
| `get_protocol_overview_for_experiment(exp_key)` | DataFrame: NTP/TCP/UDP/ICMP per exp |
| `get_src_ases_for_experiment(exp_key)` | DataFrame of source ASes |
| `get_src_ipv6_with_asn_for_experiment(exp_key)` | DataFrame of src IPs + ASN |
| `get_neighbor_scan(dst_prefix)` | Neighbor scanning within /72 |
| `get_dst_ports(expr)` | Top destination ports |
| `add_tags_by_expression(expr, tags)` | Tag matching sessions in Arkime |

Use `exp_key` (e.g. `exp-ntp-tokyo`) instead of hand-writing prefixes — the method looks
the prefix up from `data/exps.json`.

## Protocol expression cheatsheet

Already exposed as `ClassVar`s — prefer these over hand-rolled strings:

| Constant | Expression |
|---|---|
| `ntp_protocol_expression` | `protocols=ntp` |
| `tcp_protocol_expression` | `protocols=tcp` |
| `udp_protocol_expression` | `protocols=udp&&port.dst!=123` |
| `icmp_protocol_expression` | `(protocols=icmp&&icmp.type=128)` |
| `tcpudp_protocol_expression` | `protocols=tcp \|\| (protocols=udp&&port.dst!=123)` |
| `v6_scan_expression` | TCP \|\| UDP-non-NTP \|\| ICMP echo (v6) |
| `v4_scan_expression` | Same idea, ICMP echo type 0 (v4) |

## Common query recipes

### By source AS

```python
ark.get_sessions(f'source.as.number==4134 && ({ark.ntp_protocol_expression})')
```

### By destination prefix (/64 or /56)

```python
ark.get_sessions('ip.dst == 2001:200:0:c000::/56')
```

### Unique source ASes for a destination

```python
df = ark.get_src_ases('2001:200:0:c0c4::/64')   # DataFrame[as_number, as_org, count]
```

### Source IPs with ASN for an experiment

```python
df = ark.get_src_ipv6_with_asn_for_experiment('exp-ntp-tokyo')
```

### Drop-down to raw expression API

If no helper fits, use the cached request directly:

```python
params = {
    'expression': 'protocols==ntp',
    'startTime': '0',
    'stopTime': '1780297200',
    'counts': 0,
    'exp': 'source.as.full',
}
resp = ark._cached_request(ark.unique_url, params)
```

## Experiment prefixes (ENV-42 unified, port 8042)

Pulled from `data/exps.json`. Use `exp_key` with the helper methods.

### Tokyo HoneyNet (IPv6 honeynet/darknet variants)

| exp_key | Prefix |
|---------|--------|
| `exp-dark-no-expose` | `2001:200:0:c080::/64` |
| `exp-honey-no-expose` | `2001:200:0:c088::/64` |
| `exp-dark-v4rev1` | `2001:200:0:c090::/64` |
| `exp-honey-v4rev1` | `2001:200:0:c092::/64` |
| `exp-dark-v4rev5` | `2001:200:0:c098::/64` |
| `exp-honey-v4rev5` | `2001:200:0:c09a::/64` |
| `exp-dark-v6enum` | `2001:200:0:c0a0::/64` |
| `exp-honey-v6enum` | `2001:200:0:c0a2::/64` |
| `exp-dark-v6special` | `2001:200:0:c0a4::/64` |
| `exp-honey-v6special` | `2001:200:0:c0a6::/64` |
| `exp-dark-popular` | `2001:200:0:c0b0::/64` |
| `exp-honey-popular` | `2001:200:0:c0b8::/64` |
| `exp-ntp-dark` | `2001:200:0:c0c2::/64` |
| `exp-ntp-honey` | `2001:200:0:c0c4::/64` |

### AWS NTP (IPv6, by region)

| exp_key | Prefix |
|---------|--------|
| `exp-ntp-tokyo` | `2406:da14:11a0:20c4::/64` |
| `exp-ntp-california` | `2600:1f1c:601:6100::/64` |
| `exp-ntp-frankfurt` | `2a05:d014:199d:f300::/64` |
| `exp-ntp-virginia` | `2600:1f18:7ac5:b70::/64` |
| `exp-ntp-mumbai` | `2406:da1a:6f3:3daa::/64` |
| `exp-ntp-saopaulo` | `2600:1f1e:d51:f1de::/64` |
| `exp-ntp-sydney` | `2406:da1c:6ac:b6b5::/64` |

IPv4 counterparts (`exp-ntp-*-v4`) also exist — check `data/exps.json` for v4 prefixes.

## stopTime helper

`startTime='0'` covers from the beginning. `stopTime` must be in the future of the data
you want to include.

```python
from datetime import datetime
print(int(datetime(2026, 6, 1).timestamp()))   # → 1780297200
```

Common values:
- `1759276800` = 2025-10-01
- `1769871600` = 2026-02-01
- `1780297200` = 2026-06-01

## Troubleshooting

1. **`ModuleNotFoundError: analyutils`** — outdated; analyutils is gone. Import
   `from awsntpdagster.defs.arkime_resource import ArkimeResource` instead.
2. **Connection refused on 8042** — SSH tunnel down. Restart: `ssh -N -L 8042:localhost:8042 dns-analy4 &`.
3. **Redis warning** — informational only; queries still work without cache.
4. **Empty results** — check (a) `stopTime` is past your data range, (b) you're on the right port
   (8041 vs 8042), (c) prefix uses `::/64` not `:/64` (Arkime accepts both, but be consistent).
5. **Auth errors** — `ArkimeResource` defaults to `username='admin'`/`password='admin'`. Override
   via constructor if the cluster has different creds.