Arkime Query

Arkime Query Skill

Query Arkime API directly for session counts, unique source ASes, and traffic analysis on the scan-analysis-prod project.

Architecture (current)

The old analyutils package has been integrated into src/awsntpdagster/defs/arkime_resource.py. Use ArkimeResource (a Dagster ConfigurableResource) — it can be instantiated standalone outside Dagster and exposes a unified caching layer, auth, and dozens of query helpers.

SSH tunnel must be active: ssh -N -L 8042:localhost:8042 dns-analy4 & (or 8041 for dev). Check with lsof -i :8042.

Redis is optional. RedisCacheManager in arkime_resource.py falls back to no-cache mode automatically (logs Redis unavailable — running without cache). Start with brew services start redis only if you want cache hits across runs.

Canonical query pattern

uv run python3 -c "
from awsntpdagster.defs.arkime_resource import ArkimeResource

ark = ArkimeResource(port='8042', startTime='0', stopTime='1780297200')  # up to 2026-06-01

# Total sessions to a /64 prefix
total = ark.get_sessions('ip.dst == 2001:200:0:c0b0::/64')

# Per-protocol breakdown — use the ClassVar expressions
ntp  = ark.get_sessions(f'ip.dst == 2001:200:0:c0b0::/64 && ({ark.ntp_protocol_expression})')
tcp  = ark.get_sessions(f'ip.dst == 2001:200:0:c0b0::/64 && ({ark.tcp_protocol_expression})')
udp  = ark.get_sessions(f'ip.dst == 2001:200:0:c0b0::/64 && ({ark.udp_protocol_expression})')
icmp = ark.get_sessions(f'ip.dst == 2001:200:0:c0b0::/64 && ({ark.icmp_protocol_expression})')

print(f'total={total:,} ntp={ntp:,} tcp={tcp:,} udp={udp:,} icmp={icmp:,}')
"

Useful built-in methods on ArkimeResource

Most analyses don't need raw expressions — there's a method for it.

Use exp_key (e.g. exp-ntp-tokyo) instead of hand-writing prefixes — the method looks the prefix up from data/exps.json.

Protocol expression cheatsheet

Already exposed as ClassVars — prefer these over hand-rolled strings:

Common query recipes

By source AS

ark.get_sessions(f'source.as.number==4134 && ({ark.ntp_protocol_expression})')

By destination prefix (/64 or /56)

ark.get_sessions('ip.dst == 2001:200:0:c000::/56')

Unique source ASes for a destination

df = ark.get_src_ases('2001:200:0:c0c4::/64')   # DataFrame[as_number, as_org, count]

Source IPs with ASN for an experiment

df = ark.get_src_ipv6_with_asn_for_experiment('exp-ntp-tokyo')

Drop-down to raw expression API

If no helper fits, use the cached request directly:

params = {
    'expression': 'protocols==ntp',
    'startTime': '0',
    'stopTime': '1780297200',
    'counts': 0,
    'exp': 'source.as.full',
}
resp = ark._cached_request(ark.unique_url, params)

Experiment prefixes (ENV-42 unified, port 8042)

Pulled from data/exps.json. Use exp_key with the helper methods.

Tokyo HoneyNet (IPv6 honeynet/darknet variants)

AWS NTP (IPv6, by region)

IPv4 counterparts (exp-ntp-*-v4) also exist — check data/exps.json for v4 prefixes.

stopTime helper

startTime='0' covers from the beginning. stopTime must be in the future of the data you want to include.

from datetime import datetime
print(int(datetime(2026, 6, 1).timestamp()))   # → 1780297200

Common values:

• 1759276800 = 2025-10-01
• 1769871600 = 2026-02-01
• 1780297200 = 2026-06-01

Troubleshooting

1. ModuleNotFoundError: analyutils — outdated; analyutils is gone. Import from awsntpdagster.defs.arkime_resource import ArkimeResource instead.
2. Connection refused on 8042 — SSH tunnel down. Restart: ssh -N -L 8042:localhost:8042 dns-analy4 &.
3. Redis warning — informational only; queries still work without cache.
4. Empty results — check (a) stopTime is past your data range, (b) you're on the right port (8041 vs 8042), (c) prefix uses ::/64 not :/64 (Arkime accepts both, but be consistent).
5. Auth errors — ArkimeResource defaults to username='admin'/password='admin'. Override via constructor if the cluster has different creds.