Splunk AI/ML Toolkit Setup
Prerequisites
| Tool or access |
Purpose |
Verify |
| Bash and Python 3 |
Run bundled setup and validation helpers |
bash --version && python3 --version |
| Required product/platform access |
Inspect or configure the selected target |
Complete the documented preflight |
| Credential files for live modes |
Keep secrets out of chat |
Verify paths only |
Workflow Overview
┌───────────┐ ┌───────────────┐ ┌───────────────┐ ┌─────────────────┐
│ Preflight │ → │ Render/review │ → │ Apply/handoff │ → │ Validate evidence │
└───────────┘ └───────────────┘ └───────────────┘ └─────────────────┘
When to Activate
- MLTK, Splunk AI Toolkit, Machine Learning Toolkit, PSC, Python for Scientific Computing, DSDL, Deep Learning
Toolkit, Splunk anomaly detection assistants, AI Toolkit Agent Launchpad, the
aiagent command, Cisco Time
Series Model, Cisco Deep.
- Preview and review the splunk ai ml toolkit setup workflow before any live apply phase.
- Diagnose failed prerequisites, generated assets, configuration, or validation evidence.
Scope
Follow the documented read-only or render-first path whenever it is available.
This skill does not imply permission to mutate live systems. Require explicit
apply flags, protected credentials, and operator review for state changes.
Examples
Inspect the supported setup modes before selecting one:
bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh --help
Expected output: usage, supported modes, and required arguments are displayed
without changing the target environment.
Inspect validation modes before running completion checks:
bash skills/splunk-ai-ml-toolkit-setup/scripts/validate.sh --help
Expected output: offline, live, and completion options are displayed when the
skill supports them; help exits without mutation.
Troubleshooting
| Issue |
Cause |
Resolution |
| Preflight fails |
A required tool or access path is missing |
Resolve it before rendering or applying |
| Rendered assets are incomplete |
Required non-secret inputs are absent |
Complete intake and render again |
| Apply is blocked |
Review, credentials, or explicit acceptance is missing |
Use the documented handoff |
| Validation is incomplete |
Live evidence is unavailable |
Record the gap and keep completion open |
Shared add-on completion gate
Whenever this workflow installs, configures, or hands off a registry-listed
Splunk app or add-on, follow the
shared completion gate. Package delivery
alone is not success; capture applicable configuration, data/readiness, and
shipped-view evidence, or explicit package evidence that no dashboards ship.
Use this skill for Splunk-owned AI and machine-learning platform workflows
that are not Splunk AI Assistant. It owns coverage reporting, install
orchestration, compatibility validation, DSDL runtime handoffs, and migration
guidance for legacy anomaly apps.
For newer Cisco Data Fabric wording, this is the AI Toolkit / model-workflow
route. Federated search, edge/ingest pipelines, and MCP server setup remain in
their dedicated skills.
AI Toolkit Agent Launchpad is owned here. It became generally available in AI
Toolkit 6.0.0, replacing the earlier Agent Builder feature preview. It is not
Cisco Cloud Control Studio Agent Builder, which belongs to
cisco-cloud-control-setup.
Coverage Boundary
This skill covers Splunk-owned and Splunk-supported AI/ML products:
- Splunk AI Toolkit / MLTK (
Splunk_ML_Toolkit, Splunkbase 2890)
- Python for Scientific Computing (PSC) add-ons:
- Linux 64-bit (
2882, Splunk_SA_Scientific_Python_linux_x86_64)
- Windows 64-bit (
2883, Splunk_SA_Scientific_Python_windows_x86_64)
- Mac Intel (
2881, Splunk_SA_Scientific_Python_darwin_x86_64)
- Mac Apple Silicon (
6785, Splunk_SA_Scientific_Python_darwin_arm64)
- Linux 32-bit (
2884) as legacy migration/blocking coverage only
- Splunk App for Data Science and Deep Learning / DSDL (
4607,
package id mltk-container)
- AI Toolkit Smart Assistants, ML-SPL commands, model management, ONNX apply,
LLM
ai command readiness, Connections tab, Container Management tab,
external LLM/provider connection handoffs, ML alerting, and Cisco Deep Time
Series forecasting/anomaly detection readiness
- AI Toolkit Agent Launchpad readiness, generally available since
6.0.0,
including supported LLM and MCP providers, Agent Skills, the
edit_agent_connections and run_agents capabilities, the aiagent ML-SPL
command, the in-product run-history surface, and the Splunk Cloud region and
egress-allowlist prerequisites
- Cisco Time Series Model 1.0 as an available Apache-2.0 open-weight model,
kept distinct from the AI Toolkit-integrated Cisco Deep Time Series Model
and its hosted Splunk Cloud or self-hosted Enterprise paths
- Hosted foundation model readiness where available in the Splunk Platform
boundary, including Foundation-Sec and GPT-OSS review handoffs; CDTSM is not
represented as an LLM connection, and this skill never renders external
model API keys
- Legacy Splunk App for Anomaly Detection (
6843) and Smart Alerts Assistant
beta (6415) as audit and migration-only coverage
Third-party AI-tagged Splunkbase apps are out of scope unless another skill
explicitly routes them.
Splunk 10.5 Legacy Package Guardrail
Splunkbase does not list Splunk 10.5 support for PSC Linux 32-bit (2884),
Splunk App for Anomaly Detection (6843), or Smart Alerts Assistant beta
(6415). Do not install any of these three packages on a new or upgraded
Splunk 10.5 deployment. They remain in this skill only so an existing estate
can inventory dependencies and render a migration plan. Use a supported
64-bit PSC package and current Splunk AI Toolkit workflows for replacement
coverage.
Safety Rules
- Never ask for Splunk passwords, Splunkbase passwords, HEC tokens, LLM API
keys, cloud provider secrets, DSDL container credentials, or model registry
tokens in chat.
- Never pass secrets on the command line or as environment-variable prefixes.
- LLM provider credentials, HEC tokens, Splunk access tokens, Docker registry
secrets, Kubernetes kubeconfigs, and TLS key material must be file-backed or
delegated to the owning setup skill.
- Do not install legacy EOL/beta anomaly apps by default, and never install
2884, 6415, or 6843 on Splunk 10.5. Audit and migrate them to current
AI Toolkit workflows.
- Do not claim DSDL runtime automation for Docker, Kubernetes, OpenShift, HPC,
GPU, air-gapped images, Jupyter notebooks, or model governance unless the
workflow is rendered as a handoff or an owning runtime skill applies it.
- Do not report Agent Launchpad as ready from a package install alone. The
public package ships the views and the
aiagent command, but Splunk Cloud
still needs a supported AWS region plus the region's egress IP in the stack
allowlist, and Splunk Enterprise needs the Splunk Cloud Connect app.
- Do not pair AI Toolkit
6.0.2 with a PSC release below 4.3.4, and do not
downgrade the audited AI Toolkit/PSC pair based on older documentation.
- Do not conflate the open Cisco Time Series Model 1.0 release with the Cisco
Deep Time Series Model integration. Both are available, but the open model
weights, the model service, and the AI Toolkit experience remain separately
validated layers.
- Do not create or redirect the agent run-history index, and do not save
knowledge-base, MCP, LLM, or model-server credentials from this render-only
handoff.
Primary Workflow
Render and validate a complete coverage plan:
bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \
--render --validate \
--spec skills/splunk-ai-ml-toolkit-setup/template.example \
--output-dir splunk-ai-ml-toolkit-rendered
Install or update AI Toolkit with the right PSC add-on:
bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \
--install \
--psc-target linux64
Include DSDL package delivery and runtime handoff artifacts:
bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \
--render --validate \
--include-dsdl \
--dsdl-runtime kubernetes \
--output-dir splunk-ai-ml-toolkit-rendered
Audit legacy anomaly apps without installing them:
bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \
--doctor \
--legacy-anomaly-audit
Apply Model
--install renders, runs the offline structural check, delegates package
delivery, then runs a live post-install validation (skipped under
--dry-run) that confirms the PSC prerequisite is installed (a missing PSC
add-on fails) and that AI Toolkit — and DSDL when planned — are present in
Splunk. Live validation requires Splunk credentials.
--validate always renders before validating, so the offline structural
check never runs against a stale or missing rendered directory.
- Package delivery delegates to
splunk-app-install.
- Package delivery intentionally omits
--app-version so Splunkbase/ACS pulls
the latest compatible release; audited version metadata is used for reports
and regression checks, not as a live install pin.
- AI Toolkit and PSC belong on the search tier/search head cluster only.
- Install order is PSC first, AI Toolkit second, optional DSDL third.
- DSDL external runtimes are rendered handoffs by default:
docker, kubernetes, openshift, hpc, gpu, airgap, or handoff.
- Legacy Anomaly Detection and Smart Alerts beta are never part of the default
install plan; the skill emits migration reports instead.
Validation Rules
Validation must fail for:
- Unknown coverage statuses in
coverage-report.json
- AI Toolkit install plans without a selected compatible PSC target
- PSC Linux 32-bit as a new install target; on Splunk 10.5 it is inventory and
migration coverage only
- DSDL requested without AI Toolkit and PSC coverage in the same plan
- Direct-secret flags such as
--token, --api-token, --password,
--client-secret, or --llm-api-key
- Missing or unsupported upstream
product_stage values in the rendered
coverage report, including any report that fails to keep the Agent Launchpad
surfaces and integrated CDTSM at ga and open CTSM 1.0 at available for
the verified 2026-08-20 source baseline
Validation must warn for:
- DSDL Docker runtime in production because TLS, image provenance, and network
isolation must be handled by the operator
- AI Toolkit/PSC versions lower than the latest audited compatibility pair
- Any attempted Splunk 10.5 install of legacy Anomaly Detection or Smart
Alerts Assistant beta
- MLTK model objects created before the MLTK 5.3 compatibility break, which
may need retraining
- Agent Launchpad requests without a confirmed supported Splunk Cloud region
and egress allowlist entry, or without Splunk Cloud Connect on Enterprise
References
- Read
reference.md before changing product coverage, compatibility rules,
generated artifacts, or live install behavior.
- Use
scripts/render_assets.py --discover to print the built-in product
catalog and coverage surface.
1---2name: splunk-ai-ml-toolkit-setup3description: "Use when the user asks about MLTK, Splunk AI Toolkit, Machine Learning Toolkit, PSC, Python for4 Scientific Computing, DSDL, Deep Learning Toolkit, Splunk anomaly detection assistants, AI Toolkit Agent5 Launchpad or the earlier Agent Builder preview, the `aiagent` command, Cisco Time Series Model, Cisco Deep6 Time Series Model, Smart Alerts7 Assistant, or AI/ML product coverage outside Splunk AI Assistant, including Cisco Data Fabric requests8 about AI Toolkit or machine-data model workflows. Install, render, validate, and audit Cisco Data Fabric9 AI Toolkit and Splunk-owned AI and machine-learning workflows beyond Splunk AI Assistant: Splunk AI10 Toolkit / MLTK (`Splunk_ML_Toolkit`), Python for Scientific Computing (PSC), Splunk App for Data Science11 and Deep Learning (DSDL), MLTK anomaly workflows, LLM/`ai` command readiness, external model runtimes,12 and legacy anomaly app migration."13compatibility: "Splunk Cloud Platform 10.5.2605: conditional. Follow documented package, entitlement, topology, and customer-managed runtime guardrails; self-managed paths remain on the public 10.4 baseline."14metadata:15 source: https://skillmd.com/skills/chambear2809/splunk-ai-ml-toolkit-setup16 splunk_cloud_10_5: "conditional"17 compatibility_verified: "2026-08-20"18---1920# Splunk AI/ML Toolkit Setup2122## Prerequisites2324| Tool or access | Purpose | Verify |25|---|---|---|26| Bash and Python 3 | Run bundled setup and validation helpers | `bash --version && python3 --version` |27| Required product/platform access | Inspect or configure the selected target | Complete the documented preflight |28| Credential files for live modes | Keep secrets out of chat | Verify paths only |2930## Workflow Overview3132```text33┌───────────┐ ┌───────────────┐ ┌───────────────┐ ┌─────────────────┐34│ Preflight │ → │ Render/review │ → │ Apply/handoff │ → │ Validate evidence │35└───────────┘ └───────────────┘ └───────────────┘ └─────────────────┘36```3738## When to Activate3940- MLTK, Splunk AI Toolkit, Machine Learning Toolkit, PSC, Python for Scientific Computing, DSDL, Deep Learning41 Toolkit, Splunk anomaly detection assistants, AI Toolkit Agent Launchpad, the `aiagent` command, Cisco Time42 Series Model, Cisco Deep.43- Preview and review the splunk ai ml toolkit setup workflow before any live apply phase.44- Diagnose failed prerequisites, generated assets, configuration, or validation evidence.4546## Scope4748Follow the documented read-only or render-first path whenever it is available.49This skill does not imply permission to mutate live systems. Require explicit50apply flags, protected credentials, and operator review for state changes.5152## Examples5354Inspect the supported setup modes before selecting one:5556```bash57bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh --help58```5960Expected output: usage, supported modes, and required arguments are displayed61without changing the target environment.6263Inspect validation modes before running completion checks:6465```bash66bash skills/splunk-ai-ml-toolkit-setup/scripts/validate.sh --help67```6869Expected output: offline, live, and completion options are displayed when the70skill supports them; help exits without mutation.7172## Troubleshooting7374| Issue | Cause | Resolution |75|---|---|---|76| Preflight fails | A required tool or access path is missing | Resolve it before rendering or applying |77| Rendered assets are incomplete | Required non-secret inputs are absent | Complete intake and render again |78| Apply is blocked | Review, credentials, or explicit acceptance is missing | Use the documented handoff |79| Validation is incomplete | Live evidence is unavailable | Record the gap and keep completion open |8081## Shared add-on completion gate8283Whenever this workflow installs, configures, or hands off a registry-listed84Splunk app or add-on, follow the85[shared completion gate](../shared/ta_completion_gate.md). Package delivery86alone is not success; capture applicable configuration, data/readiness, and87shipped-view evidence, or explicit package evidence that no dashboards ship.8889Use this skill for Splunk-owned AI and machine-learning platform workflows90that are not Splunk AI Assistant. It owns coverage reporting, install91orchestration, compatibility validation, DSDL runtime handoffs, and migration92guidance for legacy anomaly apps.9394For newer Cisco Data Fabric wording, this is the AI Toolkit / model-workflow95route. Federated search, edge/ingest pipelines, and MCP server setup remain in96their dedicated skills.9798AI Toolkit Agent Launchpad is owned here. It became generally available in AI99Toolkit `6.0.0`, replacing the earlier Agent Builder feature preview. It is not100Cisco Cloud Control Studio Agent Builder, which belongs to101`cisco-cloud-control-setup`.102103## Coverage Boundary104105This skill covers Splunk-owned and Splunk-supported AI/ML products:106107- Splunk AI Toolkit / MLTK (`Splunk_ML_Toolkit`, Splunkbase `2890`)108- Python for Scientific Computing (PSC) add-ons:109 - Linux 64-bit (`2882`, `Splunk_SA_Scientific_Python_linux_x86_64`)110 - Windows 64-bit (`2883`, `Splunk_SA_Scientific_Python_windows_x86_64`)111 - Mac Intel (`2881`, `Splunk_SA_Scientific_Python_darwin_x86_64`)112 - Mac Apple Silicon (`6785`, `Splunk_SA_Scientific_Python_darwin_arm64`)113 - Linux 32-bit (`2884`) as legacy migration/blocking coverage only114- Splunk App for Data Science and Deep Learning / DSDL (`4607`,115 package id `mltk-container`)116- AI Toolkit Smart Assistants, ML-SPL commands, model management, ONNX apply,117 LLM `ai` command readiness, Connections tab, Container Management tab,118 external LLM/provider connection handoffs, ML alerting, and Cisco Deep Time119 Series forecasting/anomaly detection readiness120- AI Toolkit Agent Launchpad readiness, generally available since `6.0.0`,121 including supported LLM and MCP providers, Agent Skills, the122 `edit_agent_connections` and `run_agents` capabilities, the `aiagent` ML-SPL123 command, the in-product run-history surface, and the Splunk Cloud region and124 egress-allowlist prerequisites125- Cisco Time Series Model 1.0 as an available Apache-2.0 open-weight model,126 kept distinct from the AI Toolkit-integrated Cisco Deep Time Series Model127 and its hosted Splunk Cloud or self-hosted Enterprise paths128- Hosted foundation model readiness where available in the Splunk Platform129 boundary, including Foundation-Sec and GPT-OSS review handoffs; CDTSM is not130 represented as an LLM connection, and this skill never renders external131 model API keys132- Legacy Splunk App for Anomaly Detection (`6843`) and Smart Alerts Assistant133 beta (`6415`) as audit and migration-only coverage134135Third-party AI-tagged Splunkbase apps are out of scope unless another skill136explicitly routes them.137138## Splunk 10.5 Legacy Package Guardrail139140Splunkbase does not list Splunk 10.5 support for PSC Linux 32-bit (`2884`),141Splunk App for Anomaly Detection (`6843`), or Smart Alerts Assistant beta142(`6415`). Do not install any of these three packages on a new or upgraded143Splunk 10.5 deployment. They remain in this skill only so an existing estate144can inventory dependencies and render a migration plan. Use a supported14564-bit PSC package and current Splunk AI Toolkit workflows for replacement146coverage.147148## Safety Rules149150- Never ask for Splunk passwords, Splunkbase passwords, HEC tokens, LLM API151 keys, cloud provider secrets, DSDL container credentials, or model registry152 tokens in chat.153- Never pass secrets on the command line or as environment-variable prefixes.154- LLM provider credentials, HEC tokens, Splunk access tokens, Docker registry155 secrets, Kubernetes kubeconfigs, and TLS key material must be file-backed or156 delegated to the owning setup skill.157- Do not install legacy EOL/beta anomaly apps by default, and never install158 `2884`, `6415`, or `6843` on Splunk 10.5. Audit and migrate them to current159 AI Toolkit workflows.160- Do not claim DSDL runtime automation for Docker, Kubernetes, OpenShift, HPC,161 GPU, air-gapped images, Jupyter notebooks, or model governance unless the162 workflow is rendered as a handoff or an owning runtime skill applies it.163- Do not report Agent Launchpad as ready from a package install alone. The164 public package ships the views and the `aiagent` command, but Splunk Cloud165 still needs a supported AWS region plus the region's egress IP in the stack166 allowlist, and Splunk Enterprise needs the Splunk Cloud Connect app.167- Do not pair AI Toolkit `6.0.2` with a PSC release below `4.3.4`, and do not168 downgrade the audited AI Toolkit/PSC pair based on older documentation.169- Do not conflate the open Cisco Time Series Model 1.0 release with the Cisco170 Deep Time Series Model integration. Both are available, but the open model171 weights, the model service, and the AI Toolkit experience remain separately172 validated layers.173- Do not create or redirect the agent run-history index, and do not save174 knowledge-base, MCP, LLM, or model-server credentials from this render-only175 handoff.176177## Primary Workflow178179Render and validate a complete coverage plan:180181```bash182bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \183 --render --validate \184 --spec skills/splunk-ai-ml-toolkit-setup/template.example \185 --output-dir splunk-ai-ml-toolkit-rendered186```187188Install or update AI Toolkit with the right PSC add-on:189190```bash191bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \192 --install \193 --psc-target linux64194```195196Include DSDL package delivery and runtime handoff artifacts:197198```bash199bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \200 --render --validate \201 --include-dsdl \202 --dsdl-runtime kubernetes \203 --output-dir splunk-ai-ml-toolkit-rendered204```205206Audit legacy anomaly apps without installing them:207208```bash209bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \210 --doctor \211 --legacy-anomaly-audit212```213214## Apply Model215216- `--install` renders, runs the offline structural check, delegates package217 delivery, then runs a **live** post-install validation (skipped under218 `--dry-run`) that confirms the PSC prerequisite is installed (a missing PSC219 add-on fails) and that AI Toolkit — and DSDL when planned — are present in220 Splunk. Live validation requires Splunk credentials.221- `--validate` always renders before validating, so the offline structural222 check never runs against a stale or missing rendered directory.223- Package delivery delegates to `splunk-app-install`.224- Package delivery intentionally omits `--app-version` so Splunkbase/ACS pulls225 the latest compatible release; audited version metadata is used for reports226 and regression checks, not as a live install pin.227- AI Toolkit and PSC belong on the search tier/search head cluster only.228- Install order is PSC first, AI Toolkit second, optional DSDL third.229- DSDL external runtimes are rendered handoffs by default:230 `docker`, `kubernetes`, `openshift`, `hpc`, `gpu`, `airgap`, or `handoff`.231- Legacy Anomaly Detection and Smart Alerts beta are never part of the default232 install plan; the skill emits migration reports instead.233234## Validation Rules235236Validation must fail for:237238- Unknown coverage statuses in `coverage-report.json`239- AI Toolkit install plans without a selected compatible PSC target240- PSC Linux 32-bit as a new install target; on Splunk 10.5 it is inventory and241 migration coverage only242- DSDL requested without AI Toolkit and PSC coverage in the same plan243- Direct-secret flags such as `--token`, `--api-token`, `--password`,244 `--client-secret`, or `--llm-api-key`245- Missing or unsupported upstream `product_stage` values in the rendered246 coverage report, including any report that fails to keep the Agent Launchpad247 surfaces and integrated CDTSM at `ga` and open CTSM 1.0 at `available` for248 the verified 2026-08-20 source baseline249250Validation must warn for:251252- DSDL Docker runtime in production because TLS, image provenance, and network253 isolation must be handled by the operator254- AI Toolkit/PSC versions lower than the latest audited compatibility pair255- Any attempted Splunk 10.5 install of legacy Anomaly Detection or Smart256 Alerts Assistant beta257- MLTK model objects created before the MLTK 5.3 compatibility break, which258 may need retraining259- Agent Launchpad requests without a confirmed supported Splunk Cloud region260 and egress allowlist entry, or without Splunk Cloud Connect on Enterprise261262## References263264- Read `reference.md` before changing product coverage, compatibility rules,265 generated artifacts, or live install behavior.266- Use `scripts/render_assets.py --discover` to print the built-in product267 catalog and coverage surface.