Splunk Platform PKI Setup
Prerequisites
| Tool or access |
Purpose |
Verify |
| Bash and Python 3 |
Run bundled setup and validation helpers |
bash --version && python3 --version |
| Required product/platform access |
Inspect or configure the selected target |
Complete the documented preflight |
| Credential files for live modes |
Keep secrets out of chat |
Verify paths only |
Workflow Overview
┌───────────┐ ┌───────────────┐ ┌───────────────┐ ┌─────────────────┐
│ Preflight │ → │ Render/review │ → │ Apply/handoff │ → │ Validate evidence │
└───────────┘ └───────────────┘ └───────────────┘ └─────────────────┘
When to Activate
- Build Splunk PKI, mint certs, prepare third-party CA CSRs, replace default certs, configure mTLS, fix KV Store
cert validation, encrypt replication traffic, configure SAML/LDAPS trust, or rotate Splunk TLS certificates.
- Preview and review the splunk platform pki setup workflow before any live apply phase.
- Diagnose failed prerequisites, generated assets, configuration, or validation evidence.
Scope
Follow the documented read-only or render-first path whenever it is available.
This skill does not imply permission to mutate live systems. Require explicit
apply flags, protected credentials, and operator review for state changes.
Examples
Inspect the supported setup modes before selecting one:
bash skills/splunk-platform-pki-setup/scripts/setup.sh --help
Expected output: usage, supported modes, and required arguments are displayed
without changing the target environment.
Inspect validation modes before running completion checks:
bash skills/splunk-platform-pki-setup/scripts/validate.sh --help
Expected output: offline, live, and completion options are displayed when the
skill supports them; help exits without mutation.
Troubleshooting
| Issue |
Cause |
Resolution |
| Preflight fails |
A required tool or access path is missing |
Resolve it before rendering or applying |
| Rendered assets are incomplete |
Required non-secret inputs are absent |
Complete intake and render again |
| Apply is blocked |
Review, credentials, or explicit acceptance is missing |
Use the documented handoff |
| Validation is incomplete |
Live evidence is unavailable |
Record the gap and keep completion open |
Shared add-on completion gate
If this workflow installs or hands off a registry-listed certificate or health
add-on, follow the shared completion gate.
Package delivery alone is not success; validate applicable collection and
shipped views, or record explicit package evidence that no dashboards ship.
This skill owns the full TLS / PKI lifecycle for a self-managed
Splunk Enterprise deployment. It runs in either of two modes:
- Private PKI — the skill renders scripts that build an internal
Root CA (and optional Intermediate), then mint per-component
server / client certificates with the right
basicConstraints,
keyUsage, and extendedKeyUsage (including the dual serverAuth
clientAuth EKU that KV Store 7.0+ requires), with per-host
SANs.
- Public PKI — the skill renders per-host CSRs +
openssl.cnf and a handoff Markdown for the operator's
third-party CA (HashiCorp Vault PKI, ACME / cert-manager / Let's
Encrypt, Microsoft AD CS, EJBCA, or any commercial CA). It
installs and validates the returned signed PEMs but never embeds
CA credentials.
It is render-first: the default phase produces a reviewable
directory of CA scripts, CSR templates, install / verify scripts,
per-role distribution payloads (cluster bundle, SHC deployer
bundle, standalone, forwarder fleet, Edge Processor placeholders),
rotation runbooks, and operator handoff Markdown. It refuses to
apply changes until the operator passes --accept-pki-rotation.
Read this first — what this skill does NOT do
- It does not talk to a CA. Public-PKI mode renders CSRs and a
handoff Markdown; the operator submits to Vault / ACME / AD CS /
EJBCA / commercial CA out of band.
- It does not implement rolling restart or cluster bundle apply.
Both are delegated to
skills/splunk-indexer-cluster-setup
(matches the repo precedent set by pass4SymmKey rotation,
which is also operator-orchestrated).
- It does not configure Splunk Web HSTS / CSP / browser security
headers. Splunk Web has no
customHttpHeaders; those headers
come from the reverse proxy and are owned by
skills/splunk-enterprise-public-exposure-hardening.
- It never renders SSLv3, TLS 1.0, or TLS 1.1. For Splunk 10.4+, the
default TLS 1.2 floor permits both TLS 1.2 and TLS 1.3 and renders the
documented
[tls1.3] policy; --tls-version-floor tls1.3 enforces
TLS-1.3-only. Older Splunk versions remain TLS-1.2-only.
- It does not build the FIPS-validated OpenSSL module. The
operator owns the FIPS module; the skill flips FIPS on by
setting both
SPLUNK_FIPS=1 (the master enable switch) and
SPLUNK_FIPS_VERSION in splunk-launch.conf.
- It does not issue certificates for Splunk Cloud. It refuses and
emits the
Universal Forwarder Credentials Package
handoff. Splunk Cloud's ACS does not currently expose a
self-service BYOC endpoint for HEC custom-domain certificates;
operators open a Splunk Support ticket or deploy an
inputs.conf-in-app instead.
- It does not generate Java keystores / truststores (JKS / PKCS#12).
Splunk DB Connect uses those and is intentionally out of scope.
- It does not own Splunk SOAR PKI, Splunk Mobile / Secure Gateway
certs, IdP-side configuration, HSM integration for the CA private
key, or CRL / OCSP responder hosting. Each is referenced where
relevant but operator-driven.
- It does not certify compliance (PCI / HIPAA / FedRAMP / SOC 2 /
DISA STIG). It renders STIG-aligned configs (
--tls-policy stig)
and cites NIST controls in
references/fips-and-common-criteria.md
but does not attest.
Architecture the skill assumes
- Private mode builds an internal root/intermediate CA and signs
the role-specific leaf certificates.
- Public mode renders CSRs and operator handoffs for Vault PKI,
ACME, AD CS, EJBCA, or a commercial CA.
- Rendered outputs become cluster-bundle drop-ins, SHC deployer
apps, standalone overlays, and UF fleet overlays.
- Cluster bundle apply and rolling restart remain delegated to
splunk-indexer-cluster-setup; SHC app push remains delegated
to splunk-agent-management-setup.
Agent behavior — credentials
Never paste secrets into chat or pass them on argv. The skill
consumes file paths for every secret and never embeds secret
values in rendered output:
bash skills/shared/scripts/write_secret_file.sh /tmp/splunk_admin_password
bash skills/shared/scripts/write_secret_file.sh /tmp/splunk_idxc_secret
bash skills/shared/scripts/write_secret_file.sh /tmp/pki_root_ca_key_password
bash skills/shared/scripts/write_secret_file.sh /tmp/pki_intermediate_ca_key_password
bash skills/shared/scripts/write_secret_file.sh /tmp/pki_leaf_key_password
bash skills/shared/scripts/write_secret_file.sh /tmp/pki_saml_sp_key_password
Pass them in via --admin-password-file, --idxc-secret-file,
--ca-key-password-file, --intermediate-ca-key-password-file,
--leaf-key-password-file, --saml-sp-key-password-file.
The rendered pki/install/install-leaf.sh script accepts a
separate --ssl-password-file PATH flag — this is the
plaintext leaf-key passphrase the operator copies to each
target host. install-leaf.sh writes it verbatim to the
sslPassword line of the per-host overlay
($SPLUNK_HOME/etc/system/local/server.conf | web.conf | inputs.conf | outputs.conf)
and on first restart Splunk encrypts it with splunk.secret. In
typical deployments --ssl-password-file and
--leaf-key-password-file reference the same plaintext file
(the leaf key's passphrase, which is what Splunk needs to read
the key). Omit --ssl-password-file when the leaf key is
unencrypted (e.g. PKCS#8 nocrypt for Edge Processor).
For non-secret values (FQDNs, SANs, role inventory, validity days,
key algorithm, mTLS surfaces, FIPS mode, TLS preset) use
template.example.
Quick start
Render a Private PKI for a 3-peer indexer cluster + 3-member SHC
with default Splunk-modern algorithms, mTLS on S2S + HEC,
hostname validation everywhere, and the splunkd cert distributed
through the cluster bundle:
bash skills/splunk-platform-pki-setup/scripts/setup.sh \
--phase render \
--mode private \
--target indexer-cluster,shc,license-manager,deployment-server,monitoring-console \
--cm-fqdn cm01.example.com \
--peer-hosts idx01.example.com,idx02.example.com,idx03.example.com \
--shc-deployer-fqdn deployer01.example.com \
--shc-members sh01.example.com,sh02.example.com,sh03.example.com \
--lm-fqdn lm01.example.com \
--ds-fqdn ds01.example.com \
--mc-fqdn mc01.example.com \
--enable-mtls s2s,hec \
--tls-policy splunk-modern \
--include-intermediate-ca true
Render a Public PKI for the same cluster with a HashiCorp Vault PKI
operator handoff and the SAML SP signing cert:
bash skills/splunk-platform-pki-setup/scripts/setup.sh \
--phase render \
--mode public \
--target indexer-cluster,shc,license-manager,saml-sp \
--cm-fqdn cm01.example.com \
--peer-hosts idx01.example.com,idx02.example.com,idx03.example.com \
--shc-deployer-fqdn deployer01.example.com \
--shc-members sh01.example.com,sh02.example.com,sh03.example.com \
--lm-fqdn lm01.example.com \
--saml-sp true \
--public-ca-name vault \
--leaf-days 397
Render a FIPS 140-3 Private PKI with the indexer-cluster replication
port encrypted (atomic migration of [replication_port://9887] to
[replication_port-ssl://9887]):
bash skills/splunk-platform-pki-setup/scripts/setup.sh \
--phase render \
--mode private \
--target indexer-cluster \
--cm-fqdn cm01.example.com \
--peer-hosts idx01.example.com,idx02.example.com,idx03.example.com \
--fips-mode 140-3 \
--tls-policy fips-140-3 \
--encrypt-replication-port true \
--key-algorithm rsa-2048 \
--include-intermediate-ca true
Render the Edge Processor cert pair (RSA-2048 by default; pass
--key-algorithm ecdsa-p256 for ECDSA):
bash skills/splunk-platform-pki-setup/scripts/setup.sh \
--phase render \
--mode private \
--target edge-processor \
--include-edge-processor true \
--ep-fqdn ep01.example.com \
--ep-data-source-fqdn datasource01.example.com \
--key-format pkcs8
Run preflight against a live host (read-only checks; refuses to
apply):
bash skills/splunk-platform-pki-setup/scripts/setup.sh \
--phase preflight \
--mode private \
--target indexer-cluster,shc \
--cm-fqdn cm01.example.com \
--admin-password-file /tmp/splunk_admin_password
Inventory live cert posture (read-only; never writes):
bash skills/splunk-platform-pki-setup/scripts/setup.sh \
--phase inventory \
--target all \
--admin-password-file /tmp/splunk_admin_password
Apply rendered certs to a search head (mutates Splunk; requires
the explicit accept flag):
bash skills/splunk-platform-pki-setup/scripts/setup.sh \
--phase apply \
--mode private \
--target shc \
--shc-deployer-fqdn deployer01.example.com \
--leaf-target shc \
--leaf-host sh01.example.com \
--leaf-cert-file /tmp/signed/sh01.pem \
--leaf-private-key-file /tmp/signed/sh01.key \
--leaf-ca-bundle-file /tmp/signed/cabundle.pem \
--accept-pki-rotation \
--admin-password-file /tmp/splunk_admin_password \
--leaf-key-password-file /tmp/pki_leaf_key_password
Validate live state post-apply:
bash skills/splunk-platform-pki-setup/scripts/validate.sh \
--target indexer-cluster,shc \
--cm-fqdn cm01.example.com \
--admin-password-file /tmp/splunk_admin_password
What it renders
Under the project root in splunk-platform-pki-rendered/:
pki/private-ca/ — only when --mode private: create-root-ca.sh,
create-intermediate-ca.sh, sign-server-cert.sh,
sign-client-cert.sh, sign-saml-sp.sh, plus openssl-*.cnf
files with the documented basicConstraints / keyUsage /
extendedKeyUsage extensions, and a README.md that walks the
operator through CA generation. Uses
$SPLUNK_HOME/bin/splunk cmd openssl genpkey/req/x509 per
Splunk's documented workflow so the same OpenSSL build that
Splunk uses signs and verifies.
pki/csr-templates/<role>-<host>.cnf + generate-csr.sh —
emitted in both modes; per-host CSR config with SANs and EKU.
pki/install/install-leaf.sh, verify-leaf.sh,
kv-store-eku-check.sh, align-cli-trust.sh,
install-fips-launch-conf.sh, prepare-key.sh — cert
install + verify per host. kv-store-eku-check.sh runs the
documented splunk cmd openssl verify -x509_strict check from
the KV Store custom-cert prep doc and refuses to declare a host
ready unless the verification returns OK.
pki/distribute/cluster-bundle/master-apps/000_pki_trust/local/
— cluster-bundle drop-in: server.conf (with
[replication_port-ssl://9887] if --encrypt-replication-port=true),
inputs.conf ([splunktcp-ssl:9997] + [SSL]).
pki/distribute/shc-deployer/shcluster/apps/000_pki_trust/local/
— SHC deployer drop-in: server.conf, web.conf, inputs.conf.
pki/distribute/standalone/000_pki_trust/local/ — for
non-clustered roles (LM, DS, MC, single SH, HF):
server.conf, web.conf, inputs.conf, outputs.conf,
authentication.conf, deploymentclient.conf,
splunk-launch.conf (when FIPS), system-files/ldap.conf
(when LDAPS).
pki/distribute/forwarder-fleet/<group>/{outputs-overlay.conf,server-overlay.conf}
— UF / HF outputs overlay with clientCert /
sslVerifyServerCert=true / sslVerifyServerName=true and
per-indexer [tcpout-server://host:port] SAN overrides.
pki/distribute/edge-processor/ — only when
--include-edge-processor=true: 5-file PEM placeholders
(ca_cert.pem.example, edge_server_cert.pem.example,
edge_server_key.pem.example,
data_source_client_cert.pem.example,
data_source_client_key.pem.example) + upload-via-rest.sh.example
for the EP REST upload and README.md for the EP UI walkthrough.
pki/distribute/saml-sp/ — only when --saml-sp=true:
sp-signing.crt, sp-signing.key.placeholder, README.md for
re-uploading IdP metadata after rotation.
pki/rotate/{plan-rotation.md, rotate-leaf-host.sh, swap-trust-anchor.sh, swap-replication-port-to-ssl.sh, expire-watch.sh} — rotation helpers with the delegated
rolling-restart runbook.
handoff/ — CA, Cloud, FIPS, Edge Processor, native expiry monitoring,
optional legacy Splunkbase 3172, health, CIM, and operator checklists.
preflight.sh, validate.sh, inventory.sh, README.md, metadata.json.
Certificate-Monitoring Guardrail
SSL Certificate Checker (3172, app ssl_certificate_checker) stops at 9.4
and must not be installed on Cloud 10.5. Use expire-watch.sh plus
inventory.sh; see post-install monitoring.
Phases
render (default) — produce the reviewable rendered tree. No
Splunk REST calls; safe to run anywhere.
preflight — render then run the live preflight checks: cert
directory permissions, default-cert refusal,
KV-Store EKU verification (splunk cmd openssl verify -x509_strict
must return OK), splunk.secret SHA-256 parity across cluster
members, FIPS posture (refuses mid-Phase-1 / Phase-2 migration),
hostname-validation gating, TLS protocol floor check
(sslVersions = tls1.2), per-host
splunk btool server list sslConfig snapshot, replication-port
mode (cleartext vs SSL), [shclustering] pass4SymmKey
presence reminder. Refuses to mark the deployment ready when any
check fails.
apply — render then run the local-host
pki/install/install-leaf.sh + align-cli-trust.sh +
install-fips-launch-conf.sh if FIPS. Requires
explicit --leaf-target, --leaf-host, and --leaf-ca-bundle-file
inputs, plus --leaf-cert-file and --leaf-private-key-file for every
target except CA-only ldaps, and
--accept-pki-rotation (a single-flag acknowledgement that the
operator is about to swap serving certs and trigger downstream
restart).
rotate — render then emit a rotation runbook
(pki/rotate/plan-rotation.md) describing the full delegated
order. Does NOT exec the rolling restart itself (delegate
pattern, see "Rotation ownership" below), and exits nonzero so
rendering the runbook cannot be mistaken for a completed rotation.
validate — render then run the live validation probes:
REST + openssl s_client -connect per surface, KV Store
handshake check, splunk show-decrypted round-trip on
sslPassword, SAML SP signing cert exposed in IdP-metadata
endpoint.
inventory — read-only: collects
splunk btool server list sslConfig, web list sslConfig,
inputs list http SSL, dumps PEM expiry catalogue and emits
pki/inventory/<host>.json. No Splunk write operations. No
--accept-… required.
all — render + one local-host leaf apply + installed-state preflight, then
stop nonzero at the cluster-aware restart/rotation handoff. Run validate only after that
restart completes. It uses the same explicit leaf inputs and
--accept-pki-rotation gate.
Apply guard — --accept-pki-rotation
The skill refuses to run apply or all without
--accept-pki-rotation. This is a single-flag acknowledgement
that:
- The new cert chain has been verified (
verify-leaf.sh returned
OK).
- A rolling restart of the indexer cluster and SHC will follow
(delegated to
splunk-indexer-cluster-setup --phase rolling-restart).
- The SAML / LDAPS / Edge Processor / Splunk Cloud handoffs (where
applicable) will be completed.
- The operator has a rollback plan (the previous PEM directory
is preserved).
The render and preflight phases never need this flag.
Rotation ownership — delegate
The skill emits pki/rotate/plan-rotation.md, but does not run cluster
or SHC rolling restarts itself. Follow the canonical
rotation runbook for the exact staging,
bundle validation/apply, rolling-restart, SHC push, forwarder rollout,
validation, and rollback commands. That delegation preserves the
splunk-indexer-cluster-setup
ownership model instead of duplicating restart orchestration here.
Handoffs and TLS Policy
Read the cross-skill ownership matrix
before delegating restarts, bundle pushes, token lifecycle, or fleet rollout.
TLS algorithms, protocol floors, FIPS lifecycle, validity caps, key formats,
and mTLS defaults are defined in reference.md
and the topic files under references/.
The renderer consumes the machine-readable
algorithm policy and fails closed on
deprecated protocols or incomplete FIPS transitions.
References
Read reference.md before any apply. Topical deep
dives (each anchored to a specific upstream Splunk doc captured in
references/authoritative-sources.md):
- references/component-cert-matrix.md
- references/private-pki-workflow.md
- references/public-pki-workflow.md
- references/handoff-vault-pki.md
- references/handoff-acme-cert-manager.md
- references/handoff-microsoft-adcs.md
- references/handoff-ejbca.md
- references/kv-store-eku-requirements.md
- references/mtls-and-hostname-validation.md
- references/replication-port-tls.md
- references/saml-signing-certs.md
- references/ldaps-trust.md
- references/edge-processor-pki.md
- references/cli-trust-cacert-alignment.md
- references/tls-protocol-policy.md
- references/algorithm-presets.md
- algorithm-policy.json (machine-readable companion consumed by renderer + preflight)
- references/fips-and-common-criteria.md
- references/key-format-and-permissions.md
- references/rotation-runbook.md
- references/post-install-monitoring.md
- references/splunk-cloud-ufcp-handoff.md
- references/troubleshooting.md
- references/authoritative-sources.md
1---2name: splunk-platform-pki-setup3description: "Use when the user asks to build Splunk PKI, mint certs, prepare third-party CA CSRs, replace default4 certs, configure mTLS, fix KV Store cert validation, encrypt replication traffic, configure SAML/LDAPS5 trust, or rotate Splunk TLS certificates. Render, preflight, apply, validate, rotate, and inventory6 private or public PKI for Splunk Enterprise TLS surfaces: Splunk Web, splunkd REST, S2S, HEC, KV Store,7 indexer clusters, SHC, License Manager, Deployment Server, Monitoring Console, Federated Search, heavy8 forwarders, Universal Forwarders, Edge Processor, SAML SP signing, LDAPS trust, and CLI CA trust. Covers9 CSR handoffs, internal CA rendering, FIPS mode, TLS policy presets, KV Store EKU enforcement, default-10 cert refusal, SAN-aware leaf certs, mTLS, replication-port TLS, and delegated rotation runbooks."11compatibility: "Splunk Cloud Platform 10.5.2605: not applicable. This self-managed runtime workflow remains on the public Splunk Enterprise or Universal Forwarder 10.4 baseline."12metadata:13 source: https://skillmd.com/skills/chambear2809/splunk-platform-pki-setup14 splunk_cloud_10_5: "self-managed-10.4"15 compatibility_verified: "2026-08-20"16---1718# Splunk Platform PKI Setup1920## Prerequisites2122| Tool or access | Purpose | Verify |23|---|---|---|24| Bash and Python 3 | Run bundled setup and validation helpers | `bash --version && python3 --version` |25| Required product/platform access | Inspect or configure the selected target | Complete the documented preflight |26| Credential files for live modes | Keep secrets out of chat | Verify paths only |2728## Workflow Overview2930```text31┌───────────┐ ┌───────────────┐ ┌───────────────┐ ┌─────────────────┐32│ Preflight │ → │ Render/review │ → │ Apply/handoff │ → │ Validate evidence │33└───────────┘ └───────────────┘ └───────────────┘ └─────────────────┘34```3536## When to Activate3738- Build Splunk PKI, mint certs, prepare third-party CA CSRs, replace default certs, configure mTLS, fix KV Store39 cert validation, encrypt replication traffic, configure SAML/LDAPS trust, or rotate Splunk TLS certificates.40- Preview and review the splunk platform pki setup workflow before any live apply phase.41- Diagnose failed prerequisites, generated assets, configuration, or validation evidence.4243## Scope4445Follow the documented read-only or render-first path whenever it is available.46This skill does not imply permission to mutate live systems. Require explicit47apply flags, protected credentials, and operator review for state changes.4849## Examples5051Inspect the supported setup modes before selecting one:5253```bash54bash skills/splunk-platform-pki-setup/scripts/setup.sh --help55```5657Expected output: usage, supported modes, and required arguments are displayed58without changing the target environment.5960Inspect validation modes before running completion checks:6162```bash63bash skills/splunk-platform-pki-setup/scripts/validate.sh --help64```6566Expected output: offline, live, and completion options are displayed when the67skill supports them; help exits without mutation.6869## Troubleshooting7071| Issue | Cause | Resolution |72|---|---|---|73| Preflight fails | A required tool or access path is missing | Resolve it before rendering or applying |74| Rendered assets are incomplete | Required non-secret inputs are absent | Complete intake and render again |75| Apply is blocked | Review, credentials, or explicit acceptance is missing | Use the documented handoff |76| Validation is incomplete | Live evidence is unavailable | Record the gap and keep completion open |7778## Shared add-on completion gate7980If this workflow installs or hands off a registry-listed certificate or health81add-on, follow the [shared completion gate](../shared/ta_completion_gate.md).82Package delivery alone is not success; validate applicable collection and83shipped views, or record explicit package evidence that no dashboards ship.8485This skill owns the **full TLS / PKI lifecycle** for a self-managed86Splunk Enterprise deployment. It runs in either of two modes:8788- **Private PKI** — the skill renders scripts that build an internal89 Root CA (and optional Intermediate), then mint per-component90 server / client certificates with the right `basicConstraints`,91 `keyUsage`, and `extendedKeyUsage` (including the dual `serverAuth`92 + `clientAuth` EKU that **KV Store 7.0+ requires**), with per-host93 SANs.94- **Public PKI** — the skill renders per-host CSRs +95 `openssl.cnf` and a handoff Markdown for the operator's96 third-party CA (HashiCorp Vault PKI, ACME / cert-manager / Let's97 Encrypt, Microsoft AD CS, EJBCA, or any commercial CA). It98 installs and validates the returned signed PEMs but never embeds99 CA credentials.100101It is **render-first**: the default phase produces a reviewable102directory of CA scripts, CSR templates, install / verify scripts,103per-role distribution payloads (cluster bundle, SHC deployer104bundle, standalone, forwarder fleet, Edge Processor placeholders),105rotation runbooks, and operator handoff Markdown. It refuses to106apply changes until the operator passes `--accept-pki-rotation`.107108## Read this first — what this skill does NOT do109110- It does not talk to a CA. Public-PKI mode renders CSRs and a111 handoff Markdown; the operator submits to Vault / ACME / AD CS /112 EJBCA / commercial CA out of band.113- It does not implement rolling restart or cluster bundle apply.114 Both are delegated to115 [`skills/splunk-indexer-cluster-setup`](../splunk-indexer-cluster-setup/SKILL.md)116 (matches the repo precedent set by `pass4SymmKey` rotation,117 which is also operator-orchestrated).118- It does not configure Splunk Web HSTS / CSP / browser security119 headers. Splunk Web has no `customHttpHeaders`; those headers120 come from the reverse proxy and are owned by121 [`skills/splunk-enterprise-public-exposure-hardening`](../splunk-enterprise-public-exposure-hardening/SKILL.md).122- It never renders SSLv3, TLS 1.0, or TLS 1.1. For Splunk 10.4+, the123 default TLS 1.2 floor permits both TLS 1.2 and TLS 1.3 and renders the124 documented `[tls1.3]` policy; `--tls-version-floor tls1.3` enforces125 TLS-1.3-only. Older Splunk versions remain TLS-1.2-only.126- It does not build the FIPS-validated OpenSSL module. The127 operator owns the FIPS module; the skill flips FIPS on by128 setting both `SPLUNK_FIPS=1` (the master enable switch) and129 `SPLUNK_FIPS_VERSION` in `splunk-launch.conf`.130- It does not issue certificates for Splunk Cloud. It refuses and131 emits the132 [Universal Forwarder Credentials Package](https://help.splunk.com/?resourceId=Forwarder_Forwarder_ConfigSCUFCredentials)133 handoff. Splunk Cloud's ACS does not currently expose a134 self-service BYOC endpoint for HEC custom-domain certificates;135 operators open a Splunk Support ticket or deploy an136 `inputs.conf`-in-app instead.137- It does not generate Java keystores / truststores (JKS / PKCS#12).138 Splunk DB Connect uses those and is intentionally out of scope.139- It does not own Splunk SOAR PKI, Splunk Mobile / Secure Gateway140 certs, IdP-side configuration, HSM integration for the CA private141 key, or CRL / OCSP responder hosting. Each is referenced where142 relevant but operator-driven.143- It does not certify compliance (PCI / HIPAA / FedRAMP / SOC 2 /144 DISA STIG). It renders STIG-aligned configs (`--tls-policy stig`)145 and cites NIST controls in146 [references/fips-and-common-criteria.md](references/fips-and-common-criteria.md)147 but does not attest.148149## Architecture the skill assumes150151- Private mode builds an internal root/intermediate CA and signs152 the role-specific leaf certificates.153- Public mode renders CSRs and operator handoffs for Vault PKI,154 ACME, AD CS, EJBCA, or a commercial CA.155- Rendered outputs become cluster-bundle drop-ins, SHC deployer156 apps, standalone overlays, and UF fleet overlays.157- Cluster bundle apply and rolling restart remain delegated to158 `splunk-indexer-cluster-setup`; SHC app push remains delegated159 to `splunk-agent-management-setup`.160161## Agent behavior — credentials162163Never paste secrets into chat or pass them on argv. The skill164consumes **file paths** for every secret and never embeds secret165values in rendered output:166167```bash168bash skills/shared/scripts/write_secret_file.sh /tmp/splunk_admin_password169bash skills/shared/scripts/write_secret_file.sh /tmp/splunk_idxc_secret170bash skills/shared/scripts/write_secret_file.sh /tmp/pki_root_ca_key_password171bash skills/shared/scripts/write_secret_file.sh /tmp/pki_intermediate_ca_key_password172bash skills/shared/scripts/write_secret_file.sh /tmp/pki_leaf_key_password173bash skills/shared/scripts/write_secret_file.sh /tmp/pki_saml_sp_key_password174```175176Pass them in via `--admin-password-file`, `--idxc-secret-file`,177`--ca-key-password-file`, `--intermediate-ca-key-password-file`,178`--leaf-key-password-file`, `--saml-sp-key-password-file`.179180The rendered `pki/install/install-leaf.sh` script accepts a181**separate** `--ssl-password-file PATH` flag — this is the182plaintext leaf-key passphrase the operator copies to each183target host. install-leaf.sh writes it verbatim to the184`sslPassword` line of the per-host overlay185(`$SPLUNK_HOME/etc/system/local/server.conf | web.conf | inputs.conf | outputs.conf`)186and on first restart Splunk encrypts it with `splunk.secret`. In187typical deployments `--ssl-password-file` and188`--leaf-key-password-file` reference the same plaintext file189(the leaf key's passphrase, which is what Splunk needs to read190the key). Omit `--ssl-password-file` when the leaf key is191unencrypted (e.g. PKCS#8 nocrypt for Edge Processor).192193For non-secret values (FQDNs, SANs, role inventory, validity days,194key algorithm, mTLS surfaces, FIPS mode, TLS preset) use195[`template.example`](template.example).196197## Quick start198199Render a Private PKI for a 3-peer indexer cluster + 3-member SHC200with default Splunk-modern algorithms, mTLS on S2S + HEC,201hostname validation everywhere, and the splunkd cert distributed202through the cluster bundle:203204```bash205bash skills/splunk-platform-pki-setup/scripts/setup.sh \206 --phase render \207 --mode private \208 --target indexer-cluster,shc,license-manager,deployment-server,monitoring-console \209 --cm-fqdn cm01.example.com \210 --peer-hosts idx01.example.com,idx02.example.com,idx03.example.com \211 --shc-deployer-fqdn deployer01.example.com \212 --shc-members sh01.example.com,sh02.example.com,sh03.example.com \213 --lm-fqdn lm01.example.com \214 --ds-fqdn ds01.example.com \215 --mc-fqdn mc01.example.com \216 --enable-mtls s2s,hec \217 --tls-policy splunk-modern \218 --include-intermediate-ca true219```220221Render a Public PKI for the same cluster with a HashiCorp Vault PKI222operator handoff and the SAML SP signing cert:223224```bash225bash skills/splunk-platform-pki-setup/scripts/setup.sh \226 --phase render \227 --mode public \228 --target indexer-cluster,shc,license-manager,saml-sp \229 --cm-fqdn cm01.example.com \230 --peer-hosts idx01.example.com,idx02.example.com,idx03.example.com \231 --shc-deployer-fqdn deployer01.example.com \232 --shc-members sh01.example.com,sh02.example.com,sh03.example.com \233 --lm-fqdn lm01.example.com \234 --saml-sp true \235 --public-ca-name vault \236 --leaf-days 397237```238239Render a FIPS 140-3 Private PKI with the indexer-cluster replication240port encrypted (atomic migration of `[replication_port://9887]` to241`[replication_port-ssl://9887]`):242243```bash244bash skills/splunk-platform-pki-setup/scripts/setup.sh \245 --phase render \246 --mode private \247 --target indexer-cluster \248 --cm-fqdn cm01.example.com \249 --peer-hosts idx01.example.com,idx02.example.com,idx03.example.com \250 --fips-mode 140-3 \251 --tls-policy fips-140-3 \252 --encrypt-replication-port true \253 --key-algorithm rsa-2048 \254 --include-intermediate-ca true255```256257Render the Edge Processor cert pair (RSA-2048 by default; pass258`--key-algorithm ecdsa-p256` for ECDSA):259260```bash261bash skills/splunk-platform-pki-setup/scripts/setup.sh \262 --phase render \263 --mode private \264 --target edge-processor \265 --include-edge-processor true \266 --ep-fqdn ep01.example.com \267 --ep-data-source-fqdn datasource01.example.com \268 --key-format pkcs8269```270271Run preflight against a live host (read-only checks; refuses to272apply):273274```bash275bash skills/splunk-platform-pki-setup/scripts/setup.sh \276 --phase preflight \277 --mode private \278 --target indexer-cluster,shc \279 --cm-fqdn cm01.example.com \280 --admin-password-file /tmp/splunk_admin_password281```282283Inventory live cert posture (read-only; never writes):284285```bash286bash skills/splunk-platform-pki-setup/scripts/setup.sh \287 --phase inventory \288 --target all \289 --admin-password-file /tmp/splunk_admin_password290```291292Apply rendered certs to a search head (mutates Splunk; requires293the explicit accept flag):294295```bash296bash skills/splunk-platform-pki-setup/scripts/setup.sh \297 --phase apply \298 --mode private \299 --target shc \300 --shc-deployer-fqdn deployer01.example.com \301 --leaf-target shc \302 --leaf-host sh01.example.com \303 --leaf-cert-file /tmp/signed/sh01.pem \304 --leaf-private-key-file /tmp/signed/sh01.key \305 --leaf-ca-bundle-file /tmp/signed/cabundle.pem \306 --accept-pki-rotation \307 --admin-password-file /tmp/splunk_admin_password \308 --leaf-key-password-file /tmp/pki_leaf_key_password309```310311Validate live state post-apply:312313```bash314bash skills/splunk-platform-pki-setup/scripts/validate.sh \315 --target indexer-cluster,shc \316 --cm-fqdn cm01.example.com \317 --admin-password-file /tmp/splunk_admin_password318```319320## What it renders321322Under the project root in `splunk-platform-pki-rendered/`:323324- `pki/private-ca/` — only when `--mode private`: `create-root-ca.sh`,325 `create-intermediate-ca.sh`, `sign-server-cert.sh`,326 `sign-client-cert.sh`, `sign-saml-sp.sh`, plus `openssl-*.cnf`327 files with the documented `basicConstraints` / `keyUsage` /328 `extendedKeyUsage` extensions, and a `README.md` that walks the329 operator through CA generation. Uses330 `$SPLUNK_HOME/bin/splunk cmd openssl genpkey/req/x509` per331 Splunk's documented workflow so the same OpenSSL build that332 Splunk uses signs and verifies.333- `pki/csr-templates/<role>-<host>.cnf` + `generate-csr.sh` —334 emitted in both modes; per-host CSR config with SANs and EKU.335- `pki/install/install-leaf.sh`, `verify-leaf.sh`,336 `kv-store-eku-check.sh`, `align-cli-trust.sh`,337 `install-fips-launch-conf.sh`, `prepare-key.sh` — cert338 install + verify per host. `kv-store-eku-check.sh` runs the339 documented `splunk cmd openssl verify -x509_strict` check from340 the KV Store custom-cert prep doc and refuses to declare a host341 ready unless the verification returns `OK`.342- `pki/distribute/cluster-bundle/master-apps/000_pki_trust/local/`343 — cluster-bundle drop-in: `server.conf` (with344 `[replication_port-ssl://9887]` if `--encrypt-replication-port=true`),345 `inputs.conf` (`[splunktcp-ssl:9997]` + `[SSL]`).346- `pki/distribute/shc-deployer/shcluster/apps/000_pki_trust/local/`347 — SHC deployer drop-in: `server.conf`, `web.conf`, `inputs.conf`.348- `pki/distribute/standalone/000_pki_trust/local/` — for349 non-clustered roles (LM, DS, MC, single SH, HF):350 `server.conf`, `web.conf`, `inputs.conf`, `outputs.conf`,351 `authentication.conf`, `deploymentclient.conf`,352 `splunk-launch.conf` (when FIPS), `system-files/ldap.conf`353 (when LDAPS).354- `pki/distribute/forwarder-fleet/<group>/{outputs-overlay.conf,server-overlay.conf}`355 — UF / HF outputs overlay with `clientCert` /356 `sslVerifyServerCert=true` / `sslVerifyServerName=true` and357 per-indexer `[tcpout-server://host:port]` SAN overrides.358- `pki/distribute/edge-processor/` — only when359 `--include-edge-processor=true`: 5-file PEM placeholders360 (`ca_cert.pem.example`, `edge_server_cert.pem.example`,361 `edge_server_key.pem.example`,362 `data_source_client_cert.pem.example`,363 `data_source_client_key.pem.example`) + `upload-via-rest.sh.example`364 for the EP REST upload and `README.md` for the EP UI walkthrough.365- `pki/distribute/saml-sp/` — only when `--saml-sp=true`:366 `sp-signing.crt`, `sp-signing.key.placeholder`, `README.md` for367 re-uploading IdP metadata after rotation.368- `pki/rotate/{plan-rotation.md, rotate-leaf-host.sh,369 swap-trust-anchor.sh, swap-replication-port-to-ssl.sh,370 expire-watch.sh}` — rotation helpers with the delegated371 rolling-restart runbook.372- `handoff/` — CA, Cloud, FIPS, Edge Processor, native expiry monitoring,373 optional legacy Splunkbase 3172, health, CIM, and operator checklists.374- `preflight.sh`, `validate.sh`, `inventory.sh`, `README.md`, `metadata.json`.375376## Certificate-Monitoring Guardrail377378SSL Certificate Checker (`3172`, app `ssl_certificate_checker`) stops at 9.4379and must not be installed on Cloud 10.5. Use `expire-watch.sh` plus380`inventory.sh`; see [post-install monitoring](references/post-install-monitoring.md).381382## Phases383384- `render` (default) — produce the reviewable rendered tree. No385 Splunk REST calls; safe to run anywhere.386- `preflight` — render then run the live preflight checks: cert387 directory permissions, default-cert refusal,388 KV-Store EKU verification (`splunk cmd openssl verify -x509_strict`389 must return `OK`), `splunk.secret` SHA-256 parity across cluster390 members, FIPS posture (refuses mid-Phase-1 / Phase-2 migration),391 hostname-validation gating, TLS protocol floor check392 (`sslVersions = tls1.2`), per-host393 `splunk btool server list sslConfig` snapshot, replication-port394 mode (cleartext vs SSL), `[shclustering]` `pass4SymmKey`395 presence reminder. Refuses to mark the deployment ready when any396 check fails.397- `apply` — render then run the local-host398 `pki/install/install-leaf.sh` + `align-cli-trust.sh` +399 `install-fips-launch-conf.sh` if FIPS. Requires400 explicit `--leaf-target`, `--leaf-host`, and `--leaf-ca-bundle-file`401 inputs, plus `--leaf-cert-file` and `--leaf-private-key-file` for every402 target except CA-only `ldaps`, and403 `--accept-pki-rotation` (a single-flag acknowledgement that the404 operator is about to swap serving certs and trigger downstream405 restart).406- `rotate` — render then emit a rotation runbook407 (`pki/rotate/plan-rotation.md`) describing the full delegated408 order. Does NOT exec the rolling restart itself (delegate409 pattern, see "Rotation ownership" below), and exits nonzero so410 rendering the runbook cannot be mistaken for a completed rotation.411- `validate` — render then run the live validation probes:412 REST + `openssl s_client -connect` per surface, KV Store413 handshake check, `splunk show-decrypted` round-trip on414 `sslPassword`, SAML SP signing cert exposed in IdP-metadata415 endpoint.416- `inventory` — read-only: collects417 `splunk btool server list sslConfig`, `web list sslConfig`,418 `inputs list http SSL`, dumps PEM expiry catalogue and emits419 `pki/inventory/<host>.json`. No Splunk write operations. No420 `--accept-…` required.421- `all` — render + one local-host leaf apply + installed-state preflight, then422 stop nonzero at the cluster-aware restart/rotation handoff. Run `validate` only after that423 restart completes. It uses the same explicit leaf inputs and424 `--accept-pki-rotation` gate.425426## Apply guard — `--accept-pki-rotation`427428The skill refuses to run `apply` or `all` without429`--accept-pki-rotation`. This is a single-flag acknowledgement430that:431432- The new cert chain has been verified (`verify-leaf.sh` returned433 `OK`).434- A rolling restart of the indexer cluster and SHC will follow435 (delegated to `splunk-indexer-cluster-setup --phase rolling-restart`).436- The SAML / LDAPS / Edge Processor / Splunk Cloud handoffs (where437 applicable) will be completed.438- The operator has a rollback plan (the previous PEM directory439 is preserved).440441The render and preflight phases never need this flag.442443## Rotation ownership — delegate444445The skill emits `pki/rotate/plan-rotation.md`, but does not run cluster446or SHC rolling restarts itself. Follow the canonical447[rotation runbook](references/rotation-runbook.md) for the exact staging,448bundle validation/apply, rolling-restart, SHC push, forwarder rollout,449validation, and rollback commands. That delegation preserves the450[`splunk-indexer-cluster-setup`](../splunk-indexer-cluster-setup/SKILL.md)451ownership model instead of duplicating restart orchestration here.452453## Handoffs and TLS Policy454455Read the [cross-skill ownership matrix](reference.md#cross-skill-ownership)456before delegating restarts, bundle pushes, token lifecycle, or fleet rollout.457TLS algorithms, protocol floors, FIPS lifecycle, validity caps, key formats,458and mTLS defaults are defined in [reference.md](reference.md#cross-cutting-controls)459and the topic files under [references/](references/authoritative-sources.md).460The renderer consumes the machine-readable461[algorithm policy](references/algorithm-policy.json) and fails closed on462deprecated protocols or incomplete FIPS transitions.463464## References465466Read [reference.md](reference.md) before any apply. Topical deep467dives (each anchored to a specific upstream Splunk doc captured in468[references/authoritative-sources.md](references/authoritative-sources.md)):469470- [references/component-cert-matrix.md](references/component-cert-matrix.md)471- [references/private-pki-workflow.md](references/private-pki-workflow.md)472- [references/public-pki-workflow.md](references/public-pki-workflow.md)473- [references/handoff-vault-pki.md](references/handoff-vault-pki.md)474- [references/handoff-acme-cert-manager.md](references/handoff-acme-cert-manager.md)475- [references/handoff-microsoft-adcs.md](references/handoff-microsoft-adcs.md)476- [references/handoff-ejbca.md](references/handoff-ejbca.md)477- [references/kv-store-eku-requirements.md](references/kv-store-eku-requirements.md)478- [references/mtls-and-hostname-validation.md](references/mtls-and-hostname-validation.md)479- [references/replication-port-tls.md](references/replication-port-tls.md)480- [references/saml-signing-certs.md](references/saml-signing-certs.md)481- [references/ldaps-trust.md](references/ldaps-trust.md)482- [references/edge-processor-pki.md](references/edge-processor-pki.md)483- [references/cli-trust-cacert-alignment.md](references/cli-trust-cacert-alignment.md)484- [references/tls-protocol-policy.md](references/tls-protocol-policy.md)485- [references/algorithm-presets.md](references/algorithm-presets.md)486 + [algorithm-policy.json](references/algorithm-policy.json) (machine-readable companion consumed by renderer + preflight)487- [references/fips-and-common-criteria.md](references/fips-and-common-criteria.md)488- [references/key-format-and-permissions.md](references/key-format-and-permissions.md)489- [references/rotation-runbook.md](references/rotation-runbook.md)490- [references/post-install-monitoring.md](references/post-install-monitoring.md)491- [references/splunk-cloud-ufcp-handoff.md](references/splunk-cloud-ufcp-handoff.md)492- [references/troubleshooting.md](references/troubleshooting.md)493- [references/authoritative-sources.md](references/authoritative-sources.md)