ktx
Install and configure ktx, the open-source context layer for data agents.
Use this skill when a user wants an agent to add ktx to a project, connect
data sources, build initial context, install agent integration, or troubleshoot
a local ktx setup.
Operating rules
- Act autonomously when the user asks you to install or configure ktx.
The non-interactive scripted flow below is the canonical path — bare
ktx setup is interactive (clack prompts) and an agent cannot drive it.
- Setup's non-interactive flags are intentionally hidden from
--help. Use the
flags listed below; verify uncommon flags against the docs at
https://docs.kaelio.com/ktx/ or this skill — not against --help output.
- Ask only for values you cannot infer: project directory, connection targets,
credentials, account identifiers, and source selections.
- Prefer
file:/abs/path secret refs over env:VAR_NAME. env: refs are
re-resolved against the process environment on every ktx run, so a var
exported only in the setup shell is gone when ktx ingest or ktx mcp start
runs later — the secret silently resolves to empty and the connection fails.
file: refs read from disk and survive across shells. The same caveat
applies to --*-api-key-env flags: the named var must be present in every
shell that runs ktx, including the ktx mcp daemon's environment.
- A literal database URL is safe to pass —
ktx setup auto-externalizes it
into .ktx/secrets/<id>-url and rewrites ktx.yaml to a file: ref (see
workflow step 2). Source credential refs are not auto-externalized: write
the secret to a file under .ktx/secrets/ (chmod 600) and pass a file:
ref. Never ask the user to paste a secret when a file: or env: ref works.
- Do not commit
.ktx/secrets/*.
- Print each command you run and its result.
- Setup and ingest can run for many minutes (LLM-heavy source ingests take the
longest), and from the outside a slow step looks identical to a stuck one.
Don't go silent: say what's about to run and that it may take a while, then
post brief progress/liveness updates while it runs (see step 4) so the user
never has to wonder whether it stalled — otherwise they may kill it mid-run.
- If a command fails, identify the cause and change something before retrying.
Gather inputs once
Before invoking ktx setup, collect in one round:
- Project directory (default: current working directory).
- LLM backend and key strategy. In
--no-input mode the CLI defaults to
anthropic and requires an API key. When the user is inside Claude
Code, pass --llm-backend claude-code explicitly; otherwise pass
--llm-backend anthropic --anthropic-api-key-env ANTHROPIC_API_KEY.
- Embedding backend (
sentence-transformers is the local default and needs
no key; use openai only if the user already has a key, then pass
--embedding-api-key-env OPENAI_API_KEY).
- Database: driver, connection id, URL (or
env: / file: ref), and one or
more schemas.
- Optional context sources (dbt, Metabase, Looker, LookML, MetricFlow,
Notion). Add each one with a follow-up
ktx setup --source … run (see
Add context sources); use --skip-sources only
when the user has none.
Do not discover these inputs across multiple setup runs.
Install workflow
Detect the install path. If the working directory contains
packages/cli/dist/bin.js or pnpm-workspace.yaml referencing
@kaelio/ktx you are inside the ktx monorepo — build and link the
local CLI with pnpm and do not run npm install -g. Otherwise:
node --version # require >= 22; stop and ask the user if older
ktx --version || npm install -g @kaelio/ktx
Run scripted setup (canonical path):
ktx setup --no-input --yes \
--project-dir <path> \
--llm-backend claude-code \
--embedding-backend sentence-transformers \
--database <driver> --database-connection-id <id> \
--database-url '<raw-url | file:/abs/path>' \
--database-schema <schema> \
--skip-sources \
--skip-agents
--database-schema is required for scope-bearing drivers (Postgres,
MySQL, ClickHouse, SQL Server, BigQuery, Snowflake) in --no-input:
setup fails fast without it unless the connection already has scope in
ktx.yaml. SQLite needs no scope.
- Configure one new database connection per setup invocation. For multiple
connections, rerun setup once per connection.
- Pasting a literal
--database-url is safe: the CLI relocates the URL
into .ktx/secrets/<connection-id>-url and rewrites ktx.yaml to a
file: ref automatically.
ktx setup runs agent integration as its last step. In --no-input
mode with neither --target nor --skip-agents, that step has no input,
prints Run in a TTY, or pass --target <target>., and the command exits
non-zero even though every database/LLM/embedding step succeeded. Pass
--skip-agents to defer agents to step 5 (as above), or --target <agent>
to install them inline and exit 0. Judge data-layer success from
ktx status, not from this exit code.
Resumability and --skip-*. Re-running ktx setup against an existing
project resumes its config. Use --skip-llm, --skip-databases,
--skip-sources, or --skip-embeddings to leave a slice unconfigured but
let the rest complete instead of aborting on the first failure. When
resuming an existing project to change one slice (e.g. only LLM), still
pass the database flags from the previous run — setup validates current
flags, not persisted ktx.yaml state.
Build context if setup did not already complete one:
ktx ingest <connection-id> --no-input
ktx ingest always builds enriched context and requires a configured model
and embeddings (set during setup); a database connection without them fails
with an enrichment-readiness error. Note: ktx ingest rejects --yes
together with --no-input (Choose only one runtime install mode);
ktx setup accepts both. Use --no-input only for ingest.
Ingest one connection at a time. It can run for many minutes with no
stdout until it exits (LLM-heavy sources like Metabase are the slowest), so
don't assume it hung, and don't pipe it through tail/head — that buffers
all output to the end, so run it raw. Tell the user up front that the step is
slow, then keep them posted instead of blocking silently: run the ingest in
the background and poll for liveness every minute or so, reporting a one-line
update each time (which connection, roughly how long it's been running, and
that .ktx files are still changing) so a long run never looks stuck:
find <path>/.ktx/worktrees <path>/.ktx/ingest-transcripts -type f -mmin -3
On success, the Ingest finished summary table shows done in the
Source ingest and Memory update columns with no Failed sources:
section.
Install agent integration:
ktx setup --agents --target <claude-code|claude-desktop|codex|cursor|opencode|universal>
ktx mcp start --project-dir <path>
Agent integration is not usable until ktx mcp start is running. The
--agents step prints this requirement as Required before using agents.
Fall back to bare ktx setup only when a human is at the keyboard —
it uses interactive prompts an agent cannot answer.
Add context sources
Context sources (dbt, Metabase, Looker, LookML, MetricFlow, Notion) are added
one at a time — --source is not repeatable, so run ktx setup once per
source. Source setup is resumable against an existing project: pass
--skip-databases --skip-llm --skip-embeddings --skip-agents so only the source
is configured (the trailing agent step otherwise fails the run — see install
step 2). Map Metabase, Looker, and LookML to an existing database connection
with --source-warehouse-connection-id <db-connection-id> (required for those).
dbt ignores --source-warehouse-connection-id — it maps to the warehouse by
table name — so omit it for dbt. Use file:/abs/path refs for keys and tokens
(see the secrets rule above); env: refs must be exported in every later ktx
shell.
# dbt — pick exactly one of --source-path (local) or --source-git-url (remote).
# No --source-warehouse-connection-id: dbt maps to the warehouse by table name.
ktx setup --no-input --yes --skip-databases --skip-llm --skip-embeddings --skip-agents \
--source dbt --source-connection-id <id> \
--source-git-url <url> --source-branch <branch>
# Metabase
ktx setup --no-input --yes --skip-databases --skip-llm --skip-embeddings --skip-agents \
--source metabase --source-connection-id <id> \
--source-url <url> --source-api-key-ref file:/abs/path/metabase-api-key \
--source-warehouse-connection-id <db-connection-id> \
--metabase-database-id <metabase-db-id>
# Notion
ktx setup --no-input --yes --skip-databases --skip-llm --skip-embeddings --skip-agents \
--source notion --source-connection-id <id> \
--source-auth-token-ref file:/abs/path/notion-token \
--notion-crawl-mode selected_roots --notion-root-page-id <page-id>
Notes:
--metabase-database-id is the numeric id of the warehouse inside
Metabase (not the ktx connection id). Discover it from the Metabase API
(GET /api/database) or UI if the user doesn't know it.
--notion-crawl-mode selected_roots requires at least one
--notion-root-page-id (repeatable); use all_accessible to crawl
everything the token can see.
- After adding sources, ingest each new connection so its context is queryable:
ktx ingest <source-connection-id> --no-input.
Files to inspect
ktx.yaml: project configuration.
.ktx/secrets/*: local secret files. Never commit them.
semantic-layer/<connection-id>/*.yaml: semantic sources for SQL
compilation.
wiki/**/*.md: project context pages for agents.
.claude/skills/ktx/, .agents/skills/ktx/, .cursor/rules/ktx.mdc, and
.opencode/commands/ktx.md: generated agent integration files.
Verification
After setup, run:
ktx connection test <connection-id>
ktx status --json --no-input
ktx sl --output plain # lists compiled semantic sources; `ktx sl` has no --no-input
Judge readiness from ktx status --json fields, not the exit code.
ktx status exits 1 whenever the LLM is none (verdict: "blocked"), even
when embeddings and every database connection are healthy. Treat success as:
verdict: "ready" at the top of the JSON, and
- every
connections[].status === "ok" (other levels: warn, fail,
skipped), and
- every
ktx connection test <id> exited 0, and
- for each ingested source,
localStats.semanticLayer[].sourceCount > 0 and
localStats.wikiPages[].count > 0 — these confirm the source actually
produced context. Do not rely on localStats.ingest.perConnection to
confirm source ingests: it reflects only completed warehouse ingest reports
and under-reports (often lists just the warehouse connection).
If the LLM is intentionally left unconfigured, verdict is blocked and the
exit is non-zero by design — that is still a usable context layer, so report it
as "ready, LLM optional" and judge the data layer by the connection and
localStats fields above rather than retrying setup.
Troubleshooting
For known failure signatures (invalid ELF header,
Native CLI binary for <plat> not found, Missing Anthropic API key,
claude-code probe failure, ktx cannot work without a database on resume,
Run in a TTY, or pass --target <target>. with a misleading exit 1, and a
secret that resolves empty only during ktx ingest/ktx mcp), see
troubleshooting.md.
Final report
End setup work with a concise report:
ktx SETUP COMPLETE
Project: <path>
LLM: <backend> / <model>
Embeddings: <backend> / <model>
Connections: <name> (<driver>) status=<ok|warn|fail>
Sources: <list or none>
Verdict: <ready|needs action>
Next:
1. <copy-pasteable command or action>
2. <copy-pasteable command or action>
RESULT: PASS
1---
2name: ktx
3description: Installs and configures ktx, the open-source context layer for data agents — runs ktx setup non-interactively with hidden CLI flags, configures database connections and embeddings, installs agent integration, and verifies readiness. Use when the user asks an agent to add ktx to a project, connect data sources, install agent rules, ingest schema, or troubleshoot a local ktx install.
4metadata:
5 source: https://skillmd.com/skills/kaelio/ktx
6---
7
8# ktx
9
10Install and configure **ktx**, the open-source context layer for data agents.
11Use this skill when a user wants an agent to add **ktx** to a project, connect
12data sources, build initial context, install agent integration, or troubleshoot
13a local **ktx** setup.
14
15## Operating rules
16
17- Act autonomously when the user asks you to install or configure **ktx**.
18 The non-interactive scripted flow below is the canonical path — bare
19 `ktx setup` is interactive (clack prompts) and an agent cannot drive it.
20- Setup's non-interactive flags are intentionally hidden from `--help`. Use the
21 flags listed below; verify uncommon flags against the docs at
22 `https://docs.kaelio.com/ktx/` or this skill — not against `--help` output.
23- Ask only for values you cannot infer: project directory, connection targets,
24 credentials, account identifiers, and source selections.
25- Prefer `file:/abs/path` secret refs over `env:VAR_NAME`. `env:` refs are
26 re-resolved against the process environment on **every** `ktx` run, so a var
27 exported only in the setup shell is gone when `ktx ingest` or `ktx mcp start`
28 runs later — the secret silently resolves to empty and the connection fails.
29 `file:` refs read from disk and survive across shells. The same caveat
30 applies to `--*-api-key-env` flags: the named var must be present in every
31 shell that runs `ktx`, including the `ktx mcp` daemon's environment.
32- A literal database URL is safe to pass — `ktx setup` auto-externalizes it
33 into `.ktx/secrets/<id>-url` and rewrites `ktx.yaml` to a `file:` ref (see
34 workflow step 2). Source credential refs are **not** auto-externalized: write
35 the secret to a file under `.ktx/secrets/` (`chmod 600`) and pass a `file:`
36 ref. Never ask the user to paste a secret when a `file:` or `env:` ref works.
37- Do not commit `.ktx/secrets/*`.
38- Print each command you run and its result.
39- Setup and ingest can run for many minutes (LLM-heavy source ingests take the
40 longest), and from the outside a slow step looks identical to a stuck one.
41 Don't go silent: say what's about to run and that it may take a while, then
42 post brief progress/liveness updates while it runs (see step 4) so the user
43 never has to wonder whether it stalled — otherwise they may kill it mid-run.
44- If a command fails, identify the cause and change something before retrying.
45
46## Gather inputs once
47
48Before invoking `ktx setup`, collect in one round:
49
501. Project directory (default: current working directory).
512. LLM backend and key strategy. In `--no-input` mode the CLI defaults to
52 `anthropic` and **requires an API key**. When the user is inside Claude
53 Code, pass `--llm-backend claude-code` explicitly; otherwise pass
54 `--llm-backend anthropic --anthropic-api-key-env ANTHROPIC_API_KEY`.
553. Embedding backend (`sentence-transformers` is the local default and needs
56 no key; use `openai` only if the user already has a key, then pass
57 `--embedding-api-key-env OPENAI_API_KEY`).
584. Database: driver, connection id, URL (or `env:` / `file:` ref), and one or
59 more schemas.
605. Optional context sources (dbt, Metabase, Looker, LookML, MetricFlow,
61 Notion). Add each one with a follow-up `ktx setup --source …` run (see
62 [Add context sources](#add-context-sources)); use `--skip-sources` only
63 when the user has none.
64
65Do not discover these inputs across multiple setup runs.
66
67## Install workflow
68
691. **Detect the install path.** If the working directory contains
70 `packages/cli/dist/bin.js` or `pnpm-workspace.yaml` referencing
71 `@kaelio/ktx` you are inside the **ktx** monorepo — build and link the
72 local CLI with `pnpm` and do **not** run `npm install -g`. Otherwise:
73
74 ```bash
75 node --version # require >= 22; stop and ask the user if older
76 ktx --version || npm install -g @kaelio/ktx
77 ```
78
792. **Run scripted setup** (canonical path):
80
81 ```bash
82 ktx setup --no-input --yes \
83 --project-dir <path> \
84 --llm-backend claude-code \
85 --embedding-backend sentence-transformers \
86 --database <driver> --database-connection-id <id> \
87 --database-url '<raw-url | file:/abs/path>' \
88 --database-schema <schema> \
89 --skip-sources \
90 --skip-agents
91 ```
92
93 - `--database-schema` is required for scope-bearing drivers (Postgres,
94 MySQL, ClickHouse, SQL Server, BigQuery, Snowflake) in `--no-input`:
95 setup fails fast without it unless the connection already has scope in
96 `ktx.yaml`. SQLite needs no scope.
97 - Configure one new database connection per setup invocation. For multiple
98 connections, rerun setup once per connection.
99 - Pasting a literal `--database-url` is safe: the CLI relocates the URL
100 into `.ktx/secrets/<connection-id>-url` and rewrites `ktx.yaml` to a
101 `file:` ref automatically.
102 - `ktx setup` runs agent integration as its **last** step. In `--no-input`
103 mode with neither `--target` nor `--skip-agents`, that step has no input,
104 prints `Run in a TTY, or pass --target <target>.`, and the command exits
105 non-zero **even though every database/LLM/embedding step succeeded**. Pass
106 `--skip-agents` to defer agents to step 5 (as above), or `--target <agent>`
107 to install them inline and exit 0. Judge data-layer success from
108 `ktx status`, not from this exit code.
109
1103. **Resumability and `--skip-*`.** Re-running `ktx setup` against an existing
111 project resumes its config. Use `--skip-llm`, `--skip-databases`,
112 `--skip-sources`, or `--skip-embeddings` to leave a slice unconfigured but
113 let the rest complete instead of aborting on the first failure. **When
114 resuming an existing project to change one slice (e.g. only LLM), still
115 pass the database flags from the previous run** — setup validates current
116 flags, not persisted `ktx.yaml` state.
117
1184. **Build context** if setup did not already complete one:
119
120 ```bash
121 ktx ingest <connection-id> --no-input
122 ```
123
124 `ktx ingest` always builds enriched context and requires a configured model
125 and embeddings (set during setup); a database connection without them fails
126 with an enrichment-readiness error. Note: `ktx ingest` rejects `--yes`
127 together with `--no-input` (*Choose only one runtime install mode*);
128 `ktx setup` accepts both. Use `--no-input` only for ingest.
129
130 Ingest one connection at a time. It can run for many minutes with **no
131 stdout** until it exits (LLM-heavy sources like Metabase are the slowest), so
132 don't assume it hung, and don't pipe it through `tail`/`head` — that buffers
133 all output to the end, so run it raw. Tell the user up front that the step is
134 slow, then keep them posted instead of blocking silently: run the ingest in
135 the background and poll for liveness every minute or so, reporting a one-line
136 update each time (which connection, roughly how long it's been running, and
137 that `.ktx` files are still changing) so a long run never looks stuck:
138
139 ```bash
140 find <path>/.ktx/worktrees <path>/.ktx/ingest-transcripts -type f -mmin -3
141 ```
142
143 On success, the `Ingest finished` summary table shows `done` in the
144 `Source ingest` and `Memory update` columns with no `Failed sources:`
145 section.
146
1475. **Install agent integration:**
148
149 ```bash
150 ktx setup --agents --target <claude-code|claude-desktop|codex|cursor|opencode|universal>
151 ktx mcp start --project-dir <path>
152 ```
153
154 Agent integration is **not usable until `ktx mcp start` is running**. The
155 `--agents` step prints this requirement as `Required before using agents`.
156
1576. **Fall back to bare `ktx setup` only when a human is at the keyboard** —
158 it uses interactive prompts an agent cannot answer.
159
160## Add context sources
161
162Context sources (dbt, Metabase, Looker, LookML, MetricFlow, Notion) are added
163**one at a time** — `--source` is not repeatable, so run `ktx setup` once per
164source. Source setup is resumable against an existing project: pass
165`--skip-databases --skip-llm --skip-embeddings --skip-agents` so only the source
166is configured (the trailing agent step otherwise fails the run — see install
167step 2). Map Metabase, Looker, and LookML to an existing database connection
168with `--source-warehouse-connection-id <db-connection-id>` (required for those).
169**dbt ignores `--source-warehouse-connection-id`** — it maps to the warehouse by
170table name — so omit it for dbt. Use `file:/abs/path` refs for keys and tokens
171(see the secrets rule above); `env:` refs must be exported in every later `ktx`
172shell.
173
174```bash
175# dbt — pick exactly one of --source-path (local) or --source-git-url (remote).
176# No --source-warehouse-connection-id: dbt maps to the warehouse by table name.
177ktx setup --no-input --yes --skip-databases --skip-llm --skip-embeddings --skip-agents \
178 --source dbt --source-connection-id <id> \
179 --source-git-url <url> --source-branch <branch>
180
181# Metabase
182ktx setup --no-input --yes --skip-databases --skip-llm --skip-embeddings --skip-agents \
183 --source metabase --source-connection-id <id> \
184 --source-url <url> --source-api-key-ref file:/abs/path/metabase-api-key \
185 --source-warehouse-connection-id <db-connection-id> \
186 --metabase-database-id <metabase-db-id>
187
188# Notion
189ktx setup --no-input --yes --skip-databases --skip-llm --skip-embeddings --skip-agents \
190 --source notion --source-connection-id <id> \
191 --source-auth-token-ref file:/abs/path/notion-token \
192 --notion-crawl-mode selected_roots --notion-root-page-id <page-id>
193```
194
195Notes:
196
197- `--metabase-database-id` is the **numeric id of the warehouse inside
198 Metabase** (not the ktx connection id). Discover it from the Metabase API
199 (`GET /api/database`) or UI if the user doesn't know it.
200- `--notion-crawl-mode selected_roots` requires at least one
201 `--notion-root-page-id` (repeatable); use `all_accessible` to crawl
202 everything the token can see.
203- After adding sources, ingest each new connection so its context is queryable:
204 `ktx ingest <source-connection-id> --no-input`.
205
206## Files to inspect
207
208- `ktx.yaml`: project configuration.
209- `.ktx/secrets/*`: local secret files. Never commit them.
210- `semantic-layer/<connection-id>/*.yaml`: semantic sources for SQL
211 compilation.
212- `wiki/**/*.md`: project context pages for agents.
213- `.claude/skills/ktx/`, `.agents/skills/ktx/`, `.cursor/rules/ktx.mdc`, and
214 `.opencode/commands/ktx.md`: generated agent integration files.
215
216## Verification
217
218After setup, run:
219
220```bash
221ktx connection test <connection-id>
222ktx status --json --no-input
223ktx sl --output plain # lists compiled semantic sources; `ktx sl` has no --no-input
224```
225
226**Judge readiness from `ktx status --json` fields, not the exit code.**
227`ktx status` exits 1 whenever the LLM is `none` (`verdict: "blocked"`), even
228when embeddings and every database connection are healthy. Treat success as:
229
230- `verdict: "ready"` at the top of the JSON, and
231- every `connections[].status === "ok"` (other levels: `warn`, `fail`,
232 `skipped`), and
233- every `ktx connection test <id>` exited 0, and
234- for each ingested source, `localStats.semanticLayer[].sourceCount > 0` and
235 `localStats.wikiPages[].count > 0` — these confirm the source actually
236 produced context. Do **not** rely on `localStats.ingest.perConnection` to
237 confirm source ingests: it reflects only completed warehouse ingest reports
238 and under-reports (often lists just the warehouse connection).
239
240If the LLM is intentionally left unconfigured, `verdict` is `blocked` and the
241exit is non-zero by design — that is still a usable context layer, so report it
242as "ready, LLM optional" and judge the data layer by the connection and
243`localStats` fields above rather than retrying setup.
244
245## Troubleshooting
246
247For known failure signatures (`invalid ELF header`,
248`Native CLI binary for <plat> not found`, `Missing Anthropic API key`,
249`claude-code` probe failure, `ktx cannot work without a database` on resume,
250`Run in a TTY, or pass --target <target>.` with a misleading exit 1, and a
251secret that resolves empty only during `ktx ingest`/`ktx mcp`), see
252[troubleshooting.md](troubleshooting.md).
253
254## Final report
255
256End setup work with a concise report:
257
258```text
259ktx SETUP COMPLETE
260
261Project: <path>
262LLM: <backend> / <model>
263Embeddings: <backend> / <model>
264Connections: <name> (<driver>) status=<ok|warn|fail>
265Sources: <list or none>
266Verdict: <ready|needs action>
267
268Next:
2691. <copy-pasteable command or action>
2702. <copy-pasteable command or action>
271
272RESULT: PASS
273```