# Aiq Add Data Source

> Use when adding or changing an AI-Q data source under sources/, registering it as a NeMo Agent Toolkit function, wiring it into the data_source_registry for UI toggles, or validating retrieval behavior with tests.

- Skill: `nvidia-ai-blueprints-aiq/aiq-add-data-source` (Agent Skill, multi-file: 4 files)
- Install (CLI): `npx skillmds@latest add nvidia-ai-blueprints-aiq/aiq-add-data-source`
- Raw SKILL.md: https://api.skillmd.com/api/skills/nvidia-ai-blueprints-aiq/aiq-add-data-source/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- License: Apache-2.0
- Author: NVIDIA-AI-Blueprints (https://skillmd.com/u/nvidia-ai-blueprints-aiq)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/nvidia-ai-blueprints-aiq/aiq-add-data-source

---


# Add an AI-Q Data Source

Use this skill when a developer wants to add a retrieval or search source to
AI-Q and expose it as a toggleable source in the UI. A data source is a NeMo
Agent Toolkit (NAT) function package under `sources/`, registered in the
`data_source_registry`.

## Start Here

- Confirm this is a new retrieval/search source (not a UI, auth, or prompt
  change). For a general utility function, use `aiq-add-tool` instead.
- Read the authoritative files below before editing.
- Copy the closest existing source package rather than inventing a new shape.
- Never print or commit API keys; resolve secrets at runtime via `SecretStr`.

## Authoritative References

- `docs/source/extending/adding-a-data-source.md`: canonical package and
  registration walkthrough (the steps below mirror it).
- `sources/google_scholar_paper_search/`: complete example package with a
  client, a config + registration, a graceful missing-secret stub, and tests.
- `sources/tavily_web_search/`: minimal source package for comparison.
- `src/aiq_agent/common/data_source_registry.py`: the `data_source_registry`
  config (`name="data_source_registry"`) that drives `GET /v1/data_sources`.
- `docs/source/customization/tools-and-sources.md`: how the registry maps to UI
  toggles and per-request filtering.
- `frontends/ui/src/features/layout/data-sources.ts`: the UI `DataSource` type;
  sources are fetched dynamically, so usually no UI code change is needed.

Longer procedures live in this bundle:

- [references/package-layout.md](references/package-layout.md): package files,
  `pyproject.toml`, config class, `@register_function`, and the missing-secret stub.
- [references/registry-and-ui.md](references/registry-and-ui.md): registering in
  YAML and how the registry surfaces toggles and filtering.
- [references/validation.md](references/validation.md): install, test, and lint
  commands with expected results.

## Workflow

1. Pick the closest existing package under `sources/` and inspect its layout.
2. Create `sources/<my_data_source>/` with `src/register.py`, the client
   module, `pyproject.toml`, and `tests/` (see package-layout reference).
3. Define a `FunctionBaseConfig` subclass with a stable `name=` and resolve any
   API key via `SecretStr`; register it with `@register_function`.
4. Yield a graceful stub when the required secret is missing.
5. Install the package editable and add it to the `data_source_registry` in the
   relevant config under `configs/`.
6. Add focused tests; run the validation commands below.
7. Summarize changed files and paste the test/lint evidence.

## Validation

Run the narrowest commands first; broaden only if the change touches shared code.

```bash
uv pip install -e ./sources/my_data_source
uv run pytest sources/my_data_source/tests
uv run ruff check sources/my_data_source
uv run ruff format --check sources/my_data_source
```

Expected: the package installs, its tests pass, and Ruff reports no lint or
format failures for the new source package.

## Common Mistakes

- Forgetting to add the source to the `data_source_registry`, so the UI cannot
  toggle it and agents do not inherit the tool.
- Omitting the `[project.entry-points."nat.plugins"]` entry in `pyproject.toml`,
  so NAT never discovers the registration.
- Crashing on a missing API key instead of yielding a stub that returns a clear
  error message.
- Returning unstructured or citation-poor output, which weakens report grounding.
- Printing API keys or embedding secrets in YAML instead of using environment
  variables or `SecretStr`.

## Related Skills

- `aiq-configure-workflow`
- `aiq-add-tool`
- `aiq-release-qa`
- `aiq-prepare-pr`