# Topica Analysis

> Working guide for an LLM agent helping a social scientist run a topic-modeling analysis with the topica Python library — building a defensible corpus, choosing and justifying K, fitting a model, validating it, estimating covariate effects with honest uncertainty, and reporting results. Use when the user wants to USE topica to analyze a text corpus (not extend the library). The companion developer skill for adding a model is add-topic-model.

- Skill: `nealcaren/topica-analysis` (Agent Skill)
- Install (CLI): `npx skillmds@latest add nealcaren/topica-analysis`
- Raw SKILL.md: https://api.skillmd.com/api/skills/nealcaren/topica-analysis/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: nealcaren (https://skillmd.com/u/nealcaren)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/nealcaren/topica-analysis

---


# Using topica well

This is the working guide for an LLM agent (Claude Code, Cursor, or similar)
helping a social scientist run topic models with `topica`. It is a working guide,
not an API reference; the API lives at <https://nealcaren.github.io/topica/>. Read
it before you write code that fits a model or reports a result. If instead you are
*extending* topica (adding a model, a diagnostic, or a binding), this is the wrong
guide — use the `add-topic-model` skill and `CONTRIBUTING-MODELS.md`.

The one idea to carry through everything below: **topica and the agent supply the
mechanics and the honest diagnostics; the researcher owns the theory.** A topic
model is a measurement instrument, not a discovery oracle. The decisions that
make a topic-model paper credible are theoretical, and they are the researcher's:
what the corpus represents, how many topics is the right granularity for the
question, what each topic means, which covariates belong in the model, and
whether a result is substantively interesting. Your job is to make those
decisions cheap to act on and hard to get wrong, to surface uncertainty the
researcher might otherwise miss, and to refuse to manufacture certainty that the
data does not support. When one of those theory-owned decisions comes up, name it
and hand it back. Do not quietly pick `K` because a metric peaked there.

## What topica is, in one paragraph

`topica` is a Python topic-modeling library with a Rust core. It brings the
models that used to be split across R (`stm`, `keyATM`) and Java (MALLET) into
one place, runs them fast, and makes every fit reproducible from a fixed seed.
The house import is `import topica`, no alias. The core needs only NumPy. More
than a dozen models share one shape: `model.fit(docs, ...)`, then `topic_word`
(the φ matrix, topics × vocabulary), `doc_topic` (the θ matrix, documents ×
topics), `top_words(n)`, and `save` / `load`. Diagnostics are model-agnostic:
they take any fitted model's `topic_word` / `doc_topic`.

Because the count-based variational models (`CTM`, `STM`, `DTM`,
`SupervisedLDA`) are deterministic to the bit and the Gibbs models are
seed-reproducible, "run it again and see if the topics hold" is a real test here,
not a hope. Use that.

**First move at the REPL: `topica.guide()`.** It prints a one-screen cheat sheet
built live from the installed build — the canonical workflow, a goal-to-model
chooser, the read surface above, and the helper namespaces — so you do not have to
guess signatures or grep the type stub. `topica.guide("STM")` prints one model's
constructor and `fit` signatures; `topica.guide(full=True)` prints the whole
roster. `guide()` ships in the wheel, so it works from a bare `pip install topica`
with no repository checkout. For task-shaped starting points you can transplant
onto the researcher's data (explore with LDA and justify K, seed named concepts
with keyATM, short text with GSDMM, embedding clusters with BERTopic, compare
prevalence or wording across groups, track a topic over time, bootstrap topic
robustness, record provenance), copy from `examples/recipes/` in the
[repository](https://github.com/nealcaren/topica/tree/main/examples/recipes),
which has one recipe per common task.

## How topica changes the standard advice

If you have internalized the usual computational-text-analysis guidance ("STM is
R-only", "use BERTopic for the neural approach in Python", "covariates mean you
have to leave Python"), update it. With topica:

- **STM, CTM, SAGE, STS, DMR, and keyATM all run in Python**, validated against
  their reference implementations. A covariate-aware topic model, including
  covariate-driven *sentiment* (`STS`, the sentiment-discourse extension of STM),
  no longer forces a switch to R.
- **Classical and neural live side by side.** `BERTopic`, `Top2Vec`, `ETM`, and
  `FASTopic` take embedding vectors you supply (no PyTorch in the wheel) and sit
  next to LDA and STM behind the same diagnostics. You can compare them on one
  corpus without leaving the session.
- **Effect estimates carry honest uncertainty.** `estimate_effect` and
  `standard_errors` give you method-of-composition and bootstrap intervals, and
  they refuse to fabricate intervals for models that have no posterior to
  compose. Trust the refusal.

The choice of model still follows the research question, not the convenience of
the language.

## The workflow

These phases mirror how a careful text-analysis paper is actually built. At each
**handoff** the researcher makes a call you should not make for them. Stop, show
what you have, and ask.

### Phase 0: research question and method

Before any code, get the question into one of three shapes, because they need
different tools:

- **Descriptive / exploratory** ("what themes are in this corpus?") → a topic
  model: `LDA` to start, `STM` if covariates matter, `HDP` if you genuinely do
  not want to fix `K`.
- **Measurement** ("how much does theme X vary by group/time?") → `STM` or
  `DMR`, with the grouping or time variable as a prevalence covariate, plus
  `estimate_effect`. If the question is about the *tone* of a theme rather than
  its prevalence (how positively or negatively a theme is discussed, and how that
  varies by covariate), `STS` adds a sentiment-discourse latent on top of STM.
- **Confirmatory** ("do documents express concept X?") → often *not* a topic
  model. A dictionary, `KeyATM` / `SeededLDA` (guided by seed words), or a
  classifier fits a known target better than unsupervised topics do.

Topic models are a poor fit for very short, formulaic, or highly technical text,
and for questions that need crisp category boundaries. Say so when you see one.
For short text (tweets, survey answers, headlines) reach for `GSDMM` or `PT`, not
LDA.

> **Handoff.** The question framing and the model choice are the researcher's.
> Lay out the options with their trade-offs; let them choose.

### Phase 1: build a defensible corpus

The corpus defines the scope of every claim that follows, so preprocessing is a
substantive decision, not a chore. Use `topica`'s preprocessing tools and
**write down every choice** (stopwords, frequency thresholds, phrases), because
the methods section has to report them.

```python
import topica

corpus = topica.Corpus.from_documents(docs)   # docs: list of token lists
# or build from raw strings with tokenize(), learn_phrases()/apply_phrases()
```

Defaults worth stating explicitly to the researcher: lowercase, drop stopwords,
drop terms in fewer than ~5-10 documents and terms in a very large share of
documents. Prefer lemmatization to stemming (stemming hurts interpretability).
Start minimal and add preprocessing only when topics show artifacts; do not
pile on cleaning preemptively. Inspect the corpus (document count, length
distribution, vocabulary size) and report it before modeling.

**This preprocessing advice is for bag-of-words models (LDA, STM, NMF, …).**
For embedding + cluster models (`BERTopic`, `Top2Vec`), the topics are driven by
the document *embeddings*, so token cleaning does **not** change the model — the
same clusters emerge with or without stopword removal. There it only affects the
c-TF-IDF *labels* on already-formed clusters. So preprocess to get readable
labels, but do not present stopword/frequency choices as shaping the topics, and
note that the K-selection tools below (`search_k`) are LDA/STM-only. See the
[embedding-models guide](https://nealcaren.github.io/topica/guides/embedding/)
for the embedding-model workflow, including how to sweep K and avoid the `-1`
noise bucket.

> **Handoff.** Stopword lists and frequency cutoffs change what topics can
> exist. Surface them; do not bury them in a default.

### Phase 2: choose and justify K

This is where agents most often overstep. **K is a research decision about
granularity, not a hyperparameter to maximize.** K = 10 gives broad themes,
K = 50 gives fine distinctions, and several values of K are usually defensible
for the same corpus.

Do not pick the K that maximizes coherence. A model with fewer topics will often
score higher on coherence while collapsing distinctions the researcher cares
about. Use the diagnostics as evidence, not as an optimizer:

```python
rows = topica.search_k(docs, [10, 15, 20, 25, 30], iters=1000)
# coherence, exclusivity, and held-out perplexity per K
```

The honest procedure: start from a theoretically plausible K, fit a small range
around it, and for each K look at coherence **and** exclusivity **and** whether
every topic can be labeled. Count the junk topics. Then let the researcher
choose, and report sensitivity to that choice. `topica.viz.search_k(rows)` and
`quality_frontier` make the trade-off legible.

`rows.best_k()` exists, but treat it as one input, not the decision: with a
`held_out=` set it defaults to the held-out metric, which is roughly monotone in
K and so tends to return the **largest K you scanned** (it warns when it does).
Without held-out it defaults to the coherence/exclusivity frontier (a knee). Read
it as "the K this criterion prefers," report the criterion, and still hand the
choice to the researcher — never present `best_k()` as the answer.

`search_k` covers LDA and STM only (`model="lda"`/`"stm"`); it raises for
embedding + cluster models. For `BERTopic` / `Top2Vec` there is no held-out
likelihood — sweep the K knob yourself (`num_clusters` for a fixed-K clusterer,
or `min_cluster_size` for HDBSCAN) and score each fit by coherence and topic
diversity, picking the same way (a knee, not a maximum). The
[embedding-models guide](https://nealcaren.github.io/topica/guides/embedding/)
works this through.

> **Handoff.** Present the curves and the labeled topics at two or three values
> of K. The researcher picks K; you report why.

### Phase 3: fit

Always pass a seed. Always report it. For STM, covariates enter as a design
matrix (build it with `one_hot`, and `stm.spline` / `stm.interaction` for
non-linear and interaction terms), not a formula string:

```python
X, names = topica.one_hot(party)
model = topica.STM(num_topics=20, seed=13)
model.fit(docs, prevalence=X, prevalence_names=names)   # content=... for SAGE
```

A plain LDA fit is `topica.LDA(num_topics=20, seed=13).fit(docs, iters=1000)`.
Check that the fit converged (the EM models expose a bound / `converged` flag;
the Gibbs models expose log-likelihood history). A model that did not converge is
not a result. Read topics off several surfaces together — top words alone
underdetermine what a topic is:

- `model.top_words(n)` returns per-topic bare word strings, so
  `" ".join(model.top_words(n, topic=t))` works directly; pass `weights=True`
  for `(word, weight)` pairs when you need the probabilities.
- `label_topics`, `frex`, `relevance`, `find_thoughts`, `topic_table` and
  `summary` are **module-level functions, not model methods**, and take the
  model's matrices as the first argument — e.g.
  `topica.label_topics(model.topic_word, model.vocabulary)` and
  `topica.find_thoughts(model.doc_topic, texts, topic=t)` (the highest-θ
  documents for a topic).
- `label_topics` has no `method=` selector: it reports probability and FREX
  side by side. For FREX- or relevance-ranked words specifically, call the
  separate `topica.frex(model.topic_word, model.vocabulary)` or
  `topica.relevance(model.topic_word, model.vocabulary)`.

### Phase 4: validate (not optional)

Algorithmic output is not ground truth. Before any topic becomes a finding:

- **Stability.** Refit under different seeds and check the topics persist:
  `bootstrap_stability`. topica's determinism makes this a clean test.
- **Consensus across runs.** `select_model` fits several seeds and keeps the best
  by a quality score; `ensemble` goes further and combines independent runs into
  a consensus set of topics more reliable than any single fit. Reach for these
  when a topic has to be stable to carry a claim.
- **Coherence and exclusivity.** `coherence` (`u_mass`, `c_v`, `c_uci`,
  `c_npmi`), `exclusivity`, `topic_diversity`.
- **Human validation.** `word_intrusion` and `document_intrusion` build the
  intrusion tests that show a topic is coherent to a human, not just to the
  metric. Offer to generate them; the researcher (or their coders) runs them.
- **Robustness.** Different K, different preprocessing, a subset by time or
  source. Compare to an alternative model if the claim is strong.

A one-call summary is available:

```python
topica.diagnostics(model, texts)   # coherence, exclusivity, diversity table
```

> **Handoff.** Whether a topic is "real enough" to interpret is a judgment.
> Give the diagnostics; let the researcher draw the line, and report where they
> drew it.

### Phase 5: measure effects with honest uncertainty

If the question is about variation (over time, across groups), this is the
results figure. Use the method of composition, which propagates topic-estimation
uncertainty into the effect, and report intervals:

```python
from topica import stm
draws   = stm.posterior_theta_samples(model, nsims=50, seed=0)
effects = stm.estimate_effect(draws, X, feature_names=names, cluster=source_id)
# or, model-agnostic with the uncertainty kind made explicit:
se = topica.standard_errors(model, corpus, of="effect", method="composition",
                            X=X, feature_names=names)
```

`topica.viz.effect_plot(model, corpus, X=...)` draws the per-topic forest plot
and, importantly, **refuses to draw confidence bands for models that have no
posterior** (cluster models like BERTopic), drawing point estimates and saying so
instead. When you see that refusal, do not paper over it with a bootstrap unless
the researcher asks; the absence of a band is information.

### Phase 6: report

Topica produces publication-ready tables and figures (`topic_table`, `summary`,
the `topica.viz` panels and `dashboard`). The methods section must state: model,
K and the rationale, preprocessing, covariates, software version, seed, and the
validation done. The results section gives labeled topics with top words,
prevalence, covariate effects with intervals, and representative documents.

The single most important reporting discipline is language. Topics are
statistical patterns of word co-occurrence. They are **not** necessarily coherent
concepts, and they are not what a document is "about." Write:

- "Document *d* has high probability on Topic 3," not "Document *d* is about
  Topic 3."
- "Words associated with Topic 3 suggest ...," not "The model discovered that ..."
- "One interpretation of this pattern is ...," not a claim that the topic is a
  fact.

Findings apply to the analyzed corpus, not to "language" or "discourse" in
general.

## What the agent owns vs. what the researcher owns

| The agent / topica owns | The researcher owns |
|---|---|
| Running models, picking sensible defaults | The research question |
| Computing coherence, exclusivity, stability | What counts as good enough |
| Surfacing the K trade-off curves | **The choice of K** |
| Listing top words, FREX, example documents | **What each topic means** (the labels) |
| Proposing covariate specifications | **Which covariates belong in the model** |
| Reporting effect sizes with honest intervals | **Whether a result matters substantively** |
| Refusing to fabricate uncertainty | Theoretical interpretation and the argument |

When you find yourself about to make a call from the right column, stop and hand
it back. The failure mode to avoid is an agent that runs the whole pipeline,
picks K off a metric, auto-labels the topics, and presents a finished result that
looks authoritative and is theoretically ungrounded. A topica session done well
leaves the researcher having made every decision that a reviewer will question,
with the diagnostics to defend each one.

## Quick reference

- **Build:** `Corpus.from_documents`, `tokenize`, `learn_phrases` / `apply_phrases`
- **Models:** the names below are *illustrative, not the full roster* — topica ships
  ~30 models and the list grows, so do not treat this as complete. The canonical,
  always-current list is the generated roster in the docs
  (<https://nealcaren.github.io/topica/guides/models/>, from `docs/guides/models.md`)
  and the README table; consult it rather than relying on any hardcoded list here.
  Common starting points: `LDA`, `STM`, `CTM`, `STS` (sentiment), `DMR`, `HDP`,
  `KeyATM`, `SeededLDA`, `BERTopic`, `GSDMM` (short text)
- **Read topics:** `model.top_words(n)` (method → bare words; `weights=True` for
  `(word, weight)` pairs) plus
  the module functions `topica.label_topics`, `frex`, `relevance`,
  `find_thoughts`, `topic_table`, `summary` (first arg is the model's
  `topic_word` / `doc_topic` matrix, or the model)
- **Choose K:** `search_k`, `quality_frontier`, `topica.viz.search_k`
- **Validate:** `coherence`, `exclusivity`, `topic_diversity`,
  `bootstrap_stability`, `word_intrusion`, `document_intrusion`, `diagnostics`
- **Reliability across runs:** `select_model`, `ensemble`
- **Effects:** `estimate_effect`, `standard_errors`, `posterior_theta_samples`,
  `topica.viz.effect_plot`
- **Compare corpora:** `fighting_words`
- **Report:** `topica.viz.dashboard`, `summary`, `topic_table`

Everything is documented at <https://nealcaren.github.io/topica/>. When in doubt
about a signature, read the docs or the type stub (`python/topica/_topica.pyi`);
do not guess.