reindex-text — rebuild the Text row for an existing Piece
emmsap2.index_texts.main() skips any piece that already has a Text
(Piece.objects.exclude(text__isnull=False)). So updateDB will not refresh a
piece whose file changed. To reindex, delete the stale Text row and re-run the
per-piece logic.
Model facts (as of this writing — verify if models.py changed)
Text has a OneToOneField to Piece (on_delete=CASCADE), fields:
language, text, text_reg, text_no_space.
- Reusable functions live in
emmsap2/index_texts.py:
get_text_from_score(sc) — assembles lyrics across parts, adding a later
part's text only when it's <0.8 Levenshtein-similar to what's already there.
regularize_text(text_in, language) → (text_reg, text_no_space).
- module globals
ld (LanguageDetector) and mistakes (filename→language overrides).
- Piece 3758 =
Krakow_2464_10_Celebris_dies_colitur.mxl is a good smoke-test id.
Procedure
Run with uv run python manage.py shell -c "..." from the repo root. Replace
PK with the piece id (or look it up by filename with
Piece.objects.get(filename=...)).
from emmsap2.models import Piece, Text
from emmsap2 import index_texts as it
p = Piece.objects.get(pk=PK)
# show the old row so the change is auditable
old = Text.objects.filter(piece=p).first()
print('OLD language:', old.language if old else None)
print('OLD text_reg:', repr((old.text_reg or '')[:200]) if old else None)
# delete the stale Text and rebuild from the current file on disk
Text.objects.filter(piece=p).delete()
sc = p.stream()
piece_text = it.get_text_from_score(sc)
if len(piece_text) < 5:
language, text_reg, text_no_space = 'na', '', ''
else:
language = it.ld.mostLikelyLanguage(piece_text)
language = it.mistakes.get(p.filename, language) # known-mistake overrides
text_reg, text_no_space = it.regularize_text(piece_text, language)
Text(piece=p, language=language, text=piece_text,
text_reg=text_reg, text_no_space=text_no_space).save()
print('NEW language:', language)
print('NEW text_reg:', repr(text_reg[:300]))
Notes
- This mirrors
index_texts.main() exactly except (1) it deletes first so an
existing piece is rebuilt, and (2) it keys mistakes by p.filename (a str),
not the Piece object — main() passes the Piece, which never matches the
string keys, so keying by filename here is the corrected behavior.
- Always print OLD vs NEW so the user can confirm the reindex actually changed
something (e.g. extra voices' lyrics now included).
- To reindex several pieces, loop over the ids; each is independent.
- This only touches the
Text table. If tinyNotation / segment / ratio indexes
also depend on the changed file, those are separate re-runs
(index_tinyNotation, index_segments, index_ratios).
1---2name: reindex-text3description: Re-index the sung text of a Piece that already exists in the emmsap database. Use when a musicxml/mxl file was updated or re-imported and its Text row is stale, or the user says "reindex the text(s) from piece <id>", "re-run text indexing for <piece>". The normal index_texts.main() only touches pieces with NO text, so an existing Piece must have its Text row deleted and rebuilt. Runs against the live DB via Django shell.4metadata:5 source: https://skillmd.com/skills/cuthbertlab/reindex-text6---78# reindex-text — rebuild the Text row for an existing Piece910`emmsap2.index_texts.main()` **skips any piece that already has a Text**11(`Piece.objects.exclude(text__isnull=False)`). So `updateDB` will not refresh a12piece whose file changed. To reindex, delete the stale `Text` row and re-run the13per-piece logic.1415## Model facts (as of this writing — verify if models.py changed)1617- `Text` has a **OneToOneField** to `Piece` (`on_delete=CASCADE`), fields:18 `language`, `text`, `text_reg`, `text_no_space`.19- Reusable functions live in `emmsap2/index_texts.py`:20 - `get_text_from_score(sc)` — assembles lyrics across parts, adding a later21 part's text only when it's <0.8 Levenshtein-similar to what's already there.22 - `regularize_text(text_in, language)` → `(text_reg, text_no_space)`.23 - module globals `ld` (LanguageDetector) and `mistakes` (filename→language overrides).24- Piece 3758 = `Krakow_2464_10_Celebris_dies_colitur.mxl` is a good smoke-test id.2526## Procedure2728Run with `uv run python manage.py shell -c "..."` from the repo root. Replace29`PK` with the piece id (or look it up by filename with30`Piece.objects.get(filename=...)`).3132```python33from emmsap2.models import Piece, Text34from emmsap2 import index_texts as it3536p = Piece.objects.get(pk=PK)3738# show the old row so the change is auditable39old = Text.objects.filter(piece=p).first()40print('OLD language:', old.language if old else None)41print('OLD text_reg:', repr((old.text_reg or '')[:200]) if old else None)4243# delete the stale Text and rebuild from the current file on disk44Text.objects.filter(piece=p).delete()4546sc = p.stream()47piece_text = it.get_text_from_score(sc)48if len(piece_text) < 5:49 language, text_reg, text_no_space = 'na', '', ''50else:51 language = it.ld.mostLikelyLanguage(piece_text)52 language = it.mistakes.get(p.filename, language) # known-mistake overrides53 text_reg, text_no_space = it.regularize_text(piece_text, language)5455Text(piece=p, language=language, text=piece_text,56 text_reg=text_reg, text_no_space=text_no_space).save()5758print('NEW language:', language)59print('NEW text_reg:', repr(text_reg[:300]))60```6162## Notes6364- This mirrors `index_texts.main()` exactly except (1) it deletes first so an65 existing piece is rebuilt, and (2) it keys `mistakes` by `p.filename` (a str),66 not the Piece object — `main()` passes the Piece, which never matches the67 string keys, so keying by filename here is the corrected behavior.68- Always print OLD vs NEW so the user can confirm the reindex actually changed69 something (e.g. extra voices' lyrics now included).70- To reindex several pieces, loop over the ids; each is independent.71- This only touches the `Text` table. If tinyNotation / segment / ratio indexes72 also depend on the changed file, those are separate re-runs73 (`index_tinyNotation`, `index_segments`, `index_ratios`).