Prompt Cache
A lightweight caching layer that prevents regenerating identical content. Saved approximately 60% of API quota in production by catching duplicate prompts before they hit the API.
How It Works
- Normalize the prompt (lowercase, collapse whitespace)
- Combine with context keys (user name, language, model)
- SHA-256 hash the combined key
- Check cache table for existing result
- On miss: call API, store result. On hit: return cached result instantly.
Usage
import prompt_cache
# Check before calling expensive API
cached = await prompt_cache.get_cached(
prompt="Tell me a story about clouds",
child_name="Sophie",
language="fr"
)
if cached:
return cached # Free! No API call needed.
# Cache miss — call the API
result = await generate_story(prompt, child_name, language)
# Store for next time
await prompt_cache.set_cached(prompt, child_name, language, result)
Schema
CREATE TABLE IF NOT EXISTS prompt_cache (
prompt_hash TEXT NOT NULL,
child_name TEXT NOT NULL,
language TEXT NOT NULL,
story_json TEXT,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (prompt_hash, child_name, language)
);
Adapt the Keys
The default implementation uses (prompt, child_name, language) as the cache key. Adapt to your domain:
- Chat completions:
(system_prompt, user_message, model)
- TTS:
(text, voice_id, model_id)
- Image gen:
(prompt, seed, model, size)
Files
scripts/prompt_cache.py — Cache implementation (35 lines)
1---2name: prompt-cache3description: SHA-256 prompt deduplication for LLM and TTS calls — hash normalize prompts, check cache before calling APIs, store results for instant replay. Use when making repeated or similar API calls to avoid redundant spending. Works with any database backend (SQLite, Turso, Postgres).4---5
6# Prompt Cache
7
8A lightweight caching layer that prevents regenerating identical content. Saved approximately 60% of API quota in production by catching duplicate prompts before they hit the API.
9
10## How It Works
11
121. Normalize the prompt (lowercase, collapse whitespace)
132. Combine with context keys (user name, language, model)
143. SHA-256 hash the combined key
154. Check cache table for existing result
165. On miss: call API, store result. On hit: return cached result instantly.
17
18## Usage
19
20```python
21import prompt_cache
22
23# Check before calling expensive API
24cached = await prompt_cache.get_cached(
25 prompt="Tell me a story about clouds",
26 child_name="Sophie",
27 language="fr"
28)
29
30if cached:
31 return cached # Free! No API call needed.
32
33# Cache miss — call the API
34result = await generate_story(prompt, child_name, language)
35
36# Store for next time
37await prompt_cache.set_cached(prompt, child_name, language, result)
38```
39
40## Schema
41
42```sql
43CREATE TABLE IF NOT EXISTS prompt_cache (
44 prompt_hash TEXT NOT NULL,
45 child_name TEXT NOT NULL,
46 language TEXT NOT NULL,
47 story_json TEXT,
48 created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
49 PRIMARY KEY (prompt_hash, child_name, language)
50);
51```
52
53## Adapt the Keys
54
55The default implementation uses `(prompt, child_name, language)` as the cache key. Adapt to your domain:
56
57- **Chat completions:** `(system_prompt, user_message, model)`
58- **TTS:** `(text, voice_id, model_id)`
59- **Image gen:** `(prompt, seed, model, size)`
60
61## Files
62
63- `scripts/prompt_cache.py` — Cache implementation (35 lines)
modbender