Arjumaan /graphify
Turn any folder of files into a navigable knowledge graph with community detection, an honest audit trail, and three outputs: interactive HTML, GraphRAG-ready JSON, and a plain-language GRAPH_REPORT.md.
Usage
/graphify # full pipeline on current directory → Obsidian vault
/graphify <path> # full pipeline on specific path
/graphify <path> --mode deep # thorough extraction, richer INFERRED edges
/graphify <path> --update # incremental - re-extract only new/changed files
/graphify <path> --directed # build directed graph (preserves edge direction: source→target)
/graphify <path> --cluster-only # rerun clustering on existing graph
/graphify <path> --no-viz # skip visualization, just report + JSON
/graphify <path> --html # (HTML is generated by default - this flag is a no-op)
/graphify <path> --svg # also export graph.svg (embeds in Notion, GitHub)
/graphify <path> --graphml # export graph.graphml (Gephi, yEd)
/graphify <path> --neo4j # generate graphify-out/cypher.txt for Neo4j
/graphify <path> --neo4j-push bolt://localhost:7687 # push directly to Neo4j
/graphify <path> --wiki # build agent-crawlable wiki (index.md + one article per community)
/graphify <path> --obsidian --obsidian-dir ~/vaults/my-project # write vault to custom path (e.g. existing vault)
/graphify <path> --mcp # start MCP stdio server for agent access
/graphify <path> --watch # watch folder, auto-rebuild on code changes (no LLM needed)
/graphify add <url> # fetch URL, save to ./raw, update graph
/graphify add <url> --author "Name" # tag who wrote it
/graphify add <url> --contributor "Name" # tag who added it to the corpus
/graphify query "<question>" # BFS traversal - broad context
/graphify query "<question>" --dfs # DFS - trace a specific path
/graphify query "<question>" --budget 1500 # cap answer at N tokens
/graphify path "AuthModule" "Database" # shortest path between two concepts
/graphify explain "SwinTransformer" # plain-language explanation of a node
What graphify is for
graphify is built around Andrej Karpathy's /raw folder workflow: drop anything into a folder - papers, tweets, screenshots, code, notes - and get a structured knowledge graph that shows you what you didn't know was connected.
Three things it does that your AI assistant alone cannot:
- Persistent graph - relationships are stored in
graphify-out/graph.jsonand survive across sessions. Ask questions weeks later without re-reading everything. - Honest audit trail - every edge is tagged EXTRACTED, INFERRED, or AMBIGUOUS. You know what was found vs invented.
- Cross-document surprise - community detection finds connections between concepts in different files that you would never think to ask about directly.
Use it for:
- A codebase you're new to (understand architecture before touching anything)
- A reading list (papers + tweets + notes → one navigable graph)
- A research corpus (citation graph + concept graph in one)
- Your personal /raw folder (drop everything in, let it grow, query it)
What You Must Do When Invoked
If the user invoked /graphify --help or /graphify -h (with no other arguments), print the contents of the ## Usage section above verbatim and stop. Do not run any commands, do not detect files, do not default the path to .. Just print the Usage block and return.
If no path was given, use . (current directory). Do not ask the user for a path.
Follow these steps in order. Do not skip steps.
Step 1 - Ensure graphify is installed
# Detect Python with graphify — uv/pipx-aware (fixes #831)
New-Item -ItemType Directory -Force -Path graphify-out | Out-Null
$GRAPHIFY_PYTHON = $null
function Find-GraphifyPython {
# 1. uv tool install — 'uv tool dir' is authoritative, respects UV_TOOL_DIR automatically
if (Get-Command uv -ErrorAction SilentlyContinue) {
$uvDir = (uv tool dir 2>$null).Trim()
if ($uvDir) {
$py = Join-Path $uvDir "graphifyy\Scripts\python.exe"
if (Test-Path $py) {
& $py -c "import graphify" 2>$null
if ($LASTEXITCODE -eq 0) { return $py }
}
}
}
# 2. pipx install — 'pipx environment' respects PIPX_HOME automatically
if (Get-Command pipx -ErrorAction SilentlyContinue) {
$venvs = (pipx environment --value PIPX_LOCAL_VENVS 2>$null).Trim()
if ($venvs) {
$py = Join-Path $venvs "graphifyy\Scripts\python.exe"
if (Test-Path $py) {
& $py -c "import graphify" 2>$null
if ($LASTEXITCODE -eq 0) { return $py }
}
}
}
# 3. Active venv / conda / pip-into-current-env
$pyCmd = Get-Command python -ErrorAction SilentlyContinue
if ($pyCmd) {
& $pyCmd.Source -c "import graphify" 2>$null
if ($LASTEXITCODE -eq 0) {
return (& $pyCmd.Source -c "import sys; print(sys.executable)").Trim()
}
}
return $null
}
# Try to find the right Python (uv → pipx → active env)
$GRAPHIFY_PYTHON = Find-GraphifyPython
# Not found — install then re-detect
if (-not $GRAPHIFY_PYTHON) {
if (Get-Command uv -ErrorAction SilentlyContinue) {
uv tool install --upgrade graphifyy -q 2>&1 | Select-Object -Last 3
} else {
pip install graphifyy -q 2>&1 | Select-Object -Last 3
}
$GRAPHIFY_PYTHON = Find-GraphifyPython
}
# Save interpreter path — all subsequent steps read this
$GRAPHIFY_PYTHON | Out-File -FilePath graphify-out\.graphify_python -Encoding utf8 -NoNewline
If the import succeeds, print nothing and move straight to Step 2.
Step 2 - Detect files
@'
import json
from graphify.detect import detect
from pathlib import Path
result = detect(Path('INPUT_PATH'))
print(json.dumps(result, ensure_ascii=False))
'@ | Out-File -FilePath graphify-out\.graphify_step_2_detect_files_3.py -Encoding utf8
& (Get-Content graphify-out\.graphify_python) graphify-out\.graphify_step_2_detect_files_3.py | Out-File -FilePath graphify-out\.graphify_detect.json -Encoding utf8
Remove-Item -ErrorAction SilentlyContinue graphify-out\.graphify_step_2_detect_files_3.py
Replace INPUT_PATH with the actual path the user provided. Do NOT cat or print the JSON - read it silently and present a clean summary instead:
Corpus: X files · ~Y words
code: N files (.py .ts .go ...)
docs: N files (.md .txt ...)
papers: N files (.pdf ...)
images: N files
video: N files (.mp4 .mp3 ...)
Omit any category with 0 files from the summary.
Then act on it:
- If
total_filesis 0: stop with "No supported files found in [path]." - If
skipped_sensitiveis non-empty: mention file count skipped, not the file names. - If
total_words> 2,000,000 ORtotal_files> 200: show the warning and the top 5 subdirectories by file count, then ask which subfolder to run on. Wait for the user's answer before proceeding. - Otherwise: proceed directly to Step 2.5 if video files were detected, or Step 3 if not.
Step 2.5 - Transcribe video / audio files (only if video files detected)
Skip this step entirely if detect returned zero video files.
Video and audio files cannot be read directly. Transcribe them to text first, then treat the transcripts as doc files in Step 3.
Strategy: Read the god nodes from the detect output or analysis file. You are already a language model - write a one-sentence domain hint yourself from those labels. Then pass it to Whisper as the initial prompt. No separate API call needed.
However, if the corpus has only video files and no other docs/code, use the generic fallback prompt: "Use proper punctuation and paragraph breaks."
Step 1 - Write the Whisper prompt yourself.
Read the top god node labels from detect output or analysis, then compose a short domain hint sentence, for example:
- Labels:
transformer, attention, encoder, decoder->"Machine learning research on transformer architectures and attention mechanisms. Use proper punctuation and paragraph breaks." - Labels:
kubernetes, deployment, pod, helm->"DevOps discussion about Kubernetes deployments and Helm charts. Use proper punctuation and paragraph breaks."
Set it as $env:GRAPHIFY_WHISPER_PROMPT before running the transcription command.
Step 2 - Transcribe (PowerShell):
@'
import json, os
from pathlib import Path
from graphify.transcribe import transcribe_all
detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding="utf-8"))
video_files = detect.get('files', {}).get('video', [])
prompt = os.environ.get('GRAPHIFY_WHISPER_PROMPT', 'Use proper punctuation and paragraph breaks.')
transcript_paths = transcribe_all(video_files, initial_prompt=prompt)
print(json.dumps(transcript_paths, ensure_ascii=False))
'@ | Out-File -FilePath graphify-out\.graphify_step_transcribe.py -Encoding utf8
& (Get-Content graphify-out\.graphify_python) graphify-out\.graphify_step_transcribe.py | Out-File -FilePath graphify-out\.graphify_transcripts.json -Encoding utf8
Remove-Item -ErrorAction SilentlyContinue graphify-out\.graphify_step_transcribe.py
After transcription:
- Read the transcript paths from
graphify-out\.graphify_transcripts.json - Add them to the docs list before dispatching semantic subagents in Step 3B
- Print how many transcripts were created:
Transcribed N video file(s) -> treating as docs - If transcription fails for a file, print a warning and continue with the rest
Whisper model: Default is base. If the user passed --whisper-model <name>, set $env:GRAPHIFY_WHISPER_MODEL = "<name>" before running the command above.
Step 3 - Extract entities and relationships
Before starting: note whether --mode deep was given. You must pass DEEP_MODE=true to every subagent in Step B2 if it was. Track this from the original invocation - do not lose it.
This step has two parts: structural extraction (deterministic, free) and semantic extraction (your AI model, costs tokens).
Run Part A (AST) and Part B (semantic) in parallel. Dispatch all semantic subagents AND start AST extraction in the same message. Both can run simultaneously since they operate on different file types. Merge results in Part C as before.
Note: Parallelizing AST + semantic saves 5-15s on large corpora. AST is deterministic and fast; start it while subagents are processing docs/papers.
Part A - Structural extraction for code files
For any code files detected, run AST extraction in parallel with Part B subagents:
@'
import json
from graphify.extract import collect_files, extract
from pathlib import Path
def main():
code_files = []
detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding="utf-8"))
for f in detect.get('files', {}).get('code', []):
code_files.extend(collect_files(Path(f)) if Path(f).is_dir() else [Path(f)])
if code_files:
result = extract(code_files)
Path('graphify-out/.graphify_ast.json').write_text(json.dumps(result, indent=2, ensure_ascii=False), encoding="utf-8")
print(f'AST: {len(result["nodes"])} nodes, {len(result["edges"])} edges')
else:
Path('graphify-out/.graphify_ast.json').write_text(json.dumps({'nodes':[],'edges':[],'input_tokens':0,'output_tokens':0}, ensure_ascii=False), encoding="utf-8")
print('No code files - skipping AST extraction')
# Windows-spawn ProcessPoolExecutor (used inside extract()) re-imports this
# script in each worker; without an `if __name__ == "__main__":` guard the
# pool would recursively spawn itself. graphify v0.7.11+ falls back to
# sequential extraction if the pool dies, but the guard keeps multi-core
# extraction working on Windows.
if __name__ == '__main__':
main()
'@ | Out-File -FilePath graphify-out\.graphify_step_ast.py -Encoding utf8
& (Get-Content graphify-out\.graphify_python) graphify-out\.graphify_step_ast.py
Remove-Item -ErrorAction SilentlyContinue graphify-out\.graphify_step_ast.py
Part B - Semantic extraction (parallel subagents)
Fast path: If detection found zero docs, papers, and images (code-only corpus), skip Part B entirely and go straight to Part C. AST handles code - there is nothing for semantic subagents to do.
MANDATORY: You MUST use the Agent tool here. Reading files yourself one-by-one is forbidden - it is 5-10x slower. If you do not use the Agent tool you are doing this wrong.
Before dispatching subagents, print a timing estimate:
- Load
total_wordsand file counts fromgraphify-out/.graphify_detect.json - Estimate agents needed:
ceil(uncached_non_code_files / 22)(chunk size is 20-25) - Estimate time: ~45s per agent batch (they run in parallel, so total ≈ 45s × ceil(agents/parallel_limit))
- Print: "Semantic extraction: ~N files → X agents, estimated ~Ys"
Step B0 - Check extraction cache first
Before dispatching any subagents, check which files already have cached extraction results:
@'
import json
from graphify.cache import check_semantic_cache
from pathlib import Path
detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding="utf-8"))
all_files = [f for files in detect['files'].values() for f in files]
cached_nodes, cached_edges, cached_hyperedges, uncached = check_semantic_cache(all_files)
if cached_nodes or cached_edges or cached_hyperedges:
Path('graphify-out/.graphify_cached.json').write_text(json.dumps({'nodes': cached_nodes, 'edges': cached_edges, 'hyperedges': cached_hyperedges}, ensure_ascii=False), encoding="utf-8")
Path('graphify-out/.graphify_uncached.txt').write_text('\n'.join(uncached), encoding="utf-8")
print(f'Cache: {len(all_files)-len(uncached)} files hit, {len(uncached)} files need extraction')
'@ | Out-File -FilePath graphify-out\.graphify_step_3_extract_entities_and_relations_5.py -Encoding utf8
& (Get-Content graphify-out\.graphify_python) graphify-out\.graphify_step_3_extract_entities_and_relations_5.py
Remove-Item -ErrorAction SilentlyContinue graphify-out\.graphify_step_3_extract_entities_and_relations_5.py
Only dispatch subagents for files listed in .graphify_uncached.txt. If all files are cached, skip to Part C directly.
Step B1 - Split into chunks
Load files from .graphify_uncached.txt. Split into chunks of 20-25 files each. Each image gets its own chunk (vision needs separate context).
Step B2 - Dispatch ALL subagents in a single message
Call the Agent tool multiple times IN THE SAME RESPONSE - one call per chunk. This is the only way they run in parallel. If you make one Agent call, wait, then make another, you are doing it sequentially and defeating the purpose.
Concrete example for 3 chunks:
[Agent tool call 1: files 1-15]
[Agent tool call 2: files 16-30]
[Agent tool call 3: files 31-45]
All three in one message. Not three separate messages.
Each subagent receives this exact prompt (substitute FILE_LIST, CHUNK_NUM, TOTAL_CHUNKS, and DEEP_MODE):
You are a graphify extraction subagent. Read the files listed and extract a knowledge graph fragment.
Output ONLY valid JSON matching the schema below - no explanation, no markdown fences, no preamble.
Files (chunk CHUNK_NUM of TOTAL_CHUNKS):
FILE_LIST
Rules:
- EXTRACTED: relationship explicit in source (import, call, citation, "see §3.2")
- INFERRED: reasonable inference (shared data structure, implied dependency)
- AMBIGUOUS: uncertain - flag for review, do not omit
Code files: focus on semantic edges AST cannot find (call relationships, shared data, arch patterns).
Do not re-extract imports - AST already has those.
Doc/paper files: extract named concepts, entities, citations. For rationale (WHY decisions were made, trade-offs, design intent): store as a `rationale` attribute on the relevant concept node — do NOT create a separate rationale node or fragment node. Only create a node for something that is itself a named entity or concept. Use `file_type:"rationale"` for concept-like nodes (ideas, principles, mechanisms, design patterns). Do NOT invent file_types like `concept` — valid values are only `code|document|paper|image|rationale`.
Code files: when adding `calls` edges, source MUST be the caller (the function/class doing the calling), target MUST be the callee. Never reverse this direction.
Image files: use vision to understand what the image IS - do not just OCR.
UI screenshot: layout patterns, design decisions, key elements, purpose.
Chart: metric, trend/insight, data source.
Tweet/post: claim as node, author, concepts mentioned.
Diagram: components and connections.
Research figure: what it demonstrates, method, result.
Handwritten/whiteboard: ideas and arrows, mark uncertain readings AMBIGUOUS.
DEEP_MODE (if --mode deep was given): be aggressive with INFERRED edges - indirect deps,
shared assumptions, latent couplings. Mark uncertain ones AMBIGUOUS instead of omitting.
Semantic similarity: if two concepts in this chunk solve the same problem or represent the same idea without any structural link (no import, no call, no citation), add a `semantically_similar_to` edge marked INFERRED with a confidence_score reflecting how similar they are (0.6-0.95). Examples:
- Two functions that both validate user input but never call each other
- A class in code and a concept in a paper that describe the same algorithm
- Two error types that handle the same failure mode differently
Only add these when the similarity is genuinely non-obvious and cross-cutting. Do not add them for trivially similar things.
Hyperedges: if 3 or more nodes clearly participate together in a shared concept, flow, or pattern that is not captured by pairwise edges alone, add a hyperedge to a top-level `hyperedges` array. Examples:
- All classes that implement a common protocol or interface
- All functions in an authentication flow (even if they don't all call each other)
- All concepts from a paper section that form one coherent idea
Use sparingly — only when the group relationship adds information beyond the pairwise edges. Maximum 3 hyperedges per chunk.
If a file has YAML frontmatter (--- ... ---), copy source_url, captured_at, author,
contributor onto every node from that file.
confidence_score is REQUIRED on every edge - never omit it, never use 0.5 as a default:
- EXTRACTED edges: confidence_score = 1.0 always
- INFERRED edges: pick exactly ONE value from this set — never 0.5:
0.95 direct structural evidence (shared data structure, named cross-file reference).
0.85 strong inference (clear functional alignment, no direct symbol link).
0.75 reasonable inference (shared problem domain + similar shape, requires interpretation).
0.65 weak inference (thematically related, no shape evidence).
0.55 speculative but plausible (surface-level co-occurrence only).
Models follow discrete rubrics better than continuous ranges; the bimodal
distribution observed in production (>50% at 0.5, >40% at 0.85+) shows the
range guidance is being collapsed to a binary. If no value above fits, mark
the edge AMBIGUOUS rather than picking 0.4 or below.
- AMBIGUOUS edges: 0.1-0.3
Output exactly this JSON (no other text):
{"nodes":[{"id":"filestem_entityname","label":"Human Readable Name","file_type":"code|document|paper|image|rationale","source_file":"relative/path","source_location":null,"source_url":null,"captured_at":null,"author":null,"contributor":null}],"edges":[{"source":"node_id","target":"node_id","relation":"calls|implements|references|cites|conceptually_related_to|shares_data_with|semantically_similar_to|rationale_for","confidence":"EXTRACTED|INFERRED|AMBIGUOUS","confidence_score":1.0,"source_file":"relative/path","source_location":null,"weight":1.0}],"hyperedges":[{"id":"snake_case_id","label":"Human Readable Label","nodes":["node_id1","node_id2","node_id3"],"relation":"participate_in|implement|form","confidence":"EXTRACTED|INFERRED","confidence_score":0.75,"source_file":"relative/path"}],"input_tokens":0,"output_tokens":0}
Step B3 - Collect, cache, and merge
Wait for all subagents. For each result:
- Check that
graphify-out/.graphify_chunk_NN.jsonexists on disk — this is the success signal - If the file exists and contains valid JSON with
nodesandedges, include it and save to cache - If the file is missing, the subagent was likely dispatched as read-only (Explore type) — print a warning: "chunk N missing from disk — subagent may have been read-only. Re-run with general-purpose agent." Do not silently skip.
- If a subagent failed or returned invalid JSON, print a warning and skip that chunk - do not abort
If more than half the chunks failed or are missing, stop and tell the user to re-run and ensure subagent_type="general-purpose" is used.
Merge all chunk files into .graphify_semantic_new.json. After each Agent call completes, read the real token counts from the Agent tool result's usage field and write them back into the chunk JSON before merging — the chunk JSON itself always has placeholder zeros. Then run:
& (Get-Content graphify-out\.graphify_python) -c "
import json, glob
from pathlib import Path
chunks = sorted(glob.glob('graphify-out/.graphify_chunk_*.json'))
all_nodes, all_edges, all_hyperedges = [], [], []
total_in, total_out = 0, 0
for c in chunks:
d = json.loads(Path(c).read_text(encoding=\"utf-8\"))
all_nodes += d.get('nodes', [])
all_edges += d.get('edges', [])
all_hyperedges += d.get('hyperedges', [])
total_in += d.get('input_tokens', 0)
total_out += d.get('output_tokens', 0)
Path('graphify-out/.graphify_semantic_new.json').write_text(json.dumps({
'nodes': all_nodes, 'edges': all_edges, 'hyperedges': all_hyperedges,
'input_tokens': total_in, 'output_tokens': total_out,
}, indent=2, ensure_ascii=False), encoding=\"utf-8\")
print(f'Merged {len(chunks)} chunks: {total_in:,} in / {total_out:,} out tokens')
"
Save new results to cache:
@'
import json
from graphify.cache import save_semantic_cache
from pathlib import Path
new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding="utf-8")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]}
saved = save_semantic_cache(new.get('nodes', []), new.get('edges', []), new.get('hyperedges', []))
print(f'Cached {saved} files')
'@ | Out-File -FilePath graphify-out\.graphify_step_3_extract_entities_and_relations_6.py -Encoding utf8
& (Get-Content graphify-out\.graphify_python) graphify-out\.graphify_step_3_extract_entities_and_relations_6.py
Remove-Item -ErrorAction SilentlyContinue graphify-out\.graphify_step_3_extract_entities_and_relations_6.py
Merge cached + new results into graphify-out/.graphify_semantic.json:
@'
import json
from pathlib import Path
cached = json.loads(Path('graphify-out/.graphify_cached.json').read_text(encoding="utf-8")) if Path('graphify-out/.graphify_cached.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]}
new = json.loads(Path('graphify-out/.graphify_semantic_new.json').read_text(encoding="utf-8")) if Path('graphify-out/.graphify_semantic_new.json').exists() else {'nodes':[],'edges':[],'hyperedges':[]}
all_nodes = cached['nodes'] + new.get('nodes', [])
all_edges = cached['edges'] + new.get('edges', [])
all_hyperedges = cached.get('hyperedges', []) + new.get('hyperedges', [])
seen = set()
deduped = []
for n in all_nodes:
if n['id'] not in seen:
seen.add(n['id'])
deduped.append(n)
merged = {
'nodes': deduped,
'edges': all_edges,
'hyperedges': all_hyperedges,
'input_tokens': new.get('input_tokens', 0),
'output_tokens': new.get('output_tokens', 0),
}
Path('graphify-out/.graphify_semantic.json').write_text(json.dumps(merged, indent=2, ensure_ascii=False), encoding="utf-8")
print(f'Extraction complete - {len(deduped)} nodes, {len(all_edges)} edges ({len(cached["nodes"])} from cache, {len(new.get("nodes",[]))} new)')
'@ | Out-File -FilePath graphify-out\.graphify_step_3_extract_entities_and_relations_7.py -Encoding utf8
& (Get-Content graphify-out\.graphify_python) graphify-out\.graphify_step_3_extract_entities_and_relations_7.py
Remove-Item -ErrorAction SilentlyContinue graphify-out\.graphify_step_3_extract_entities_and_relations_7.py
Clean up temp files: Remove-Item -ErrorAction SilentlyContinue graphify-out\.graphify_cached.json, graphify-out\.graphify_uncached.txt, graphify-out\.graphify_semantic_new.json
Part C - Merge AST + semantic into final extraction
@'
import sys, json
from pathlib import Path
ast = json.loads(Path('graphify-out/.graphify_ast.json').read_text(encoding="utf-8"))
sem = json.loads(Path('graphify-out/.graphify_semantic.json').read_text(encoding="utf-8"))
# Merge: AST nodes first, semantic nodes deduplicated by id
seen = {n['id'] for n in ast['nodes']}
merged_nodes = list(ast['nodes'])
for n in sem['nodes']:
if n['id'] not in seen:
merged_nodes.append(n)
seen.add(n['id'])
merged_edges = ast['edges'] + sem['edges']
merged_hyperedges = sem.get('hyperedges', [])
merged = {
'nodes': merged_nodes,
'edges': merged_edges,
'hyperedges': merged_hyperedges,
'input_tokens': sem.get('input_tokens', 0),
'output_tokens': sem.get('output_tokens', 0),
}
Path('graphify-out/.graphify_extract.json').write_text(json.dumps(merged, indent=2, ensure_ascii=False), encoding="utf-8")
total = len(merged_nodes)
edges = len(merged_edges)
print(f'Merged: {total} nodes, {edges} edges ({len(ast["nodes"])} AST + {len(sem["nodes"])} semantic)')
'@ | Out-File -FilePath graphify-out\.graphify_step_3_extract_entities_and_relations_8.py -Encoding utf8
& (Get-Content graphify-out\.graphify_python) graphify-out\.graphify_step_3_extract_entities_and_relations_8.py
Remove-Item -ErrorAction SilentlyContinue graphify-out\.graphify_step_3_extract_entities_and_relations_8.py
Step 4 - Build graph, cluster, analyze, generate outputs
New-Item -ItemType Directory -Force -Path graphify-out | Out-Null
@'
import sys, json
from graphify.build import build_from_json
from graphify.cluster import cluster, score_all
from graphify.analyze import god_nodes, surprising_connections, suggest_questions
from graphify.report import generate
from graphify.export import to_json
from pathlib import Path
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding="utf-8"))
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding="utf-8"))
G = build_from_json(extraction)
communities = cluster(G)
cohesion = score_all(G, communities)
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
gods = god_nodes(G)
surprises = surprising_connections(G, communities)
labels = {cid: 'Community ' + str(cid) for cid in communities}
# Placeholder questions - regenerated with real labels in Step 5
questions = suggest_questions(G, communities, labels)
report = generate(G, communities, cohesion, labels, gods, surprises, detection, tokens, 'INPUT_PATH', suggested_questions=questions)
Path('graphify-out/GRAPH_REPORT.md').write_text(report, encoding="utf-8")
to_json(G, communities, 'graphify-out/graph.json')
analysis = {
'communities': {str(k): v for k, v in communities.items()},
'cohesion': {str(k): v for k, v in cohesion.items()},
'gods': gods,
'surprises': surprises,
'questions': questions,
}
Path('graphify-out/.graphify_analysis.json').write_text(json.dumps(analysis, indent=2, ensure_ascii=False), encoding="utf-8")
if G.number_of_nodes() == 0:
print('ERROR: Graph is empty - extraction produced no nodes.')
print('Possible causes: all files were skipped, binary-only corpus, or extraction failed.')
raise SystemExit(1)
print(f'Graph: {G.number_of_nodes()} nodes, {G.number_of_edges()} edges, {len(communities)} communities')
'@ | Out-File -FilePath graphify-out\.graphify_step_4_build_graph_cluster_analyze_ge_9.py -Encoding utf8
& (Get-Content graphify-out\.graphify_python) graphify-out\.graphify_step_4_build_graph_cluster_analyze_ge_9.py
Remove-Item -ErrorAction SilentlyContinue graphify-out\.graphify_step_4_build_graph_cluster_analyze_ge_9.py
If this step prints ERROR: Graph is empty, stop and tell the user what happened - do not proceed to labeling or visualization.
Replace INPUT_PATH with the actual path.
Step 5 - Label communities
Read graphify-out/.graphify_analysis.json. For each community key, look at its node labels and write a 2-5 word plain-language name (e.g. "Attention Mechanism", "Training Pipeline", "Data Loading").
Then regenerate the report and save the labels for the visualizer:
@'
import sys, json
from graphify.build import build_from_json
from graphify.cluster import score_all
from graphify.analyze import god_nodes, surprising_connections, suggest_questions
from graphify.report import generate
from pathlib import Path
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding="utf-8"))
detection = json.loads(Path('graphify-out/.graphify_detect.json').read_text(encoding="utf-8"))
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding="utf-8"))
G = build_from_json(extraction)
communities = {int(k): v for k, v in analysis['communities'].items()}
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
tokens = {'input': extraction.get('input_tokens', 0), 'output': extraction.get('output_tokens', 0)}
# LABELS - replace these with the names you chose above
labels = LABELS_DICT
# Regenerate questions with real community labels (labels affect question phrasing)
questions = suggest_questions(G, communities, labels)
report = generate(G, communities, cohesion, labels, analysis['gods'], analysis['surprises'], detection, tokens, 'INPUT_PATH', suggested_questions=questions)
Path('graphify-out/GRAPH_REPORT.md').write_text(report, encoding="utf-8")
Path('graphify-out/.graphify_labels.json').write_text(json.dumps({str(k): v for k, v in labels.items()}, ensure_ascii=False), encoding="utf-8")
print('Report updated with community labels')
'@ | Out-File -FilePath graphify-out\.graphify_step_5_label_communities_10.py -Encoding utf8
& (Get-Content graphify-out\.graphify_python) graphify-out\.graphify_step_5_label_communities_10.py
Remove-Item -ErrorAction SilentlyContinue graphify-out\.graphify_step_5_label_communities_10.py
Replace LABELS_DICT with the actual dict you constructed (e.g. {0: "Attention Mechanism", 1: "Training Pipeline"}).
Replace INPUT_PATH with the actual path.
Step 6 - Generate Obsidian vault (opt-in) + HTML
Generate HTML always (unless --no-viz). Obsidian vault only if --obsidian was explicitly given — skip it otherwise, it generates one file per node.
If --obsidian was given:
- If
--obsidian-dir <path>was also given, use that path as the vault directory. Otherwise default tographify-out/obsidian.
@'
import sys, json
from graphify.build import build_from_json
from graphify.export import to_obsidian, to_canvas
from pathlib import Path
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding="utf-8"))
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding="utf-8"))
labels_raw = json.loads(Path('graphify-out/.graphify_labels.json').read_text(encoding="utf-8")) if Path('graphify-out/.graphify_labels.json').exists() else {}
G = build_from_json(extraction)
communities = {int(k): v for k, v in analysis['communities'].items()}
cohesion = {int(k): v for k, v in analysis['cohesion'].items()}
labels = {int(k): v for k, v in labels_raw.items()}
obsidian_dir = 'OBSIDIAN_DIR' # replace with --obsidian-dir value, or 'graphify-out/obsidian' if not given
n = to_obsidian(G, communities, obsidian_dir, community_labels=labels or None, cohesion=cohesion)
print(f'Obsidian vault: {n} notes in {obsidian_dir}/')
to_canvas(G, communities, f'{obsidian_dir}/graph.canvas', community_labels=labels or None)
print(f'Canvas: {obsidian_dir}/graph.canvas - open in Obsidian for structured community layout')
print()
print(f'Open {obsidian_dir}/ as a vault in Obsidian.')
print(' Graph view - nodes colored by community (set automatically)')
print(' graph.canvas - structured layout with communities as groups')
print(' _COMMUNITY_* - overview notes with cohesion scores and dataview queries')
'@ | Out-File -FilePath graphify-out\.graphify_step_6_generate_obsidian_vault_opt_in_11.py -Encoding utf8
& (Get-Content graphify-out\.graphify_python) graphify-out\.graphify_step_6_generate_obsidian_vault_opt_in_11.py
Remove-Item -ErrorAction SilentlyContinue graphify-out\.graphify_step_6_generate_obsidian_vault_opt_in_11.py
Generate the HTML graph (always, unless --no-viz):
@'
import sys, json
from graphify.build import build_from_json
from graphify.export import to_html
from pathlib import Path
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding="utf-8"))
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding="utf-8"))
labels_raw = json.loads(Path('graphify-out/.graphify_labels.json').read_text(encoding="utf-8")) if Path('graphify-out/.graphify_labels.json').exists() else {}
G = build_from_json(extraction)
communities = {int(k): v for k, v in analysis['communities'].items()}
labels = {int(k): v for k, v in labels_raw.items()}
if G.number_of_nodes() > 5000:
print(f'Graph has {G.number_of_nodes()} nodes - too large for HTML viz. Use Obsidian vault instead.')
else:
to_html(G, communities, 'graphify-out/graph.html', community_labels=labels or None)
print('graph.html written - open in any browser, no server needed')
'@ | Out-File -FilePath graphify-out\.graphify_step_6_generate_obsidian_vault_opt_in_12.py -Encoding utf8
& (Get-Content graphify-out\.graphify_python) graphify-out\.graphify_step_6_generate_obsidian_vault_opt_in_12.py
Remove-Item -ErrorAction SilentlyContinue graphify-out\.graphify_step_6_generate_obsidian_vault_opt_in_12.py
Step 7 - Neo4j export (only if --neo4j or --neo4j-push flag)
If --neo4j - generate a Cypher file for manual import:
@'
import sys, json
from graphify.build import build_from_json
from graphify.export import to_cypher
from pathlib import Path
G = build_from_json(json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding="utf-8")))
to_cypher(G, 'graphify-out/cypher.txt')
print('cypher.txt written - import with: cypher-shell < graphify-out/cypher.txt')
'@ | Out-File -FilePath graphify-out\.graphify_step_7_neo4j_export_only_if_neo4j_or__13.py -Encoding utf8
& (Get-Content graphify-out\.graphify_python) graphify-out\.graphify_step_7_neo4j_export_only_if_neo4j_or__13.py
Remove-Item -ErrorAction SilentlyContinue graphify-out\.graphify_step_7_neo4j_export_only_if_neo4j_or__13.py
If --neo4j-push <uri> - push directly to a running Neo4j instance. Ask the user for credentials if not provided:
@'
import sys, json
from graphify.build import build_from_json
from graphify.cluster import cluster
from graphify.export import push_to_neo4j
from pathlib import Path
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding="utf-8"))
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding="utf-8"))
G = build_from_json(extraction)
communities = {int(k): v for k, v in analysis['communities'].items()}
result = push_to_neo4j(G, uri='NEO4J_URI', user='NEO4J_USER', password='NEO4J_PASSWORD', communities=communities)
print(f'Pushed to Neo4j: {result["nodes"]} nodes, {result["edges"]} edges')
'@ | Out-File -FilePath graphify-out\.graphify_step_7_neo4j_export_only_if_neo4j_or__14.py -Encoding utf8
& (Get-Content graphify-out\.graphify_python) graphify-out\.graphify_step_7_neo4j_export_only_if_neo4j_or__14.py
Remove-Item -ErrorAction SilentlyContinue graphify-out\.graphify_step_7_neo4j_export_only_if_neo4j_or__14.py
Replace NEO4J_URI, NEO4J_USER, NEO4J_PASSWORD with actual values. Default URI is bolt://localhost:7687, default user is neo4j. Uses MERGE - safe to re-run without creating duplicates.
Step 7b - SVG export (only if --svg flag)
@'
import sys, json
from graphify.build import build_from_json
from graphify.export import to_svg
from pathlib import Path
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding="utf-8"))
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding="utf-8"))
labels_raw = json.loads(Path('graphify-out/.graphify_labels.json').read_text(encoding="utf-8")) if Path('graphify-out/.graphify_labels.json').exists() else {}
G = build_from_json(extraction)
communities = {int(k): v for k, v in analysis['communities'].items()}
labels = {int(k): v for k, v in labels_raw.items()}
to_svg(G, communities, 'graphify-out/graph.svg', community_labels=labels or None)
print('graph.svg written - embeds in Obsidian, Notion, GitHub READMEs')
'@ | Out-File -FilePath graphify-out\.graphify_step_7b_svg_export_only_if_svg_flag_15.py -Encoding utf8
& (Get-Content graphify-out\.graphify_python) graphify-out\.graphify_step_7b_svg_export_only_if_svg_flag_15.py
Remove-Item -ErrorAction SilentlyContinue graphify-out\.graphify_step_7b_svg_export_only_if_svg_flag_15.py
Step 7c - GraphML export (only if --graphml flag)
@'
import json
from graphify.build import build_from_json
from graphify.export import to_graphml
from pathlib import Path
extraction = json.loads(Path('graphify-out/.graphify_extract.json').read_text(encoding="utf-8"))
analysis = json.loads(Path('graphify-out/.graphify_analysis.json').read_text(encoding="utf-8"))
G = build_from_json(extraction)
communities = {int(k): v for k, v in analysis['communities'].items()}
to_graphml(G, communities, 'graphify-out/graph.graphml')
print('graph.graphml written - open in Gephi, yEd, or any GraphML tool')
'@ | Out-File -FilePath graphify-out\.graphify_step_7c_graphml_export_only_if_graphml_16.py -Encoding utf8
& (Get-Content graphify-out\.graphify_python) graphify-out\.graphify_step_7c_graphml_export_only_if_graphml_16.py
Remove-Item -ErrorAction SilentlyContinue graphify-out\.graphify_step_7c_graphml_export_only_if_graphml_16.py
Step 7d - MCP server (only if --mcp flag)
& (Get-Content graphify-out\.graphify_python) -m graphify.serve graphify-out/graph.json
This starts a stdio MCP server that exposes tools: query_graph, get_node, get_neighbors, get_community, god_nodes, graph_stats, shortest_path. Add to Claude Desktop or any MCP-compatible agent orchestrator so other agents can query the graph live.
To configure in Claude Desktop, add to claude_desktop_config.json:
{
"mcpServers": {
"graphify": {
"command": "python",
"args": ["-m", "graphify.serve", "/absolute/path/to/graphify-out/graph.json"]
}
}
}
Step 8 - Token reduction benchmark (only if total_words > 5000)
If total_words from graphify-out/.graphify_detect.json is greater than 5,000, run:
@'
import json
from graphify.benchmark import run_benchmark, print_benchmark
from pathlib import Path
dete
…(truncated)