Wiki Generator with LSP
Usage
python scripts/generate_docs.py /path/to/project --output WIKI.md
Workflow
- Detect language from config files (
package.json, pyproject.toml, go.mod, etc.)
- Start LSP server for the detected language
- Query LSP for symbols, references, types, and call hierarchy
- Generate WIKI.md with Mermaid diagrams
LSP Servers
| Language |
Server |
| TypeScript/JS |
typescript-language-server |
| Python |
pylsp |
| Go |
gopls |
| Rust |
rust-analyzer |
| C/C++ |
clangd |
If LSP unavailable, scripts fall back to AST/regex analysis.
Output Sections
- Project Overview (tech stack, dependencies)
- Architecture (Mermaid flowchart)
- Project Structure (directory tree)
- Core Components (classes, functions, APIs)
- Data Flow (Mermaid sequenceDiagram)
- Data Model (Mermaid erDiagram, classDiagram)
- API Reference
- Configuration
- Getting Started
- Development Guide
See references/wiki-template.md for section templates.
Tips
Analysis:
- Exclude:
node_modules/, venv/, .git/, dist/, build/
- Prioritize entry points:
main.py, index.ts, App.tsx
- For large codebases, focus on
src/ or lib/
Formatting:
- Use Mermaid diagrams extensively (flowchart, sequenceDiagram, classDiagram, erDiagram)
- Include code snippets where helpful
- Use tables for structured information
- Add navigation links between sections
- Keep explanations clear and concise
Language Support Details
| Language |
Config Files |
Entry Points |
LSP Server |
Installation |
| TypeScript/JS |
package.json, tsconfig.json |
index.ts, main.ts, App.tsx |
typescript-language-server |
npm i -g typescript-language-server typescript |
| Python |
pyproject.toml, setup.py, requirements.txt |
main.py, app.py, __main__.py |
pylsp |
pip install python-lsp-server |
| Go |
go.mod |
main.go, cmd/*/main.go |
gopls |
go install golang.org/x/tools/gopls@latest |
| Rust |
Cargo.toml |
main.rs, lib.rs |
rust-analyzer |
rustup component add rust-analyzer |
| Java |
pom.xml, build.gradle |
Main.java, Application.java |
jdtls |
Eclipse JDT Language Server |
| C/C++ |
CMakeLists.txt, Makefile |
main.c, main.cpp |
clangd |
apt install clangd or brew install llvm |
Fallback Analysis
When LSP is unavailable, the scripts use fallback analyzers:
- AST Parsing - For Python (
ast module) and JavaScript (regex-based)
- Regex Patterns - For function/class detection in other languages
- Import Analysis - Parse import statements for dependency graphs
Fallback provides ~70% of LSP accuracy but works without server setup.
Troubleshooting
LSP Server Not Found
Error: Could not start LSP server for TypeScript
Solution: Install the required LSP server globally. See Installation column in Language Support table.
Timeout During Analysis
Error: LSP request timed out
Solutions:
- Increase timeout:
--timeout 120
- Reduce scope:
--path src/ instead of entire project
- Check if LSP server is overloaded (large codebase)
Empty or Incomplete Output
Possible causes:
- Project has no recognizable entry points
- LSP server crashed during analysis
- File permissions prevent reading
Solutions:
- Specify entry point:
--entry src/main.py
- Check LSP server logs
- Run with
--verbose for debug output
Mermaid Diagrams Not Rendering
Mermaid diagrams require a compatible viewer:
- GitHub/GitLab markdown preview
- VS Code with Mermaid extension
- Mermaid Live Editor: https://mermaid.live
Performance Considerations
| Project Size |
Files |
Estimated Time |
Memory |
| Small |
< 50 |
< 30s |
~100MB |
| Medium |
50-500 |
1-5 min |
~500MB |
| Large |
500-5000 |
5-30 min |
1-2GB |
| Very Large |
> 5000 |
Consider splitting |
2GB+ |
Tips for large codebases:
- Analyze specific directories:
--path src/core
- Skip test files:
--exclude "**/test/**"
- Use incremental analysis if available
- Consider generating separate docs per module
Known Limitations
- Monorepos: May need per-package analysis
- Dynamic imports: Not fully detected (Python
importlib, JS import())
- Metaprogramming: Dynamically generated code may be missed
- Private symbols: Some LSP servers don't expose private members
- Cross-language projects: Analyze each language separately
Best Practices
- Run from project root - Ensures correct path resolution
- Commit WIKI.md - Track documentation changes with code
- Regenerate periodically - Keep docs in sync with codebase
- Review output - AI-generated docs may need human refinement
- Customize template - Adapt
wiki-template.md to your needs
1---2name: wiki-generator-lsp3description: Generate comprehensive WIKI.md documentation for any codebase using LSP for precise code analysis. Use when user asks to document a codebase, generate technical documentation, create architecture diagrams, or analyze code structure and dependencies. Supports TypeScript, JavaScript, Python, Go, Rust, Java, and C/C++ projects.4license: MIT5---6
7# Wiki Generator with LSP
8
9## Usage
10
11```bash
12python scripts/generate_docs.py /path/to/project --output WIKI.md
13```
14
15## Workflow
16
171. **Detect language** from config files (`package.json`, `pyproject.toml`, `go.mod`, etc.)
182. **Start LSP server** for the detected language
193. **Query LSP** for symbols, references, types, and call hierarchy
204. **Generate WIKI.md** with Mermaid diagrams
21
22## LSP Servers
23
24| Language | Server |
25|----------|--------|
26| TypeScript/JS | `typescript-language-server` |
27| Python | `pylsp` |
28| Go | `gopls` |
29| Rust | `rust-analyzer` |
30| C/C++ | `clangd` |
31
32If LSP unavailable, scripts fall back to AST/regex analysis.
33
34## Output Sections
35
361. Project Overview (tech stack, dependencies)
372. Architecture (Mermaid flowchart)
383. Project Structure (directory tree)
394. Core Components (classes, functions, APIs)
405. Data Flow (Mermaid sequenceDiagram)
416. Data Model (Mermaid erDiagram, classDiagram)
427. API Reference
438. Configuration
449. Getting Started
4510. Development Guide
46
47See [references/wiki-template.md](references/wiki-template.md) for section templates.
48
49## Tips
50
51**Analysis:**
52- Exclude: `node_modules/`, `venv/`, `.git/`, `dist/`, `build/`
53- Prioritize entry points: `main.py`, `index.ts`, `App.tsx`
54- For large codebases, focus on `src/` or `lib/`
55
56**Formatting:**
57- Use Mermaid diagrams extensively (flowchart, sequenceDiagram, classDiagram, erDiagram)
58- Include code snippets where helpful
59- Use tables for structured information
60- Add navigation links between sections
61- Keep explanations clear and concise
62
63## Language Support Details
64
65| Language | Config Files | Entry Points | LSP Server | Installation |
66|----------|--------------|--------------|------------|--------------|
67| TypeScript/JS | `package.json`, `tsconfig.json` | `index.ts`, `main.ts`, `App.tsx` | `typescript-language-server` | `npm i -g typescript-language-server typescript` |
68| Python | `pyproject.toml`, `setup.py`, `requirements.txt` | `main.py`, `app.py`, `__main__.py` | `pylsp` | `pip install python-lsp-server` |
69| Go | `go.mod` | `main.go`, `cmd/*/main.go` | `gopls` | `go install golang.org/x/tools/gopls@latest` |
70| Rust | `Cargo.toml` | `main.rs`, `lib.rs` | `rust-analyzer` | `rustup component add rust-analyzer` |
71| Java | `pom.xml`, `build.gradle` | `Main.java`, `Application.java` | `jdtls` | Eclipse JDT Language Server |
72| C/C++ | `CMakeLists.txt`, `Makefile` | `main.c`, `main.cpp` | `clangd` | `apt install clangd` or `brew install llvm` |
73
74## Fallback Analysis
75
76When LSP is unavailable, the scripts use fallback analyzers:
77
781. **AST Parsing** - For Python (`ast` module) and JavaScript (regex-based)
792. **Regex Patterns** - For function/class detection in other languages
803. **Import Analysis** - Parse import statements for dependency graphs
81
82Fallback provides ~70% of LSP accuracy but works without server setup.
83
84## Troubleshooting
85
86### LSP Server Not Found
87
88```
89Error: Could not start LSP server for TypeScript
90```
91
92**Solution:** Install the required LSP server globally. See Installation column in Language Support table.
93
94### Timeout During Analysis
95
96```
97Error: LSP request timed out
98```
99
100**Solutions:**
101- Increase timeout: `--timeout 120`
102- Reduce scope: `--path src/` instead of entire project
103- Check if LSP server is overloaded (large codebase)
104
105### Empty or Incomplete Output
106
107**Possible causes:**
108- Project has no recognizable entry points
109- LSP server crashed during analysis
110- File permissions prevent reading
111
112**Solutions:**
113- Specify entry point: `--entry src/main.py`
114- Check LSP server logs
115- Run with `--verbose` for debug output
116
117### Mermaid Diagrams Not Rendering
118
119Mermaid diagrams require a compatible viewer:
120- GitHub/GitLab markdown preview
121- VS Code with Mermaid extension
122- Mermaid Live Editor: https://mermaid.live
123
124## Performance Considerations
125
126| Project Size | Files | Estimated Time | Memory |
127|--------------|-------|----------------|--------|
128| Small | < 50 | < 30s | ~100MB |
129| Medium | 50-500 | 1-5 min | ~500MB |
130| Large | 500-5000 | 5-30 min | 1-2GB |
131| Very Large | > 5000 | Consider splitting | 2GB+ |
132
133**Tips for large codebases:**
134- Analyze specific directories: `--path src/core`
135- Skip test files: `--exclude "**/test/**"`
136- Use incremental analysis if available
137- Consider generating separate docs per module
138
139## Known Limitations
140
141- **Monorepos**: May need per-package analysis
142- **Dynamic imports**: Not fully detected (Python `importlib`, JS `import()`)
143- **Metaprogramming**: Dynamically generated code may be missed
144- **Private symbols**: Some LSP servers don't expose private members
145- **Cross-language projects**: Analyze each language separately
146
147## Best Practices
148
1491. **Run from project root** - Ensures correct path resolution
1502. **Commit WIKI.md** - Track documentation changes with code
1513. **Regenerate periodically** - Keep docs in sync with codebase
1524. **Review output** - AI-generated docs may need human refinement
1535. **Customize template** - Adapt `wiki-template.md` to your needs
majiayu000