1---2name: virtuoso-rdf-bulk-loader3description: Generate and run Virtuoso isql bulk-load scripts for RDF directories. Takes a source folder plus optional graph IRI, file pattern/formats, and load mode (directory-wide via ld_dir + rdf_loader_run, or per-file named graphs via TTLP_MT with an auto prefix preamble), and emits a ready-to-run .sql. Use when bulk-loading RDF files from a folder into a Virtuoso quad store via isql, regenerating loader scripts after store changes, or simplifying repeated folder loads.4---56# Virtuoso RDF Bulk Loader — isql loader generation from a source folder78Generate a ready-to-run isql `.sql` from a local folder of RDF files. Two modes:910| Mode | Mechanism | Use for |11|---|---|---|12| `dir` (default) | `ld_dir` + `rdf_loader_run` — one named graph for the whole folder; Virtuoso auto-detects format (N-Triples, Turtle, RDF/XML, N-Quads, TriG, JSON-LD, Notation3) and reads gzip/bzip2 natively | Simple bulk ingestion, mixed formats, compressed archives |13| `per-file` | `TTLP_MT` — one named graph per file (Turtle family: `.ttl`, `.nt`, `.trig`), optional auto prefix-preamble and CLEAR-before-load | Exact per-document graphs (e.g. an agent memory store), re-loadable deltas |1415## Workflow16171. **Elicit** (never guess): source folder (must be accessible to the **Virtuoso server process** — `ld_dir` registers server-side paths), target graph IRI (dir mode), file pattern or format list.182. **Generate**:19 ```bash20 python3 scripts/generate-bulk-load-sql.py \21 --source-dir /path/to/rdf/ \22 --graph https://example.com/my-graph \23 --pattern '*.ttl' \24 --out bulk-load.sql25 ```263. **Run** (your credentials, never displayed):27 ```bash28 isql 1111 dba <dba-password> -f bulk-load.sql29 ```304. **Monitor** (the generated script ends with the load_list check):31 ```sql32 select * from db.dba.load_list where ll_state <> 2;33 ```34 `ll_state 2` = done; rows remaining = still pending/errored (`ll_error` column).355. **Verify** (generated script ends with per-graph counts):36 ```sql37 SPARQL SELECT ?g (COUNT(*) AS ?t) WHERE { GRAPH ?g { ?s ?p ?o } } GROUP BY ?g;38 ```3940## Options (`generate-bulk-load-sql.py`)4142| Flag | Default | Meaning |43|---|---|---|44| `--source-dir` | *(required)* | Folder containing the RDF files (server-visible path) |45| `--graph` | `urn:dav:/DAV/home/kidehen/bulk-load/` | Target named graph (dir mode) |46| `--pattern` | `*.*` | `ld_dir` pattern, e.g. `*.ttl`, `*.nt.gz`, `*.nq` |47| `--formats` | — | Shorthand expanding to patterns, e.g. `ttl,nt,nq` |48| `--mode` | `dir` | `dir` (ld_dir) or `per-file` (TTLP_MT per document) |49| `--graph-base` | `urn:dav:/DAV/home/kidehen/bulk-load/` | IRI prefix for per-file graphs (file relpath appended) |50| `--preamble` | off | per-file mode: prepend store-wide prefix union so prefix-undeclared Turtle loads |51| `--clear` | off | Emit `SPARQL CLEAR GRAPH` before loading (idempotent re-runs) |52| `--out` | `bulk-load.sql` | Output path |53| `--dry-run` | off | Print the script instead of writing |5455## Notes5657- **Server-side paths**: `ld_dir` paths are read by the Virtuoso server, not your client. For a remote Virtuoso, stage files on the server first.58- **Idempotency**: `ld_dir` does not double-load identical files; `rdf_loader_run()` may be re-run safely. `--clear` makes full re-runs replace the graph.59- **Troubleshooting**: load failures, permission issues, DAV paths, and named-graph management — see the `virtuoso-rdf-loader` skill (ld_dir + rdf_loader_run deep-dive).60- This skill is the generator generalization of the agent-rdf-memory loader (`agent-rdf-memory/scripts/generate-loader-sql.py`); that store's `refresh-loader.sh` is the same pattern in action.
OpenLinkSoftware