# Project Structure

> Project layout, module architecture, compiler pipeline, and design patterns.

- Skill: `luisagroup/project-structure` (Agent Skill)
- Install (CLI): `npx skillmds@latest add luisagroup/project-structure`
- Raw SKILL.md: https://api.skillmd.com/api/skills/luisagroup/project-structure/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: luisagroup (https://skillmd.com/u/luisagroup)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/luisagroup/project-structure

---


# LuisaCompute Project Structure

Layered architecture: **Core** → **AST/IR** → **DSL/Runtime** → **Backends**. `src/` + public headers `include/luisa/`. Dual build: CMake + XMake. Frontends: C++, Python, Rust.

## Top-Level Directory Map

```
src/
├── api/          C API & runtime API layer
├── ast/          AST (expressions, statements, types, function builder)
├── backends/     Plugins: CUDA, DX, Metal, CPU, Vulkan, HIP, remote, fallback, common
├── clangcxx/     Clang-based C++→GPU shader compiler (experimental)
├── core/         Foundation: types, math, logging, platform, STL wrappers
├── dsl/          Embedded C++ DSL (kernel/callable lambda tracing)
├── ext/          Third-party deps (git submodules)
├── gui/          Windowing, ImGui, framerate
├── ir/           IR bridge: AST↔IR transforms
├── osl/          Open Shading Language parser
├── py/           Python bindings (pybind11 + pure Python)
├── runtime/      Unified runtime: device, buffer, image, stream, RTX, raster
├── rust/         Rust workspace: IR, CPU backend, remote
├── tensor/       Tensor ops & compute graph
├── tests/        Unit/integration/example tests
├── vstl/         Virtual STL: custom containers, allocators, hashes
└── xir/          Extended IR: SSA, basic blocks, passes, translators

include/luisa/    Public headers mirroring src/ layout (+ ir_v2/)

Root also has: examples/, tests/, tutorials/, utils/, docs/
```

## Modules

### `src/core/` — Foundation
Platform abstractions, math, logging, binary I/O, dynamic modules.
- `basic_types.cpp` — vector/matrix instantiations
- `logging.cpp` — spdlog-based logging
- `platform.cpp` — OS abstraction (paths, threads, DLL)
- `dynamic_module.cpp` — cross-platform shared library loader
- `binary_io.cpp`, `binary_file_stream.cpp` — binary serialization
- `first_fit.cpp`, `pool.cpp`, `string_scratch.cpp` — allocators/scratch buffers
- `stl/` — custom STL: `vector`, `string`, `unordered_map`, `optional`, `variant`, etc.
- `generate_swizzles.py` — swizzle codegen

### `src/vstl/` — Virtual STL
High-perf containers beyond `core/stl`: `stack_allocator`, `string_builder`, `lmdb`, `md5`, `v_guid`. Headers: `include/luisa/vstl/*` (hash maps, arenas, lockfree queues, ranges).

### `src/ast/` — Abstract Syntax Tree
DSL traces C++ lambdas → AST nodes.
- `expression.cpp` — literal, binary, unary, call, swizzle, member
- `statement.cpp` — if, loop, switch, break, return, ray_query
- `type.cpp` — scalars, vectors, matrices, buffers, textures, structs
- `function.cpp` — kernel/callable metadata
- `function_builder.cpp` — manual AST construction API
- `variable.cpp` — local variables
- `op.cpp` — `BinaryOp`, `UnaryOp`, `CallOp`
- `ast2json.cpp` — AST→JSON serialization
- `constant_data.cpp`, `callable_library.cpp`, `external_function.cpp`, `function_duplicator.cpp`, `atomic_ref_node.cpp`

### `src/xir/` — Extended IR (Next-Gen)
SSA IR with basic blocks, instructions, optimization passes. Receives AST via `ast2xir`.
- `instructions/` — 30+ types: arithmetic, memory, control flow, resource, autodiff, atomic
- `passes/` — DCE, mem2reg, SROA, autodiff, outline, dom-tree, GEP tracing, local load/store elimination, ray-query lowering, unused callable removal, LICM, GVN, SCCP, inlining, CFG simplification
- `translators/` — `ast2xir`, `xir2json`, `json2xir`, `xir2text`, `xir2ast`
- `metadata/` — source locations, names, comments, curve basis
- `tests/` — XIR unit tests (enabled by `LUISA_COMPUTE_ENABLE_XIR_TESTS`)
- Key classes: `Module`, `Function`, `BasicBlock`, `Instruction`, `Value`, `Use`, `Builder`

### `src/ir/` — IR Bridge (Legacy)
AST↔IR transforms, high-level transforms: `ast2ir.cpp`, `ir2ast.cpp`, `transform.cpp`.

### `src/dsl/` — Embedded DSL
GPU kernels via lambda tracing.
- `func.cpp` — `Kernel1D/2D/3D`, `Callable`
- `builtin.cpp` — `dispatch_id`, `thread_id`, math
- `resource.cpp` — buffer/image/volume/bindless DSL wrappers
- `local.cpp` — local/thread storage helpers
- `sugar.cpp` — `$if`, `$for`, `$while`
- `rtx/` — ray tracing: `Accel`, `Ray`, `RayQuery`, `Curve`, `TriangleHit`
- `raster/` — `RasterKernel`
- `ext/` — DSL extensions
- `soa.cpp`, `polymorphic.cpp`, `dispatch_indirect.cpp`

### `src/runtime/` — Unified Runtime
Resource management, command scheduling, RHI abstraction.
- `device.cpp`, `context.cpp` — device creation, backend loading
- `stream.cpp`, `command_list.cpp` — command batching/submission
- `buffer.cpp`, `image.cpp`, `volume.cpp` — GPU memory
- `byte_buffer.cpp`, `dispatch_buffer.cpp`, `mipmap.cpp` — auxiliary buffers
- `sparse_buffer.cpp`, `sparse_texture.cpp`, `sparse_heap.cpp`, `sparse_command_list.cpp` — sparse resources
- `bindless_array.cpp`, `swapchain.cpp`, `event.cpp`, `builtin_kernel.cpp`
- `rhi/` — `device_interface.h`, `command.h`, `command_encoder.h`, `resource.h`
- `rtx/` — `accel.cpp`, `mesh.cpp`, `curve.cpp`, `motion_instance.cpp`, `procedural_primitive.cpp`
- `raster/` — `raster.cpp`, `depth_buffer.cpp`
- `remote/` — remote device client/server

### `src/backends/` — Backend Plugins
Dynamically loaded (`luisa-backend-<name>.dll/.so`). Each: codegen (AST/XIR→native) + compiler (NVRTC/DXC/etc.) + resources + command encoder.

| Backend | Technology |
|---|---|
| **CUDA** (`cuda/`) | NVRTC + OptiX + CUDA driver |
| **DirectX** (`dx/`) | DX12 + DXR + HLSL DXC |
| **Metal** (`metal/`) | Metal 3 + MSL |
| **CPU** (`cpu/`) | Rust-based (via `src/rust/`) |
| **Vulkan** (`vk/`) | Vulkan + SPIR-V |
| **HIP** (`hip/`) | AMD HIP |
| **Remote** (`remote/`) | Network-distributed |
| **Fallback** (`fallback/`) | Reference interpreter |
| **Common** (`common/`) | `c_codegen/`, `hlsl/`, `spirv/`, `spirv_llvm/`, Vulkan swapchain helpers |
| **Validation** (`validation/`) | Debug layer |
| **Toy C** (`toy_c/`) | Minimal C codegen |

### `src/rust/` — Rust Workspace
- `luisa_compute_ir` — core IR: AST→IR, analysis, transforms (DCE, inliner, SSA, autodiff, vectorize)
- `luisa_compute_ir_v2` — IR v2 bindings
- `luisa_compute_ir_staticlib` — static library wrapper for C++ linking
- `luisa_compute_backend` — backend proxy/message protocol
- `luisa_compute_backend_impl` — CPU backend: LLVM JIT, C++ codegen, texture sampling, remote backend
- `luisa_compute_cpu_kernel_defs` — CPU kernel ABI definitions
- `luisa_compute_api_types` — C API types
- Built via CMake/Cargo interop, linked into C++

### `src/api/` — C API
Stable C API for language bindings: `runtime.cpp`, `logging.cpp`, Rust binding/RPC generators.

### `src/py/` — Python
- `lcapi.cpp` — pybind11 entry; `export_*.cpp` — per-component bindings
- `luisa/` — pure Python package: `buffer.py`, `accel.py`, `autodiff.py`, `gui.py`, `types.py`
- `interop.cpp/h` — PyTorch/DLPack

### `src/tensor/` — Tensor & Compute Graph
High-level tensor ops, expression DAG, graph passes.
- `fallback/` — CPU kernels (matmul, softmax)
- `pass/` — graph passes

### `src/clangcxx/`, `src/osl/`, `src/gui/`, `src/ext/`
- **clangcxx**: Clang/libTooling C++→GPU compiler (experimental)
- **osl**: OSO bytecode parser for shader interop
- **gui**: Cross-platform windowing + ImGui
- **ext**: git submodules: EASTL, glfw, glslang, imgui, pybind11, spdlog, reproc, SPIRV-Tools, spirv-headers, stb, volk, yyjson, xxhash, marl, half, HIPRT, liblmdb, magic_enum

### `src/tests/`
- `unit/{core,ast,dsl,runtime,ext,xir}/` — unit tests by layer
- `integration/{runtime,ir}/` — cross-cutting integration tests
- `common/` — shared headers (`test_device.h`, `ut/`, asset loaders)
- `cxx_shaders/` — C++ shader tests
- `python/` — Python frontend tests
- `ut/` — extra UT harness directory
- Root: integration tests (`test_path_tracing`, `test_dsl`, `test_rtx`, `test_raster`, `test_tensor`, `test_autodiff`, etc.)

## Build System

- **CMake (primary)**: root + `src/CMakeLists.txt`, targets: `luisa-compute-<name>`, alias: `luisa::compute`. Backends as `MODULE` plugins named `luisa-backend-<name>`. Options: `LUISA_COMPUTE_ENABLE_CUDA|DX|METAL|CPU|VULKAN|HIP|DSL|RUST|TENSOR|GUI|...`
- **XMake (secondary)**: `xmake.lua` in root + `src/` and subdirs
- **Bootstrap**: `bootstrap.py` at repo root
- **IntelliSense**: `update_intellisense.lua`

## Compiler Pipeline

```
DSL Tracing (src/dsl/) → AST (src/ast/)
                              │
                    ┌─────────┴─────────┐
                    ▼                   ▼
              XIR (src/xir/)       IR (src/ir/ → src/rust/)
                    │                   │
                    └─────────┬─────────┘
                              ▼
                    Backend Codegen (src/backends/<name>/)
                              │
                              ▼
                    GPU Execution (src/runtime/)
```

Rust IR path: `luisa_compute_ir` does autodiff, DCE, SSA, vectorize before codegen.

## Key Headers

| Header | Scope |
|---|---|
| `<luisa/luisa-compute.h>` | Core + AST + DSL + Runtime + GUI |
| `<luisa/dsl/syntax.h>` | DSL core |
| `<luisa/dsl/sugar.h>` | Sugar macros |
| `<luisa/runtime/context.h>` | Runtime entry |
| `<luisa/runtime/device.h>` | Device & resources |

## Design Patterns

1. **RHI**: `src/runtime/rhi/` abstracts GPU APIs into common interfaces
2. **Plugin Architecture**: Backends as runtime-loaded dynamic modules
3. **RAII Resources**: Move-only handles (`Buffer`, `Image`, `Stream`, `Accel`)
4. **Command-Based**: Work encoded as `Command` → `CommandList` → `Stream`
5. **DSL Tracing**: Operator overloading + lambda capture builds AST at definition time
6. **Dual IR**: AST (frontend tree) + XIR (SSA backend with optimization passes)
7. **Rust + C++ Hybrid**: IR/CPU backend in Rust; API/DSL in C++

## Naming

| Convention | Example |
|---|---|
| CMake target | `luisa-compute-core` |
| Backend binary | `luisa-backend-cuda` |
| PCH | `lc_core_pch.h` |
| Integration test | `test_path_tracing.cpp` |
| Python export | `export_runtime.cpp` |

## Maintenance

- **New backend**: `src/backends/<name>/`, implement `DeviceInterface`, register in `src/backends/CMakeLists.txt`
- **New XIR pass**: `src/xir/passes/`, register in `src/xir/CMakeLists.txt`
- **New runtime resource**: define in `rhi/resource.h`, implement per-backend, expose in `runtime/` + `include/luisa/runtime/`
- `tensor/` is opt-in, less mature. `clangcxx/` is experimental.