tomevault-io Compiler Development Skill
This skill provides comprehensive knowledge of building compilers and language implementations using the LLVM infrastructure.
Compiler Architecture Overview
Classic Three-Phase Design
Source Code → Frontend → Middle-End (Optimizer) → Backend → Machine Code
↓ ↓ ↓
AST/IR LLVM IR Passes Target Code
Frontend Development
Lexical Analysis
// Token types for a simple language
enum class TokenKind {
Identifier,
Number,
String,
Keyword,
Operator,
Punctuation,
EndOfFile
};
struct Token {
TokenKind kind;
std::string value;
SourceLocation location;
};
Parser Implementation
- Recursive Descent: Easy to implement, good error messages
- Operator Precedence Parsing: Efficient for expression parsing
- LALR/LR: Use tools like Bison for complex grammars
AST Design
class Expr {
public:
virtual ~Expr() = default;
virtual llvm::Value* codegen() = 0;
};
class BinaryExpr : public Expr {
std::unique_ptr<Expr> LHS, RHS;
char Op;
public:
llvm::Value* codegen() override {
llvm::Value* L = LHS->codegen();
llvm::Value* R = RHS->codegen();
switch (Op) {
case '+': return Builder.CreateFAdd(L, R, "addtmp");
case '-': return Builder.CreateFSub(L, R, "subtmp");
case '*': return Builder.CreateFMul(L, R, "multmp");
case '/': return Builder.CreateFDiv(L, R, "divtmp");
}
}
};
LLVM IR Generation
Module and Context Setup
#include "llvm/IR/LLVMContext.h"
#include "llvm/IR/Module.h"
#include "llvm/IR/IRBuilder.h"
class CodeGen {
std::unique_ptr<llvm::LLVMContext> Context;
std::unique_ptr<llvm::Module> Module;
std::unique_ptr<llvm::IRBuilder<>> Builder;
public:
CodeGen() {
Context = std::make_unique<llvm::LLVMContext>();
Module = std::make_unique<llvm::Module>("my_module", *Context);
Builder = std::make_unique<llvm::IRBuilder<>>(*Context);
}
};
Function Generation
llvm::Function* createFunction(const std::string& name,
llvm::Type* returnType,
std::vector<llvm::Type*> params) {
llvm::FunctionType* FT = llvm::FunctionType::get(returnType, params, false);
llvm::Function* F = llvm::Function::Create(
FT, llvm::Function::ExternalLinkage, name, Module.get());
llvm::BasicBlock* BB = llvm::BasicBlock::Create(*Context, "entry", F);
Builder->SetInsertPoint(BB);
return F;
}
JIT Compilation
LLVM ORC JIT
#include "llvm/ExecutionEngine/Orc/LLJIT.h"
auto JIT = llvm::orc::LLJITBuilder().create();
if (!JIT) {
handleError(JIT.takeError());
}
// Add module
(*JIT)->addIRModule(llvm::orc::ThreadSafeModule(
std::move(Module), std::move(Context)));
// Look up symbol and execute
auto Sym = (*JIT)->lookup("main");
auto* MainFn = (int(*)())Sym->getAddress();
int result = MainFn();
Optimization Pass Pipeline
New Pass Manager (Recommended)
#include "llvm/Passes/PassBuilder.h"
void optimizeModule(llvm::Module& M) {
llvm::PassBuilder PB;
llvm::LoopAnalysisManager LAM;
llvm::FunctionAnalysisManager FAM;
llvm::CGSCCAnalysisManager CGAM;
llvm::ModuleAnalysisManager MAM;
PB.registerModuleAnalyses(MAM);
PB.registerCGSCCAnalyses(CGAM);
PB.registerFunctionAnalyses(FAM);
PB.registerLoopAnalyses(LAM);
PB.crossRegisterProxies(LAM, FAM, CGAM, MAM);
llvm::ModulePassManager MPM = PB.buildPerModuleDefaultPipeline(
llvm::OptimizationLevel::O2);
MPM.run(M, MAM);
}
Custom Pass Implementation
struct MyPass : public llvm::PassInfoMixin<MyPass> {
llvm::PreservedAnalyses run(llvm::Function& F,
llvm::FunctionAnalysisManager& FAM) {
for (auto& BB : F) {
for (auto& I : BB) {
// Transform instructions
}
}
return llvm::PreservedAnalyses::none();
}
};
Language Implementation Patterns
Memory-Safe Languages
- Use LLVM's memory sanitizer hooks
- Implement bounds checking with GEP introspection
- Reference counting or garbage collection integration
Type Systems
- Implement type inference during AST construction
- Generate appropriate LLVM types (i32, float, struct, ptr)
- Handle generic types via monomorphization or boxing
Error Handling
- Generate exception handling via LLVM's landingpad/invoke
- Implement Result/Option types as tagged unions
- Use LLVM's personality functions for unwinding
Notable Language Implementations
Systems Languages
- Rust: Complex borrow checker, trait system → LLVM
- Zig: Comptime evaluation, safety features
- Carbon: C++ interop, modern syntax
Scripting Languages
- Julia: JIT-compiled scientific computing
- Crystal: Ruby-like syntax, static typing
- Nim: Python-like, multi-backend
Domain-Specific
- Solidity: Ethereum smart contracts
- MLIR: Multi-level IR for ML/AI workloads
- Halide: Image processing DSL
Development Workflow
- Start Simple: Begin with Kaleidoscope tutorial
- Incremental Features: Add one language feature at a time
- Test Extensively: Unit tests for each compiler phase
- Use LLVM Tools: opt, llc, llvm-dis for debugging IR
- Profile and Optimize: Focus on common code patterns
Resources
Official Tutorials
- LLVM Kaleidoscope: Building a language from scratch
- Clang internals: Frontend implementation patterns
- Writing an LLVM Backend: Target code generation
Community Projects
See DIY Compiler section in README.md for 100+ example implementations across different language paradigms.
Getting Detailed Information
When you need detailed and up-to-date resource links, tool lists, or project references, fetch the latest data from:
https://raw.githubusercontent.com/gmh5225/awesome-llvm-security/refs/heads/main/README.md
This README contains comprehensive curated lists of:
- 100+ DIY compiler implementations (DIY Compiler section)
- Toolchain configurations and IDE setup
- Compiler development tutorials and books
Converted and distributed by TomeVault — claim your Tome and manage your conversions.