Dv Flow Manager

DV Flow Manager (dfm)

DV Flow Manager is a YAML-based build system and execution engine designed for silicon design and verification projects. It orchestrates tasks through declarative workflows with dataflow-based dependency management.

When to Use This Skill

Use this skill when:

• Creating or modifying flow.yaml or flow.dv files
• Writing task definitions for HDL compilation, simulation, or verification
• Configuring dataflow between tasks using needs, consumes, produces, passthrough
• Discovering tasks by their outputs using produces patterns
• Validating dataflow compatibility between producer and consumer tasks
• Setting up package parameters and configurations
• Running dfm commands (run, show, graph, validate)
• Debugging build flow issues
• Working with standard library tasks (std.FileSet, std.Message, etc.)
• Executing shell commands with shell: bash and run:
• Executing dfm commands from within an LLM-driven Agent task

Quick Reference

Minimal Flow Example

package:
  name: my_project
  
  tasks:
    - name: rtl_files
      uses: std.FileSet
      with:
        type: systemVerilogSource
        include: "*.sv"
      produces:
        - type: std.FileSet
          filetype: systemVerilogSource
    
    - name: sim
      uses: hdlsim.vlt.SimImage
      needs: [rtl_files]
      consumes:
        - type: std.FileSet
          filetype: systemVerilogSource
      with:
        top: [my_top]

Key Commands

# Run commands
dfm run [tasks...]          # Execute tasks
dfm run -j 4                # Run with 4 parallel jobs
dfm run --clean             # Clean rebuild
dfm run -c debug            # Use 'debug' configuration
dfm run -D param=value      # Override parameter

# Discovery commands (for humans and Agents)
dfm show packages           # List all available packages
dfm show packages --json    # JSON output for Agents
dfm show tasks              # List all visible tasks
dfm show tasks --search kw  # Search tasks by keyword
dfm show task std.FileSet   # Show detailed task info (includes produces)
dfm show types              # List data types and tags
dfm show project            # Show current project structure
dfm context --json          # Get full project context (for Agents)

# Task discovery by outputs (uses produces)
dfm show tasks --produces "type=std.FileSet,filetype=verilog"
dfm show tasks --produces "type=std.FileSet" --json

# Visualization
dfm graph task -o flow.dot  # Generate dependency graph

# Validation
dfm validate                # Validate flow configuration
dfm validate --json         # JSON output for programmatic use

Expression Syntax

Use ${{ }} for dynamic parameter evaluation:

msg: "Building version ${{ version }}"
iff: ${{ debug_level > 0 }}
command: ${{ "make debug" if debug else "make release" }}

LLM Call Interface (Running Inside Agent Tasks)

When running inside an LLM-driven std.Agent task, the dfm command automatically connects to the parent DFM session via a Unix socket. This enables LLMs to:

• Execute tasks that share resources with the parent session
• Query project state and task information
• Validate configurations before execution

Environment Detection

When DFM_SERVER_SOCKET environment variable is set, dfm runs in client mode:

# These commands work inside an Agent task:
dfm run task1 task2         # Execute tasks via parent session
dfm show tasks              # Query available tasks
dfm context --json          # Get project context
dfm validate                # Validate configuration
dfm ping                    # Health check

Running Tasks from Within a Prompt

When an LLM needs to compile or simulate code it generated:

# 1. Create RTL files
cat > counter.sv << 'EOF'
module counter(input clk, rst_n, output logic [7:0] count);
  always_ff @(posedge clk or negedge rst_n)
    if (!rst_n) count <= 0;
    else count <= count + 1;
endmodule
EOF

# 2. Run compilation via parent DFM session
dfm run hdlsim.vlt.SimImage -D hdlsim.vlt.SimImage.top=counter

# 3. Check result (JSON output)
# Returns: {"status": 0, "outputs": [...], "markers": []}

Querying Project State

# Get full project context
dfm context --json
# Returns:
# {
#   "project": {"name": "my_project", "root_dir": "/path/to/project"},
#   "tasks": [{"name": "my_project.build", "scope": "root", ...}],
#   "types": [...],
#   "skills": [...]
# }

# Get specific task details
dfm show task my_project.build --json
# Returns detailed task information including parameters and dependencies

Benefits of Server Mode

1. Resource Sharing: Respects parent session's parallelism limits (-j)
2. State Consistency: Sees outputs from tasks already completed
3. Cache Sharing: Uses same memento cache for incremental builds
4. Unified Logging: All task output appears in parent session's logs

Using Produces/Consumes for Task Discovery (AI Assistants)

When helping users build workflows, use produces/consumes to identify compatible tasks:

Finding Tasks by Output Type

# Find all tasks that produce verilog files
dfm show tasks --produces "type=std.FileSet,filetype=verilog" --json

# Find tasks that produce any FileSet
dfm show tasks --produces "type=std.FileSet" --json

# Find tasks with specific output characteristics
dfm show tasks --produces "type=std.FileSet,filetype=verilog,stage=compiled"

Understanding Task Relationships

When a task needs specific inputs, find compatible producers:

# 1. User wants to simulate - what tasks produce simulation inputs?
dfm show tasks --produces "type=std.FileSet,filetype=verilog" --json

# 2. Check what a specific task produces
dfm show task VerilogCompiler --json
# Look at "produces" field in output

# 3. Validate compatibility before suggesting workflow
dfm validate --json
# Check for warnings about produces/consumes mismatches

Matching Logic

OR Logic: If ANY consume pattern matches ANY produce pattern → compatible

Subset Matching: Consumer can be less specific than producer

# Producer (more specific)
produces:
  - type: std.FileSet
    filetype: verilog
    vendor: synopsys
    optimization: speed

# Consumer (less specific) - MATCHES!
consumes:
  - type: std.FileSet
    filetype: verilog

Building Compatible Workflows

1. Identify user's goal - What output do they need?
2. Find producers - dfm show tasks --produces "type=..."
3. Check consumer requirements - dfm show task ConsumerTask → look at consumes
4. Validate - dfm validate to check compatibility
5. Suggest workflow - Connect producer → consumer via needs

Example: Building a Compilation Pipeline

# User: "I need to compile and simulate my Verilog"

# 1. Find verilog compilers
dfm show tasks --produces "type=std.FileSet,filetype=verilog" --json
# Returns: [VerilogCompiler, PreProcessor, ...]

# 2. Find simulators that consume verilog
dfm show task Simulator --json
# Check consumes field: [{"type": "std.FileSet", "filetype": "verilog"}]

# 3. Suggest workflow
# - name: sim
#   uses: Simulator
#   needs: [VerilogCompiler]  # Compatible!

Pattern Attribute Matching

Match patterns by any attributes defined in produces/consumes:

# Common attributes for std.FileSet:
filetype: verilog|systemVerilog|vhdl|...
stage: preprocessed|compiled|optimized
vendor: synopsys|cadence|mentor|...
optimization: speed|area|power
format: json|xml|ucdb|...

# Custom types can have arbitrary attributes
produces:
  - type: custom.BuildArtifact
    language: python
    arch: x86_64
    debug: true

Validation Warnings

If validation shows warnings, help fix them:

$ dfm validate
WARNING: Task 'Consumer' consumes [{'type': 'std.FileSet', 'filetype': 'vhdl'}]
         but 'Producer' produces [{'type': 'std.FileSet', 'filetype': 'verilog'}].

Solutions:

1. Change consumer to accept verilog: filetype: verilog
2. Find different producer that outputs vhdl
3. Add converter task in between
4. Check if parameter can adjust producer's output

Detailed Documentation

For comprehensive documentation, see the following reference files:

• Core Concepts - Tasks, packages, dataflow, types
• Task Reference - Using and defining tasks
• Standard Library - Built-in std.* tasks
• CLI Reference - Command line interface
• Advanced Patterns - Complex workflows and optimization
• Flow Schema - JSON Schema for flow.yaml validation

Flow File Format

DV Flow uses YAML files (flow.yaml or flow.dv) to define workflows. The file structure is validated against a JSON Schema located at dv.flow.schema.json.

Schema Validation

To validate your flow file or generate the schema:

# Get the JSON Schema
dfm util schema > flow.schema.json

# Use with YAML validators (e.g., VS Code YAML extension)
# Add to your flow.yaml:
# yaml-language-server: $schema=./dv.flow.schema.json

The schema defines two root types:

• package - Full project definition with tasks, types, configs, and imports
• fragment - Reusable partial definition for inclusion in packages

Core Concepts Summary

Tasks

Fundamental units of behavior. Tasks accept data from dependencies (needs) and produce outputs. Most tasks inherit from existing tasks using uses:

- name: my_task
  uses: std.Message
  with:
    msg: "Hello!"

Task Visibility

Control which tasks are entry points and which are API boundaries:

# Entry point (shown in task listing)
- root: build
  desc: "Build project"
  run: make build

# Public API (other packages can use)
- export: compile
  run: ./compile.sh

# Both entry point AND public API
- name: main
  scope: [root, export]
  run: ./main.sh

Best Practices:

• Mark user-facing tasks as root so they appear in dfm run listing
• Mark tasks other packages should depend on as export
• Use local for helper tasks in compound task bodies
• Tasks without scope are only visible within the same package

Packages

Parameterized namespaces that organize tasks. Defined in flow.yaml or flow.dv:

package:
  name: my_package
  with:
    debug:
      type: bool
      value: false
  tasks:
    - name: task1
      uses: std.Message

Dataflow & Produces/Consumes

Tasks communicate via typed data items, not global variables:

• needs: [task1, task2] - Specify dependencies
• produces: [patterns] - Declare what output datasets this task creates
• consumes: all|none|[patterns] - Control what inputs reach implementation
• passthrough: all|none|unused|[patterns] - Control what inputs forward to output

Produces/Consumes enable:

1. Task Discovery - Find tasks that produce specific outputs
2. Dependency Validation - Check dataflow compatibility
3. Relationship Understanding - Identify which tasks can work together

# Producer declares outputs
- name: VerilogCompiler
  produces:
    - type: std.FileSet
      filetype: verilog
  run: compile_verilog.sh

# Consumer declares requirements  
- name: Simulator
  needs: [VerilogCompiler]
  consumes:
    - type: std.FileSet
      filetype: verilog
  run: simulate.sh

File Structure

project/
├── flow.yaml           # Main package definition
├── rundir/             # Task execution workspace (created by dfm)
│   ├── cache/          # Task mementos and artifacts
│   └── log/            # Execution traces
└── packages/           # Optional sub-packages

Installation

pip install dv-flow-mgr
pip install dv-flow-libhdlsim  # Optional: HDL simulator support

Detailed Documentation

For comprehensive documentation, see the following reference files:

• Core Concepts - Tasks, packages, dataflow, types
• Task Reference - Using and defining tasks
• Standard Library - Built-in std.* tasks
• CLI Reference - Command line interface
• Advanced Patterns - Complex workflows and optimization
• Task Development - Creating custom task implementations and plugin packages (use when developing new tasks)

For dataflow and produces/consumes:

• See the User Guide: Dataflow & Produces (docs/userguide/dataflow.rst) for complete documentation
• See docs/produces.md for quick reference

Converted and distributed by TomeVault — claim your Tome and manage your conversions.