How To Build A Graph Database With Flutter

How to build a graph database with Flutter

In this article I will go over how to create and use a graph database with Flutter.

TLDR The final source here and an online demo.

Prerequisites

Flutter installed and setup (Refer to this article if you need help).

Basic knowledge of SQLite.

Basic knowledge of Graph Databases (Refer to this video if you need to learn more).

Overview

First of all, why do we need a graph database when other storage options exist?

Why not use key value stores, document stores, or relational databases?

Well, the answer is that it depends on the problem you are trying to solve.

Graph databases are great for modeling relationships between data.

A couple examples:

• A social network app can model the relationships between users and posts
• A game can model the relationships between players and items
• A blog can model the relationships between posts and comments

The possibilities are endless.

Instead of storing data in a table for each collection we store the data as a graph in a nodes and edges table with some additional extensions in SQLite to make it easier.

Here is a page that goes in to detail about it and showcases what we are trying to build.

Getting Started

First we need to create a new Flutter project.

mkdir flutter_graph_database
cd flutter_graph_database
flutter create .

After the project is created open it in your favorite code editor.

code .

Creating the Database

We are going to use the drift package to create the database.

Update the pubspec.yaml file with the following:

name: flutter_graph_database
description: A new Flutter package project.
version: 0.0.1
publish_to: none

environment:
  sdk: ">=2.19.0-238.0.dev <3.0.0"
  flutter: ">=1.17.0"

dependencies:
  flutter:
    sdk: flutter
  drift: ^2.1.0
  sqlite3_flutter_libs: ^0.5.5
  http: ^0.13.5
  path_provider: ^2.0.0
  path: ^1.8.2
  sqlite3: ^1.7.0

dev_dependencies:
  flutter_test:
    sdk: flutter
  flutter_lints: ^2.0.0
  build_runner: ^2.2.0
  drift_dev: ^2.1.0

flutter:

Database Connection

Next we need to create the database.

Create a new file at lib/database/connection/unsupported.dart and update it with the following:

import 'package:drift/drift.dart';
import 'package:drift/native.dart';

DatabaseConnection connect(
  String dbName, {
  bool useWebWorker = false,
  bool logStatements = false,
}) {
  return DatabaseConnection(NativeDatabase.memory(
    logStatements: logStatements,
  ));
}

Create a new file at lib/database/connection/native.dart and update it with the following:

import 'dart:io';
import 'dart:isolate';

import 'package:drift/drift.dart';
import 'package:drift/isolate.dart';
import 'package:drift/native.dart';
import 'package:path_provider/path_provider.dart';
import 'package:path/path.dart' as p;

DatabaseConnection connect(
  String dbName, {
  bool useWebWorker = false,
  bool logStatements = false,
}) {
  return DatabaseConnection.delayed(Future.sync(() async {
    final appDir = await getApplicationDocumentsDirectory();
    final dbPath = p.join(appDir.path, dbName);

    final receiveDriftIsolate = ReceivePort();
    await Isolate.spawn(_entrypointForDriftIsolate,
        _IsolateStartRequest(receiveDriftIsolate.sendPort, dbPath));

    final driftIsolate = await receiveDriftIsolate.first as DriftIsolate;
    return driftIsolate.connect();
  }));
}

class _IsolateStartRequest {
  final SendPort talkToMain;
  final String databasePath;

  _IsolateStartRequest(this.talkToMain, this.databasePath);
}

void _entrypointForDriftIsolate(_IsolateStartRequest request) {
  final databaseImpl = NativeDatabase(
    File(request.databasePath),
    logStatements: false,
  );

  final driftServer = DriftIsolate.inCurrent(
    () => DatabaseConnection(databaseImpl),
  );

  request.talkToMain.send(driftServer);
}

Create a new file at lib/database/connection/web.dart and update it with the following:

import 'dart:async';

// ignore: avoid_web_libraries_in_flutter
import 'dart:html';

import 'package:drift/drift.dart';
import 'package:drift/remote.dart';
import 'package:drift/web.dart';
import 'package:drift/wasm.dart';
import 'package:http/http.dart' as http;
import 'package:sqlite3/wasm.dart';

DatabaseConnection connect(
  String dbName, {
  bool useWebWorker = false,
  bool logStatements = false,
}) {
  if (useWebWorker) {
    final worker = SharedWorker('shared_worker.dart.js');
    return remote(worker.port!.channel());
  } else {
    return DatabaseConnection.delayed(Future.sync(() async {
      final response = await http.get(Uri.parse('sqlite3.wasm'));
      final fs = await IndexedDbFileSystem.open(dbName: '/db/');
      final path = '/drift/db/$dbName';
      final sqlite3 = await WasmSqlite3.load(
        response.bodyBytes,
        SqliteEnvironment(fileSystem: fs),
      );
      final databaseImpl = WasmDatabase(
        sqlite3: sqlite3,
        path: path,
        fileSystem: fs, // <- this is required but not documented
        logStatements: logStatements,
      );
      return DatabaseConnection(databaseImpl);
    }));
  }
}

Create a new file at lib/database/connection/connection.dart and update it with the following:

export 'unsupported.dart'
    if (dart.library.js) 'web.dart'
    if (dart.library.ffi) 'native.dart';

Database SQL Files

Schema

Create a new file at lib/database/sql/schema.drift and update it with the following:

CREATE TABLE IF NOT EXISTS nodes (
    body TEXT,
    id   TEXT GENERATED ALWAYS AS (json_extract(body, '$.id')) VIRTUAL NOT NULL UNIQUE
);

CREATE INDEX IF NOT EXISTS id_idx ON nodes(id);

CREATE TABLE IF NOT EXISTS edges (
    source     TEXT,
    target     TEXT,
    properties TEXT,
    UNIQUE(source, target, properties) ON CONFLICT REPLACE,
    FOREIGN KEY(source) REFERENCES nodes(id),
    FOREIGN KEY(target) REFERENCES nodes(id)
);

CREATE INDEX IF NOT EXISTS source_idx ON edges(source);
CREATE INDEX IF NOT EXISTS target_idx ON edges(target);

The ID column is a virtual column that is generated from the body column. This is done so that we can query the database by ID without having to parse the JSON body column.

Queries

Create a new file at lib/database/sql/queries.drift and update it with the following:

import 'schema.drift';

getAllNodes: 
    SELECT * FROM nodes;
getAllEdges: 
    SELECT * FROM edges;

Delete Edge

Create a new file at lib/database/sql/delete-edge.drift and update it with the following:

import 'schema.drift';

deleteEdge:
    DELETE FROM edges 
    WHERE source = ? OR target = ?;

Delete Node

Create a new file at lib/database/sql/delete-node.drift and update it with the following:

import 'schema.drift';

deleteNode: 
    DELETE FROM nodes 
    WHERE id = ?;

Insert Edge

Create a new file at lib/database/sql/insert-edge.drift and update it with the following:

import 'schema.drift';

insertEdge(:source as TEXT, :target as TEXT, :body as TEXT): 
    INSERT INTO edges VALUES(:source, :target, json(:body));

Search Edges Inbound

Create a new file at lib/database/sql/search-edges-inbound.drift and update it with the following:

import 'schema.drift';

searchEdgesInbound: 
    SELECT * FROM edges 
    WHERE source = ?;

Search Edges Outbound

Create a new file at lib/database/sql/search-edges-outbound.drift and update it with the following:

import 'schema.drift';

searchEdgesOutbound: 
    SELECT * FROM edges 
    WHERE target = ?;

Search Edges

Create a new file at lib/database/sql/search-edges.drift and update it with the following:

import 'schema.drift';

searchEdges: 
    SELECT * FROM edges WHERE source = ? 
    UNION 
    SELECT * FROM edges WHERE target = ?;

Search Node By ID

Create a new file at lib/database/sql/search-node-by-id.drift and update it with the following:

import 'schema.drift';

searchNodeById: 
    SELECT body FROM nodes 
    WHERE id = ?;

Search Node

Create a new file at lib/database/sql/search-node.drift and update it with the following:

import 'schema.drift';

-- Create a text index of entries, see https://www.sqlite.org/fts5.html#external_content_tables
CREATE VIRTUAL TABLE node_entries USING fts5 (
    body,
    content=nodes,
    content_rowid=id
);

-- Triggers to keep entries and fts5 index in sync.
CREATE TRIGGER nodes_insert AFTER INSERT ON nodes BEGIN
  INSERT INTO node_entries(rowid, body) VALUES (new.id, new.body);
END;

CREATE TRIGGER nodes_delete AFTER DELETE ON nodes BEGIN
  INSERT INTO node_entries(node_entries, rowid, body) VALUES ('delete', old.id, old.body);
END;

CREATE TRIGGER nodes_update AFTER UPDATE ON nodes BEGIN
  INSERT INTO node_entries(node_entries, rowid, body) VALUES ('delete', new.id, new.body);
  INSERT INTO node_entries(rowid, body) VALUES (new.id, new.body);
END;

-- Full text search query.
searchNode: SELECT r.** FROM node_entries
    INNER JOIN nodes r ON r.id = node_entries.rowid
    WHERE node_entries MATCH :query
    ORDER BY rank;

Here we are using the fts5 extension to create a full text search index. This is a very powerful feature that allows us to search for nodes by their body text.

Traverse Inbound

Create a new file at lib/database/sql/traverse-inbound.drift and update it with the following:

import 'schema.drift';

traverseInbound(:source AS TEXT): 
  WITH RECURSIVE traverse(id) AS (
  SELECT :source
  UNION
  SELECT source FROM edges JOIN traverse ON target = id
) SELECT id FROM traverse;

Traverse Outbound

Create a new file at lib/database/sql/traverse-outbound.drift and update it with the following:

import 'schema.drift';

traverseOutbound(:source AS TEXT): 
  WITH RECURSIVE traverse(id) AS (
  SELECT :source
  UNION
  SELECT target FROM edges JOIN traverse ON source = id
) SELECT id FROM traverse;

Traverse Bodies Inbound

Create a new file at lib/database/sql/traverse-with-bodies-inbound.drift and update it with the following:

import 'schema.drift';

traverseWithBodiesInbound(:source AS TEXT): 
  WITH RECURSIVE traverse(x, y, obj) AS (
  SELECT :source, '()', '{}'
  UNION
  SELECT id, '()', body FROM nodes JOIN traverse ON id = x
  UNION
  SELECT source, '<-', properties FROM edges JOIN traverse ON target = x
) SELECT x, y, obj FROM traverse;

Traverse Bodies Outbound

Create a new file at lib/database/sql/traverse-with-bodies-outbound.drift and update it with the following:

import 'schema.drift';

traverseWithBodiesOutbound(:source AS TEXT): 
  WITH RECURSIVE traverse(x, y, obj) AS (
  SELECT :source, '()', '{}'
  UNION
  SELECT id, '()', body FROM nodes JOIN traverse ON id = x
  UNION
  SELECT target, '->', properties FROM edges JOIN traverse ON source = x
) SELECT x, y, obj FROM traverse;

Traverse Bodies

Create a new file at lib/database/sql/traverse-bodies.drift and update it with the following:

import 'schema.drift';

traverseWithBodies(:source AS TEXT): 
  WITH RECURSIVE traverse(x, y, obj) AS (
  SELECT :source, '()', '{}'
  UNION
  SELECT id, '()', body FROM nodes JOIN traverse ON id = x
  UNION
  SELECT source, '<-', properties FROM edges JOIN traverse ON target = x
  UNION
  SELECT target, '->', properties FROM edges JOIN traverse ON source = x
) SELECT x, y, obj FROM traverse;

Traverse

Create a new file at lib/database/sql/traverse.drift and update it with the following:

import 'schema.drift';

traverse(:source AS TEXT): 
  WITH RECURSIVE traverse(id) AS (
  SELECT :source
  UNION
  SELECT source FROM edges JOIN traverse ON target = id
  UNION
  SELECT target FROM edges JOIN traverse ON source = id
) SELECT id FROM traverse;

Update Node

Create a new file at lib/database/sql/update-node.drift and update it with the following:

import 'schema.drift';

updateNode: 
    UPDATE nodes SET body = json(?) 
    WHERE id = ?;

Database Setup

Create a new file at lib/database/database.dart and update it with the following:

import 'dart:convert';

import 'package:drift/drift.dart';
import 'package:flutter/foundation.dart';

import 'connection/connection.dart' as impl;

part 'database.g.dart';

@DriftDatabase(include: {
  'sql/schema.drift',
  'sql/queries.drift',
  'sql/delete-edge.drift',
  'sql/delete-node.drift',
  'sql/insert-edge.drift',
  'sql/insert-node.drift',
  'sql/search-edges-inbound.drift',
  'sql/search-edges-outbound.drift',
  'sql/search-edges.drift',
  'sql/search-node-by-id.drift',
  'sql/search-node.drift',
  'sql/traverse-inbound.drift',
  'sql/traverse-outbound.drift',
  'sql/traverse-with-bodies-inbound.drift',
  'sql/traverse-with-bodies-outbound.drift',
  'sql/traverse-with-bodies.drift',
  'sql/traverse.drift',
  'sql/update-node.drift',
})
class GraphDatabase extends _$GraphDatabase {
  GraphDatabase({
    String dbName = 'graph_db.db',
    DatabaseConnection? connection,
    bool useWebWorker = false,
    bool logStatements = false,
  }) : super.connect(
          connection ??
              impl.connect(
                dbName,
                useWebWorker: useWebWorker,
                logStatements: logStatements,
              ),
        );

  @override
  int get schemaVersion => 1;

  /// Helper method to add graph data from json
  Future<void> addGraphData(
    Map<String, dynamic> data, {
    bool shouldBatch = false,
  }) {
    return transaction(() async {
      try {
        final localNodes = data['nodes'] as List<dynamic>;
        final localEdges = data['edges'] as List<dynamic>;
        // Update nodes
        for (final node in localNodes) {
          final id = node['id'] as String?;
          if (id != null) {
            final current = await searchNodeById(id).getSingleOrNull();
            final body = jsonEncode(node);
            if (current != null) {
              await updateNode(id, body);
            } else {
              await insertNode(body);
            }
          }
        }
        // Update edges
        for (final edge in localEdges) {
          final source = edge['from'] ?? edge['source'] as String?;
          final target = edge['to'] ?? edge['target'] as String?;
          if (source != null && target != null) {
            final body = jsonEncode(edge);
            await insertEdge(source, target, body);
          }
        }
      } catch (e) {
        debugPrint('Error adding graph data: $e');
      }
    });
  }

  Future<void> deleteAll() {
    return transaction(() async {
      try {
        await deleteAllEdges();
        await deleteAllNodes();
      } catch (e) {
        debugPrint('Error clearing graph data: $e');
      }
    });
  }

  Future<void> deleteAllEdges() {
    return transaction(() async {
      final edges = await getAllEdges().get();
      for (final edge in edges) {
        await deleteEdge(edge.source, edge.target);
      }
    });
  }

  Future<void> deleteAllNodes() {
    return transaction(() async {
      final nodes = await getAllNodes().get();
      for (final node in nodes) {
        await deleteNode(node.id);
      }
    });
  }
}

Create a new file at build.yaml and update it with the following:

targets:
  $default:
    sources:
      - lib/**
      - web/**
      - "tool/**"
      - pubspec.yaml
      - lib/$lib$
      - $package$
    builders:
      drift_dev:
        options:
          sql:
            dialect: sqlite
            options:
              version: "3.38"
              modules:
                - json1
                - fts5
          generate_connect_constructor: true
          apply_converters_on_variables: true
          generate_values_in_copy_with: true
          scoped_dart_components: true

Now run the following command to generate the database files:

flutter pub run build_runner build --delete-conflicting-outputs

Connecting to the Database

Add a new dependency to your pubspec.yaml file:

flutter pub add graphview

This will be used for the graph visualization.

Create a new file at lib/main.dart and update it with the following:

import 'dart:convert';

import 'package:flutter/material.dart';
import 'package:flutter_graph_database/flutter_graph_database.dart' as db;
import 'package:graphview/GraphView.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Graph Database',
      debugShowCheckedModeBanner: false,
      theme: ThemeData.dark(),
      home: const Example(),
    );
  }
}

class Example extends StatefulWidget {
  const Example({Key? key}) : super(key: key);

  @override
  State<Example> createState() => _ExampleState();
}

class _ExampleState extends State<Example> {
  final database = db.GraphDatabase();
  Graph graph = Graph();
  Algorithm builder = FruchtermanReingoldAlgorithm();

  final nodes = <String, db.Node>{};
  bool loaded = false;

  @override
  void initState() {
    super.initState();
    WidgetsBinding.instance.addPostFrameCallback((_) => loadData());
  }

  @override
  void reassemble() {
    super.reassemble();
    // Needed to reset graph on hot reload
    loadData();
  }

  void setLoadedState(bool value) {
    if (mounted) {
      setState(() {
        loaded = value;
      });
    }
  }

  Future<void> addDummyData() async {
    // Load example data
    try {
      // Optionally reset data
      await database.deleteAll();
      // Add example data to database
      await database.addGraphData({
        "nodes": [
          {"id": '1', "label": 'circle'},
          {"id": '2', "label": 'ellipse'},
          {"id": '3', "label": 'database'},
          {"id": '4', "label": 'box'},
          {"id": '5', "label": 'diamond'},
          {"id": '6', "label": 'dot'},
          {"id": '7', "label": 'square'},
          {"id": '8', "label": 'triangle'},
          {"id": '9', "label": "star"},
        ],
        "edges": [
          {"from": '1', "to": '2'},
          {"from": '2', "to": '3'},
          {"from": '2', "to": '4'},
          {"from": '2', "to": '5'},
          {"from": '5', "to": '6'},
          {"from": '5', "to": '7'},
          {"from": '6', "to": '8'},
          {"from": '2', "to": '8'},
          {"from": '1', "to": '8'},
          {"from": '1', "to": '7'},
          {"from": '1', "to": '6'},
          {"from": '1', "to": '5'},
          {"from": '1', "to": '4'},
          {"from": '1', "to": '3'},
          {"from": '1', "to": '9'},
          {"from": '9', "to": '8'},
          {"from": '9', "to": '5'},
          {"from": '9', "to": '3'},
        ]
      });
      loadData();
    } catch (e) {
      debugPrint('Error loading example data: $e');
    }
  }

  Future<void> loadData() async {
    setLoadedState(false);

    final nodeMap = <String, Node>{};
    this.nodes.clear();
    graph = Graph();
    builder = FruchtermanReingoldAlgorithm();

    // Load graph data
    final nodes = await database.getAllNodes().get();
    final edges = await database.getAllEdges().get();

    for (final node in nodes) {
      final newNode = Node.Id(node.id);
      nodeMap[node.id] = newNode;
      this.nodes[node.id] = node;
      graph.addNode(newNode);
    }
    for (final edge in edges) {
      final source = nodeMap[edge.source];
      final target = nodeMap[edge.target];
      if (source != null && target != null) {
        graph.addEdge(source, target);
      }
    }

    setLoadedState(true);
  }

  Widget buildNode(Node node) {
    final dbNode = nodes[node.key!.value];
    final data = jsonDecode(dbNode?.body ?? '{}') as Map<String, dynamic>;
    final label = data['label'] ?? '';
    return SizedBox(
      width: 80,
      height: 80,
      child: Center(
        child: Text(
          label,
          textAlign: TextAlign.center,
        ),
      ),
    );
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('Flutter Graph Database'),
        actions: [
          IconButton(
            icon: const Icon(Icons.restore),
            onPressed: addDummyData,
          ),
        ],
      ),
      body: !loaded
          ? const Center(child: CircularProgressIndicator())
          : nodes.isEmpty
              ? const Center(child: Text('No Data Loaded'))
              : LayoutBuilder(builder: (context, dimens) {
                  return SizedBox.expand(
                    child: InteractiveViewer(
                      constrained: false,
                      boundaryMargin: EdgeInsets.symmetric(
                        horizontal: dimens.maxWidth * 0.75,
                        vertical: dimens.maxHeight * 0.75,
                      ),
                      minScale: 0.01,
                      maxScale: 5.6,
                      child: GraphView(
                        key: UniqueKey(),
                        graph: graph,
                        algorithm: builder,
                        paint: Paint()
                          ..color = Colors.green
                          ..strokeWidth = 1
                          ..style = PaintingStyle.stroke,
                        builder: buildNode,
                      ),
                    ),
                  );
                }),
    );
  }
}

When you run the flutter app you should see the following:

Conclusion

If you want to learn more about building a graph database in Flutter, check out the source code.