Networkxternal 0 3 0

NetworkXternal 0.3.0

Overview

NetworkXternal provides a NetworkX-like API for graphs persisted in external databases, enabling you to scale from megabyte-sized in-memory graphs to terabyte-petabyte graphs that exceed available RAM — without changing application code. It wraps five database backends (SQLite, PostgreSQL, MySQL, MongoDB, Neo4J) behind a unified BaseAPI abstract class partially compatible with NetworkX's MultiDiGraph.

The library is written in Python (98.6%), uses SQLAlchemy for ORM-based SQL backends, PyMongo for MongoDB, and the official Neo4J Bolt driver for Neo4J. It comes with a performance penalty compared to pure in-memory NetworkX, but provides a practical starting point for database-backed graph applications.

When to Use

• Graphs too large to fit in RAM (terabyte to petabyte scale)
• Persistent graph storage across application restarts
• Multi-edge graphs where edges carry unique identity via key/label
• Migrating from in-memory NetworkX to database-backed storage with minimal code changes
• Applications needing concurrent read access to graph data (__is_concurrent__ = True on SQL, MongoDB, Neo4J backends)

Core Concepts

BaseAPI: Abstract base class defining the graph interface. All backends inherit from it. Designed for directed weighted multigraphs by default. Supports multi-edges (multiple edges between same node pair). Edge IDs can be auto-generated by hashing connected node IDs using the Cantor pairing function.

Node: Dataclass with _id: int, weight: float, label: int, and payload: dict. Non-integer node names are hashed into integer IDs automatically.

Edge: Dataclass with _id: int, first: int, second: int, weight: float, label: int, is_directed: bool, and payload: dict. Supports tuple-like indexing (edge[0] returns first, edge[1] returns second) for NetworkX compatibility. Edge identity is deterministic via Edge.identify_by_members(first, second) using a modular Cantor pairing function.

GraphDegree: Simple dataclass holding count: int and weight: float, returned by reduction operations.

Architecture

The library uses a two-tier architecture:

1. BaseAPI — abstract class defining all graph operations (metadata, bulk reads, random reads/writes, bulk operations)
2. Backend implementations — concrete classes for each database:
   • BaseSQL → SQLite, SQLiteMem, PostgreSQL, MySQL (SQLAlchemy-based)
   • MongoDB (PyMongo-based)
   • Neo4J (Bolt driver + Cypher-based)

SQL backends share a common BaseSQL parent that handles SQLAlchemy session management, ORM mapping (NodeSQL, EdgeSQL), and SQL query generation. MongoDB and Neo4J implement BaseAPI directly with database-specific query languages.

Advanced Topics

Core API Reference: BaseAPI methods, Node/Edge dataclasses, GraphDegree → Core API Reference SQL Backends: SQLAlchemy ORM mapping, SQLite pragmas, PostgreSQL optimizations, MySQL tuning → SQL Backends MongoDB Backend: PyMongo integration, aggregation pipelines, bulk operations → MongoDB Backend Neo4J Backend: Cypher queries, Bolt protocol, labels and constraints, CSV import → Neo4J Backend Usage Examples: Common patterns for each backend, data loading, querying → Usage Examples