Lineage document. This page intentionally references the old “Grimoire” codebase to document the architectural evolution to Hyle. Old type names (
FriendUtterance,ClaudeUtterance,GraphModel,MemoryType) are historical — see the current system in Hyle Graph-Native ORM with Dynamic Schema Registry.
Hyle: The Matter Substrate of Eidos
Architectural design document — Sókrates internal March 2026
Naming and Ontological Position
Eidos is the knowledge graph — the form layer that gives structure, relationships, and meaning to organisational data. Hyle is its complement: the matter substrate. In Aristotelian metaphysics, hyle (ὕλη) is formless until eidos acts upon it. Raw schemas become meaningful only when the knowledge graph provides relational structure. The name is not decorative. It encodes the architectural relationship: Hyle receives and holds data; Eidos interprets it.
Hyle’s job is to be the schema backbone for Eidos. It defines what a node is — structurally, in Python’s type system — while Eidos defines what a node means in the organisational graph. Everything that follows is the machinery for making that division clean, dynamic, and self-healing.
Lineage: What Grimoire Already Does
Hyle does not start from zero. Grimoire — the production knowledge graph built at Wise and now adapted for Sókrates — already contains the structural DNA. The GraphModel base class in grimoire.domain.models.base is the ancestor:
class GraphModel(BaseModel):
"""Base class for all Neo4j entities with discriminated union support."""
id: UUID = Field(default_factory=uuid4)
memory_type: MemoryType
timestamp: datetime = Field(default_factory=utc_now)
@classmethod
def labels(cls) -> list[str]:
"""Get Neo4j labels for this entity."""
...
def to_neo4j_properties(self) -> dict:
"""Convert to Neo4j-compatible property dict."""
...
@classmethod
def from_neo4j_record(cls, record):
"""Create instance from Neo4j record."""
...And the discriminated union in grimoire.domain.models.memories already routes deserialization by type:
Memory = Annotated[
FriendUtterance | ClaudeUtterance | SystemNote | TopicCluster
| OntologyNode | GitRepository | CodeFile | GitCommit,
Field(discriminator="memory_type"),
]Each concrete model uses a Literal type on the discriminator field, which is exactly the pattern Pydantic v2 uses to resolve which branch of the union to instantiate:
class FriendUtterance(GraphModel):
memory_type: Literal[MemoryType.FRIEND_UTTERANCE] = MemoryType.FRIEND_UTTERANCE
content: str
embedding: list[float] | None = None
...This works. But it is static. Every node type is hand-written Python, the union is manually maintained, and adding a new type requires editing source code, redeploying, and updating the union. For a personal knowledge graph, that is fine. For an outsourced AI department serving dozens of customers with different ERP systems, each with their own schema landscape that evolves independently — it is structurally insufficient.
Hyle solves this by making the discriminated union a living registry that populates itself at class creation time, accepts dynamically generated models, and provides inherited query and persistence methods to every node type automatically.
The Two Input Channels
Hyle encodes organisational topology. That data arrives through two channels with fundamentally different trust profiles and evolution dynamics.
Channel 1: OpenAPI → DMCG → BaseNode. Customer systems expose OpenAPI specifications. These specs are pre-validated — the contract is the schema, and if the OpenAPI spec is wrong, that is the source system’s problem. Datamodel Code Generator (hereafter DMCG) consumes these specs and produces Pydantic v2 models. The --base-class flag means you point DMCG at hyle.BaseNode and everything inherits the metaclass, registry, and query builder for free. No post-processing. The pipeline is:
OpenAPI spec (from ERP, CRM, etc.)
→ DMCG with --base-class hyle.BaseNode
→ Python module on disk
→ importlib loads it
→ metaclass fires during class creation
→ node type auto-registered in NodeRegistry
→ queryable and persistable immediately
DMCG’s Python API makes this programmable:
from datamodel_code_generator import generate, DataModelType, PythonVersion
result = generate(
openapi_spec_string,
input_file_type="openapi",
output_model_type=DataModelType.PydanticV2BaseModel,
target_python_version=PythonVersion.PY_311,
base_class="hyle.BaseNode",
)
# Write to disk, then importlib loads itChannel 2: Sókrates-designed ontology elements. These are hand-crafted, semantically loaded nodes created by the Socratic Workflow Archaeologist or by human operators. They encode things like “this department has this reporting structure” or “this workflow bottleneck connects these two ERP entities in a way the ERP itself does not know about.” These nodes are not in any OpenAPI spec because they describe relationships between systems, not within any single system.
The two channels share the same metaclass and the same registry but require different evolution policies. Discovered nodes heal automatically when the OpenAPI spec changes — DMCG regenerates, importlib reloads, done. Designed nodes require deliberation. The Philosopher King or a human operator must approve schema changes because the semantics are load-bearing in a way that machine-discovered schemas are not. The @node decorator carries a source attribute that encodes this distinction.
The Metaclass + Decorator Architecture
The design separates two concerns: structural identity (what is a node in Hyle) and semantic configuration (what does this node mean in Eidos). The metaclass handles the first. The decorator handles the second.
Why a Metaclass, Not __init_subclass__
__init_subclass__ treats registration as a side effect of inheritance. The class exists whether registration succeeds or not. It is a notification: “a new subclass appeared.” If registration fails — a duplicate node_type literal, a missing discriminator field, a conflict with an existing version — you have a zombie class that lives in Python’s type system but is absent from the registry. In a static codebase where a developer catches that immediately, this is a style preference. In a system where DMCG is producing classes that get hot-loaded via importlib into a running knowledge graph, it is the difference between a circuit breaker and a post-mortem.
The metaclass intercepts __new__ before the class is finalised. It can enforce structural contracts and refuse to create the class if the contract fails. Registration is a construction step, not a side effect. The class cannot exist without being registered.
Pydantic v2 already uses ModelMetaclass, so HyleMeta extends it rather than replacing it:
from pydantic._internal._model_construction import ModelMetaclass
class NodeRegistry:
"""The living registry. Separate from the metaclass so it's
injectable and testable."""
def __init__(self):
self._types: dict[str, type[BaseNode]] = {}
self._versions: dict[str, dict[int, type[BaseNode]]] = {}
self._hooks: list[callable] = []
def register(self, node_type: str, cls: type, version: int = 1):
self._types[node_type] = cls
self._versions.setdefault(node_type, {})[version] = cls
for hook in self._hooks:
hook(node_type, cls, version)
def deregister(self, node_type: str):
self._types.pop(node_type, None)
self._versions.pop(node_type, None)
def resolve(self, data: dict):
node_type = data.get("node_type")
version = data.get("_schema_version", 1)
registry = self._versions.get(node_type, {})
cls = registry.get(version) or self._types.get(node_type)
if cls is None:
raise KeyError(
f"No registered node: type={node_type!r} v{version}"
)
return cls.model_validate(data)
def on_register(self, hook: callable):
"""Observer pattern — Eidos gets notified when Hyle's
schema changes."""
self._hooks.append(hook)
@property
def union_type(self):
"""Dynamically build the discriminated union from
whatever is registered."""
from typing import Annotated, Union
from pydantic import Field
types = tuple(self._types.values())
if not types:
return BaseNode
return Annotated[
Union[types], Field(discriminator="node_type")
]
_default_registry = NodeRegistry()
class HyleMeta(ModelMetaclass):
"""Intercepts class creation to:
1. Validate the node contract (must have node_type if concrete)
2. Inject query/repository methods from the class's bindings
3. Auto-register unless the @node decorator will handle it
"""
def __new__(mcs, name, bases, namespace, **kwargs):
cls = super().__new__(mcs, name, bases, namespace, **kwargs)
if name == "BaseNode":
return cls
if "node_type" in cls.model_fields:
field = cls.model_fields["node_type"]
args = get_args(field.annotation)
if args:
if not getattr(cls, "_defer_registration", False):
registry = getattr(cls, "_registry", _default_registry)
version = getattr(cls, "_schema_version", 1)
registry.register(args[0], cls, version)
if hasattr(cls, "_repository_class") and cls._repository_class:
repo_cls = cls._repository_class
cls.query = classmethod(
lambda klass: QueryBuilder(klass, repo_cls)
)
return clsThe BaseNode
This is the Hyle equivalent of Grimoire’s GraphModel, but designed for dynamic registration:
class BaseNode(BaseModel, metaclass=HyleMeta):
"""The matter. Formless until Eidos gives it meaning."""
node_type: str
_schema_version: ClassVar[int] = 1
_registry: ClassVar[NodeRegistry] = _default_registry
_repository_class: ClassVar[Any] = None
def save(self, repo=None):
r = repo or self._repository_class
if r is None:
raise RuntimeError(
"No repository bound. Use @node(repository=...) "
"or pass repo="
)
return r.save(self)
def delete(self, repo=None):
r = repo or self._repository_class
return r.delete(self)The Decorator
The @node decorator handles semantic graph configuration — labels, indexes, constraints, repository binding, source provenance. It is opt-in configuration that layers on top of the metaclass’s structural registration:
def node(
label: str | None = None,
indexes: list[str] | None = None,
constraints: list[str] | None = None,
repository: type | None = None,
registry: NodeRegistry | None = None,
version: int = 1,
source: str = "designed",
spec_origin: str | None = None,
designed_by: str | None = None,
):
def wrapper(cls: type[BaseNode]) -> type[BaseNode]:
cls._graph_label = label or cls.__name__
cls._graph_indexes = indexes or []
cls._graph_constraints = constraints or []
cls._schema_version = version
cls._source = source
cls._spec_origin = spec_origin
cls._designed_by = designed_by
if repository:
cls._repository_class = repository
reg = registry or cls._registry
node_type_field = cls.model_fields.get("node_type")
if node_type_field:
args = get_args(node_type_field.annotation)
if args:
reg.register(args[0], cls, version)
return cls
return wrapperThe two channels then look like this in practice:
# Channel 1: DMCG-generated from OpenAPI (auto-registered by metaclass)
# The decorator adds provenance metadata and repository binding
@node(source="discovered", spec_origin="erp_openapi_v3.json",
repository=Neo4jRepository, version=2)
class SalesOrderNode(BaseNode):
node_type: Literal["sales_order"] = "sales_order"
order_id: str
customer_ref: str
total_amount: float
# ... fields from DMCG
# Channel 2: Sókrates-designed ontology element
@node(source="designed", designed_by="archaeologist",
repository=Neo4jRepository, version=1)
class WorkflowBottleneckNode(BaseNode):
node_type: Literal["workflow_bottleneck"] = "workflow_bottleneck"
description: str
affected_departments: list[str]
estimated_cost_per_occurrence: float | None = NoneThe Observer Hook
The NodeRegistry.on_register method is where Eidos and Hyle communicate. When Hyle registers a new node type, Eidos can automatically create Neo4j constraints, update its schema cache, and trigger relationship inference. The form learning about new matter:
def on_new_node_type(node_type: str, cls: type, version: int):
"""Called by Hyle's registry whenever a new node type appears."""
# Create Neo4j constraints for the new type
ensure_constraints(cls._graph_label, cls._graph_indexes)
# Update Eidos's schema cache
eidos_schema_cache.invalidate(node_type)
# If this is a discovered type, trigger relationship inference
if getattr(cls, '_source', None) == 'discovered':
schedule_relationship_discovery(node_type)
_default_registry.on_register(on_new_node_type)Hot-Loading via importlib
When the OpenAPI spec changes — a customer adds a field to their ERP, removes an entity, or restructures an API — DMCG regenerates the models and importlib loads them into the running process. This is the self-healing mechanism.
import importlib.util
import sys
def load_generated_model(filepath: str, module_name: str):
spec = importlib.util.spec_from_file_location(
module_name, filepath
)
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module) # Metaclass fires here
return module
def heal_schema(node_type: str, new_openapi_spec: str):
"""Schema drift detected → regenerate → hot-swap."""
from datamodel_code_generator import (
generate, DataModelType, PythonVersion
)
code = generate(
new_openapi_spec,
input_file_type="openapi",
output_model_type=DataModelType.PydanticV2BaseModel,
target_python_version=PythonVersion.PY_311,
base_class="hyle.BaseNode",
)
# Write to disk (audit trail + rollback)
version = get_next_version(node_type)
module_name = f"hyle.generated.{node_type}_v{version}"
filepath = GENERATED_DIR / f"{node_type}_v{version}.py"
filepath.write_text(code)
# Load — metaclass auto-registers on class creation
load_generated_model(str(filepath), module_name)
# Eidos gets notified via the observer hook
# → updates graph constraints, reindexes, migratesWhy importlib over exec: files on disk provide an audit trail, proper __module__ attributes on classes enable debuggability, importlib.reload() enables hot-swapping, and sys.modules namespacing enables version isolation. The one thing to manage: when hot-swapping a schema version, explicitly remove the old module from sys.modules before loading the new one. Version-namespace your module names (hyle.generated.customer_v2) and it stays clean.
GQL as the Compilation Target
Grimoire’s query builder currently compiles to Cypher. The CypherQueryBuilder in grimoire.infrastructure.neo4j.query_builder already provides a fluent interface with a state machine that validates clause ordering:
builder = CypherQueryBuilder()
builder.match(lambda p: p.node("Memory", "m"))
builder.where("m.salience >= 0.7")
builder.return_clause("m.id AS id", "m.content AS content")
builder.order_by("m.timestamp DESC")
builder.limit(20)
query, params = builder.build()And the specification pattern in grimoire.domain.specifications already bridges domain logic and Cypher generation:
class SalientMemorySpecification(BaseSpecification):
type: Literal["salience"] = "salience"
min_salience: float = Field(0.5, ge=0.0, le=1.0)
def to_cypher(self) -> str:
return f"m.salience >= {self.min_salience}"For Hyle, the query builder should compile to GQL (ISO/IEC 39075:2024) rather than Cypher. GQL is the first new ISO database language since SQL in 1987, published in April 2024. It was built by the same standards committee that maintains SQL, and it descends directly from Cypher — Neo4j is actively converging Cypher toward GQL. The syntax is nearly identical with a few keyword changes: INSERT instead of CREATE, FOR instead of UNWIND.
Three structural reasons to target GQL:
First, Neo4j is converging on GQL anyway, so Cypher is a moving target in the GQL direction. Writing a Cypher backend today means rewriting it toward GQL tomorrow.
Second, Microsoft Fabric speaks GQL natively. Fabric has a GQL language guide and is building graph-in-Fabric with GQL as the query language. This is directly relevant given Hyle’s lineage as the extracted soul of a Fabric lakehouse ETL pipeline. A GQL backend gives Eidos both Neo4j compatibility and Fabric portability for free.
Third, the ISO standard means the portability surface area grows over time without Sókrates doing anything. Other vendors — AWS Neptune, TigerGraph, and others — are implementing GQL. Any customer on any graph database that supports GQL can use Eidos.
The architecture: BaseNode.query() returns a builder. The builder produces a GQL AST (abstract syntax tree). A driver serialises the AST to the target dialect. Neo4j gets Cypher (which is converging to GQL), Fabric gets native GQL, and if a customer is on Neptune, the same AST goes through a different serialiser. The Grimoire specification pattern (to_cypher() methods on each specification) becomes to_gql(), and the composite specifications compose GQL expressions exactly as they currently compose Cypher expressions.
The migration path from Grimoire’s existing query infrastructure is mostly find-and-replace at the specification level, plus adding a GQL AST layer between the builder and the string output.
The Hypergraph Metalayer: Queries as Identity
Eidos is a hypergraph. A hyperedge connects an arbitrary set of nodes — not just pairs. The critical architectural insight is that a hyperedge in Eidos is not a stored list of connected nodes. It is its generating query.
Consider an ordinary graph edge: “Alice reports to Bob.” This is a stored fact. Now consider a hyperedge like “all purchase orders over 500k ISK that traverse more than two departments before reaching a budget holder.” That is not a fact to be stored. It is a computation over the underlying data. The set of nodes connected by this hyperedge changes whenever the underlying data changes. The hyperedge is alive.
The analogy to SQL’s Common Table Expressions (CTEs) makes this precise. A CTE is a named computation that produces a result set. Other CTEs can reference it by name. The result of evaluating all the CTEs is the query’s answer. In Eidos, a hyperedge definition is the analog of a CTE: a named computation whose result set is the hyperedge’s membership. Other hyperedge definitions can reference it. The metalayer is the language for composing these definitions.
This gives Eidos a three-layer architecture:
Layer 0 — Hyle (ground facts). BaseNode instances from DMCG-generated models and Sókrates-designed ontology elements. Sales orders, invoices, employees, departments. These exist independently. They are the data.
Layer 1 — Generating queries (hyperedges as computation). A hyperedge like bottleneck_approval_chain is stored not as a static membership list but as a GQL query:
MATCH (po:PurchaseOrder)-[:REQUIRES_APPROVAL]->(d:Department)
WHERE po.value > d.approval_threshold
MATCH (d)-[:REPORTS_TO*]->(budget_holder:Role {type: "budget_holder"})
RETURN po, d, budget_holder
The result set of this query is the hyperedge. Re-execute it tomorrow and the membership changes if the underlying data has changed. The self-healing property is definitional, not mechanical. There is no reconciliation process that detects drift and patches the graph. You re-evaluate the generating queries and the graph is whatever they produce.
Layer 2 — The metalayer (DSL over queries). Metalayer expressions compose hyperedges. A metalayer expression can reference other hyperedges by name, and the system resolves them by executing the generating queries:
DEFINE bottleneck_chains AS (
-- generating query over Layer 0 nodes
)
DEFINE cross_department_friction AS (
-- generating query that references bottleneck_chains
-- this hyperedge depends on another hyperedge
)
The metalayer expression is a program in a DSL where the primitives are generating queries and the composition operators build higher-order hyperedges from lower-order ones. The organisational topology is not modeled. It is computed. And recomputed. And the computation is the model.
This has a recursive consequence for Sókrates. The Philosopher King lives at Layer 2. It does not just execute metalayer queries — it writes new ones. The Socratic Workflow Archaeologist discovers a pattern in the data (“every purchase order over 500k ISK traverses four departments but only needs two”). That insight gets encoded as a new generating query, which becomes a new hyperedge, which becomes a new fact that other generating queries can reference. The basis grows.
Datalog: The Theory That Was Already There
This architecture — ground facts, derived facts via rules, rules composing into higher-order derivations, evaluation to a fixed point — is Datalog. It was invented in the 1980s as a logic programming language for database theory. The correspondence is exact:
| Datalog | Eidos |
|---|---|
| Ground facts | Layer 0 — Hyle BaseNode instances |
| Rules | Generating queries that define hyperedges |
| Rule heads (derived predicates) | Hyperedge names |
| Stratification | The metalayer (layered composition) |
| Minimal model | The organisational topology at any given moment |
| Fixed-point evaluation | Self-healing |
In Datalog, facts and rules live in the same universe. A rule produces new facts; those facts can trigger other rules; you iterate to a fixed point. The stratification — the layering that prevents paradox — comes from restricting negation. In Eidos, the metalayer is the stratification: Layer N can reference hyperedges from Layer N-1 but not from its own layer.
The key properties Datalog guarantees and Eidos inherits:
Monotonicity guarantees termination. As long as generating queries only add membership (no negation, no deletion), the fixed point is guaranteed to exist and is unique.
Recursion is native. A generating query can reference itself. reachable(X, Z) :- reachable(X, Y), edge(Y, Z) computes transitive closure — the kind of query you need for “trace the approval chain through whatever departments it crosses.”
Queries and data live in the same universe. A generating query’s output is indistinguishable from a ground fact to any other generating query. There is no ontological distinction between stored data and derived data.
Differential Datalog
The practical concern with fixed-point evaluation is cost: if you naively re-evaluate all generating queries every time a ground fact changes, the system does not scale. Differential Datalog solves this. Its core thesis: when a few input facts change, recompute only the affected derived facts. This is exactly the self-healing semantics Eidos needs — schema drift in an ERP changes a few Layer 0 nodes, and only the hyperedges whose generating queries touch those nodes get re-evaluated.
The modern implementations worth tracking: Differential Datalog (incremental computation, the evaluation engine most directly applicable to Hyle), Soufflé (high-performance Datalog compiler with C++ output), and RelationalAI (commercial Datalog-as-a-database).
Evaluation Strategy
Not all hyperedges should be evaluated the same way. Some are stable (the reporting structure changes quarterly) and should be materialised — computed once, cached, refreshed on a schedule. Others are volatile (active purchase orders in flight) and should be virtual — computed on demand from current data. The metalayer DSL needs a keyword for this:
DEFINE MATERIALIZED org_reporting_structure AS (...)
DEFINE VIRTUAL active_bottlenecks AS (...)
Stratification and Cycle Detection
If hyperedge A’s generating query references hyperedge B, and B’s references A, you have a cycle. Datalog handles this with stratified negation. Eidos has two options: enforce a DAG on hyperedge dependencies (simple, restrictive), or allow well-founded recursion with fixed-point semantics (powerful, complex). Given that circular reporting structures are a real organisational pathology you would want to detect rather than silently permit, cycle detection in the dependency graph is a feature, not a bug.
The Specification Pattern as Precursor
Grimoire’s existing specification pattern is the direct precursor to Hyle’s generating queries. The MemorySpecification discriminated union already provides composable, type-safe query predicates:
MemorySpecification = Annotated[
SalientMemorySpecification
| TopicMemorySpecification
| ConversationMemorySpecification
| RecentMemorySpecification
| EmotionalMemorySpecification
| OntologyPathSpecification
| ...,
Field(discriminator="type"),
]Each specification can be composed via and_() and or_() methods and compiled to Cypher via to_cypher(). The unified query endpoint already accepts a QueryDSL model that combines specifications with similarity search, relationship traversal, and pagination.
In Hyle, this pattern generalises. A generating query is a composition of specifications, evaluated against the node registry. The specification pattern does not change — it extends. Each specification gains a to_gql() method alongside to_cypher(), and the metalayer DSL becomes the language for naming and composing specification trees into hyperedge definitions.
Summary of Moving Parts
The complete Hyle architecture involves these components:
BaseNode — the Pydantic base model, inheriting from BaseModel via HyleMeta. Carries node_type as discriminator, _schema_version for evolution, inherited save() and delete() methods, and a query() classmethod injected by the metaclass.
HyleMeta — the metaclass extending Pydantic’s ModelMetaclass. Validates node contracts at class creation time, auto-registers concrete node types in the NodeRegistry, and injects query builder access.
@node decorator — semantic configuration for graph mapping. Attaches labels, indexes, constraints, repository binding, source provenance (discovered vs designed), and version metadata.
NodeRegistry — the living registry. Maps node_type strings to Python classes, supports version-aware resolution, provides observer hooks for Eidos integration, and dynamically generates discriminated union types.
DMCG integration — consumes OpenAPI specifications, produces BaseNode subclasses with --base-class hyle.BaseNode, writes to disk for audit trail.
importlib loader — hot-loads generated modules. The metaclass fires during exec_module, so registration happens as a construction step. Version-namespaced module names enable rollback.
GQL query builder — evolution of Grimoire’s CypherQueryBuilder. Produces a GQL AST that serialises to Cypher (Neo4j), native GQL (Fabric), or other dialects. The specification pattern compiles to GQL expressions.
Metalayer DSL — the language for defining hyperedges as generating queries. Supports MATERIALIZED vs VIRTUAL evaluation, dependency-ordered stratification, and composition of named hyperedges.
Differential evaluation engine — inspired by Differential Datalog. When ground facts change, recomputes only affected derived facts (hyperedge memberships). This is the self-healing mechanism at the evaluation level.
Observer hooks — NodeRegistry.on_register notifies Eidos when Hyle’s schema changes. Eidos responds by creating graph constraints, updating its schema cache, and triggering relationship inference for discovered types.
What Comes Next
This document captures the architectural decisions made so far. The immediate next steps are:
-
Implement
BaseNode,HyleMeta,NodeRegistry, and@nodeas a standalone Python package (hyle), tested against Pydantic v2’sModelMetaclass. -
Build the GQL AST layer by refactoring Grimoire’s
CypherQueryBuilderto produce an intermediate representation that serialises to both Cypher and GQL. -
Prototype the metalayer DSL — start with a minimal version that supports
DEFINE ... AS (...)with named query composition andMATERIALIZED/VIRTUALkeywords. -
Evaluate Differential Datalog — specifically, how much of its incremental maintenance algorithm can be adopted versus how much needs to be built from scratch for the GQL-over-Neo4j context.
-
Tackle edges.
BaseEdgewith its own discriminated union, metaclass registration, and generating-query semantics. This is the next design session.
The foundation is the HyleMeta metaclass and the NodeRegistry. Everything else builds on top of it. Get that right and the rest is engineering. Get it wrong and every layer above inherits the mistake.