Sókrates Wiki

Refactoring the Eidos Filter Compiler and Query Builder

2 min read

Refactoring the Eidos Filter Compiler and Query Builder

Summary

This process documents the identification and resolution of structural bugs within the Eidos query builder’s filter compiler. The refactoring addressed issues with logical operator grouping ($or, $and), clause scoping, and the removal of legacy closure-based state management.

Details

The refactoring of the filter compiler was triggered by the discovery of several functional bugs and string formatting errors in the generated queries. Initially, the query builder was producing malformed Cypher/SQL strings, such as inserting double spaces after keywords (e.g., "WHERE n.age").

Identification of the Scoping Bug

The primary architectural flaw identified was in how process_filters handled clause generation. The compiler utilized a top-level clauses: list[str] list. While simple equality filters worked, complex filters involving $or groups or NULL checks utilized a local_clauses list. However, the results of process_filters for these sub-groups were not being correctly fed back into the primary clauses list, leading to discarded filter logic.

A deeper issue existed within the recursive calls of the compiler. The system used a closure-based function called add_clause that appended directly to the outer clauses list. When an $or filter was processed, its sub-items would trigger add_clause and append themselves to the global list immediately, bypassing the structured grouping required for logical OR operations. This resulted in filters that were logically flat when they should have been nested.

Implementation of the Fix

The resolution involved making process_filters entirely self-contained and functional. The reliance on an outer scope list via closure was eliminated. The refactored logic ensures that:

  1. Every recursive call to process_filters returns its generated clauses.
  2. The caller is responsible for wrapping these clauses (e.g., in parentheses for an OR block) before merging them into the parent scope.
  3. The local_clauses pattern is used consistently to maintain logical boundaries between nested operators.

Code Cleanup and Testing

Following the logic fix, several legacy components were identified as dead code and removed:

  • The original add_clause closure.
  • The handle_field_op function.
  • The initial clauses: list[str] = [] declaration at the top level of the compiler.

The test suite for the filter compiler was updated to assert the correct behavior of $or paths and NULL checks. These updates verified that parameters are correctly mapped and that the resulting query strings contain the appropriate grouping syntax without redundant spacing.

Graph View