Legacy Code Cleanup (SOK-78)

Summary

The removal of the “DreamJob” and “Memory Palace” legacy systems from the Sokrates codebase to reduce technical debt. This process involved stripping out obsolete orchestrators, specific Neo4j queries, and personal migration scripts while preserving core infrastructure like cache statistics and repository indexing tools.

Details

As part of the SOK-78 task, a comprehensive cleanup was performed to remove legacy references and components that were no longer aligned with the current architecture of the Sokrates project. The primary targets for removal were the “DreamJob” feature set and references to the “Memory Palace” concept.

Component Removal

The cleanup targeted several key areas of the Eidos API and supporting infrastructure:

  • DreamJobOrchestrator: The entire service file services/dream_jobs.py was removed. Corresponding imports, global variables, and shutdown blocks in main.py were deleted.
  • Neo4j Queries: The DreamJobQueries class, located in infrastructure/neo4j/queries.py (specifically lines 271-372), was excised. The cleanup was careful to preserve adjacent classes such as CacheQueries, VectorIndexQueries, and the QueryFactory.
  • Admin Endpoints: The api/endpoints/admin.py file was refactored to remove all DreamJob-related endpoints. The cache statistics endpoint (line 78+) was retained as it remains useful for system monitoring.
  • Documentation: Legacy docstrings in specifications/memory.py referring to “querying the palace” were updated or removed to reflect the transition away from the “Memory Palace” terminology.

Script Management

A distinction was made between “personal/migration” scripts and “useful/product” scripts. The following scripts were identified as obsolete and removed from the scripts/ directory:

  • import_friendship_memories.py
  • import_curated_memories.py
  • import_tiered_memories.py
  • review_memories.py
  • migrate_memory_types.py
  • migrate_conversation_turns.py

Conversely, scripts essential for the ongoing development and operation of the knowledge graph were kept, including ingest_repo.py, process_repository.py, and search_code.py.

Rationale

The “DreamJob” functionality represented an earlier iteration of the project’s logic that had been superseded by the Hyle type system and the more generalized Eidos knowledge graph architecture. Removing these components reduces the maintenance burden and prevents confusion regarding the current state of the system’s ontology and service layers.