Hermes Agent Model Configuration
Summary
The Sókrates system uses a two-tier model architecture: Gemma 4 31B Dense as the production target for local inference on DGX Spark appliances (with several fine-tuned variants planned), and OpenRouter as the cloud inference provider for development and for hardware without local GPU capacity. The Hermes Agent’s chat loop currently runs via OpenRouter during the development phase.
Details
Production Model: Gemma 4 31B Dense
The target production model is Gemma 4 31B Dense, running locally on NVIDIA DGX Spark appliances. This model was selected for:
- Local inference: Data never leaves the customer’s premises for reasoning tasks.
- Fine-tuning potential: Multiple domain-specific fine-tuned variants are planned (e.g., Socratic dialogue, workflow discovery, Basis consultation).
- Performance: Strong reasoning and tool-use capabilities sufficient for the Sókrates Agent’s interaction with the Eidos knowledge graph and MCP servers.
On pilot/development hardware without GPU capacity, complex reasoning is routed to cloud providers via OpenRouter.
Development Configuration: OpenRouter
During development, the Hermes Agent uses OpenRouter as its LLM provider. The current development models are configured in hermes.nix and routed through OpenRouter. Model selection has evolved through several iterations — the specific models in use should be checked in the live hermes.nix configuration rather than relying on this page.
Secret Management and Deployment
Secrets for the Hermes Agent are managed directly on the physical hardware rather than through encrypted repository-based solutions like SOPS. On the sokrates-dev appliance (the GMKtec node), the OpenRouter API key and related environment variables are stored in a specific secrets directory.
- Secret Path:
/var/lib/sokrates/secrets/hermes/hermes.env - Configuration Variable:
OPENROUTER_API_KEYTo apply changes to the model configuration or secrets, the operator must update the.envfile on the box and execute a NixOS rebuild:
sudo nixos-rebuild switch --flake .#sokrates-devThis process ensures that the nix-hermes module picks up the updated environment variables and restarts the systemd service with the new configuration.