Sókrates Wiki

Hermes Agent Model Evaluation and Update (v0.6.0)

2 min read

Hermes Agent Model Evaluation and Update (v0.6.0)

Summary

A technical evaluation of the Hermes Agent’s model configuration and version status revealed a critical mismatch between desired agentic capabilities and OpenRouter provider support. Research confirmed that Hermes-family models currently lack tool-use support on OpenRouter, necessitating a migration to alternative models (e.g., Qwen, Gemini) and a version upgrade to Hermes v0.6.0 to resolve security vulnerabilities and enable provider fallback chains.

Details

OpenRouter Model Research

Investigation into the “No endpoints found that support tool use” error confirmed that while the Hermes family of models (including Hermes 3 and Hermes 4) are natively capable of function calling, no providers currently serving these models on OpenRouter have enabled the tools or tool_choice parameters. This includes nousresearch/hermes-4-70b and nousresearch/hermes-3-llama-3.1-405b.

To maintain agentic functionality within the Sokrates stack, a shortlist of alternative models with verified tool-use support and high cost-efficiency (under $5.00 per 1M output tokens) was generated:

Model IDContextInput $/MOutput $/MNotes
qwen/qwen3.5-122b-a10b262k$0.26$2.08Recommended “Sweet Spot” for intelligence/cost.
google/gemini-2.0-flash-0011M$0.10$0.40Excellent tool use and massive context.
xiaomi/mimo-v2-flash262k$0.09$0.29High SWE-Bench performance, extremely cheap.
qwen/qwen3-235b-a22b-2507262k$0.07$0.10MoE architecture, very high value.
qwen/qwen3.6-plus-preview:free1M$0.00$0.00Current free preview for testing.

Hermes Agent Version Audit

The Sokrates environment was found to be running an outdated version of the Hermes Agent, locked to a commit (9d4b3e54) between v0.4.0 and v0.5.0. This lag introduced significant security risks and missed architectural improvements.

Key Changes in v0.5.0 and v0.6.0:

  • Supply Chain Security: Removal of the compromised litellm dependency and implementation of hash-verified lockfiles (uv.lock).
  • SSRF Protection: Hardened protections for browser_navigate, vision_tools, and web_tools.
  • Provider Routing: v0.6.0 introduced ordered fallback provider chains, allowing the agent to automatically failover between providers (e.g., from a primary provider to OpenRouter) rather than failing on a single point of congestion.
  • Multi-Instance Support: The introduction of “Profiles” allows running isolated Hermes instances from a single installation.
  • MCP Integration: A new hermes mcp serve mode allows Hermes conversations to be exposed to other MCP-compliant clients.

Migration and Update Path

The update requires a two-step process involving the Nix flake and the system configuration.

  1. Flake Update: Update the nix-hermes input to pull the latest v0.6.0 release. 
    cd /home/rationallyprime/projects/sokrates
nix flake update nix-hermes
  2. Configuration Adjustment: Modify nixos/modules/hermes.nix to switch the model from the non-functional Hermes 4 ID to a supported tool-use model like qwen/qwen3.5-122b-a10b or google/gemini-2.0-flash-001.
  3. System Rebuild: Apply the changes to the sokrates-dev environment. 
    sudo nixos-rebuild switch --flake .#sokrates-dev

Graph View