Sókrates Wiki

OpenAPI Corpus for Gemma Fine-Tuning

6 min read

OpenAPI Corpus for Gemma Fine-Tuning

Mass-collecting public OpenAPI specifications as training data for the Gemma 4 31B fine-tune.

Context: The Hyle pipeline (DMCG) ingests OpenAPI 3.0 specs and produces typed BaseNode subclasses, ontological classifications, and edge manifests. The quality of downstream reasoning — semantic enrichment, generating queries, inefficiency surfacing — depends on how well the model understands API schemas. A purpose-built training corpus dramatically improves this.

The Idea

Public OpenAPI specifications are abundant, machine-readable, and structurally identical to the customer API specs that Sókrates ingests during onboarding. By collecting thousands of public specs and running them through the Hyle pipeline, we generate a massive corpus of (input spec → typed ontology → classified schemas → edge relationships) pairs. This corpus is the fine-tuning dataset for Gemma 4 31B.

The model learns:

  • How to classify schemas into ontological primitives (Entity, Process, Law, Observation) across arbitrary domains.
  • How API structures map to real-world business relationships (a CustomerID FK implies a participation edge, not just a data dependency).
  • Cross-schema entity resolution patterns (recognising that user_id in one API is the same entity as customer_ref in another).
  • Common structural patterns across industries (every ERP has purchase orders; every CRM has contacts; every HR system has departments and employees).

This is not generic language model fine-tuning. It is training the model on the exact task it will perform at every customer site: understanding operational system schemas and reasoning about their ontological structure.

Sources of Public OpenAPI Specs

Primary Sources

  • APIs.guru — the largest directory of public OpenAPI specs. 2,500+ APIs with machine-readable spec URLs. Crawlable via their GitHub repo or API. Specs are normalised to OpenAPI 3.0.
  • SwaggerHub Public Registry — thousands of publicly shared API definitions. Searchable and downloadable.
  • GitHub search — openapi.json, swagger.json, openapi.yaml files in public repos. Millions of results. Quality varies but volume compensates.
  • Postman Public Collections — many include OpenAPI exports. Accessible via Postman API.

Domain-Specific Sources

  • Plaid, Stripe, Square — FinTech APIs with rich schema structures (accounts, transactions, transfers, disputes). Directly relevant to the beachhead market.
  • Salesforce, HubSpot, Zoho — CRM APIs with entity-relationship-heavy schemas. Customer, Deal, Contact, Activity patterns.
  • SAP, Oracle, Microsoft Dynamics — ERP APIs. Purchase orders, invoices, inventory, production orders. The exact domain AdventureWorks models.
  • Workday, BambooHR, Personio — HR APIs. Employees, departments, org charts, time tracking.
  • Jira, Asana, Monday.com — Project management APIs. Tasks, sprints, boards, workflows.

Quality Criteria

Not all specs are useful. Filter for:

  • Completeness — specs with components.schemas containing real business domain models, not just request/response wrappers.
  • Relationship density — specs with $ref pointers between schemas (indicating FK-like relationships). Flat schemas with no cross-references have limited training value.
  • Domain diversity — balance across FinTech, ERP, CRM, HR, logistics, healthcare, etc. The model should generalise, not overfit to one vertical.
  • Schema size — specs with 10-200 schemas are the sweet spot. Trivial APIs (2-3 schemas) and mega-APIs (1000+ schemas) are less useful.

Pipeline: From Spec to Training Data

The Hyle pipeline already does most of the work:

Public OpenAPI spec
    ↓ [Stage 1: SpecParser]
SpecInfo (schemas, $ref relationships)
    ↓ [Stage 2: Generator (DMCG)]
Raw Pydantic models
    ↓ [Stage 3: Classifier — initially Gemini, later self-supervised]
ClassificationResult (entity/process/law/observation per schema)
    ↓ [Stage 4: Transformer]
Typed, decorated models
    ↓ [Stage 5: EdgeMapper]
EdgeManifest (typed relationships)

Each spec produces a training example: the input is the OpenAPI spec (or its SpecInfo representation), and the target is the classification + edge manifest. For the initial corpus, Gemini 3.1 Pro provides the ground-truth classifications. As the corpus grows and the Gemma fine-tune improves, the model bootstraps its own training data — Gemini validates edge cases, Gemma handles the bulk.

Training Data Format

Each training example is a tuple:

  1. Input: The OpenAPI components.schemas section — schema names, properties, types, $ref relationships.
  2. Output (classification): {"SchemaName": "entity"|"process"|"law"|"observation", ...} for every schema.
  3. Output (edges): The typed edge manifest — source, target, edge type, cardinality.
  4. Output (enrichment): Semantic field descriptions, cross-schema entity resolution candidates, ontological reasoning traces.

The enrichment output (4) is the highest-value training signal — it teaches the model not just what the classification is, but why. Gemini’s chain-of-thought reasoning for each classification decision becomes the training target for Gemma’s reasoning ability.

Scale and Economics

  • APIs.guru alone provides 2,500+ specs. At an average of 30 schemas per spec, that is 75,000 schema classification examples.
  • GitHub search can yield 10,000+ additional specs with modest crawling effort.
  • Gemini classification cost at 7.50. The entire corpus can be classified for under $100.
  • LoRA fine-tuning on DGX Spark (128 GB unified memory) can train on the full corpus in hours. Base Gemma 4 31B (~62 GB) leaves 66 GB for adapter weights, optimiser states, and gradient buffers.

This is not a “nice to have” dataset. It is the difference between a generic language model that happens to see API schemas and a purpose-built ontological reasoning engine that has seen every common business domain pattern before it encounters a customer’s systems.

The LoRA Registry: Specialised Models at Fleet Scale

The OpenAPI corpus enables not just one fine-tune but a registry of specialised LoRA adapters — see LoRA Adapter Registry and Fleet Command.

Rather than training a single generalist adapter, the corpus can be segmented by domain:

  • FinTech adapter — trained on Plaid, Stripe, Square, banking APIs. Deployed to FinTech customers.
  • ERP adapter — trained on SAP, Dynamics, Navision APIs. Deployed to manufacturing and logistics.
  • HR adapter — trained on Workday, BambooHR, Personio APIs. Deployed to companies with complex org structures.
  • CRM adapter — trained on Salesforce, HubSpot APIs. Deployed to sales-driven organisations.

Each adapter is a few hundred MB — trivially storable, distributable, and hot-swappable. Fleet Command maintains the registry and routes the appropriate adapter to each customer deployment based on their connected systems.

Implementation Phases

  1. Crawl — collect 5,000-10,000 public OpenAPI specs from APIs.guru, GitHub, and domain-specific sources. Store as a corpus.
  2. Classify — run each spec through the Hyle pipeline with Gemini providing ground-truth classifications. Store (spec, classification, edges, reasoning traces) tuples.
  3. Curate — filter low-quality examples (trivial schemas, malformed specs, pure CRUD wrappers with no domain semantics). Target 3,000-5,000 high-quality examples.
  4. Train — LoRA fine-tune Gemma 4 31B on the curated corpus. Evaluate on held-out specs.
  5. Segment — cluster the corpus by domain and train specialised adapters.
  6. Deploy — integrate adapter selection into Fleet Command. Customer onboarding selects adapters based on connected systems.

Graph View

Backlinks