Skip to content

Schema Enforcement ​

This document describes how Hegemony maintains type consistency across the full stack without schema drift.

The Problem ​

In a full-stack application with separate layers, schemas can drift across:

  • Database (SQLAlchemy models + Alembic migrations)
  • Backend Contract (Pydantic request/response schemas)
  • API (FastAPI endpoints / OpenAPI)
  • UI (TypeScript types)
  • Import/Export (YAML serialization format)

Drift usually happens when:

  • DB columns are added/changed but API schemas are not updated
  • Endpoints return untyped dict payloads
  • UI maintains hand-written API DTO types
  • Import/export format is documented but not strictly validated or versioned
  • Generated types are bypassed by direct imports

Key Principle: What is the Single Source of Truth? ​

Hegemony uses Pydantic request/response schemas as the Single Source of Truth for the API contract.

  • DB truth: SQLAlchemy models + Alembic migrations (persistence)
  • API contract truth: Pydantic schemas (what the API guarantees)
  • OpenAPI spec: derived from Pydantic via FastAPI
  • UI contract types: derived from OpenAPI via code generation
  • UI-only view models: local types for state/rendering that never cross the wire

Important: SQLAlchemy is not the SSoT for what the API returns. The API intentionally exposes a subset of DB fields.


Architecture: Contract-Driven and Deterministic ​

text
┌─────────────────┐     ┌────────────────────────┐     ┌─────────────────┐
│  SQLAlchemy     │────▶│  Alembic Migrations     │────▶│   Database      │
│  Models         │     │  (schema matches models)│     │   Schema        │
└─────────────────┘     └────────────────────────┘     └─────────────────┘

┌─────────────────┐     ┌─────────────────┐     ┌───────────────────────┐
│   Pydantic      │────▶│    FastAPI       │────▶│  OpenAPI Spec         │
│   Schemas       │     │  Endpoints       │     │  (exported JSON file) │
│ (contract SSoT) │     │                 │     └──────────┬────────────┘
└─────────────────┘     └─────────────────┘                │
                                                           ▼
                                                 ┌───────────────────────┐
                                                 │  TypeScript Types      │
                                                 │  (openapi-typescript)  │
                                                 └──────────┬────────────┘
                                                            │
                                                            ▼
                                                 ┌───────────────────────┐
                                                 │ UI Contract Facade     │
                                                 │ (clean aliases only)   │
                                                 └───────────────────────┘

Non-Negotiable Rules ​

1) No untyped API payloads ​

Endpoints must not return untyped dict payloads. Every response must have a Pydantic response model, even if it contains flexible sub-objects.

Examples:

  • Bad: return {"valid": True, "errors": []}
  • Good: return FlowValidationResponse(valid=True, errors=[], ...)

For flexible data (e.g., metrics), keep the envelope typed and allow controlled flexibility:

  • Prefer dict[str, int | float | str | bool | None] over dict[str, Any]
  • Or introduce a JsonValue union type for controlled JSON payloads

2) UI must not define hand-written API DTOs ​

UI must not maintain manual “API-shaped” types for backend payloads. API DTO types come from OpenAPI generation, then are re-exported via the UI contract facade.

One payload has no generated type by design: the responses of an extension's own routes. Extensions are installed separately, so their routes are not in the platform's openapi.json: scripts/export-openapi.py loads no extension, even one installed in the development environment (the registry extension, which the API ships with), so the contract stays the platform's own. The page an extension describes (ExtensionPageRead, which is generated and re-exported like any other schema) names the keys to read, and extensionRequest in api/client.ts returns unknown; the page renderer reads values by those keys and never casts the payload to a hand-written type.

3) Generated types must be quarantined and never imported directly ​

UI code must not import from generated/api-types.ts directly. All UI imports go through a single facade (contract file).

4) Import/Export must be a first-class schema ​

Import/export is validated using Pydantic schemas. YAML is only a serialization format for a schema-defined bundle (versioned).


Enforcement Mechanisms ​

1) Deterministic OpenAPI export → TypeScript generation ​

The API's openapi.json is the single source of truth for type generation:

  • openapi.json is generated by FastAPI from Pydantic schemas (no manual editing)
  • TypeScript types are generated from openapi.json using openapi-typescript

File locations:

  • apps/api/openapi.json - API OpenAPI spec (generated by task api:regenerate)
  • apps/ui/src/api/generated/api-types.ts - Generated TypeScript types (from OpenAPI, DO NOT EDIT)

The type generation script always reads from apps/api/openapi.json - no running API required.

Example:

bash
# Two step action that combines export + generation (no containers needed) - recommended for local development
task api:regenerate

The local export option is useful for:

  • CI environments without Docker
  • Development agents that cannot spin up containers
  • Quick schema regeneration during development

The contract as a reference ​

The same file is the API reference people read: Swagger UI at /api/docs, ReDoc at /api/redoc, and the documentation site, which renders the committed apps/api/openapi.json. What a reader needs around the paths and schemas is in apps/api/openapi_docs.py:

  • The introduction (info.description): base URL, authentication, organizations, list parameters and errors.
  • A display name (x-displayName) and a one-sentence description for every tag, and the groups ReDoc lists them under (x-tagGroups). The tag itself stays what the router sets. A new router's tag needs a row in _TAG_CATALOG: Spectral (operation-tag-defined) and tests/api/test_openapi_docs.py fail without one. A tag from an installed extension is listed under "Extensions" at run time.
  • x-internal: true on every operation that only the platform's own worker and UI call: all that take the internal token, the stream-ticket endpoints, and the live streams that need a ticket. The operations stay in the contract, because the type generation and the route checks read it. public_openapi() returns the contract without them; the API serves that at /api/openapi.public.json, and ReDoc reads it. Swagger UI shows the complete contract.
  • Two servers entries: the relative /api, which makes "Try it out" work on an instance, and https://{host}/api for a reader elsewhere.

The export sets a placeholder Keycloak address (https://keycloak.example.com/realms/hegemony), so the committed contract names no development host; a running instance serves its own. info.version is packages/core/version.py, which the release sets: between releases it reads 0.0.0.

2) UI Contract Facade (single import surface) ​

UI structure:

text
apps/ui/src/api/
  generated/api-types.ts      # generated, DO NOT EDIT
  contract.ts                 # curated type aliases, EDIT
  client.ts                   # typed client, EDIT
apps/ui/src/types/ui/         # UI-only view models, EDIT

contract.ts:

  • Defines clean aliases (Device, DeviceCreate, FlowDefinition, …)
  • Covers read models as well as entities: aggregate response DTOs that have no CRUD counterpart (dashboard and platform-health snapshots, for example) and the outcomes of action endpoints (a connection or login test's result, or a run's moved maintenance window) are aliased here too, so no screen is tempted to hand-write the shape of a response the API already describes
  • Exposes “advanced” raw types under branded names (ApiOperations, ApiPaths) to discourage casual usage
  • Contains any migration aliases in a clearly marked “temporary” section

3) Enforce imports via linting (no bypass) ​

Oxlint is configured with no-restricted-imports to block direct imports from generated types:

jsonc
// .oxlintrc.json
{
  "rules": {
    "no-restricted-imports": [
      "error",
      {
        "patterns": [
          {
            "group": ["*/api/generated/*", "../api/generated/*"],
            "message": "Import from '../api/contract' instead of directly from generated types."
          }
        ]
      }
    ]
  },
  "overrides": [
    {
      "files": ["src/api/contract.ts"],
      "rules": {
        "no-restricted-imports": "off"
      }
    }
  ]
}

Only contract.ts is allowed to import from generated/. All other code must import from ../api/contract.

4) Schema alignment tests (intent-based, not brittle) ​

tests/api/test_schema_alignment.py verifies intent-based invariants:

  • Response schemas are an allowed subset of DB model fields (not necessarily equal)
  • Sensitive fields are denylisted and never exposed
  • Create schema fields are a subset of writable model fields
  • Update schemas are optional versions of create schemas
  • Import schemas match create schemas (or differences are explicitly documented)
  • Export schemas match response schemas (or differences are explicitly documented)

This prevents accidental drift without forcing “every DB column must be exposed.”

5) Import/Export validation with a versioned bundle schema ​

Import/export is validated with explicit versioned schemas — ImportBundle and ExportBundle plus ImportResult and ImportSummary in apps/api/schemas_config_exchange.py. YAML is only the serialization format:

  • ExportBundle is serialized to YAML
  • ImportBundle is parsed from YAML and validated

Benefits:

  • Safe evolvability via schema_version
  • Better error messages
  • Full end-to-end type generation for UI

Workflow for Schema Changes ​

Adding a new field ​

  1. DB model (models.py): add column (if persisted)
  2. Migration: task db:revision -- "short message" (see migrations for the naming and review rules)
  3. Pydantic schemas (schemas.py): update relevant Create/Update/Response models (API contract)
  4. Alignment tests: update rules/allowlists/denylists if needed
  5. OpenAPI export and Generate UI types: task api:regenerate
  6. Contract facade (contract.ts): add the components['schemas'][…] alias for any new schema. Rule 3 quarantines the generated file, so a new request or response model is unusable until it has an alias here; task ui:check:contract-types fails on a hand-written one.
  7. UI: update components importing types from the contract facade (@api/contract)
  8. Import/Export: update bundle schemas if field should be importable/exportable

Making a field nullable ​

  1. DB model: set nullable=True (if persisted)
  2. Migration: generate migration for nullability change
  3. Schema: change to FieldType | None in Pydantic
  4. Types: regenerate TS (may become field?: T | null)
  5. UI: handle null/optional cases explicitly

Files Reference ​

PurposeLocation
SQLAlchemy modelsapps/api/models.py
Alembic migrationsapps/api/alembic/versions/
Pydantic schemas (contract SSoT)apps/api/schemas.py
OpenAPI spec export (deterministic)apps/api/openapi.json
TS types (generated, DO NOT EDIT)apps/ui/src/api/generated/api-types.ts
UI contract facade (clean aliases)apps/ui/src/api/contract.ts
Typed UI clientapps/ui/src/api/client.ts
UI-only view models (local state)apps/ui/src/types/ui/*
Import/Export routerapps/api/routers/config_exchange.py
Import/Export bundle schemasapps/api/schemas_config_exchange.py (or schemas.py)
Schema alignment teststests/api/test_schema_alignment.py
Type generation scriptapps/ui/scripts/generate-types.mjs

generate-types.mjs runs the locally installed openapi-typescript and oxfmt binaries directly, passing arguments as argv rather than through a shell. Run npm ci (or task ui:install) first: the script resolves both tools from the UI's installed dependencies and does not fetch them on demand.


CI Integration ​

Required CI steps (implemented in .github/workflows/ci.yml) ​

  1. Schema alignment tests (in lint-and-test job):
yaml
- name: Run tests
  run: uv run pytest tests/ -v --tb=short
  1. Verify generated types are up-to-date (in typecheck-ui job):
yaml
- name: Verify generated types are up-to-date
  run: |
    npm run generate:types
    git diff --exit-code src/api/generated/ || (echo "::error::Generated types are out of sync." && exit 1)
  1. Oxlint enforces import restrictions (in typecheck-ui job):
yaml
- name: Run linter (oxlint)
  run: npm run lint

Additional recommendations ​

  • Pin Node + openapi-typescript versions to reduce churn
  • Ensure OpenAPI export is stable and not environment-dependent

Manual vs Generated Types (Policy) ​

API DTO types ​

  • Must come from OpenAPI generation
  • Must be imported only from apps/ui/src/api/contract.ts
  • No hand-written API DTOs in the UI codebase

UI-only types ​

Allowed only for:

  • Table state, filters, form drafts, component props not matching API DTOs
  • Derived view models (e.g., “row models”) that do not cross the network boundary

UI-only types must live under:

  • apps/ui/src/types/ui/*

Notes for Contributors ​

  • If you add/modify an API schema, you must regenerate UI types.
  • If you see an endpoint returning dict, convert it to a response model immediately.
  • If you need a UI-only type, put it in apps/ui/src/types/ui/ and keep it clearly separated from API DTOs.

Released as open source under the AGPL-3.0-or-later license. Development is sponsored by Rexonix s.r.o. Contact — [email protected].