Schema Enforcement
This document describes how Hegemony maintains type consistency across the full stack without schema drift.
The Problem
In a full-stack application with separate layers, schemas can drift across:
- Database (SQLAlchemy models + Alembic migrations)
- Backend Contract (Pydantic request/response schemas)
- API (FastAPI endpoints / OpenAPI)
- UI (TypeScript types)
- Import/Export (YAML serialization format)
Drift usually happens when:
- DB columns are added/changed but API schemas are not updated
- Endpoints return untyped
dictpayloads - UI maintains hand-written API DTO types
- Import/export format is documented but not strictly validated or versioned
- Generated types are bypassed by direct imports
Key Principle: What is the Single Source of Truth?
Hegemony uses Pydantic request/response schemas as the Single Source of Truth for the API contract.
- DB truth: SQLAlchemy models + Alembic migrations (persistence)
- API contract truth: Pydantic schemas (what the API guarantees)
- OpenAPI spec: derived from Pydantic via FastAPI
- UI contract types: derived from OpenAPI via code generation
- UI-only view models: local types for state/rendering that never cross the wire
Important: SQLAlchemy is not the SSoT for what the API returns. The API intentionally exposes a subset of DB fields.
Architecture: Contract-Driven and Deterministic
┌─────────────────┐ ┌────────────────────────┐ ┌─────────────────┐
│ SQLAlchemy │────▶│ Alembic Migrations │────▶│ Database │
│ Models │ │ (schema matches models)│ │ Schema │
└─────────────────┘ └────────────────────────┘ └─────────────────┘
┌─────────────────┐ ┌─────────────────┐ ┌───────────────────────┐
│ Pydantic │────▶│ FastAPI │────▶│ OpenAPI Spec │
│ Schemas │ │ Endpoints │ │ (exported JSON file) │
│ (contract SSoT) │ │ │ └──────────┬────────────┘
└─────────────────┘ └─────────────────┘ │
▼
┌───────────────────────┐
│ TypeScript Types │
│ (openapi-typescript) │
└──────────┬────────────┘
│
▼
┌───────────────────────┐
│ UI Contract Facade │
│ (clean aliases only) │
└───────────────────────┘Non-Negotiable Rules
1) No untyped API payloads
Endpoints must not return untyped dict payloads. Every response must have a Pydantic response model, even if it contains flexible sub-objects.
Examples:
- Bad:
return {"valid": True, "errors": []} - Good:
return FlowValidationResponse(valid=True, errors=[], ...)
For flexible data (e.g., metrics), keep the envelope typed and allow controlled flexibility:
- Prefer
dict[str, int | float | str | bool | None]overdict[str, Any] - Or introduce a
JsonValueunion type for controlled JSON payloads
2) UI must not define hand-written API DTOs
UI must not maintain manual “API-shaped” types for backend payloads. API DTO types come from OpenAPI generation, then are re-exported via the UI contract facade.
One payload has no generated type by design: the responses of an extension's own routes. Extensions are installed separately, so their routes are not in the platform's openapi.json: scripts/export-openapi.py loads no extension, even one installed in the development environment (the registry extension, which the API ships with), so the contract stays the platform's own. The page an extension describes (ExtensionPageRead, which is generated and re-exported like any other schema) names the keys to read, and extensionRequest in api/client.ts returns unknown; the page renderer reads values by those keys and never casts the payload to a hand-written type.
3) Generated types must be quarantined and never imported directly
UI code must not import from generated/api-types.ts directly. All UI imports go through a single facade (contract file).
4) Import/Export must be a first-class schema
Import/export is validated using Pydantic schemas. YAML is only a serialization format for a schema-defined bundle (versioned).
Enforcement Mechanisms
1) Deterministic OpenAPI export → TypeScript generation
The API's openapi.json is the single source of truth for type generation:
openapi.jsonis generated by FastAPI from Pydantic schemas (no manual editing)- TypeScript types are generated from
openapi.jsonusingopenapi-typescript
File locations:
apps/api/openapi.json- API OpenAPI spec (generated bytask api:regenerate)apps/ui/src/api/generated/api-types.ts- Generated TypeScript types (from OpenAPI, DO NOT EDIT)
The type generation script always reads from apps/api/openapi.json - no running API required.
Example:
# Two step action that combines export + generation (no containers needed) - recommended for local development
task api:regenerateThe local export option is useful for:
- CI environments without Docker
- Development agents that cannot spin up containers
- Quick schema regeneration during development
The contract as a reference
The same file is the API reference people read: Swagger UI at /api/docs, ReDoc at /api/redoc, and the documentation site, which renders the committed apps/api/openapi.json. What a reader needs around the paths and schemas is in apps/api/openapi_docs.py:
- The introduction (
info.description): base URL, authentication, organizations, list parameters and errors. - A display name (
x-displayName) and a one-sentence description for every tag, and the groups ReDoc lists them under (x-tagGroups). The tag itself stays what the router sets. A new router's tag needs a row in_TAG_CATALOG: Spectral (operation-tag-defined) andtests/api/test_openapi_docs.pyfail without one. A tag from an installed extension is listed under "Extensions" at run time. x-internal: trueon every operation that only the platform's own worker and UI call: all that take the internal token, the stream-ticket endpoints, and the live streams that need a ticket. The operations stay in the contract, because the type generation and the route checks read it.public_openapi()returns the contract without them; the API serves that at/api/openapi.public.json, and ReDoc reads it. Swagger UI shows the complete contract.- Two
serversentries: the relative/api, which makes "Try it out" work on an instance, andhttps://{host}/apifor a reader elsewhere.
The export sets a placeholder Keycloak address (https://keycloak.example.com/realms/hegemony), so the committed contract names no development host; a running instance serves its own. info.version is packages/core/version.py, which the release sets: between releases it reads 0.0.0.
2) UI Contract Facade (single import surface)
UI structure:
apps/ui/src/api/
generated/api-types.ts # generated, DO NOT EDIT
contract.ts # curated type aliases, EDIT
client.ts # typed client, EDIT
apps/ui/src/types/ui/ # UI-only view models, EDITcontract.ts:
- Defines clean aliases (
Device,DeviceCreate,FlowDefinition, …) - Covers read models as well as entities: aggregate response DTOs that have no CRUD counterpart (dashboard and platform-health snapshots, for example) and the outcomes of action endpoints (a connection or login test's result, or a run's moved maintenance window) are aliased here too, so no screen is tempted to hand-write the shape of a response the API already describes
- Exposes “advanced” raw types under branded names (
ApiOperations,ApiPaths) to discourage casual usage - Contains any migration aliases in a clearly marked “temporary” section
3) Enforce imports via linting (no bypass)
Oxlint is configured with no-restricted-imports to block direct imports from generated types:
// .oxlintrc.json
{
"rules": {
"no-restricted-imports": [
"error",
{
"patterns": [
{
"group": ["*/api/generated/*", "../api/generated/*"],
"message": "Import from '../api/contract' instead of directly from generated types."
}
]
}
]
},
"overrides": [
{
"files": ["src/api/contract.ts"],
"rules": {
"no-restricted-imports": "off"
}
}
]
}Only contract.ts is allowed to import from generated/. All other code must import from ../api/contract.
4) Schema alignment tests (intent-based, not brittle)
tests/api/test_schema_alignment.py verifies intent-based invariants:
- Response schemas are an allowed subset of DB model fields (not necessarily equal)
- Sensitive fields are denylisted and never exposed
- Create schema fields are a subset of writable model fields
- Update schemas are optional versions of create schemas
- Import schemas match create schemas (or differences are explicitly documented)
- Export schemas match response schemas (or differences are explicitly documented)
This prevents accidental drift without forcing “every DB column must be exposed.”
5) Import/Export validation with a versioned bundle schema
Import/export is validated with explicit versioned schemas — ImportBundle and ExportBundle plus ImportResult and ImportSummary in apps/api/schemas_config_exchange.py. YAML is only the serialization format:
ExportBundleis serialized to YAMLImportBundleis parsed from YAML and validated
Benefits:
- Safe evolvability via
schema_version - Better error messages
- Full end-to-end type generation for UI
Workflow for Schema Changes
Adding a new field
- DB model (
models.py): add column (if persisted) - Migration:
task db:revision -- "short message"(see migrations for the naming and review rules) - Pydantic schemas (
schemas.py): update relevant Create/Update/Response models (API contract) - Alignment tests: update rules/allowlists/denylists if needed
- OpenAPI export and Generate UI types:
task api:regenerate - Contract facade (
contract.ts): add thecomponents['schemas'][…]alias for any new schema. Rule 3 quarantines the generated file, so a new request or response model is unusable until it has an alias here;task ui:check:contract-typesfails on a hand-written one. - UI: update components importing types from the contract facade (
@api/contract) - Import/Export: update bundle schemas if field should be importable/exportable
Making a field nullable
- DB model: set
nullable=True(if persisted) - Migration: generate migration for nullability change
- Schema: change to
FieldType | Nonein Pydantic - Types: regenerate TS (may become
field?: T | null) - UI: handle null/optional cases explicitly
Files Reference
| Purpose | Location |
|---|---|
| SQLAlchemy models | apps/api/models.py |
| Alembic migrations | apps/api/alembic/versions/ |
| Pydantic schemas (contract SSoT) | apps/api/schemas.py |
| OpenAPI spec export (deterministic) | apps/api/openapi.json |
| TS types (generated, DO NOT EDIT) | apps/ui/src/api/generated/api-types.ts |
| UI contract facade (clean aliases) | apps/ui/src/api/contract.ts |
| Typed UI client | apps/ui/src/api/client.ts |
| UI-only view models (local state) | apps/ui/src/types/ui/* |
| Import/Export router | apps/api/routers/config_exchange.py |
| Import/Export bundle schemas | apps/api/schemas_config_exchange.py (or schemas.py) |
| Schema alignment tests | tests/api/test_schema_alignment.py |
| Type generation script | apps/ui/scripts/generate-types.mjs |
generate-types.mjs runs the locally installed openapi-typescript and oxfmt binaries directly, passing arguments as argv rather than through a shell. Run npm ci (or task ui:install) first: the script resolves both tools from the UI's installed dependencies and does not fetch them on demand.
CI Integration
Required CI steps (implemented in .github/workflows/ci.yml)
- Schema alignment tests (in
lint-and-testjob):
- name: Run tests
run: uv run pytest tests/ -v --tb=short- Verify generated types are up-to-date (in
typecheck-uijob):
- name: Verify generated types are up-to-date
run: |
npm run generate:types
git diff --exit-code src/api/generated/ || (echo "::error::Generated types are out of sync." && exit 1)- Oxlint enforces import restrictions (in
typecheck-uijob):
- name: Run linter (oxlint)
run: npm run lintAdditional recommendations
- Pin Node +
openapi-typescriptversions to reduce churn - Ensure OpenAPI export is stable and not environment-dependent
Manual vs Generated Types (Policy)
API DTO types
- Must come from OpenAPI generation
- Must be imported only from
apps/ui/src/api/contract.ts - No hand-written API DTOs in the UI codebase
UI-only types
Allowed only for:
- Table state, filters, form drafts, component props not matching API DTOs
- Derived view models (e.g., “row models”) that do not cross the network boundary
UI-only types must live under:
apps/ui/src/types/ui/*
Notes for Contributors
- If you add/modify an API schema, you must regenerate UI types.
- If you see an endpoint returning
dict, convert it to a response model immediately. - If you need a UI-only type, put it in
apps/ui/src/types/ui/and keep it clearly separated from API DTOs.