Multi-Tenancy

Hegemony isolates business data by organization (tenant). A single deployment hosts many orgs that share the same Keycloak realm, Temporal namespace, secret store, and Postgres database; rows are partitioned by an org_id column and every request runs in the context of exactly one active organization.

This document is the reference for how that isolation works. For the authentication layer it builds on (JWT validation, PATs, the BFF ticket flow, the RBAC middleware), see Authentication & Authorization.

Model at a glance

• Tenant boundary: the organizations table. Every org has an immutable, URL-safe slug (regex ^[a-z0-9][a-z0-9-]{0,61}[a-z0-9]$) and a UUID id.
• Membership: org_memberships binds a user to an org with a single per-org role (admin, operator, approver, auditor, viewer). A user may belong to many orgs.
• Active org per request: resolved by the RBAC middleware from the X-Org-Id header (or a PAT binding / sole membership) and validated against the caller's membership. It is exposed to handlers as CurrentOrg and mirrored into a request-scoped current_org_id ContextVar that drives ORM-level scoping.
• Default org: a fixed seed row (slug = "default", id = 00000000-0000-0000-0000-000000000001) created by migration 026. All pre-tenancy rows are backfilled to it, so existing single-tenant deployments keep working with no changes.

Request → active-org resolution

The middleware (apps/api/auth/rbac_middleware.py) resolves the active org after the principal is established, via resolve_org_context() (apps/api/auth/org_resolution.py). The fallback order is:

1. PAT binding. A PAT bound to an org (personal_access_tokens.org_id) locks the request to that org. A conflicting X-Org-Id header → 403. A PAT with NULL org is a platform token (admin-only to mint); it uses the header to select an org.
2. X-Org-Id header. Accepts a UUID or a slug. Unknown org → 404; deactivated org → 403; caller is not a member and not a platform admin → 403.
3. Sole membership. No header and the caller belongs to exactly one org → that org.
4. Platform-admin default. A platform admin (see below) with no header and no single membership falls back to the default org.
5. Unresolvable. No memberships → 403; multiple memberships with no header → 400 (X-Org-Id required).

Resolution results are cached briefly (see Caching).

Header contract

• Header name: X-Org-Id. Value is a UUID or a slug.
• CORS allows arbitrary request headers (allow_headers=["*"]). Custom reverse proxies must forward X-Org-Id — call this out in your proxy config.
• In dev mode (HEGEMONY_AUTH_DISABLED=true) a default-org platform-admin context is synthesized without a database lookup, and the header can select an org for local testing.

Roles & enforcement

Enforcement happens in one place — the RBAC middleware — which classifies every route and applies the matching policy.

Route classification

classify_route() (apps/api/auth/org_scope.py) is the single source of truth: it labels each path org-scoped or platform-scoped. Only org-scoped routes whose permission policy is role-based or authenticated require a resolved active org; public, internal_token, and custom policies (webhook triggers, worker callbacks, SSE) never hit org resolution — they derive the org from the target row server-side instead.

Org-scoped prefixes include /sites, /devices, /flows, /runs, /schedules, /approvals, /secrets, /variables, /webhooks, /notifications, /git-repositories, /file-repositories, /files, /registry-credentials, plus the /inventory/objects exception. Everything else — /auth, /orgs, /api-tokens, /audit-logs, /settings, /permissions, /workers, /inventory provider configs, /config-exchange, /platform-sync, /internal, /hooks, /bff — is platform-scoped. The tests/api/tenancy/test_route_classification.py invariant asserts every registered route classifies.

Per-org RBAC

For an org-scoped roles route, the required role is checked against the caller's effective org roles (effective_org_roles()), not raw Keycloak realm roles:

• Platform admin — a principal with the Keycloak realm role admin — is a cross-org super-admin: effective roles = the full set. Platform admins manage orgs and memberships and may act in any org.

• Members get their single membership role expanded per ORG_ROLE_EXPANSION:

  Membership role	Effective roles
  admin	admin, operator, approver, auditor, viewer
  operator	operator, auditor, viewer
  approver	approver, viewer
  auditor	auditor, viewer
  viewer	viewer

• PAT effective roles = the token's scopes, each expanded the same way.

On org-scoped routes the caller's roles come from per-org membership; the policy vocabulary of apps/api/auth/actions.yaml and per-route required-role sets are the same for every route. Non-admin realm roles are advisory on org routes; no Keycloak realm configuration is needed for org role assignments.

Platform-scoped routes keep realm-role semantics unchanged.

Shared vs isolated at a glance

A single deployment hosts many orgs on shared infrastructure, with all business data partitioned by org_id and a small platform layer that is intentionally common.

Infrastructure — shared, with an isolation mechanism

Resource	Shared?	How tenants are kept apart
Keycloak realm (issuer, login)	Shared	Authentication is common; authorization is per-org via org_memberships + per-org effective roles
Postgres database	Shared	org_id column + session-level query net + explicit org_filter()
Temporal namespace + task queue	Shared	workflow_id prefixed ‹slug›--‹run_id›; org_id/org_slug in workflow memo; the runs row is org-scoped
Secret store (OpenBao)	Shared backend	Secret paths namespaced orgs/‹slug›/secrets/…; browse/discovery filters foreign namespaces
Object storage (S3)	Shared bucket	Each org gets an org-scoped file_repositories row pointing at the shared bucket, provisioned automatically on org creation; non-default orgs get an isolated key prefix ‹s3_prefix›/orgs/‹slug›/…. Objects also hang off org-scoped rows (stored_files, run_artifacts)
Worker pool / scheduler	Shared	Execution infra is org-agnostic; each run carries its org context

Explicitly not per-tenant (by design): per-org Keycloak realms, per-org Temporal namespaces/queues, per-org secret backends.

Data — isolated per tenant (NOT shared)

Every row belongs to exactly one org; cross-org reads return 404/empty, except read-only access to the designated shared organization's resources. See ORG_SCOPED_DIRECT (NOT NULL org_id) and ORG_SCOPED_DERIVED (org via parent) in the scope matrix below.

Platform-global — shared across all tenants

Table	What it is
users	Shared identity directory (org access via memberships)
organizations	The tenant registry
org_memberships	User→org role grants
org_idp_mappings	IdP group-claim → org-role rules
secrets_backends	Secret-backend configuration (platform-admin managed)
permission_overrides	Route-level authorization policy overrides (one method + path)
permission_action_overrides	Action-level authorization policy overrides (a resource:verb action and all its routes)
worker_heartbeats, maintenance_jobs	Shared worker/scheduler state
config_exchange_locks	Cross-org CX coordination locks

Hybrid — nullable org_id (per-org or platform)

Table	Behavior
audit_logs	Stamped with the acting org; platform actions have none
personal_access_tokens	Bound to an org, or NULL = platform PAT (admin-only)
config_exchange_runs, config_exchange_import_plans	Org-scoped in an org, else platform

Scope matrix

The classification of every ORM model lives in registries in apps/api/tenancy.py and is asserted exhaustive by tests/api/tenancy/test_model_scope_registry.py (every Base subclass is classified exactly once; direct models must carry OrgScopedMixin, so a new model cannot silently dodge tenancy).

• ORG_SCOPED_DIRECT — carry a NOT NULL org_id FK (ondelete=RESTRICT) via OrgScopedMixin: sites, devices, inventory_objects, inventory_sync_history, inventory_discovered_object_types, flow_definitions, runs, schedules, approval_requests, secrets, variables, webhook_endpoints, notification_destinations, flow_notification_subscriptions, git_repositories, file_repositories, file_repository_folders, stored_files, registry_credentials, inventory_provider_configs (destination org), platform_sync_profiles.
• ORG_SCOPED_NULLABLE — nullable plain org_id column: audit_logs, personal_access_tokens, config_exchange_runs, config_exchange_import_plans. (NULL = platform-level.)
• ORG_SCOPED_DERIVED — no column; scoped through a parent (e.g. run children via run.org_id, flow versions/attachments via the flow, webhook deliveries via the endpoint).
• PLATFORM_GLOBAL — no tenant: users, organizations, org_memberships, org_idp_mappings, secrets_backends, permission_overrides, permission_action_overrides, worker_heartbeats, maintenance_jobs, config_exchange_locks.

Unique constraints on org-scoped tables include org_id (e.g. unique(org_id, name) on secrets/variables/file_repositories/ notification_destinations; (org_id, name) WHERE deleted_at IS NULL on git_repositories; unique(org_id, host) on registry_credentials), so the same name may coexist across orgs.

Query scoping — two layers

Isolation is enforced by two complementary layers (apps/api/tenancy_orm.py):

1. Explicit filters (primary). Handlers add org_filter(Model, org) / org_scoped(stmt, Model, org) to queries and stamp org_id=org.org_id on writes. These are greppable, reviewable, and use the composite (org_id, …) indexes directly.
2. Session guards (safety net). install_tenancy_guards() registers Session-level events, active only while current_org_id holds a real org:
   • do_orm_execute attaches with_loader_criteria(OrgScopedMixin, …) to every ORM SELECT, so a missed explicit filter degrades to "no cross-tenant rows" rather than a leak.
   • before_flush stamps org_id on new OrgScopedMixin rows that were not stamped explicitly.

Outside a request (worker callbacks, maintenance, startup, tests) the ContextVar is UNSCOPED and the guards are inert.

Known gaps of the safety net — why explicit filters stay mandatory:

• Core update() / delete() statements are not touched by with_loader_criteria.
• session.get() identity-map hits bypass the SELECT criteria — use a filtered select or follow with require_same_org.
• Column refreshes and relationship lazy-loads are deliberately excluded.

The DB NOT NULL + FK is the final backstop against an unscoped write.

Cross-org integrity

References between org-scoped rows are validated in the application at every write point (not with composite FKs): require_same_org(entity, org) raises 404 when a referenced entity belongs to another org. Examples: device.site_id, subscription.flow_id + destination_id, webhook_endpoints.flow_id. Cross-org access always reads as 404 — a resource's existence in another tenant is never disclosed. Missing membership (as opposed to cross-org reference) is 403.

Shared organization (cross-org read-only content)

At most one org may carry the organizations.is_shared flag (partial unique index). While that org is active, everything it owns is readable — read-only — from every other org. It is an ordinary organization: its members manage its content with their normal per-org roles; everyone else gets an implicit read-only view. Isolation between regular orgs is completely unaffected.

Designation is data, not deployment configuration. A "Shared Organization" (slug shared, fixed UUID …0002) is seeded by migration 031 on every deployment — mirroring the default org — but starts deactivated, so sharing is off out of the box. A platform admin enables it by activating the org in Settings → Organizations (or re-designates any other org via the is_shared toggle; clearing the current designation first — an intentional two-step). The flag travels in config-exchange bundles (organizations[].is_shared, applied authoritatively on import) and there is no environment variable.

Scope — no object-type allowlist. Every org-scoped model participates: whatever the shared org's members create (flows, variables, secrets, destinations, git repos, sites, devices, …) is visible to other orgs where the read paths admit it. What to put there is up to the users of the shared org.

Mechanics (apps/api/tenancy.py + tenancy_orm.py):

• RBACMiddleware resolves the shared org through the TTL org cache (get_shared_org: flagged AND active) and sets the shared_org_id ContextVar only for interactive org-scoped requests. Config-exchange import/export, platform sync, git push, and background bindings never get the fallback — a tenant's export can never serialize shared-org rows as its own (fail closed).
• With the fallback active, the SELECT net widens to "own org OR shared org" for every OrgScopedMixin model. Read paths with explicit filters opt in via org_read_filter / require_readable_org; mutations keep the strict helpers.
• A before_flush write guard rejects any insert into, update/delete of, or re-parenting out of the shared org from another org's context with 403 "Shared organization resources are read-only" — mutations of shared rows are structurally impossible, not just unrouted.

Execution semantics — the shared org owns templates, never execution. A run of a shared flow executes as its consumer org: run.org_id, the Temporal workflow_id prefix/memo, device targeting, variable resolution and the secret namespace are all the consumer's (create_run_from_request stamps org / run_org_id / the parent run's org, and rejects any other cross-org flow with 404). Schedules and webhook endpoints of a consumer org may target shared flows; their runs execute as the consumer. On-run git freshness sync is skipped for shared flows (it would write to the shared row); the shared org's own activity keeps them fresh.

Resolution semantics — {{ vars.NAME }} merges the shared org's variables underneath the run org's own (own always wins on a name collision; see GET /internal/variables). The worker-side secret() guard admits orgs/‹shared-slug›/… in addition to the run org's own namespace: the API resolves the active shared org at run creation and stamps its slug into the Temporal workflow input (shared_org_slug), so the worker needs no configuration or DB access of its own. Whether the slug is stamped at all is governed by HEGEMONY_SHARED_ORG_SECRET_SCOPE: shared_resources (the default) stamps it only when the work touches a shared resource — a shared-org flow, a shared-org device the run executes against (a device parked in a hidden or disabled target field does not count), a shared-org destination the flow is subscribed to notify, or a shared-org destination of a notification dispatch (shared_secret_namespace_admitted() in apps/api/services/shared_org.py). The admission is run-wide: one shared device among the targets lets every step of that run read the namespace. always stamps it for every run and notification in every org, so any flow author can resolve any shared-org secret value — the behaviour from before the setting existed, kept only when set explicitly. With an unknown run org the whole orgs/ namespace — shared included — stays denied. Consumers may also wire their own (or shared) notification destinations to a shared flow; the subscription row lives in the consumer org, so consumer runs notify consumer-configured destinations.

Surfaces — list/read responses carry a shared: bool flag across the board: flows, variables, secrets, notification destinations, git repositories, sites, devices, runs, schedules, and webhook endpoints. The UI badges those rows "Shared" and swaps mutations for duplicate-into-my-org (or nothing, for view-only rows like runs). Duplicating is the supported way to fork shared content into a tenant. Mutations of shared rows keep failing — 404 from the strict write filters or 403 from the flush guard — and manual triggers of shared schedules/webhooks stay strict, since those would execute as the shared org. Settings → Organizations shows the designation and lets platform admins toggle it per org.

PATs and tenancy

personal_access_tokens.org_id binds a token to an org:

• Bound PAT — locked to its org; a conflicting X-Org-Id → 403. Org admins may mint only org-bound PATs.
• Platform PAT (NULL org) — admin-only to mint; selects an org via X-Org-Id.

Existing PATs are backfilled to the default org by migration 027, so automation keeps working after upgrade.

A run step's token (run_access_tokens, org derived through its run) is bound the same way: to its run's organization, with no header override. Managed Terraform/OpenTofu states (tf_states, org-scoped; tf_state_versions, derived) hold secrets, so every state query filters on the caller's own org explicitly: the shared-organization read fallback never applies to them.

Temporal / worker propagation

Runs carry their org through Temporal:

• workflow_id = f"{org_slug}--{run_id}"; nothing parses the run id back out of the workflow id — signal paths use the Run.workflow_id column.
• org_id and org_slug travel in the workflow input payload and the Temporal memo.
• /internal/* callbacks derive the org from the Run row server-side and never trust worker-supplied org values. Nested runs inherit the parent run's org and validate the child flow is in the same org.
• Workers read org from input with dict.get(...) fallbacks so in-flight pre-upgrade workflows (which lack the org keys) keep running — determinism/compat is preserved.

Secrets / secret-store paths

Secrets are stored at orgs/<org_slug>/secrets/<name> in the secret store. Because the default org's slug is default — the segment used before tenancy — no path migration is required for tenancy. The policies already glob orgs/*, so no policy change is needed either.

That covers paths and policies only. Migrating from a pre-OpenBao bundled store is a separate matter and does require moving the secret values: the new store starts empty, and there is no in-place upgrade. See the runbook in Internal OpenBao.

Runtime template enforcement (worker)

Flow templates can build strings dynamically, so save-time scanning cannot protect the per-org namespaces — the worker enforces tenancy at resolution time, on every path (step templates, device credentials, notification formatting, sync thread-pool helpers):

• {{ secret('…') }} — the run's org_slug travels in the workflow input; TemplateResolver rejects any path under orgs/<other-slug>/… before touching the backend (no existence oracle). Paths with empty, . or .. segments are rejected outright. Legacy non-orgs/ paths stay resolvable. With no org context (only possible for pre-tenancy workflow histories or a lost thread context), the whole orgs/ namespace is denied — fail closed.
• {{ env() }} / {{ file() }} — refused in every run: they read the worker's own environment and secrets directory, which hold platform credentials. A TemplateResolver refuses both unless its caller opts in (allow_env_and_file), and only the platform's own configuration opts in: secret backend configs and inventory provider tokens. The API refuses them the same way in git repository, file repository and registry credential refs, and save-time checks (422) and imports reject them in all org content.
• {{ vars.NAME }} — the worker fetches /internal/variables?org_id=… per run org and caches per org. No org context means an empty variable set, except for legacy pre-tenancy histories which use the API's documented default-org fallback (those runs are default-org by migration definition).

Audit

emit_audit_log() stamps audit_logs.org_id from the current_org_id ContextVar by default (NULL for platform/unscoped requests), so every audit entry written inside an org-scoped request is attributed to its tenant with no caller changes. Callers may pass an explicit org_id to override. The /audit-logs list endpoint accepts an optional org_id filter for platform admins auditing a single tenant.

Caching

Org lookups and membership roles are cached in-process with a short TTL (apps/api/auth/org_cache.py):

• ORG_CACHE_TTL_SECONDS = 30 — org-by-ref and (sub, org_id) → role.
• USER_PROVISION_THROTTLE_SECONDS = 60 — login-time user upsert throttle.

Tradeoff: a revoked membership or deactivated org is honored within ≤30 s across replicas (same model as the permission-overrides cache). Mutations that change membership or org state call invalidate_org_caches().

User provisioning & membership bootstrap

On the first authenticated JWT request, the middleware (best-effort, never fatal) upserts the user by sub and then, in order:

1. IdP group-claim sync (if HEGEMONY_ORG_IDP_SYNC=true) — reconciles idp-sourced memberships from the token's group claim (see below).
2. Default-org auto-join (if the user still has zero memberships and HEGEMONY_ORG_AUTO_JOIN=true, the default) — joins the default org with a role mapped from the caller's realm roles (highest of admin/operator/approver/auditor, else viewer).

Set HEGEMONY_ORG_AUTO_JOIN=false after onboarding to require explicit membership (manual grant or IdP mapping).

Federated identity — IdP → org mapping

Org membership can be derived from identity-provider group claims so any brokered IdP (Microsoft Entra ID / Azure AD, Okta, another Keycloak) or Keycloak-local groups drive access without manual invites. This follows the Grafana/GitLab/Vault pattern: the IdP is only an authentication source; Hegemony's database stays the source of truth for org membership.

• Trust anchor: Keycloak (single realm, single issuer). Upstream IdPs are brokered in Keycloak; every user receives a Keycloak-signed JWT, so Hegemony validates one issuer regardless of the upstream IdP.
• Mappings: org_idp_mappings rows say "a token whose <claim> contains <value> grants <role> in this org". (claim, value, org_id) is unique. The table is platform-global: org_id is a managed reference to the target org (like org_memberships), not a tenant stamp the session net filters on.
• Reconciliation: on login, reconcile_idp_memberships() (apps/api/services/org_provisioning.py) reads the configured claim (default groups), matches active mappings, and computes the desired {org → strongest role}. It adds/upgrades idp-sourced memberships, never touches manual ones (explicit grants always win), and — in authoritative mode — removes idp memberships the claims no longer justify. Org caches are invalidated on any change.
• Membership source: org_memberships.source is manual or idp. The admin UI shows idp members read-only (managed via IdP Mappings) so the two systems never overwrite each other.

Admin management lives under /api/orgs/{org_ref}/idp-mappings (org admin or platform admin) and the Settings → Organizations → IdP Mappings UI. The full operator guide, including Azure AD / generic OIDC / SAML brokering and a demo walk-through, is IdP → Organization Mapping.

Plugin & sibling-repo boundary

Tenancy is enforced entirely host-side; the out-of-tree plugin wheels are org-agnostic by design:

• Inventory plugins treat provider_id as an opaque string (never parsed), keep no local or module-level state, and derive external_id only from source-system records — the org-embedding provider-id form is transparent to them. Git-provider workdirs are host-managed, keyed by the per-org git_repo_id.
• Secret backends are leaf components: the host validates the orgs/<slug>/secrets/… namespace before any backend call. The store's backend maps the namespace onto a real path hierarchy (its admin-configured path_prefix relocates the whole tree uniformly and cannot be influenced by tenants). The 1Password backends map paths as <vault>/<item> — isolation holds (the slug is part of the item title) but org folder browsing is not supported; see that plugin's README.
• Notification transports are stateless per send, never call platform APIs, and delegate all secret resolution/templating to injected host services.
• Step handlers receive no org context; every platform call goes through the host-built internal client, and the worker binds the run's org before resolving any secret or variable (set_run_org_context).
• Demo data declares organization: default plus the organizations directory in its bundle, and every demo secret folder is namespaced under orgs/default/…, so the bootstrap import binds deterministically to the default org.

Settings

Setting	Env var	Default	Purpose
default_org_slug	HEGEMONY_DEFAULT_ORG_SLUG	default	Slug of the seed org used for backfill and platform-admin fallback
org_auto_join	HEGEMONY_ORG_AUTO_JOIN	true	Auto-join new users to the default org on first login
shared_org_secret_scope	HEGEMONY_SHARED_ORG_SECRET_SCOPE	shared_resources	Which runs may resolve the shared org's secrets: shared_resources (only runs executing a shared flow, targeting a shared device, or notifying a shared destination) or always (every run - the legacy behaviour)

The shared-org designation is deliberately not a setting — it lives on organizations.is_shared (see the shared-organization section above) and is managed via the org admin API/UI or config-exchange import. | org_idp_sync | HEGEMONY_ORG_IDP_SYNC | false | Derive org membership from IdP group claims on login | | org_idp_group_claim | HEGEMONY_ORG_IDP_GROUP_CLAIM | groups | JWT claim read for group values | | org_idp_sync_authoritative | HEGEMONY_ORG_IDP_SYNC_AUTHORITATIVE | true | Revoke idp memberships the claims no longer grant (mirror); false = additive |

Testing

The tenancy suite (tests/api/tenancy/) is the regression net:

• test_org_isolation.py — a parameterized harness over org-scoped endpoints: seed rows in two orgs, assert header-filtered results and cross-org GET → 404. This is the missed-spot detector.
• test_model_scope_registry.py — blocks unclassified new models.
• test_tenancy_guards.py, test_org_resolution.py, test_org_rbac.py, test_route_classification.py, test_cross_org_references.py, test_secrets_org_paths.py, test_sse_org_isolation.py, test_audit_org_stamping.py, test_user_provisioning.py, test_idp_membership_sync.py (JIT reconciliation: matching grant, strongest role, manual-preserved, authoritative removal, additive keep, multi-org).

Out of scope (by design)

• Per-org Temporal namespaces or queues; per-org Keycloak realms; per-org secret backends. (Cross-org membership is derived from JWT group claims — see Federated identity — but each org does not get its own realm or issuer.)

Migrations 026–032 introduce the schema for all of the above; a fresh deployment gets it by running the standard migrations.