Security Model¶
Verity implements a defence-in-depth security architecture spanning authentication, authorisation, network isolation, secrets management, data encryption, and continuous security scanning.
Authentication¶
Azure AD / Microsoft Entra ID — OAuth 2.0 (OIDC)¶
All API requests are authenticated via JSON Web Tokens (JWT) issued by Microsoft Entra ID (formerly Azure AD). The API Gateway validates every incoming token before routing to downstream services.
sequenceDiagram
autonumber
participant User as User / Client
participant Entra as Microsoft Entra ID
participant API as API Gateway
participant SVC as Downstream Service
User->>Entra: Authenticate (OAuth 2.0 / OIDC)
Entra-->>User: Access Token (JWT)
User->>API: Request + Bearer Token
API->>API: Validate JWT signature (JWKS)
API->>API: Verify issuer, audience, expiry
API->>API: Extract roles from token claims
API->>SVC: Forward request + security context
SVC-->>API: Response
API-->>User: Response
Token Validation¶
| Check | Details |
|---|---|
| Signature | RS256 — verified against the Entra ID JWKS endpoint. Keys are cached and rotated automatically. |
Issuer (iss) |
Must match https://login.microsoftonline.com/{tenant_id}/v2.0. |
Audience (aud) |
Must match the Verity application registration's client ID. |
Expiry (exp) |
Tokens are rejected if expired. Clock skew tolerance: 30 seconds. |
Roles (roles) |
Extracted from the roles claim; mapped to Verity RBAC roles. |
Configuration¶
| Environment Variable | Description | Example |
|---|---|---|
AZURE_TENANT_ID |
Entra ID tenant identifier. | a1b2c3d4-... |
AZURE_CLIENT_ID |
Application (client) ID for Verity. | e5f6g7h8-... |
AZURE_AUTHORITY |
OIDC authority URL. | https://login.microsoftonline.com/{tenant} |
API_AUTH_DISABLED |
Dev mode only — disables authentication. | true / false |
Development Mode¶
Development Only
Setting API_AUTH_DISABLED=true disables all authentication and authorisation
checks. This must never be used in production environments.
When API_AUTH_DISABLED=true:
- All requests are treated as authenticated.
- A synthetic admin identity is injected into the security context.
- RBAC checks are bypassed.
- An audit event with
action=auth.dev_bypassis logged on every request.
Authorisation — Role-Based Access Control (RBAC)¶
Verity enforces RBAC using four roles defined as Entra ID App Roles. Roles are
carried in the JWT roles claim and evaluated by the API Gateway middleware.
Role Definitions¶
| Role | Scope | Description |
|---|---|---|
verity.admin |
Full platform | Full administrative access — manage connectors, policies, users, and all API endpoints. |
verity.reviewer |
Reviews | Access to review queues, submit decisions, view scores and grant details for assigned reviews. |
verity.reader |
Read-only | Read-only access to dashboards, scores, grants, and audit logs. Cannot submit decisions or modify configuration. |
verity.connector |
Machine-to-machine | Service account role for connectors — limited to event ingestion and checkpoint management endpoints. |
Role-to-Endpoint Mapping¶
| Endpoint Group | verity.admin |
verity.reviewer |
verity.reader |
verity.connector |
|---|---|---|---|---|
GET /api/v1/principals |
✅ | ✅ | ✅ | ❌ |
GET /api/v1/assets |
✅ | ✅ | ✅ | ❌ |
GET /api/v1/grants |
✅ | ✅ | ✅ | ❌ |
GET /api/v1/scores |
✅ | ✅ | ✅ | ❌ |
GET /api/v1/reviews |
✅ | ✅ | ✅ | ❌ |
POST /api/v1/reviews/{id}/decide |
✅ | ✅ | ❌ | ❌ |
GET /api/v1/audit |
✅ | ❌ | ✅ | ❌ |
GET /api/v1/reports |
✅ | ❌ | ✅ | ❌ |
POST /api/v1/connectors/*/sync |
✅ | ❌ | ❌ | ✅ |
PUT /api/v1/connectors/*/checkpoint |
✅ | ❌ | ❌ | ✅ |
POST /api/v1/events |
✅ | ❌ | ❌ | ✅ |
*/admin/* (configuration) |
✅ | ❌ | ❌ | ❌ |
GET /api/v1/metrics |
✅ | ✅ | ✅ | ✅ |
GET /healthz |
✅ | ✅ | ✅ | ✅ |
RBAC Middleware¶
flowchart TD
A[Incoming Request] --> B{JWT Present?}
B -->|No| C[401 Unauthorized]
B -->|Yes| D{JWT Valid?}
D -->|No| C
D -->|Yes| E[Extract roles claim]
E --> F{Role authorized<br/>for endpoint?}
F -->|No| G[403 Forbidden]
F -->|Yes| H[Forward to handler]
H --> I[Emit audit event]
Network Policies¶
Verity deploys 21 Kubernetes NetworkPolicies that enforce a zero-trust network model within the cluster. All traffic is denied by default; only explicitly permitted flows are allowed.
Policy Summary¶
| # | Policy Name | Source | Destination | Port(s) | Purpose |
|---|---|---|---|---|---|
| 1 | default-deny-ingress |
* |
verity/* |
— | Deny all ingress by default. |
| 2 | default-deny-egress |
verity/* |
* |
— | Deny all egress by default. |
| 3 | allow-api-ingress |
Ingress controller | api-gateway |
8000 | External API traffic. |
| 4 | allow-dashboard-ingress |
Ingress controller | dashboard-ui |
3000 | Dashboard traffic. |
| 5 | allow-api-to-pg |
api-gateway |
postgresql |
5432 | API reads/writes. |
| 6 | allow-api-to-redis |
api-gateway |
redis |
6379 | Cache lookups. |
| 7 | allow-connectors-to-kafka |
connector-* |
kafka |
9092 | Produce raw events. |
| 8 | allow-connectors-egress |
connector-* |
External APIs | 443 | Reach source systems. |
| 9 | allow-ingest-to-kafka |
ingest-worker |
kafka |
9092 | Consume/produce events. |
| 10 | allow-normalise-to-kafka |
normalise-engine |
kafka |
9092 | Consume/produce events. |
| 11 | allow-normalise-to-pg |
normalise-engine |
postgresql |
5432 | Write normalised data. |
| 12 | allow-decay-to-kafka |
decay-engine |
kafka |
9092 | Consume/produce events. |
| 13 | allow-decay-to-pg |
decay-engine |
postgresql |
5432 | Read grants, write scores. |
| 14 | allow-review-to-kafka |
review-generator |
kafka |
9092 | Consume/produce events. |
| 15 | allow-review-to-pg |
review-generator |
postgresql |
5432 | Write review packets. |
| 16 | allow-workflow-to-temporal |
workflow-engine |
temporal |
7233 | Workflow orchestration. |
| 17 | allow-workflow-to-kafka |
workflow-engine |
kafka |
9092 | Produce decided events. |
| 18 | allow-remediation-to-kafka |
remediation-executor |
kafka |
9092 | Consume decided events. |
| 19 | allow-remediation-egress |
remediation-executor |
External APIs | 443 | Execute remediations. |
| 20 | allow-audit-to-kafka |
audit-writer |
kafka |
9092 | Consume audit events. |
| 21 | allow-audit-to-clickhouse |
audit-writer |
clickhouse |
8123 | Write audit records. |
Network Architecture Diagram¶
flowchart TB
subgraph EXTERNAL["External"]
ING[Ingress Controller]
SRC[Source Systems<br/>Entra ID / Snowflake / etc.]
end
subgraph VERITY["namespace: verity"]
API[api-gateway :8000]
DUI[dashboard-ui :3000]
CON[connectors]
IW[ingest-worker]
NE[normalise-engine]
DE[decay-engine]
RG[review-generator]
WE[workflow-engine]
RE[remediation-executor]
AW[audit-writer]
end
subgraph INFRA["namespace: verity-infra"]
PG[(PostgreSQL :5432)]
CH[(ClickHouse :8123)]
KF[[Kafka :9092]]
RD[(Redis :6379)]
TM[Temporal :7233]
end
ING -->|443→8000| API
ING -->|443→3000| DUI
CON -->|443| SRC
RE -->|443| SRC
API --> PG
API --> RD
CON --> KF
IW --> KF
NE --> KF
NE --> PG
DE --> KF
DE --> PG
RG --> KF
RG --> PG
WE --> TM
WE --> KF
RE --> KF
AW --> KF
AW --> CH
Secrets Management¶
Kubernetes Secrets¶
All sensitive configuration is stored in Kubernetes Secret objects and mounted
as environment variables. No secrets are committed to source control.
| Secret Name | Contents | Used By |
|---|---|---|
verity-db-credentials |
POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB |
All services with DB access |
verity-clickhouse-credentials |
CLICKHOUSE_USER, CLICKHOUSE_PASSWORD |
audit-writer, compliance-reporter |
verity-kafka-credentials |
KAFKA_SASL_USERNAME, KAFKA_SASL_PASSWORD |
All Kafka consumers/producers |
verity-redis-credentials |
REDIS_PASSWORD |
api-gateway |
verity-azure-credentials |
AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET |
api-gateway, connectors |
verity-snowflake-credentials |
SNOWFLAKE_ACCOUNT, SNOWFLAKE_USER, SNOWFLAKE_PASSWORD |
connector-snowflake |
verity-databricks-credentials |
DATABRICKS_HOST, DATABRICKS_TOKEN |
connector-databricks |
verity-fabric-credentials |
FABRIC_TENANT_ID, FABRIC_CLIENT_ID, FABRIC_CLIENT_SECRET |
connector-fabric |
verity-temporal-credentials |
TEMPORAL_ADDRESS, TEMPORAL_NAMESPACE |
workflow-engine |
Helm Integration¶
All Helm charts support an existingSecret pattern to reference pre-created
secrets rather than generating them:
# values.yaml
postgresql:
auth:
existingSecret: verity-db-credentials
secretKeys:
userPasswordKey: POSTGRES_PASSWORD
redis:
auth:
existingSecret: verity-redis-credentials
secretKeys:
passwordKey: REDIS_PASSWORD
Data Encryption¶
Encryption in Transit¶
| Path | Protocol | Details |
|---|---|---|
| Client → Ingress | TLS 1.3 | Terminated at the Kubernetes Ingress controller (cert-manager + Let's Encrypt). |
| Ingress → API Gateway | TLS 1.2+ | Cluster-internal mTLS via service mesh (optional) or plain HTTP behind the ingress. |
| Service → PostgreSQL | TLS 1.2+ | sslmode=require enforced on all connection strings. |
| Service → ClickHouse | TLS 1.2+ | HTTPS interface (port 8443) in production. |
| Service → Kafka | TLS 1.2+ | SASL_SSL security protocol with SCRAM-SHA-512. |
| Service → Redis | TLS 1.2+ | TLS-enabled Redis with password authentication. |
| Service → Temporal | TLS 1.2+ | Temporal Cloud or self-hosted with mTLS. |
Encryption at Rest¶
| Store | Method | Details |
|---|---|---|
| PostgreSQL | ADE (cloud) / LUKS (self-hosted) | Transparent data encryption at the volume level. |
| ClickHouse | ADE (cloud) / LUKS (self-hosted) | Volume-level encryption for all data directories. |
| Kafka | ADE (cloud) / LUKS (self-hosted) | Broker log directories encrypted at rest. |
| Kubernetes Secrets | etcd encryption | Secrets encrypted in etcd using aescbc or KMS provider. |
Security Scanning¶
Verity integrates three layers of security scanning into the CI/CD pipeline.
Scanning Tools¶
| Tool | Type | Stage | Scope | Configuration |
|---|---|---|---|---|
| Trivy | Container scanning | CI — post-build | All Docker images | Scans for OS and library CVEs. Blocks CRITICAL and HIGH severity. |
| Bandit | SAST (Python) | CI — pre-build | All Python source | Detects common Python security anti-patterns (SQL injection, hardcoded passwords, etc.). |
| Semgrep | Pattern-based SAST | CI — pre-build | All source code | Custom and community rules for secrets-in-code, insecure deserialization, SSRF. |
CI Pipeline Integration¶
flowchart LR
A[Git Push] --> B[Lint & Format]
B --> C[Bandit SAST]
B --> D[Semgrep Scan]
C --> E{Security<br/>Gate}
D --> E
E -->|Pass| F[Unit Tests]
E -->|Fail| G[❌ Block Merge]
F --> H[Build Docker Images]
H --> I[Trivy Scan]
I --> J{Vulnerability<br/>Gate}
J -->|Pass| K[Push to Registry]
J -->|CRITICAL/HIGH| G
K --> L[Deploy to Staging]
Scanning Policies¶
| Policy | Rule |
|---|---|
| Bandit severity threshold | medium — all medium and above findings block the pipeline. |
| Bandit confidence threshold | medium — low-confidence findings are reported but do not block. |
| Semgrep severity threshold | warning — warnings and errors block the pipeline. |
| Trivy severity threshold | HIGH,CRITICAL — only high and critical CVEs block the pipeline. |
| Trivy ignore unfixed | true — unfixed vulnerabilities are reported but do not block. |
| Scan frequency | Every pull request + nightly full scans on main. |
Audit Trail¶
ClickHouse Audit Log¶
Every significant action in the platform is recorded in the verity_audit table
in ClickHouse. The audit log is append-only and immutable — no UPDATE or
DELETE operations are permitted on the audit table.
See Database Schema — ClickHouse for the complete schema definition.
What Is Audited¶
| Category | Events |
|---|---|
| Authentication | Login success, login failure, token refresh, dev-mode bypass. |
| Authorisation | Access denied (403), role escalation attempts. |
| Data Access | API reads of principals, assets, grants, scores, reviews. |
| Score Changes | Score computed, score threshold crossed. |
| Review Lifecycle | Review created, assigned, notified, decided, escalated, closed. |
| Remediation | Remediation requested, executed, confirmed, failed. |
| Configuration | Connector created, policy changed, threshold updated. |
| Connector Sync | Sync started, completed, failed, checkpoint updated. |
Audit Retention¶
| Requirement | Implementation |
|---|---|
| Minimum retention | 7 years (SOX, SOC 2 compliance). |
| ClickHouse TTL | occurred_at + INTERVAL 7 YEAR DELETE — automatic partition expiry. |
| Kafka retention | verity.audit.all topic: 7-year retention for replay capability. |
| Immutability | Enforced at application level — audit-writer only executes INSERT. |
| Tamper evidence | Audit events include cryptographic checksums (planned: Merkle tree). |
Compliance Considerations¶
| Regulation | Controls |
|---|---|
| SOX | Segregation of duties (reviewer ≠ admin), complete audit trail, access certification workflows. |
| SOC 2 | Encryption in transit and at rest, RBAC, continuous monitoring, incident response audit trail. |
| GDPR | Data minimisation in audit logs, right-to-erasure support for PII fields (anonymisation), data classification tagging. |
| HIPAA | Access logging, encryption, minimum necessary access principle enforced by decay scoring. |
Security Checklist¶
Production Deployment
Use this checklist before deploying Verity to production.
-
API_AUTH_DISABLEDis set tofalse(or unset). - All Kubernetes Secrets are created and referenced via
existingSecret. - TLS certificates are provisioned for all external endpoints.
-
sslmode=requireis set on all PostgreSQL connection strings. - Kafka is configured with
SASL_SSLand SCRAM-SHA-512. - All 21 NetworkPolicies are applied to the
verityandverity-infranamespaces. - Trivy scanning is enabled in CI with
CRITICAL,HIGHblocking. - Bandit and Semgrep are running on every pull request.
- Nightly security scans are scheduled on
main. - ClickHouse audit table has no
UPDATEorDELETEgrants for any user. - etcd encryption is enabled for Kubernetes Secrets.
- Ingress controller enforces TLS 1.2+ with strong cipher suites.