Cloud Data Platform Security¶

The Challenge¶

Modern data teams move fast — spinning up Databricks notebooks, Synapse SQL pools, and Fabric lakehouses to meet business deadlines. Access is granted liberally to unblock work, but almost never revoked once the project ends.

How Verity Helps¶

Verity connects to each data platform, correlates usage telemetry with permission grants, and scores every grant on a continuous 0–100 decay scale.

flowchart LR
    subgraph Data Platforms
        DB["Databricks"]
        SY["Azure Synapse"]
        FB["Microsoft Fabric"]
    end

    subgraph Verity
        I["Ingest Plane"]
        N["Normalise Plane"]
        S["Score Plane"]
        R["Review Plane"]
        X["Remediation Plane"]
    end

    DB -->|REST API| I
    SY -->|REST API| I
    FB -->|REST API| I
    I --> N --> S --> R --> X
    X -->|Revoke / Downgrade| DB
    X -->|Revoke / Downgrade| SY
    X -->|Revoke / Downgrade| FB

    style I fill:#7c4dff,color:#fff,stroke:none
    style N fill:#651fff,color:#fff,stroke:none
    style S fill:#536dfe,color:#fff,stroke:none
    style R fill:#448aff,color:#fff,stroke:none
    style X fill:#40c4ff,color:#000,stroke:none

Key Capabilities¶

Example Scenario¶

Northwind Analytics — 800-Person Data Organisation¶

Northwind connects Databricks, Azure Synapse, and Microsoft Fabric to Verity.

Discovery¶

Scoring Results (Week 1)¶

pie title Decay Score Distribution — 8,600 Grants
    "Low (0–29)" : 3400
    "Medium (30–59)" : 2900
    "High (60–79)" : 1500
    "Critical (80–100)" : 800

Review & Remediation¶

Verity generates 2,300 review packets (High + Critical), routed to 85 workspace owners.

Within 14 days:

• 780 Critical grants auto-remediated (revoked or downgraded).
• 1,220 High grants decided by data owners.
• 300 remaining escalated per SLA.

Before & After¶

Connector Deep Dive¶

Databricks Connector¶

# config/connectors/databricks.yaml
connector:
  type: databricks
  workspace_url: https://adb-1234567890.azuredatabricks.net
  auth:
    method: service_principal
    client_id: ${DATABRICKS_CLIENT_ID}
    client_secret: ${DATABRICKS_CLIENT_SECRET}
  sync:
    schedule: "0 */6 * * *"       # Every 6 hours
    resources:
      - workspace_permissions
      - cluster_permissions
      - sql_warehouse_permissions
      - unity_catalog_grants

Synapse Connector¶

# config/connectors/synapse.yaml
connector:
  type: synapse
  workspace_name: northwind-analytics
  auth:
    method: managed_identity
  sync:
    schedule: "0 */6 * * *"
    resources:
      - sql_pool_permissions
      - spark_pool_permissions
      - pipeline_permissions
      - workspace_roles

Fabric Connector¶

# config/connectors/fabric.yaml
connector:
  type: fabric
  tenant_id: ${AZURE_TENANT_ID}
  auth:
    method: service_principal
    client_id: ${FABRIC_CLIENT_ID}
    client_secret: ${FABRIC_CLIENT_SECRET}
  sync:
    schedule: "0 */6 * * *"
    resources:
      - workspace_permissions
      - lakehouse_permissions
      - warehouse_permissions

Deployment Topology¶

For cloud data-platform use cases, Verity is typically deployed in the same Azure region as the data platforms to minimise API latency.

graph TB
    subgraph "Azure Region — East US 2"
        subgraph "Data Platforms"
            DB["Databricks<br/>Workspace"]
            SY["Synapse<br/>Workspace"]
            FB["Fabric<br/>Workspace"]
        end

        subgraph "AKS Cluster"
            V["Verity<br/>(19 microservices)"]
            PG["PostgreSQL +<br/>TimescaleDB"]
            CH["ClickHouse"]
        end
    end

    DB <--> V
    SY <--> V
    FB <--> V

Getting Started¶

Related¶

• Glossary → Connector
• Connector SDK
• Enterprise IAM Governance
• Regulatory Compliance