Skip to content

Compliance Reporter

Path: services/audit/compliance-reporter/ · Type: Worker

The Compliance Reporter generates regulatory compliance reports by querying the immutable audit trail in ClickHouse. It supports multiple compliance frameworks and produces structured reports suitable for auditors and regulators.

Architecture

graph LR
    AG[API Gateway] -->|report request| CR[Compliance Reporter]
    CR --> CH[(ClickHouse\nverity_audit)]
    CR --> PG[(PostgreSQL)]
    CR -->|completed report| AG

Supported Frameworks

Framework Report Type Key Focus Areas
GDPR Data Subject Access Report Personal data access, consent, right-to-be-forgotten compliance
SOX Access Control Attestation Segregation of duties, access reviews, change management
SOC 2 Trust Services Criteria Security, availability, processing integrity, confidentiality
HIPAA PHI Access Audit Protected health information access and disclosure logging
PCI-DSS Cardholder Data Access Access to payment card data, authentication events

Report Generation Flow

sequenceDiagram
    participant User
    participant API as API Gateway
    participant CR as Compliance Reporter
    participant CH as ClickHouse
    participant PG as PostgreSQL

    User->>API: POST /v1/reports (framework, date range)
    API->>CR: Queue report generation
    CR->>CH: Query audit events
    CR->>PG: Query principal & asset metadata
    CR->>CR: Compile report sections
    CR->>PG: Store report (status: COMPLETED)
    CR-->>API: Report ready notification
    User->>API: GET /v1/reports/{id}/download
    API-->>User: Report file (PDF / CSV / JSON)

Report Structure

Each generated report contains:

Section Description
Executive Summary High-level compliance posture with key metrics
Scope Time period, systems covered, principal/asset counts
Access Review Summary Reviews completed, SLA compliance rate, decisions breakdown
Risk Distribution Score distribution across CRITICAL / HIGH / MEDIUM / LOW
Remediation Activity Revocations executed, average time-to-remediation
Exceptions & Findings SLA breaches, escalation-exhausted reviews, anomalies
Evidence Trail Detailed audit events supporting each finding

Async Generation

Reports are generated asynchronously to avoid blocking the API:

  1. Request: The API Gateway accepts the report request and returns a report_id with status PENDING.
  2. Processing: The Compliance Reporter picks up the job and begins querying.
  3. Completion: The report is stored in PostgreSQL with status COMPLETED.
  4. Download: The user retrieves the finished report via the API.

Report statuses: PENDING → PROCESSING → COMPLETED | FAILED

Output Formats

Format Use Case
PDF Formal auditor-facing reports with charts and tables
CSV Raw data export for custom analysis
JSON Machine-readable format for integration with GRC tools

Configuration

Variable Required Default Description
REPORTER_DATABASE_URL Yes — PostgreSQL connection string
REPORTER_CLICKHOUSE_URL Yes — ClickHouse HTTP URL
REPORTER_OUTPUT_DIR No /tmp/reports Directory for generated report files
REPORTER_MAX_CONCURRENT No 3 Maximum concurrent report generation jobs
REPORTER_LOG_LEVEL No INFO Python log level

Observability

Metric Type Description
reporter_reports_generated_total Counter Reports generated (by framework, format)
reporter_generation_duration_seconds Histogram Report generation time
reporter_query_duration_seconds Histogram ClickHouse query time per report
reporter_errors_total Counter Failed report generation attempts