Skip to content

Workflow Engine

Path: services/decision/workflow-engine/ · Type: Worker (Temporal)

The Workflow Engine orchestrates the access-review lifecycle using Temporal durable workflows. It handles reviewer notification, SLA enforcement, escalation, decision capture, and audit trail generation.

Architecture

graph LR
    K1[verity.reviews.created] --> WE[Workflow Engine]
    WE --> TS[Temporal Server]
    WE -->|email| SMTP[SMTP Server]
    WE -->|webhook| Teams[Microsoft Teams]
    WE -->|webhook| Slack[Slack]
    WE --> K2[verity.audit.all]
    WE --> RM[Remediation Service]

AccessReviewWorkflow

The core workflow is AccessReviewWorkflow — a Temporal durable workflow that manages a single review packet from creation to completion.

stateDiagram-v2
    [*] --> LoadReview
    LoadReview --> NotifyApprover
    NotifyApprover --> WaitForDecision
    WaitForDecision --> DecisionReceived: submit_decision signal
    WaitForDecision --> Escalate: SLA timeout
    Escalate --> NotifyEscalation
    NotifyEscalation --> WaitForDecision
    DecisionReceived --> CheckDecision
    CheckDecision --> ExecuteRevocation: decision = REVOKE
    CheckDecision --> WriteAudit: decision = APPROVE
    ExecuteRevocation --> WriteAudit
    WriteAudit --> [*]

Activities

Each step in the workflow is implemented as a Temporal activity with automatic retry:

Activity Description Retry Policy
load_review_packet Loads the review packet and evidence from PostgreSQL 3 retries, 5 s backoff
notify_approver Sends notification via configured channels 3 retries, 10 s backoff
escalate_review Reassigns to next-level reviewer and resets SLA 3 retries, 5 s backoff
execute_revocation Triggers the Remediation Service to revoke access 5 retries, 30 s backoff
write_audit_record Publishes an immutable audit record to Kafka 5 retries, 10 s backoff

Signal-Based Decision Input

Reviewer decisions are submitted via the submit_decision Temporal signal:

@workflow.signal
async def submit_decision(self, decision: ReviewDecision) -> None:
    self.decision = decision

The ReviewDecision payload:

Field Type Description
decision enum APPROVE or REVOKE
reviewer_id UUID Who made the decision
justification str Free-text rationale
decided_at datetime Decision timestamp

Notification Channels

The workflow supports three notification channels, configured independently:

Channel Configuration Format
Email (SMTP) WORKFLOW_SMTP_HOST, WORKFLOW_SMTP_PORT, WORKFLOW_SMTP_FROM HTML email with review summary and deep link
Microsoft Teams WORKFLOW_TEAMS_WEBHOOK_URL Adaptive Card with approve/revoke action buttons
Slack WORKFLOW_SLACK_WEBHOOK_URL Block Kit message with approve/revoke buttons

Notifications include:

  • Principal name and email
  • Asset name, type, and sensitivity
  • Current decay score and risk level
  • SLA deadline
  • Direct link to the review in the Dashboard UI

SLA Enforcement

The workflow enforces review SLAs using Temporal timers:

flowchart TD
    A[Review created] --> B[Start SLA timer]
    B --> C{Decision received before timeout?}
    C -->|Yes| D[Process decision]
    C -->|No| E[Escalate to next reviewer]
    E --> F[Reset SLA timer]
    F --> C
Risk Level SLA Escalation Behaviour
CRITICAL 48 hours Escalate to manager → platform admin
HIGH 7 days Escalate to manager → platform admin
MEDIUM 30 days Escalate to platform admin
LOW 90 days Escalate to platform admin

After maximum escalations (configurable, default: 3), the review is flagged as ESCALATION_EXHAUSTED and an alert is sent to the platform admin.

Audit Records

Every workflow state transition produces an audit record published to verity.audit.all:

Field Description
event_type REVIEW_STARTED, NOTIFICATION_SENT, DECISION_SUBMITTED, ESCALATED, REVOCATION_EXECUTED
packet_id Review packet ID
actor_id User or system that triggered the event
timestamp UTC timestamp
details JSON with event-specific context

Configuration

Variable Required Default Description
WORKFLOW_TEMPORAL_HOST Yes Temporal server address
WORKFLOW_TEMPORAL_NAMESPACE No verity Temporal namespace
WORKFLOW_TEMPORAL_TASK_QUEUE No access-review Temporal task queue
WORKFLOW_KAFKA_BOOTSTRAP Yes Kafka bootstrap servers
WORKFLOW_DATABASE_URL Yes PostgreSQL connection string
WORKFLOW_SMTP_HOST No SMTP server hostname
WORKFLOW_SMTP_PORT No 587 SMTP server port
WORKFLOW_SMTP_FROM No Sender email address
WORKFLOW_TEAMS_WEBHOOK_URL No Microsoft Teams incoming webhook
WORKFLOW_SLACK_WEBHOOK_URL No Slack incoming webhook
WORKFLOW_MAX_ESCALATIONS No 3 Maximum escalation attempts
WORKFLOW_LOG_LEVEL No INFO Python log level

Observability

Metric Type Description
workflow_reviews_started_total Counter Workflows started (by risk level)
workflow_decisions_total Counter Decisions received (by decision type)
workflow_escalations_total Counter SLA-triggered escalations
workflow_review_duration_seconds Histogram Time from creation to decision
workflow_notification_errors_total Counter Failed notification deliveries