CI/CD Pipeline¶

Verity uses GitHub Actions for continuous integration and deployment. Two workflow files define the build pipeline:

• .github/workflows/ci.yml — Main CI/CD pipeline: lint, test, build, deploy
• .github/workflows/security.yml — Security scanning: vulnerabilities, SAST, container images

Pipeline Overview¶

graph TB
    subgraph Trigger["Trigger"]
        PUSH["Push to main"]
        PR["Pull Request"]
    end

    subgraph CI["CI Pipeline (.github/workflows/ci.yml)"]
        LINT["Lint & Type Check<br/><i>Ruff + Mypy</i>"]
        UNIT["Unit Tests<br/><i>pytest + coverage ≥90%</i>"]
        INTEGRATION["Integration Tests<br/><i>Docker Compose + pytest</i>"]
        SECURITY_CI["Security Scan<br/><i>Bandit + Trivy</i>"]
        BUILD["Build & Push Images<br/><i>Multi-arch: amd64 + arm64</i>"]
        HELM_LINT["Helm Lint<br/><i>helm lint + kubeconform</i>"]
        DEPLOY["Deploy to Staging<br/><i>Helm upgrade</i>"]
    end

    subgraph Security["Security Pipeline (.github/workflows/security.yml)"]
        TRIVY["Trivy Scan<br/><i>Filesystem + Config</i>"]
        BANDIT["Bandit SAST<br/><i>Python security</i>"]
        SEMGREP["Semgrep SAST<br/><i>Cross-language</i>"]
        IMAGE_SCAN["Container Image Scan<br/><i>Trivy per service</i>"]
    end

    PUSH --> LINT
    PR --> LINT
    PUSH --> TRIVY & BANDIT & SEMGREP

    LINT --> UNIT
    LINT --> SECURITY_CI
    UNIT --> INTEGRATION
    UNIT & INTEGRATION & SECURITY_CI --> BUILD
    BUILD --> DEPLOY
    PUSH --> HELM_LINT
    BUILD & HELM_LINT --> DEPLOY
    PUSH --> IMAGE_SCAN

    style LINT fill:#6366f1,color:#fff
    style UNIT fill:#6366f1,color:#fff
    style INTEGRATION fill:#6366f1,color:#fff
    style BUILD fill:#10b981,color:#fff
    style DEPLOY fill:#f59e0b,color:#000
    style TRIVY fill:#ef4444,color:#fff
    style BANDIT fill:#ef4444,color:#fff
    style SEMGREP fill:#ef4444,color:#fff
    style IMAGE_SCAN fill:#ef4444,color:#fff

CI Pipeline (ci.yml)¶

Triggered on pushes and pull requests to main.

Environment¶

env:
  PYTHON_VERSION: "3.12"
  ACR_REGISTRY: verityacr.azurecr.io

Jobs¶

1. Lint & Type Check¶

Runs on every push and PR:

2. Unit Tests¶

Depends on: Lint & Type Check

pytest tests/unit/ \
  --cov=services/ --cov=libs/ \
  --cov-report=xml \
  --cov-fail-under=90 \
  -v

• Coverage threshold: 90% minimum
• Coverage report uploaded as build artifact

3. Integration Tests¶

Depends on: Unit Tests

# Start test infrastructure
docker compose -f tests/integration/docker-compose.yml up -d --wait

# Run integration tests
pytest tests/integration/ -x -v

# Tear down
docker compose -f tests/integration/docker-compose.yml down -v

• Uses a dedicated tests/integration/docker-compose.yml for isolated infrastructure
• Timeout: 10 minutes for tests, 5 minutes for infrastructure startup

4. Security Scan (in CI)¶

Depends on: Lint & Type Check (runs in parallel with tests)

5. Build & Push Images¶

Depends on: Unit Tests + Integration Tests + Security Scan

Build matrix — 7 service images built in parallel:

Multi-architecture builds:

platforms: linux/amd64,linux/arm64

Image tags:

verityacr.azurecr.io/verity/{service}:{git-sha}
verityacr.azurecr.io/verity/{service}:latest

Caching:

cache-from: type=gha
cache-to: type=gha,mode=max

Uses GitHub Actions cache for Docker layer caching, significantly reducing build times for incremental changes.

6. Helm Lint¶

Runs on every push and PR (independent of other jobs):

# Lint the chart
helm lint infra/helm/verity/ --values infra/helm/verity/values.yaml

# Validate generated manifests
helm template verity infra/helm/verity/ --values infra/helm/verity/values.yaml \
  | kubeconform -strict -summary -output json

7. Deploy to Staging¶

Depends on: Build & Push + Helm Lint

helm upgrade --install verity infra/helm/verity/ \
  --namespace verity \
  --create-namespace \
  --values infra/helm/verity/values.yaml \
  --values infra/helm/verity/values-staging.yaml \
  --set global.image.tag=${GITHUB_SHA} \
  --wait \
  --timeout 10m

Authentication flow:

1. Azure Login via OIDC (federated credentials — no stored secrets)
2. Set AKS context for aks-verity-staging in rg-verity-staging
3. Helm upgrade with staging values

Security Pipeline (security.yml)¶

Triggered on pushes to main, pull requests, and weekly on Monday at 06:00 UTC.

Jobs¶

1. Trivy Vulnerability Scan¶

Scans two targets:

Results are uploaded to GitHub's Security tab via CodeQL SARIF integration.

2. Bandit Python Security¶

bandit -r services/ libs/ -ll -f sarif -o bandit-results.sarif

Scans all Python code for security issues (SQL injection, hardcoded credentials, unsafe deserialization, etc.). Results uploaded as SARIF to GitHub Security.

3. Semgrep SAST¶

semgrep scan --config auto --sarif --output semgrep-results.sarif .

Cross-language static analysis using Semgrep's auto ruleset. Runs in the official Semgrep container. Results uploaded as SARIF to GitHub Security.

4. Container Image Scan¶

Builds and scans each service image with Trivy:

strategy:
  matrix:
    service:
      - connectors
      - ingestion
      - decision
      - remediation
      - api-gateway
      - analytics
      - audit

Each image is built locally (not pushed), scanned for HIGH/CRITICAL vulnerabilities, and results uploaded as SARIF.

GitHub Permissions¶

permissions:
  contents: read        # Checkout code
  packages: write       # Push container images
  id-token: write       # OIDC for Azure login
  security-events: write  # Upload SARIF results

Container Registry¶

Pipeline Duration¶

Typical pipeline execution times: