Backup And Restore

Backups are an operations responsibility. The repository provides the data layout and runtime services, but the final backup tooling depends on the target host or Kubernetes platform.

What To Back Up

AssetWhy it matters
PostgreSQL application databaseTenants, organizations, users, proofs, certificates, jobs, DLQ, audit, receipts, licenses, idempotency, verification logs.
Keycloak database/configRealms, clients, users, roles, federation, credentials metadata.
Object storage bucketsRaw .der proofs, PDFs, canonical JSON, binaries, exports, monthly reports.
Environment and secretsRuntime config, DB/S3/SMTP/Keycloak credentials, signer mTLS material, frontend session secret.
Signer key custody recordsProduction signer owns cryptographic keys; back up according to signer/HSM policy, not application DB policy.
Nginx/TLS configPublic routing, certificates, redirects, callback behavior.
Generated release artifactsImage tags/digests, SBOM/provenance, Swagger/client versions.

Do not place API key raw secrets, JWTs, signer private keys, or production license signing material in application backups.

PostgreSQL Backup

For the local compose database:

  docker compose exec postgres pg_dump -U wipe -d wipe -Fc -f /tmp/wipe.dump
docker compose cp postgres:/tmp/wipe.dump ./wipe.dump
  

For production, use managed snapshots or pg_dump/WAL archiving according to the RPO/RTO target. The cahier target is RPO <= 15 minutes and RTO <= 4 hours.

Object Storage Backup

Back up all configured buckets:

BucketContents
S3_BUCKET_PROOFSSubmitted .der proof payloads.
S3_BUCKET_CERTIFICATESCertificate PDFs and canonical JSON.
S3_BUCKET_BINARIESAgent binaries or static download metadata targets.
S3_BUCKET_EXPORTSBilling exports and monthly reports.
S3_BUCKET_BILLING_DROPOptional write-only billing delivery target.

Keep bucket versioning and retention policy aligned with legal and contractual requirements. WORM/Object Lock is not active in this repository, so enforce any immutability at the storage provider only after product approval.

Restore Order

  1. Restore PostgreSQL and run migrations to the expected application version.
  2. Restore object storage buckets and verify object keys referenced by database rows.
  3. Restore Keycloak realms/clients/users or connect to the managed Keycloak backup.
  4. Restore environment secrets and frontend session secret.
  5. Start API first, then workers.
  6. Verify /readyz, login, proof list, certificate PDF download, canonical JSON download, billing report download, and /verify.
  7. Resume workers and inspect DLQ before replay.

Consistency Checks After Restore

CheckExpected result
Proof object keysEvery recent proof row has a readable object in the proofs bucket.
Certificate artifactsPDF and canonical JSON keys resolve for sampled certificates.
Consumption receiptsSequence continuity and hash-chain verification pass.
Audit logMaintenance worker reports no unexpected chain breaks.
Idempotency keysRecent in-progress keys are resolved or expired before replaying writes.
Queue stateJobs/DLQ are reviewed before workers are scaled up.

Restore Drill Cadence

Run restore drills before production launch and after any change to database schema, object storage policy, Keycloak realm structure, signer integration, or release pipeline. Keep drill evidence with the operations log.