# Data Retention and Deletion Standard

**Owner:** platform engineering · **Review cadence:** quarterly, and on any
change to `web/data_retention.py`, the backup scripts, or the third-party
credential set · **Primary production verification:** 2026-08-22 · **Offsite
storage metadata re-read:** 2026-08-25 UTC · **Published at:** `/static/data-retention.md`

This document states where customer data goes, what deletes it, **how each of
those statements was verified**, and what remains afterwards. Every row of the
sink table names its verification method, because a deletion claim with no
method behind it is the failure this document exists to prevent.

Two ground rules govern everything below.

**A verification method is not a promise.** Where a sink is verified by a test,
the test is named. Where it is verified by reading production, the reading and
its date are given. Where it could not be verified, the row says so and names the
open ticket — it is never softened into coverage.

**Nothing here is a delivery commitment.** Dates and windows describe what is
configured today. They are not contractual retention terms; a buyer needing a
guaranteed window should fix it in the agreement.

---

## 1. What triggers deletion

| Trigger | Entry point | Effect |
|---|---|---|
| A customer deletes their account | `purge_user_account` in `web/data_retention.py` | Hard-deletes the account and the whole graph of records it owns, its generated chart artifacts, and its tenant copies in the vector store when one is operated |
| A council session ages out, or is soft-deleted | `purge_sessions` / `purge_session` | Removes the session row, its artifacts, and its vector copy |
| Scheduled retention | the retention job, on the windows in §3 | Removes rows past their window regardless of any customer action |

**The purge fails closed.** If an external sink cannot be cleared — the vector
store refuses, for example — `ExternalCleanupError` is raised, the database
transaction is rolled back, **and the artifacts are not deleted either**. Data
survives until every sink can be cleared, rather than the canonical record
disappearing while a copy remains somewhere else. Verified by
`tests/test_data_retention.py::test_failed_vector_cleanup_does_not_delete_canonical_data`,
which asserts the session row still holds its original content after a failed
cleanup and that no artifact deletion was attempted.

A purge also refuses to race a session that is still running
(`test_account_deletion_refuses_to_race_active_session`).

**Dormant execution Core tables — source candidate, NOT DEPLOYED.** The
cross-agent foundations are outside the ORM registry and fail closed under a
separate exact inventory. Account deletion first removes append-only route
decisions and their cycles, then capacity authorities, then kernel rows, all
while the exact owner fence is `deleting`; only then can the account and fence
retire. `tests/test_execution_account_erasure.py` seeds every inventoried table
for both the deleted owner and a survivor, runs with SQLite foreign keys both
on and off, proves the target reaches zero, proves the survivor is unchanged,
and proves a later purge failure rolls the entire transaction back.
These owner-bound cycle and decision rows have **no scheduled TTL in this
dormant source candidate**: they are retained until account deletion. That is
the current source policy, not a deployed or contractual retention promise.

---

## 2. The sinks

| Sink | What lands there | Deletion mechanism | How that was verified | Residual after deletion |
|---|---|---|---|---|
| **PostgreSQL** | Every canonical record: accounts, sessions, queries and outputs, tickets, marketing history, stored tenant credentials, content-free fixed-API reservation/request-capability evidence, saved monitors, audit rows | Explicit per-model deletion in `web/data_retention.py`, leaf-first so foreign keys hold. Append-only fixed-API request capabilities are deleted before their reservation bindings after the owner fence enters deletion | `tests/test_purge_completeness.py` reflects over the model registry and requires **every** user-scoped model to be purged, anonymised, ORM-cascaded, or recorded as retained with a written reason — a model in none of those four fails the build. It **refuses to accept a database-level `ondelete=CASCADE` as coverage**, because that clause does not fire on SQLite with foreign keys off. A second guard covers transitively-owned models, and further guards assert the reflection actually inspected the registry rather than passing over an empty set. Behavioural proof is separate: `tests/test_data_retention.py` seeds real rows and asserts absence, with foreign keys enforced; `tests/test_execution_fixed_authority.py` proves the request evidence is append-only while active and deletable only in the owner-deletion phase | None |
| **Chart artifacts** | Generated SVG charts, their JSON specs, and per-artifact access sidecars, on the server's local disk under `data/artifacts` | `delete_owner_artifacts` for the account sweep and `delete_session_artifacts` per session, both in `web/viz_store.py` | **Two links, verified separately, because no single test spans them.** That the purge calls the helpers with the right account and session identifiers: `tests/test_data_retention.py` records the calls and asserts on the arguments. That the helpers actually remove files from disk: `tests/test_viz_artifacts.py::test_store_hard_deletes_by_session_and_owner`, plus a case covering an artifact whose spec is corrupted | None expected. The account sweep finds artifacts through their access sidecar; an artifact predating sidecars would be reachable only through the session sweep's legacy fallback. Measured on production 2026-08-22: **26 of 26 artifacts carry a sidecar**, so the account sweep is complete there today |
| **Vector store** | Would hold embedded copies of completed sessions, if operated | `delete_indexed_session` and `delete_tenant_data`; every vector delete carries a tenant predicate, and a session identifier alone is never sufficient authority to remove points | **Not operated in production.** Read from the production host 2026-08-22: neither `QDRANT_URL` nor `ENABLE_KNOWLEDGE_INDEXING` is present in the environment file, the service unit declares exactly one environment file, and nothing is listening on the vector store's port. `_tenant_vector_store_configured()` is therefore false and no vector call is made on purge — asserted by `test_unconfigured_vector_store_does_not_block_local_deletion` | None, because nothing is written there. This says the store is **not operated in production**, which is checkable. It does not claim it never held data |
| **Application logs** | Recipient email addresses in authentication, outreach and bounce-handling log lines; a small number of lines carrying account or session identifiers | Time-bounded expiry, not per-customer deletion | Read from the production host 2026-08-22. The journal is capped at `SystemMaxUse=200M` **and** `MaxRetentionSec=2week`; the cron output files under `/var/log/metacouncil-*.log` are rotated daily and kept 14 days by an installed logrotate rule | **Up to 14 days.** A deleted customer's address can appear in logs for that long. Logs are not searched or edited on a deletion request; they expire |
| **Local database backups** | A full compressed database dump, so any row present at 03:17 that day | Rotation by the backup scripts on the server | Read from the production host 2026-08-22. `backup_pg.sh` deletes local dumps older than **7 days**; `backup_pg_gcs.sh` keeps the newest **7** of its own dumps. Note these are different rules: one is time-based, the other count-based, so if backups run more often than daily the count-based window is shorter than seven days | Up to 7 days locally |
| **Predecessor rollback source archives — source candidate, NOT DEPLOYED** | Built-in application source needed for an exact predecessor rollback. Customer/runtime paths, tenant-marked workflow YAML, root and script output trees, and `.meta-council-activation-attempt.json` are excluded by design. Retired legacy crash artifacts are different: `/tmp/mc-env-backup` may contain credentials, and exact `/tmp/meta-council-previous-stage.<six alphanumeric>.tar.gz` partial stages may contain the legacy tree | After verified activation completes, graph retirement keeps the live state/referenced archive/control slot and, when the predecessor has its own complete versioned state, the same three artifacts for that predecessor; it removes only unreferenced older artifacts. On the first completed migration it also retires the exact trusted legacy `/tmp/meta-council-previous.tar.gz`, exact `/tmp/mc-env-backup`, and exact six-alphanumeric partial stages. Every legacy candidate must pass one whole-set, no-link, root-owned mode-`0600` regular-file preflight before any deletion | `tests/test_predecessor_archive.py` verifies source-only membership, tenant-marker omission, exact-validator compatibility, and fail-closed handling. `tests/test_release_artifact_pruning.py` verifies the established bounded C→B→A graph, the first-migration C→B graph, all three legacy classes, preflight-before-delete behavior, and legacy retirement. Unexpected or ambiguous workflow catalog content aborts deployment rather than being silently retained or omitted | No Customer Content is intended to land in the new archives. When B has complete versioned artifacts, retained C state refers to the B archive and retained B state refers to the A archive, supporting C→B and B→A. The first legacy migration has no B state or slot, so retained C state/B archive/C slot supports C→B only. A post-completion retirement failure needs reconciliation but does not stop the healthy candidate. These are source-candidate properties, not claims about the verified live legacy artifacts |
| **Release-exchange and private staging residues — source candidate, NOT DEPLOYED** | Activation or rollback can briefly duplicate `.env`, `data`, `.cache`, tenant workflows, or staged credentials in these exact classes: `/opt/.meta-council-activate.pending`; `/opt/.meta-council-rollback.pending`; private `/tmp/meta-council-release.<six alphanumeric>`, `/tmp/meta-council-rollback.<six alphanumeric>`, and `/tmp/meta-council-rollback-env.<six alphanumeric>` directories; a mode-`0600` `/tmp/meta-council-env.<six alphanumeric>` file; and live mode-`0600` `.env.ticket-roots-stage.<six alphanumeric>`, `.env.bootstrap.<six alphanumeric>`, or `.env.bootstrap.<six alphanumeric>.ticket-roots-stage.<six alphanumeric>` files | Immediate cleanup uses the authenticated installed cleaner under the deployment and ticket-root locks. An application `ExecStartPre` gate runs it before service start, and an enabled timer retries after boot and every five minutes. Every eligible candidate is preflighted before the first deletion. Exact roots and environment files have strict owner, mode, type, link-count, filesystem, and mount requirements. An unprivileged exact-name `/tmp` decoy is preserved. Within a trusted private tree, historical `0775` subdirectories and hard-linked regular files are accepted; entries must still be root-owned, be a regular file, directory, or symlink, and stay on the same filesystem and Linux mount. Symlinks are unlinked without being followed | `tests/test_release_exchange_cleanup.py` verifies every exact class, authenticated installation, whole-set preflight, strict root/file metadata, accepted nested historical modes and hardlinks, unprivileged-decoy preservation, no-follow deletion, lock serialization, device/mount rejection, timer refusal of a one-use start token, and start-gate cleanup. The deployment and rollback integration suites exercise success and cleanup failure paths | Account/session erasure does not edit a duplicate while the application is stopped. The source candidate removes every authenticated safe stale candidate before the application can start, while an owning deployment or rollback may preserve only its inode-bound active scratch. The timer provides independent retry. An unsafe root-owned residue or mount ambiguity blocks service start and requires approved recovery. This is not a claim about the currently verified live deployment |
| **Offsite database backups** | The same dumps, uploaded to our own storage bucket | Nothing in this repository deletes a remote object. Provider metadata currently makes live objects lifecycle-eligible for deletion at age **90 days** and configures a **seven-day soft-delete period** | **Provider metadata read on 2026-08-25 UTC** reported: versioning disabled; 88 live distinct objects and 88 total versions; zero noncurrent versions; bucket retention policy and default event-based hold absent; zero objects with an individual hold or retention-expiration value; zero soft-deleted objects; the age-90 lifecycle condition; and the seven-day soft-delete setting. This is a dated configuration observation, not a source-enforced, approved, or contractual retention policy | The configured lifecycle age and soft-delete period are finite values, but actual and contractual unrecoverability upper bounds remain unverified. A frozen customer row remains until its archive is actually unrecoverable, and there is no per-customer archive erasure. Lifecycle eligibility and the configured soft-delete period do not establish execution timing or a 97-day upper bound. Approval and ongoing verification remain tracked in [ENT-SEC-118] |
| **Model provider (inference)** | The question, the context supplied with it, and the model's output, for platform-supplied inference | The provider's own retention terms; we hold no delete API for it | Contract citation, published on the trust page: under commercial terms the provider is contractually prohibited from training on customer content, and inputs and outputs are deleted within 30 days, with the exceptions the trust page states rather than omits. A second model-provider credential exists in the production environment but addresses a **loopback endpoint on the server itself, with nothing listening on it** — read 2026-08-22 — so no customer content leaves the host through it | Up to 30 days at the provider, per its terms. Zero-retention processing is offered by the provider per organisation and is available as a negotiated addendum; whether this account holds such an arrangement is not established here, and we publish neither that we have it nor that we lack it |
| **Legal research provider** | The question text, and any context marked unverified, **only** when the optional legal research route is used and the operator gate is on | The provider's own terms | Credential presence read from the production environment 2026-08-22; the data sent is the question and context assembled in `web/routers/legal.py`. No deletion route is defined on our side | Retained under the provider's terms. Not covered by our purge |
| **Payment processor** | The customer's email address, their payment history (amounts, dates, what was bought), and checkout labels carrying our internal account id | **No automatic purge call.** For a formal erasure request, the account owner verifies the requester and exact processor object, then manually deletes each matching active Customer object that exists; one-time guest checkouts can have no Customer object to delete. The routine account purge remains unchanged | Verified by search 2026-09-02: no deletion, modification, or cancellation call exists anywhere in the application, and the purge path contains no processor reference. The provider's Customer-deletion, Checkout guest-customer, and privacy documentation were re-read the same day: deleting a Customer removes its stored card details and prevents future operations, guest payments may have no Customer object, and transaction history remains retrievable under the provider's retention obligations | On a formal request, each verified active Customer object and its stored card details are deleted manually when one exists; otherwise the record and reply state that no Customer object existed. The operating business's retention horizon is **7 years from the transaction** under its tax and accounting record-keeping basis; the provider may retain its copy longer under independent obligations, so no provider-side deletion at year seven is promised. The reply identifies what was deleted, what remains, why, and the business horizon. The owner-confirmed route and request record are defined in `docs/PAYMENT_PROCESSOR_ERASURE_ROUTE.md` ([ENT-SEC-120]) |
| **Email delivery vendor** | Message bodies and recipient addresses for verification, reset and automated reply mail | None available | The account has been **suspended since 2026-06-12**; nobody can sign in to inspect what is held, request deletion, or read the vendor's retention window | Unknown and currently unclearable. Filed as [ENT-SEC-119] |
| **Outreach enrichment vendors** | Prospect names and company domains for outbound lead research | Not applicable to customer deletion | Verified by search: these credentials are read only by operator-run enrichment scripts under `scripts/`, never by the serving application, and what they send is prospect records the platform researched. No call site reads from a customer's sessions, documents, or workspace, so a customer's own data is not sent to them | Not a customer-data sink for deletion purposes. Listed so the enumeration is complete rather than selective. Prospect data has its own lifecycle, which this standard does not cover |
| **Identity provider (single sign-on)** | Nothing we send. The customer authenticates with the provider directly | Not applicable | The sign-on flow returns an identity to us; we do not transmit customer content to the provider | Not a customer-data sink |

---

## 3. Retention windows

The application rows are values the source sets. **Each is overridable by
environment variable**, so the operative window is a property of the deployment
rather than of the code. The offsite row is different: it is external provider
configuration and is dated separately rather than presented as a source default.

| Data | Default or provider configuration | Override | Production observation |
|---|---|---|---|
| Council sessions | **90 days** | `SESSION_RETENTION_DAYS` | Not overridden — the default is operative |
| Site page views | **730 days** | `PAGEVIEW_RETENTION_DAYS` | Not overridden — the default is operative |
| Workflow definition snapshots | **365 days** | `DEFINITION_SNAPSHOT_RETENTION_DAYS` | Not overridden — the default is operative |
| Dormant execution cycles and route decisions | **Until account deletion; no scheduled TTL** | None in this source candidate | **NOT DEPLOYED.** Exact owner-scoped erasure is covered by `tests/test_execution_account_erasure.py`; no age-based purge exists |
| Fixed-API reservation and exact request-capability evidence | **Until account deletion** | None | Dormant/code-disabled; owner erasure explicitly deletes request capabilities before reservation bindings |
| Local database backups | **7 days**, and separately the newest **7** dumps | script constants | Both families present, 16 archives on disk |
| Predecessor rollback source artifacts | **Live release plus at most one versioned predecessor graph** | Source-candidate post-completion retirement | **NOT DEPLOYED.** When B already has complete versioned artifacts, live C retains C state/B archive/C slot and B state/A archive/B slot, supporting C→B→A. The first legacy migration retains C state/B archive/C slot only, supporting C→B, then safely retires all three exact legacy crash-artifact classes |
| Release-exchange and private staging residues | **Activation or rollback duration; start-gated cleanup with boot/five-minute retry** | Source-candidate installed cleanup unit | **NOT DEPLOYED.** Only the exact `/opt`, `/tmp`, and live secret-stage classes named above are eligible. Authentication, unsafe root-owned metadata, or mount ambiguity blocks application start and needs approved recovery; unprivileged exact-name `/tmp` decoys are preserved |
| Journal logs | **2 weeks**, capped at 200 MB | `journald.conf.d` drop-in | Both limits installed |
| Cron output logs | **14 days** | logrotate rule | Installed |
| Offsite database backups | **Lifecycle-eligible at age 90 days**; **seven-day soft-delete period configured** | Storage-provider configuration | Read from provider metadata on 2026-08-25 UTC. The configured values are finite, but actual and contractual unrecoverability upper bounds remain unverified; this point-in-time reading is not an approved policy or a 97-day upper bound. See [ENT-SEC-118] |

Overrides accept only values between 1 and 3650 days; an invalid or
out-of-range value is logged and the default is used rather than silently
accepted.

`tests/test_retention_deletion_standard.py` pins the application numbers in this
table to source constants, so document and code cannot drift without failing the
build. The offsite values have no source constant: the test instead requires a
dated provider reading and forbids language that turns it into an approved or
contractual promise. That external metadata must be re-read periodically.

---

## 4. What is deliberately not claimed

* **Per-customer erasure from a backup archive is not possible and is not
  claimed in any form.** A backup is a frozen copy; a customer deleted today
  can still exist inside last night's copy, and the actual time at which each
  copy becomes unrecoverable is unverified. The mechanism that would change
  this is a restore-safe erasure ledger, which exists only as a frozen research
  checkpoint in `docs/RESTORE_SAFE_ACCOUNT_ERASURE.md` and creates no ledger,
  key, schema, backup marker or restore gate today. It is cited here as future
  work, not as a capability.
* **The source-only rollback archive and bounded artifact graph are NOT
  DEPLOYED.** The source candidate excludes tenant-marked workflows,
  output/customer/runtime paths, and the activation-attempt marker, and fails a
  deployment on ambiguous or unexpected workflow catalog content. Do not apply
  those statements to the verified live legacy archive until exact-main
  deployment evidence exists.
* **Logs are not searched or redacted on a deletion request.** They expire.
* **No certification, audit, or compliance posture is claimed here.** This is a
  description of mechanisms, not an attestation.
* **Nothing in this document addresses protected health information.** The
  platform is not offered for it.

---

## 5. Open gaps

Each of these is a sink where deletion cannot currently be proven. Per the
acceptance criteria of [ENT-SEC-108], each is filed as its own blocking ticket
rather than described as covered.

| Gap | Ticket | Why it is blocked |
|---|---|---|
| Offsite lifecycle and soft-delete settings exist, but per-customer archive erasure and an approved retention decision do not | [ENT-SEC-118] | Provider metadata was read on 2026-08-25 UTC and reported the finite configured values of age-90 lifecycle eligibility plus a seven-day soft-delete period. It also reported versioning disabled, 88 live objects/88 total versions, no noncurrent or soft-deleted objects, no bucket retention/default event hold, and no individual holds or retention-expiration values. Actual and contractual unrecoverability upper bounds remain unverified: this point-in-time configuration does not prove deletion timing or a 97-day upper bound, does not delete one customer from a frozen archive, and must be reverified |
| Historical email content at the suspended delivery vendor cannot be inspected or deleted | [ENT-SEC-119] | The account has been suspended since 2026-06-12; only the account owner can restore or replace it |

Related, and tracked separately: clean-host restore and a measured recovery
objective remain open under [ENT-SEC-113]. Being able to restore a backup is a
different question from what a backup retains, but a buyer reading this document
usually asks both.