Skip to content

Data and retention

AlertBird initially keeps seven days of compact monitoring history and bounded diagnostic evidence in PostgreSQL. This is not seven days of every raw trace or log. Analysis and enabled log rules process supported received observations before retention reduction.

What is kept

The system retains service/request/status/latency summaries, source-aware graph and native metric history, coverage gaps, grouped error observations and selected bounded evidence. Diagnostic traces never add to request counts. Log rules count matching arrivals prospectively; retained evidence is not a searchable log lake.

Counter intervals and histogram buckets are consolidated without averaging percentiles. Gauge history records sample minimum, maximum and latest values; it does not claim continuous measurement. Sources and uncertainty stay distinct through retention.

Change the policy

Open Data & retention as an admin to review and change the persisted monitoring policy. Retention is configurable from one to seven days in the current contract. The evidence budget ranges from zero to the installation's 1 GiB ceiling. The app displays the active limits and policy version. A saved change records the actor and old/new policy atomically; conflicting edits require reloading the current version.

The worker expires eligible data in bounded batches, so a shorter policy does not reclaim everything immediately. Longer retention does not restore data already deleted. Active episodes and pending delivery work pin their exact evidence and configuration references outside ordinary monitoring expiry. Recovery or acknowledgement alone does not release a pin: the episode must be inactive and have no pending work.

Configuration versions, secret references, configuration audits, silences and instance state have their own lifecycle rather than the ordinary monitoring TTL. This means the monitoring retention value is not a deadline for every database row or credential reference.

Measure actual storage

The page shows received and retained logical bytes and available retention lag. PostgreSQL heap, indexes, WAL, allocated disk, backups and compression are not inferred from those logical estimates. Some retention components have no measured lag; a zero for one source does not prove the entire system is caught up. Unknown queue/drop/pairing counters remain unavailable unless measured.

Size and monitor the actual database and persistent volume. Seven days and the chart's default 16 GiB volume are configuration defaults, not a capacity or cost guarantee. The evidence budget is a logical-data bound, not a physical disk cap. Admission/cardinality limits can reject data before disk is full; inspect coverage and pipeline failures as well as storage use.

Backup, restart and restore

Bundled PostgreSQL retains its PVC when its StatefulSet is deleted or scaled. A retained volume is not a backup. Use PostgreSQL's consistent backup/restore tools, test recovery into a separate database and verify schema compatibility before starting API/worker processes.

For exact pending-work recovery, pause the worker and quiesce collection before backup. Record pending delivery identities, rule/destination versions and evidence pins, then compare them after restore. Preserve referenced credential versions separately in your secret manager: a database reference cannot recover credential bytes. Keep the original database until the restore is checked.

Collector export queues and graph pairing live in memory, independent of database backups. Restart can lose them; file logs start at end on a fresh reader. Collection-mode changes need a planned gap and must avoid overlapping pairing gateways. An arbitrary old binary may not understand a newer schema. Follow the upgrade and restore procedure before changing versions or modes.

View Markdown source