Skip to content

Install AlertBird

Install the API, alerting worker, web app, OpenTelemetry gateway and PostgreSQL in your Kubernetes cluster. Choose included Kubernetes collection or connect an existing Collector. Both use the same service-health and alerting engine.

AlertBird is an early self-hosted project. The repository is currently private, licensing is being prepared, and application images must be built from source. You need authorized repository access to follow this guide.

Commands below run from the repository root and use release alertbird, namespace monitoring, workload namespace applications, cluster my-cluster and environment production. Change these together if your installation uses different names.

Prepare your cluster and images

You need Kubernetes administrator access for installation, Helm 4.0.1, kubectl, Docker, Python 3, a registry reachable by your nodes, persistent storage for bundled PostgreSQL, and an HTTPS ingress controller. Included mode also needs Linux nodes and trusted kubelet certificates. These settings are a starting envelope; size storage and resources for your own workload.

Build and push both application images. Replace the registry below with yours. Build for your cluster's CPU architecture and arrange node registry access before installing; the chart has no application image-pull-secret setting.

docker build -f apps/api/Dockerfile -t registry.example.com/alertbird-api:initial apps/api
docker build -f apps/app/Dockerfile -t registry.example.com/alertbird-app:initial apps/app
docker push registry.example.com/alertbird-api:initial
docker push registry.example.com/alertbird-app:initial
kubectl create namespace monitoring
kubectl create namespace applications
helm dependency build charts/alertbird

Record the pushed digests. Use registry/name@sha256:... references in values, rather than the mutable tags used for building. Upstream Collector, Operator, instrumentation and PostgreSQL images are already pinned by the chart.

Create credentials outside Helm

Use your secret manager or protected local files. This example generates separate bootstrap, ingress, ingest-role and bundled-database credentials without printing their values. It creates a private directory outside the checkout; preserve it securely or import the values into your secret manager. Do not commit it.

export ALERTBIRD_SECRETS_DIR="$(mktemp -d "${TMPDIR:-/tmp}/alertbird-secrets.XXXXXX")"
chmod 700 "$ALERTBIRD_SECRETS_DIR"
python3 - <<'PY'
import json, os, secrets
from pathlib import Path
from urllib.parse import quote

root = Path(os.environ['ALERTBIRD_SECRETS_DIR'])
def write(name, value):
    path = root / name
    path.write_text(value)
    path.chmod(0o600)

roles = {role: secrets.token_urlsafe(32) for role in ('workload', 'graph', 'diagnostic')}
for role, token in roles.items():
    write(role, token)
write('keys.json', json.dumps({'keys': [
    {'id': role + '-1', 'role': role, 'secret': token} for role, token in roles.items()
]}))
write('bootstrap-password', secrets.token_urlsafe(32))
write('ingress-token', secrets.token_urlsafe(32))
write('username', 'alertbird')
write('database', 'alertbird')
password = secrets.token_urlsafe(32)
write('database-password', password)
write('database-url', 'postgresql://alertbird:' + quote(password, safe='') +
      '@alertbird-postgres.monitoring.svc:5432/alertbird')
PY
kubectl -n monitoring create secret generic owner-bootstrap \
  --from-file=password="$ALERTBIRD_SECRETS_DIR/bootstrap-password"
kubectl -n monitoring create secret generic owner-ingest \
  --from-file=keys.json="$ALERTBIRD_SECRETS_DIR/keys.json"
kubectl -n monitoring create secret generic alertbird-otel-ingress \
  --from-file=token="$ALERTBIRD_SECRETS_DIR/ingress-token"
kubectl -n monitoring create secret generic alertbird-otel-api-roles \
  --from-file=workload="$ALERTBIRD_SECRETS_DIR/workload" \
  --from-file=graph="$ALERTBIRD_SECRETS_DIR/graph" \
  --from-file=diagnostic="$ALERTBIRD_SECRETS_DIR/diagnostic"

The role map must match all three gateway tokens. The gateway ingress token is separate. Application users sign in with local sessions; ingest tokens do not grant access to the product UI.

Supply gateway and kubelet trust

Obtain a server certificate and private key from your PKI for alertbird-otel-gateway.monitoring.svc. Its SAN must match that DNS name and its chain must be trusted by exporters. Place them in protected gateway.crt and gateway.key files, with the issuing CA bundle in gateway-ca.crt.

kubectl -n monitoring create secret tls alertbird-otel-gateway-tls \
  --cert=gateway.crt --key=gateway.key

Included mode also needs the CA that verifies each node's kubelet serving certificate for its node name, plus working node-name DNS. Obtain that bundle from your cluster administrator as kubelet-ca.pem; do not substitute an unrelated API-server CA or disable hostname verification.

kubectl -n monitoring create secret generic alertbird-otel-gateway-ca \
  --from-file=ca.crt=gateway-ca.crt
kubectl -n monitoring create configmap alertbird-kubelet-ca \
  --from-file=ca.pem=kubelet-ca.pem

Endpoint mode does not need those last two objects. Give existing exporters the gateway CA and ingress token through their own protected mounts. HTTP port 4318 is plaintext: keep it private or put it behind an authenticated TLS boundary. gRPC port 4317 uses the gateway certificate. The chart does not issue or renew these certificates.

Choose PostgreSQL

For bundled persistent PostgreSQL, create the Secret below and save database-values.yaml with the shown content. The URL must match the Secret's username, password and database. Bundled connections are private and explicitly disable database TLS.

kubectl -n monitoring create secret generic owner-database \
  --from-file=url="$ALERTBIRD_SECRETS_DIR/database-url" \
  --from-file=username="$ALERTBIRD_SECRETS_DIR/username" \
  --from-file=database="$ALERTBIRD_SECRETS_DIR/database" \
  --from-file=password="$ALERTBIRD_SECRETS_DIR/database-password"
database:
  mode: bundled
  secret: owner-database
  storage: 16Gi
  # storageClass: your-persistent-storage-class

For external PostgreSQL, instead put your connection URI in a protected external-database-url file and its CA bundle in external-database-ca.crt. Use a TCP hostname matching the database certificate and URI-encode credentials. The database must be reachable from the release namespace and permit the migration role to create/update AlertBird's schema.

kubectl -n monitoring create secret generic owner-database \
  --from-file=url=external-database-url
kubectl -n monitoring create secret generic owner-database-ca \
  --from-file=ca.crt=external-database-ca.crt

Use this alternative database-values.yaml:

database:
  mode: external
  secret: owner-database
  caSecret: owner-database-ca

External mode creates no PostgreSQL workload or PVC. All migration, API and worker connections enforce verify-full with the mounted CA, including when URL options request weaker TLS. A PVC is not a backup. Changing an existing database Secret does not rotate the database role's password; coordinate rotation separately.

Save owner values

Save the following as owner-values.yaml, replacing image digests and origin. Keep this nonsecret file, database-values.yaml and your chosen collection profile on every install or upgrade.

core:
  backendImage: registry.example.com/alertbird-api@sha256:REPLACE_WITH_PUSHED_DIGEST
  appImage: registry.example.com/alertbird-app@sha256:REPLACE_WITH_PUSHED_DIGEST
  publicOrigin: https://monitoring.example.com
  bootstrapSecret: owner-bootstrap
  ingestSecret: owner-ingest
installation:
  clusterId: my-cluster
kube-stack:
  resourceAttributes:
    k8s.cluster.name: my-cluster
    deployment.environment.name: production
  collectors:
    node:
      workloadNamespace: applications

Install one collection mode

Existing Collector or SDK

This profile installs the core and singleton gateway without node collectors, Operator, CRDs or injection.

helm lint charts/alertbird -f owner-values.yaml -f database-values.yaml \
  -f charts/alertbird/values-endpoint.yaml
helm install alertbird charts/alertbird --namespace monitoring \
  -f owner-values.yaml -f database-values.yaml \
  -f charts/alertbird/values-endpoint.yaml

Configure exporters using the collection guide. Use alertbird-otel-gateway.monitoring.svc:4317 for in-cluster TLS gRPC, with the matching CA and ingress bearer token. Forward supported complete traces before sampling or exception-only filtering; do not add a second graph connector.

Included Kubernetes collection

Included mode adds node collectors for selected pod logs and kubelet metrics, plus singleton Deployment/event collection. It needs an upstream OTel Operator even when application instrumentation is disabled.

For a new Operator, install its prerequisites first:

helm install alertbird charts/alertbird --namespace monitoring \
  -f owner-values.yaml -f database-values.yaml \
  -f charts/alertbird/values-included.yaml \
  -f charts/alertbird/values-prerequisites.yaml
kubectl wait --for=condition=Established --timeout=120s \
  crd/opentelemetrycollectors.opentelemetry.io \
  crd/instrumentations.opentelemetry.io \
  crd/opampbridges.opentelemetry.io \
  crd/targetallocators.opentelemetry.io
kubectl -n monitoring get deployments,pods

Before continuing, verify the ready Operator runs pinned 0.161.0, all CRD schemas are compatible, admission CA/serving certificate/DNS/validity agree, and server-side CREATE and UPDATE of an owner-scoped test Collector CR succeed. Use the Operator verification procedure for the required checks and certificate lifecycle limits. The prerequisite stage contains no API, database, app or collection resources.

Then enable the complete installation:

helm upgrade alertbird charts/alertbird --namespace monitoring \
  -f owner-values.yaml -f database-values.yaml \
  -f charts/alertbird/values-included.yaml

For an existing compatible Operator, verify the same prerequisites and install directly with the reuse profile:

helm install alertbird charts/alertbird --namespace monitoring \
  -f owner-values.yaml -f database-values.yaml \
  -f charts/alertbird/values-included.yaml \
  -f charts/alertbird/values-existing-operator.yaml

Retain the reuse file on every upgrade. Shared Operator/CRDs remain externally owned. Do not apply the prerequisite stage to a running core release. Supported Python/Node injection is a separate opt-in and workload rollout; see collection.

Open the app

kubectl -n monitoring get jobs,pods,pvc
kubectl -n monitoring rollout status deployment/alertbird-api --timeout=300s
kubectl -n monitoring rollout status deployment/alertbird-worker --timeout=300s
kubectl -n monitoring rollout status deployment/alertbird-app --timeout=300s
kubectl -n monitoring rollout status deployment/alertbird-otel-gateway --timeout=300s

Require the current alertbird-migrate-<Helm revision> Job to complete, not just the gateway listener to be ready. Failed first migrations need the underlying credentials/connection corrected and a Helm upgrade for a fresh migration Job; restart waiting API/worker pods if their credential environment changed.

Create your owner-managed HTTPS ingress to Service alertbird-app, port 3000, at the exact core.publicOrigin hostname. Configure its certificate, DNS and ingress class for your cluster. The chart creates ClusterIP Services only. Its app NetworkPolicy allows same-namespace traffic: an ingress controller in another namespace needs an additional policy allowing only that controller's namespace and pods to the app's app.kubernetes.io/instance=alertbird and app.kubernetes.io/component=app labels on TCP 3000. Verify your CNI enforces NetworkPolicy. Keep the API, database and plaintext OTLP listener private.

Visit https://monitoring.example.com/login and sign in as admin using your protected bootstrap password. There is no default password. An existing user is not reset by changing the bootstrap Secret.

Verify your first service and alert

  1. Instrument one test service with service.name, cluster, environment and namespace attributes matching your installation. Send known successful and failing HTTP requests. To discover a connection, instrument both caller and server, propagate trace context, and send a complete CLIENT/SERVER pair.
  2. Open Service health, select the matching scope and a recent whole-minute window. Check observed request/error/status counts and coverage reasons. A service with no traffic, missing instrumentation or stale telemetry is not proof of a healthy service. Configure source-quality assumptions only after validating the source path, as described in collection.
  3. Open Alerts → New rule. Choose pipeline_stale, source trace_server, your exact scope, comparison Greater than, threshold 120 seconds, window 60, sustained duration 0, and minimum samples 1. Enable the rule with no destination for an evaluation-only check.
  4. After a valid observation, stop only the test service's telemetry exporter. Check that the rule opens an episode for stale receipts. Resume telemetry and verify observed recovery. This tests telemetry arrival, not HTTP error rate.
  5. To verify notification delivery, mount an owner-approved credential reference, create an enabled destination in Integrations, and use Test. Confirm provider acceptance in delivery activity and actual receipt at the destination. Then save a new rule version selecting it and repeat the test. See alerting for credentials, numeric-rule verification and provider limits. PagerDuty tests can create real incidents.

Keep an independent availability/worker/telemetry check outside this installation: a stopped API or worker cannot reliably notify you of its own outage. Read architecture and retention before relying on coverage or storage figures. Use the deployment lifecycle guide for upgrades, mode changes, backups and removal; do not reset a PVC to repair an installation.

View Markdown source