Skip to content

Collect telemetry

Both collection modes send supported data through one AlertBird gateway. Follow the quickstart for images, credentials, trust, database and Helm installation before configuring workloads.

Connect an existing Collector

The existing Collector configuration shows a loopback OTLP/HTTP receiver forwarding traces, logs and native metrics through authenticated TLS gRPC. Adapt its exporter into your existing pipelines after resource enrichment and before sampling or exception-only filtering.

Set the following locations and identity; mount the token and CA privately in the existing Collector:

export ALERTBIRD_GATEWAY_GRPC_ENDPOINT=alertbird-otel-gateway.monitoring.svc:4317
export ALERTBIRD_GATEWAY_CA_FILE=/etc/alertbird/ca.crt
export ALERTBIRD_INGRESS_TOKEN_FILE=/etc/alertbird/token

The token must match alertbird-otel-ingress in the installation namespace. The CA must verify the gateway certificate and its DNS name. Keep TLS verification enabled. An exporter outside the cluster needs owner-provided routing and a certificate matching the address it actually uses; the chart creates no public OTLP endpoint. HTTP 4318 requires a private network or trusted TLS boundary. Do not add another graph connector or count an existing spanmetrics stream as AlertBird's canonical request population.

Connect an SDK

The SDK environment example documents standard exporter settings. Set a stable service.name and resource attributes matching your cluster, environment and namespaces. For example:

export OTEL_SERVICE_NAME=checkout
export OTEL_RESOURCE_ATTRIBUTES=k8s.cluster.name=my-cluster,deployment.environment.name=production,k8s.namespace.name=applications,service.namespace=commerce
export OTEL_TRACES_SAMPLER=always_on
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_ENDPOINT=http://alertbird-otel-gateway.monitoring.svc:4318
export OTEL_TRACES_EXPORTER=otlp
export OTEL_LOGS_EXPORTER=none
export OTEL_METRICS_EXPORTER=none

This endpoint is for private in-cluster traffic. Add OTEL_EXPORTER_OTLP_HEADERS in the application launcher from the owner's mounted ingress token, using Authorization=Bearer%20<token>. Keep credentials out of values, annotations, source control and public browser variables. SDKs do not uniformly support header-file variables. For a trusted HTTPS boundary use its actual origin and configure its CA. Per-signal HTTP endpoints include /v1/traces, /v1/metrics or /v1/logs.

Use AlwaysOn for the supported complete-source path; ParentBased(AlwaysOn) can still suppress spans with unsampled remote parents. Both callers and servers need compatible instrumentation and trace-context propagation to form graph connections. Existing SDKs, runtime libraries and platform combinations require their own verification; accepting OTLP does not certify every instrument.

Included Kubernetes signals

The included profile selects one workload namespace with kube-stack.collectors.node.workloadNamespace, default applications. Linux node collectors read its pod stdout and selected kubelet CPU/memory metrics. The gateway collects selected Deployment desired/available metrics and Kubernetes events once. Resource cluster and environment values must describe the actual source. Unknown service identities remain unknown.

Node collectors use read-only host logs and narrow read-only Kubernetes access. Owners supply a kubelet CA that verifies each node hostname and a gateway CA that verifies the gateway. The node reader requires UID 0 for host logs, with capabilities dropped and privilege escalation disabled. Checkpoints are disabled and filelog starts at the end; a fresh reader does not replay old files.

Choose one log owner per workload population. In the included stdout namespace, set SDK OTEL_LOGS_EXPORTER=none to avoid exporting the same log twice. The chart's ownership declaration does not reconfigure existing SDKs or deduplicate arbitrary records. Known Collector containers are excluded to prevent recursion; owners must exclude custom external Collector container names before adopting them.

Opt-in application instrumentation

The verified injection scope is Linux arm64/glibc with Python 3.12.16 + Flask 3.1.3 (instrumentation 0.66b1), or Node 22.23.4 built-in HTTP (instrumentation 0.78.0). Other runtime, library and platform combinations are unverified. There is no blanket workload instrumentation or privileged Go injection.

After verifying Operator/CRD/admission prerequisites, create alertbird-instrumentation-auth in each opted-in workload namespace. Its key headers contains Authorization=Bearer%20<matching-ingress-token>. The following uses the protected ingress file from the quickstart without printing the token:

python3 - <<'PY'
import os
from pathlib import Path
root = Path(os.environ['ALERTBIRD_SECRETS_DIR'])
path = root / 'headers'
path.write_text('Authorization=Bearer%20' + (root / 'ingress-token').read_text().strip())
path.chmod(0o600)
PY
kubectl -n applications create secret generic alertbird-instrumentation-auth \
  --from-file=headers="$ALERTBIRD_SECRETS_DIR/headers"
helm upgrade alertbird charts/alertbird --namespace monitoring \
  -f owner-values.yaml -f database-values.yaml \
  -f charts/alertbird/values-included.yaml \
  -f charts/alertbird/values-instrumentation.yaml \
  --set kube-stack.instrumentation.exporter.endpoint=http://alertbird-otel-gateway.monitoring.svc:4318

For a reused Operator, also retain -f charts/alertbird/values-existing-operator.yaml after the included profile on every upgrade. Endpoint mode does not offer chart-managed injection.

Set annotations on the chosen workload's pod template, under spec.template.metadata.annotations, replacing app with its actual container name:

instrumentation.opentelemetry.io/inject-python: monitoring/alertbird
instrumentation.opentelemetry.io/container-names: app

For Node, use instrumentation.opentelemetry.io/inject-nodejs instead of inject-python. The Instrumentation resource must exist before new pods start. Roll out the selected workload, verify new pod UIDs and injected init image, mounts and environment, then check actual complete CLIENT/SERVER trace pairs. A running pod can still be uninstrumented if injection failed open. Keep unselected workloads unchanged.

Source quality and coverage

Fresh observations alone do not prove a complete source path. Numeric alert evaluation also checks the owner's explicit scoped source-quality assumptions. After validating an unsampled path, add a declaration to owner-values.yaml:

installation:
  clusterId: my-cluster
  sourceQuality:
    - source: trace_server
      scope:
        cluster_id: my-cluster
        environment: production
        k8s_namespace: applications
        service_namespace: commerce
        service_name: checkout
      quality: supported_unsampled
      effective_from: 'REPLACE_WITH_ACTUAL_UTC_TIME'

Replace the effective time with an aware UTC ISO8601 timestamp for when that source path was established, such as 2026-11-16T12:00:00Z. Do not backdate it to make old windows eligible. Apply the same chosen collection/database profiles on upgrade. Declarations apply only to windows wholly after that time. Mark a sampled source sampled; leave uncertain sources unknown.

These declarations are assumptions, not measured proof. Sampled/unknown declarations dominate overlapping supported declarations. Actual missing data, stale receipts, open windows, drops or resets can still prevent numeric evaluation. Use Data & retention to inspect source freshness and coverage reasons; use Service health to check scoped measurements. Never infer whole-cluster completeness from a connected listener or one observed service.

View Markdown source