Security monitoring and log collection

You can use monitoring and logging to gain insight into your deployment for proactive security and continuous compliance activities, or for incident investigation. Monitoring and logging span two distinct domains.

The control plane runs as pods on Red Hat OpenShift Container Platform (RHOCP), and its metrics and logs are collected through the native RHOCP Observability stack. The data plane consists of Red Hat Enterprise Linux 9.4 nodes where telemetry is collected by agents such as ceilometer and sg-core.

These metrics are forwarded to a dedicated monitoring stack instance, which uses the same standard tools as the RHOCP observability stack but is exclusively reserved for RHOSO administrator infrastructure. This architecture replaces the standalone Service Telemetry Framework (STF) and per-node log file approaches from Red Hat Openstack Platform (RHOSP) 17.1.

Monitoring and logging infrastructure security
Copy link

Centralized logging and metrics systems are high-value targets because they gather important information about your deployment. Apply the following hardening measures to the observability stack.

Secure the RHOSO monitoring and logging stack
  • Restrict access to observability namespaces: The Prometheus, AlertManager, and Loki instances run in Red Hat OpenShift Container Platform (RHOCP) and the RHOSO namespaces such as openshift-monitoring, openshift-logging,openstack. Use RHOCP RBAC to limit who can query, modify alert rules, or access raw log data. Limit access to cluster-admin and designated security operators.
  • Enable Transport Layer Security (TLS) for all observability endpoints: RHOCP monitoring components use TLS by default for internal scrape targets. RHOSO components also enable TLS by default. You can verify this by ensuring the TLS config is present and that podLevel TLS is enabled in the OpenStackControlPlane custom resource (CR).
  • Protect persistent storage: Prometheus metrics are stored on persistent volumes, and Loki log data is stored on a supported object storage solution. Ensure that the StorageClass uses encrypted volumes, for example, LUKS-encrypted LVM with the LVM Storage Operator, or Red Hat OpenShift Data Foundation with encryption enabled.
  • Back up monitoring data: Implement regular backups of Prometheus time series database (TSDB) snapshots and Loki chunks to an offsite or object-storage target. If a control plane compromise occurs, you can use these backups to preserve the forensic record.
  • Set retention policies: Configure Prometheus and Loki retention windows to comply with your organization’s log retention policies. Balance your retention policy based on your needs and available resources. Insufficient retention loses forensic history; excessive retention increases storage costs and breach exposure.
Secure data plane log transport
  • Use TLS for syslog forwarding: If forwarding data plane logs to the RHOCP logging stack or an external Security Information and Event Management (SIEM), configure the syslog transport with TLS encryption and mutual authentication by configuring the OpenStackDataPlaneNodeSet CR with tlsEnabled: true. Unencrypted syslog exposes log content on the wire.
  • Restrict log destination endpoints: Ensure that outputs of the ClusterLogForwarder point only to approved, internal log aggregation endpoints. An attacker who modifies the forwarder CR could redirect logs to an external server.
  • Observe the logging pipeline and the monitoring stack: Create PrometheusRules to notify you when the log forwarder pods are down, when Loki ingestion rate drops to zero, or when Prometheus scrape failures exceed a threshold.
Note
The absence of log and metrics generation might indicate a high-priority security event, such as a service failure or an intruder who has disabled logging or modified the log level. Alert on silence by using the absent_over_time Prometheus query.

For information on how to set up alerts with metrics collected by Telemetry Operator, see the Red Hat Knowledgebase article Alerts in the Telemetry Operator.

For information about how to enable logging on the control plane and data plane, see Enable observability logging.

API audit trails for compliance
Copy link

The Cloud Auditing Data Federation (CADF) standard generates audit trails from API calls. CADF events conform to the Distributed Management Task Force (DMTF) CADF specification.

CADF provides normalized audit data that features the seven W’s of auditing:

  • What
  • When
  • Who
  • On What
  • Where
  • From Where
  • To Where