Chapter 12. Forward from pods and projects


You can configure log forwarding to send logs from specific pods or projects, allowing you to route logs based on application labels or namespace ownership.

12.1. Forward application logs from specific pods

As a cluster administrator, you can use Kubernetes pod labels to gather log data from specific pods and forward it to a log collector.

Suppose that you have an application composed of pods running alongside other pods in various namespaces. If those pods have labels that identify the application, you can gather and output their log data to a specific log collector.

To specify the pod labels, you use one or more matchLabels key-value pairs. If you specify many key-value pairs, the collector selects only pods that match all of them.

Procedure

  1. Create or edit a YAML file that defines the ClusterLogForwarder CR object. In the file, specify the pod labels using simple equality-based selectors under inputs[].name.application.selector.matchLabels, as shown in the following example.

    apiVersion: observability.openshift.io/v1
    kind: ClusterLogForwarder
    metadata:
      name: <log_forwarder_name>
      namespace: <log_forwarder_namespace>
    spec:
      serviceAccount:
        name: <service_account_name>
      outputs:
        - <output_name>
        # ...
      inputs:
      - name: exampleAppLogData
        type: application
        application:
          includes:
          - namespace: app1
          - namespace: app2
          selector:
            matchLabels:
              environment: production
              app: nginx
      pipelines:
      - inputRefs:
        - exampleAppLogData
        outputRefs:
        # ...
    serviceAccount.name
    Specify the service account name.
    inputs.name
    Specify a name for the input.
    type
    Specify the type as application to collect logs from applications.
    includes
    Specify the set of namespaces to include when collecting logs.
    matchLabels
    Specify the key-value pairs of pod labels whose log data you want to gather. You must specify both a key and value, not just a key. The collector selects only pods that match all the key-value pairs.
  2. Optional: You can send log data from additional applications that have different pod labels to the same pipeline.

    1. For each unique combination of pod labels, create an additional inputs[].name section similar to the one shown.
    2. Update the selectors to match the pod labels of this application.
    3. Add the new inputs[].name value to inputRefs. For example:

      - inputRefs: [ myAppLogData, myOtherAppLogData ]
  3. Create the CR object:

    $ oc create -f <file_name>.yaml

12.1.1. Forward logs by using the syslog protocol

You can use the syslog RFC3164 or RFC5424 protocol to send a copy of your logs to an external log aggregator. You are responsible for configuring the external log aggregator, such as a syslog server, to receive the logs from OpenShift Container Platform.

To configure log forwarding by using the syslog protocol, you must create a ClusterLogForwarder custom resource (CR) with one or more outputs to the syslog servers, and pipelines that use those outputs. The syslog output can use a UDP, TCP, or TLS connection.

Prerequisites

  • You must configure a logging server to receive the logging data by using the specified protocol or format.

Procedure

  1. Create or edit a YAML file that defines the ClusterLogForwarder CR object:

    apiVersion: observability.openshift.io/v1
    kind: ClusterLogForwarder
    metadata:
      name: collector
    spec:
      managementState: Managed
      outputs:
      - name: rsyslog-east
        syslog:
          appName: <app_name>
          enrichment: KubernetesMinimal
          facility: <facility_value>
          msgId: <message_ID>
          payloadKey: <record_field>
          procId: <process_ID>
          rfc: <RFC3164_or_RFC5424>
          severity: informational
          tuning:
            deliveryMode: <AtLeastOnce_or_AtMostOnce>
          url: <url>
        tls:
          ca:
            key: ca-bundle.crt
            secretName: syslog-secret
        type: syslog
      pipelines:
      - inputRefs:
        - application
        name: syslog-east
        outputRefs:
        - rsyslog-east
      serviceAccount:
        name: logcollector
    outputs.name
    Specify a name for the output.
    appName
    Optional: Specify the value for the APP-NAME part of the syslog message header. The value must conform with The Syslog Protocol. The value can be a combination of static and dynamic values consisting of field paths followed by ||, and then followed by another field path or a static value. The collector truncates the final values to a maximum length of 48 characters. You must encase a dynamic value in curly brackets and follow the value with a static fallback value separated with ||. Static values can only contain alphanumeric characters along with dashes, underscores, dots and forward slashes. Example value: <value1>-{.<value2>||"none"}.
    enrichment
    Optional: Specify the enrichment level for Kubernetes metadata fields added to log records. The value can be KubernetesMinimal to add minimal Kubernetes fields such as namespace, pod name, and container name. This parameter replaces the addLogSource parameter that was removed in Logging 6.0.
    facility
    Optional: Specify the value for Facility part of the syslog-msg header.
    msgId
    Optional: Specify the value for MSGID part of the syslog-msg header. The value can be a combination of static and dynamic values consisting of field paths followed by ||, and then followed by another field path or a static value. The collector truncates the final values to a maximum length of 32 characters. You must encase a dynamic value in curly brackets and follow the value with a static fallback value separated with ||. Static values can only contain alphanumeric characters along with dashes, underscores, dots and forward slashes. Example value: <value1>-{.<value2>||"none"}.
    payloadKey
    Optional: Specify the record field to use as the payload. The payloadKey value must be a single field path encased in single curly brackets {}. Example: {.<value>}.
    procId
    Optional: Specify the value for the PROCID part of the syslog message header. The value must conform with The Syslog Protocol. The value can be a combination of static and dynamic values consisting of field paths followed by ||, and then followed by another field path or a static value. The collector truncates the final values to a maximum length of 48 characters. You must encase a dynamic value in curly brackets and follow the value with a static fallback value separated with ||. Static values can only contain alphanumeric characters along with dashes, underscores, dots and forward slashes. Example value: <value1>-{.<value2>||"none"}.
    rfc
    Optional: Set the RFC that the generated messages conform to. The value can be RFC3164 or RFC5424.
    severity
    Optional: Set the severity level for the message. For more information, see The Syslog Protocol.
    deliveryMode
    Optional: Set the delivery mode for log forwarding. The value can be either AtLeastOnce, or AtMostOnce.
    url
    Specify the absolute URL with a scheme. Valid schemes are: tcp, tls, and udp. For example: tls://syslog-receiver.example.com:6514.
    tls
    Specify the settings for controlling options of the transport layer security (TLS) client connections.
    inputRefs
    Specify which log types to forward by using the pipeline: application, infrastructure, or audit.
    pipelines.name
    Specify a name for the pipeline.
    serviceAccount
    The name of your service account.
  2. Create the CR object:

    $ oc create -f <filename>.yaml

12.1.2. Default field mappings for RFC5424

When using the RFC5424 protocol, the collector automatically populates the APP-NAME, PROCID, and MSGID fields with default values based on the log source type. You can override these defaults by specifying appName, procId, or msgId in the syslog output configuration.

The following table shows the default field mappings:

Table 12.1. Default RFC5424 field values by log source
Log sourceAPP-NAMEPROCIDMSGID

Infrastructure (journal logs)

SYSLOG_IDENTIFIER

_PID

.log_source

Infrastructure (container logs)

namespace_pod_container

pod_id

.log_source

Application (container logs)

namespace_pod_container

pod_id

.log_source

Audit logs

.log_source

.auditID (if available)

.log_source

Note

Field values can use template syntax for dynamic per-event values. The maximum length for APP-NAME and PROCID is 48 characters, and for MSGID is 32 characters. The collector truncates values that exceed these limits.

12.1.3. Syslog output format and field mappings

When you forward logs to a syslog server, the log collector transforms OpenShift log records into RFC 5424 or RFC 3164 syslog format. Understanding this transformation helps you configure syslog server parsing rules and log routing.

The log collector sends logs by using the syslog protocol over TCP or UDP. You can configure the RFC format, severity mapping, and facility in the ClusterLogForwarder output configuration.

12.1.3.2. RFC 3164 format (legacy)

RFC 3164 is the older syslog format without structured data support. Use this format for compatibility with legacy syslog servers.

RFC 3164 message structure:

<PRIORITY>TIMESTAMP HOSTNAME TAG: MESSAGE

The following table shows how OpenShift log record fields map to RFC 3164 components:

RFC 3164 fieldOpenShift fieldNotes

PRIORITY

Calculated: (facility * 8) + severity

Default facility: 1 (user-level). Severity mapped from log level.

TIMESTAMP

@timestamp

MMM DD HH:MM:SS format (for example, Jan 15 10:30:00)

HOSTNAME

hostname

Kubernetes node name

TAG

Log type-dependent

Namespace/pod for application logs, process name for infrastructure logs

MESSAGE

message or complete log record

UTF-8 encoded log message

The same OpenShift log record in RFC 3164 format:

<11>Jan 15 10:30:00 ip-10-0-1-42.ec2.internal my-app/web-1: Error connecting to database

12.1.3.3. Severity mapping

The log collector maps OpenShift log levels to syslog severity values:

OpenShift levelSyslog severityValue

trace, debug

Debug

7

info, default

Informational

6

warn, warning

Warning

4

err, error

Error

3

crit, critical

Critical

2

alert

Alert

1

emerg, emergency, fatal

Emergency

0

If the log record does not have a level field, the collector defaults to severity 6 (Informational).

12.1.3.4. Facility codes

The syslog facility indicates the source of the log message. You can configure the facility in the ClusterLogForwarder output configuration.

Common facility values:

ValueFacilityDescription

0

kern

kernel messages

1

user

User-level messages (default)

3

daemon

System daemons

4

auth

Security/authentication messages

16-23

local0-local7

Locally-defined facilities

The default facility is 1 (user-level). For infrastructure logs, you might configure facility 3 (daemon) or a local facility.

The following infrastructure log from systemd journal:

{
  "message": "Started container",
  "SYSLOG_IDENTIFIER": "kubelet",
  "hostname": "ip-10-0-1-42.ec2.internal",
  "@timestamp": "2024-01-15T10:30:00.000Z"
}

The log collector transforms this into RFC 5424 syslog:

<14>1 2024-01-15T10:30:00.000Z ip-10-0-1-42.ec2.internal kubelet - - - Started container

Priority calculation: (1 * 8) + 6 = 14 (facility 1 = user, severity 6 = info)

12.1.3.5. Transport protocols

The log collector supports the following transport protocols for syslog:

TCP
Reliable delivery with connection-oriented transport. Recommended for production environments.
UDP
Connectionless transport that reduces network traffic but provides no delivery guarantees. Use for high-volume, non-critical logs.
TLS over TCP
Encrypted transport with certificate-based authentication. Required for secure log forwarding across untrusted networks.

Configure the transport protocol in the ClusterLogForwarder output specification.

You can add namespace_name, pod_name, and container_name elements to the message field of the record by adding the enrichment field to your ClusterLogForwarder custom resource (CR).

# ...
  spec:
    outputs:
    - name: syslogout
      syslog:
        enrichment: KubernetesMinimal
        facility: user
        payloadKey: message
        rfc: RFC3164
        severity: debug
      type: syslog
      url: tls://syslog-receiver.example.com:6514
    pipelines:
    - inputRefs:
      - application
      name: test-app
      outputRefs:
      - syslogout
# ...
Note

This configuration is compatible with both RFC3164 and RFC5424.

Example syslog message output with enrichment: None

 2025-03-03T11:48:01+00:00  example-worker-x  syslogsyslogserverd846bb9b: {...}

Example syslog message output with enrichment: KubernetesMinimal

2025-03-03T11:48:01+00:00  example-worker-x  syslogsyslogserverd846bb9b: namespace_name=cakephp-project container_name=mysql pod_name=mysql-1-wr96h,message: {...}

You can forward a copy of the application logs from specific projects to an external log aggregator, in addition to, or instead of, using the internal log store. You must also configure the external log aggregator to receive log data from OpenShift Container Platform.

To configure forwarding application logs from a project, you must create a ClusterLogForwarder custom resource (CR) with at least one input from a project, optional outputs for other log aggregators, and pipelines that use those inputs and outputs.

Prerequisites

  • You must configure a logging server to receive the logging data using the specified protocol or format.

Procedure

  1. Create or edit a YAML file that defines the ClusterLogForwarder CR:

    Example ClusterLogForwarder CR

    apiVersion: observability.openshift.io/v1
    kind: ClusterLogForwarder
    metadata:
      name: <log_forwarder_name>
      namespace: <log_forwarder_namespace>
    spec:
      serviceAccount:
        name: <service_account_name>
      outputs:
      - name: <output_name>
        type: <output_type>
      inputs:
      - name: my-app-logs
        type: application
        application:
          includes:
          - namespace: my-project
      filters:
      - name: my-project-labels
        type: openshiftLabels
        openshiftLabels:
          project: my-project
      - name: cluster-labels
        type: openshiftLabels
        openshiftLabels:
          clusterId: C1234
      pipelines:
      - name: <pipeline_name>
        inputRefs:
        - my-app-logs
        outputRefs:
        - <output_name>
        filterRefs:
        - my-project-labels
        - cluster-labels

    inputs.name
    Specify the name for the input.
    type
    Specify the type as application to collect logs from applications.
    includes
    Specify the set of namespaces and containers to include when collecting logs.
    openshiftLabels
    Specify the labels to apply to log records passing through this pipeline. These labels appear in the openshift.labels map in the log record.
    pipelines.name
    Specify a name for the pipeline.
  2. Apply the ClusterLogForwarder CR by running the following command:

    $ oc apply -f <filename>.yaml
Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat Documentation

Legal Notice

Theme

© 2026 Red Hat
Back to top