Chapter 8. Forward to Amazon CloudWatch


You can forward logs to Amazon CloudWatch and Amazon S3 using AWS authentication credentials.

8.1. Forward logs to Amazon CloudWatch

To configure log forwarding to CloudWatch, create a ClusterLogForwarder custom resource (CR) with an output for CloudWatch and a pipeline that uses the output.

Prerequisites

  • You have installed Red Hat OpenShift Logging Operator.
  • You have administrator access to OpenShift Container Platform.
  • You have installed OpenShift CLI (oc).

Procedure

  1. Create a Secret YAML file that uses the aws_access_key_id and aws_secret_access_key fields to specify your base64-encoded AWS credentials. For example:

    apiVersion: v1
    kind: Secret
    metadata:
      name: aws-secret
      namespace: <namespace_name>
    data:
      aws_access_key_id: QUtJQUlPU0ZPRE5ON0VYQU1QTEUK
      aws_secret_access_key: d0phbHJYVXRuRkVNSS9LN01ERU5HL2JQeFJmaUNZRVhBTVBMRUtFWQo=
    namespace
    Specify the namespace where your ClusterLogForwarder CR is deployed. The secret must be in the same namespace as the ClusterLogForwarder CR.
  2. Create the secret. For example:

    $ oc apply -f aws-secret.yaml
  3. Create or edit a YAML file that defines the ClusterLogForwarder CR object. In the file, specify the name of the secret. For example:

    apiVersion: observability.openshift.io/v1
    kind: ClusterLogForwarder
    metadata:
      name: <log_forwarder_name>
      namespace: <log_forwarder_namespace>
    spec:
      serviceAccount:
        name: <service_account_name>
      outputs:
       - name: cw-output
         type: cloudwatch
         cloudwatch:
           authentication:
             type: awsAccessKey
             awsAccessKey:
               keyId:
                 key: <key>
                 secretName: cw-secret
               keySecret:
                 key: <key>
                 secretName: cw-secret
           groupName: <prefix-{.dynamic_value||"none"}>
           region: us-east-2
      pipelines:
        - name: infra-logs
          inputRefs:
            - infrastructure
            - audit
            - application
          outputRefs:
            - cw-output
    serviceAccount.name
    Specify the name of your service account.
    outputs.name
    Specify a name for the output.
    type
    Specify the cloudwatch type.
    secretName
    Specify the name of the secret that has your AWS credentials.
    groupName
    Specify the strategy for grouping log streams. The value can be a combination of static and dynamic values consisting of field paths separated by "||". Encase dynamic values in single curly brackets "{}". Follow dynamic values with a static fallback value.
    region
    Specify the AWS region.
    pipelines.name
    Specify a name for the pipeline.
    inputRefs
    Specify the names of inputs defined in the input.name field for this pipeline. You can also use the built-in values application, infrastructure, audit.
    output
    Specify the names of outputs defined in the outputs.name field for this pipeline.
  4. Apply the CR object:

    $ oc apply -f <file_name>.yaml

8.2. CloudWatch output format and field mappings

When you forward logs to Amazon CloudWatch, the log collector transforms OpenShift log records into the CloudWatch Logs API format. Understanding this transformation helps you configure CloudWatch queries and dashboards effectively.

The log collector sends logs to CloudWatch by using the PutLogEvents API. Each log record becomes a CloudWatch LogEvent with two main components:

message
The entire OpenShift log record serialized as a JSON string. This field contains all log data including Kubernetes metadata, application messages, and timestamps.
timestamp
The log event timestamp as UNIX milliseconds (milliseconds since January 1, 1970 00:00:00 UTC). The collector converts the log record’s @timestamp field to this format.

The following table shows how OpenShift log record fields map to CloudWatch LogEvents:

OpenShift fieldCloudWatch fieldTransformation

Complete log record

message

JSON-serialized string containing the entire log record

@timestamp

timestamp

Converted to UNIX milliseconds

n/a

logGroupName

Configured in ClusterLogForwarder output

n/a

logStreamName

Configured in ClusterLogForwarder output, supports template syntax

{
  "kubernetes": {
    "namespace_name": "my-app",
    "pod_name": "web-1",
    "container_name": "nginx"
  },
  "message": "Error connecting to database",
  "level": "error",
  "@timestamp": "2024-01-15T10:30:00Z"
}

The log collector transforms this into a CloudWatch LogEvent:

{
  "logEvents": [
    {
      "message": "{\"kubernetes\":{\"namespace_name\":\"my-app\",\"pod_name\":\"web-1\",\"container_name\":\"nginx\"},\"message\":\"Error connecting to database\",\"level\":\"error\",\"@timestamp\":\"2024-01-15T10:30:00Z\"}",
      "timestamp": 1705318200000
    }
  ],
  "logGroupName": "/openshift/logging",
  "logStreamName": "my-app"
}

The group_name and stream_name fields in your ClusterLogForwarder output configuration control how logs are organized in CloudWatch:

  • group_name: Determines the CloudWatch log group
  • stream_name: Determines the CloudWatch log stream, supports template syntax such as {{ kubernetes.namespace_name }}

Both fields can use dynamic values from the log record. The collector evaluates these templates for each log event.

Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat Documentation

Legal Notice

Theme

© 2026 Red Hat
Back to top