Chapter 8. Forward to Amazon CloudWatch
You can forward logs to Amazon CloudWatch and Amazon S3 using AWS authentication credentials.
8.1. Forward logs to Amazon CloudWatch Copy linkLink copied to clipboard!
To configure log forwarding to CloudWatch, create a ClusterLogForwarder custom resource (CR) with an output for CloudWatch and a pipeline that uses the output.
Prerequisites
- You have installed Red Hat OpenShift Logging Operator.
- You have administrator access to OpenShift Container Platform.
-
You have installed OpenShift CLI (
oc).
Procedure
Create a
SecretYAML file that uses theaws_access_key_idandaws_secret_access_keyfields to specify your base64-encoded AWS credentials. For example:apiVersion: v1 kind: Secret metadata: name: aws-secret namespace: <namespace_name> data: aws_access_key_id: QUtJQUlPU0ZPRE5ON0VYQU1QTEUK aws_secret_access_key: d0phbHJYVXRuRkVNSS9LN01ERU5HL2JQeFJmaUNZRVhBTVBMRUtFWQo=namespace-
Specify the namespace where your
ClusterLogForwarderCR is deployed. The secret must be in the same namespace as theClusterLogForwarderCR.
Create the secret. For example:
$ oc apply -f aws-secret.yamlCreate or edit a YAML file that defines the
ClusterLogForwarderCR object. In the file, specify the name of the secret. For example:apiVersion: observability.openshift.io/v1 kind: ClusterLogForwarder metadata: name: <log_forwarder_name> namespace: <log_forwarder_namespace> spec: serviceAccount: name: <service_account_name> outputs: - name: cw-output type: cloudwatch cloudwatch: authentication: type: awsAccessKey awsAccessKey: keyId: key: <key> secretName: cw-secret keySecret: key: <key> secretName: cw-secret groupName: <prefix-{.dynamic_value||"none"}> region: us-east-2 pipelines: - name: infra-logs inputRefs: - infrastructure - audit - application outputRefs: - cw-outputserviceAccount.name- Specify the name of your service account.
outputs.name- Specify a name for the output.
type-
Specify the
cloudwatchtype. secretName- Specify the name of the secret that has your AWS credentials.
groupName- Specify the strategy for grouping log streams. The value can be a combination of static and dynamic values consisting of field paths separated by "||". Encase dynamic values in single curly brackets "{}". Follow dynamic values with a static fallback value.
region- Specify the AWS region.
pipelines.name- Specify a name for the pipeline.
inputRefs-
Specify the names of inputs defined in the
input.namefield for this pipeline. You can also use the built-in valuesapplication,infrastructure,audit. output-
Specify the names of outputs defined in the
outputs.namefield for this pipeline.
Apply the CR object:
$ oc apply -f <file_name>.yaml
8.2. CloudWatch output format and field mappings Copy linkLink copied to clipboard!
When you forward logs to Amazon CloudWatch, the log collector transforms OpenShift log records into the CloudWatch Logs API format. Understanding this transformation helps you configure CloudWatch queries and dashboards effectively.
The log collector sends logs to CloudWatch by using the PutLogEvents API. Each log record becomes a CloudWatch LogEvent with two main components:
message- The entire OpenShift log record serialized as a JSON string. This field contains all log data including Kubernetes metadata, application messages, and timestamps.
timestamp-
The log event timestamp as UNIX milliseconds (milliseconds since January 1, 1970 00:00:00 UTC). The collector converts the log record’s
@timestampfield to this format.
The following table shows how OpenShift log record fields map to CloudWatch LogEvents:
| OpenShift field | CloudWatch field | Transformation |
|---|---|---|
| Complete log record |
| JSON-serialized string containing the entire log record |
|
|
| Converted to UNIX milliseconds |
| n/a |
|
Configured in |
| n/a |
|
Configured in |
{
"kubernetes": {
"namespace_name": "my-app",
"pod_name": "web-1",
"container_name": "nginx"
},
"message": "Error connecting to database",
"level": "error",
"@timestamp": "2024-01-15T10:30:00Z"
}
The log collector transforms this into a CloudWatch LogEvent:
{
"logEvents": [
{
"message": "{\"kubernetes\":{\"namespace_name\":\"my-app\",\"pod_name\":\"web-1\",\"container_name\":\"nginx\"},\"message\":\"Error connecting to database\",\"level\":\"error\",\"@timestamp\":\"2024-01-15T10:30:00Z\"}",
"timestamp": 1705318200000
}
],
"logGroupName": "/openshift/logging",
"logStreamName": "my-app"
}
The group_name and stream_name fields in your ClusterLogForwarder output configuration control how logs are organized in CloudWatch:
-
group_name: Determines the CloudWatch log group -
stream_name: Determines the CloudWatch log stream, supports template syntax such as{{ kubernetes.namespace_name }}
Both fields can use dynamic values from the log record. The collector evaluates these templates for each log event.