Chapter 7. Configure inputs
Inputs define which logs the collector gathers from your cluster. Configure inputs to select specific namespaces, containers, infrastructure components, or audit logs. By filtering at the input stage, you control what enters your logging pipeline and reduce downstream processing costs.
7.1. ClusterLogForwarder input types Copy linkLink copied to clipboard!
Input types define the source of log data for the collector to gather and forward. You can configure inputs to select specific log streams from different parts of your cluster.
applicationSelects logs from user-deployed application containers. This input type excludes logs from infrastructure namespaces. You can further customize which application logs to collect by specifying namespace and pod label selectors.
Use case: Developers need access to their application logs for debugging and troubleshooting. Use custom inputs with label selectors to route logs from specific applications to dedicated outputs that development teams can access.
The
applicationinput type can be configured with the following:-
selector: A label selector to collect logs only from pods with matching labels -
includes: A list of namespaces and container patterns to include -
excludes: A list of namespaces and container patterns to exclude -
tuning: Configuration for rate limiting per container and maximum message size
-
infrastructureSelects logs from infrastructure components and system services. This includes logs from two sources:
-
container: Container logs from workloads deployed in namespaces with thedefault,kube*, oropenshift*prefix -
node: Journal logs from the cluster nodes
If you do not specify sources, the collector gathers logs from both container and node sources.
Use case: Site reliability engineers need infrastructure logs for troubleshooting cluster issues, analyzing node performance, and monitoring Red Hat OpenShift Logging component health.
-
auditSelects audit logs that track security-relevant events in the cluster. Audit logs are sourced from multiple components:
-
kubeAPI: Kubernetes API server audit logs -
openshiftAPI: Red Hat OpenShift Logging API server audit logs -
auditd: Node audit daemon logs -
ovn: Open Virtual Network audit logs
If you do not specify sources, the collector gathers logs from all audit sources.
Use case: Security officers require audit logs for compliance, security investigations, and maintaining an immutable record of API activity. Audit logs are typically more voluminous and require careful consideration of storage costs and retention policies.
-
receiverCreates a network listener that receives logs from sources outside the cluster. This input type can accept logs from remote systems or applications that cannot use the standard collector deployment.
Supported receiver types include:
-
http: Receives logs over HTTP (currently supports Kubernetes audit logs only) -
syslog: Receives logs using the syslog protocol (currently supports infrastructure logs only)
Each receiver requires a port number between 1024 and 65535. The operator can request TLS certificates from the cluster certificate signing service or you can provide your own certificates.
-
7.2. Filter application logs by namespace or container Copy linkLink copied to clipboard!
You can include or exclude the application logs based on the namespace and container name by using the input selector.
Procedure
Add a configuration to include or exclude the namespace and container names in the
ClusterLogForwarderCR.apiVersion: observability.openshift.io/v1 kind: ClusterLogForwarder # ... spec: serviceAccount: name: <service_account_name> inputs: - name: mylogs application: includes: - namespace: "my-project" container: "my-container" excludes: - container: "other-container*" namespace: "other-namespace" type: application # ...includes.namespace- Specifies that the logs are only collected from these namespaces.
includes.container- Specifies that the logs are only collected from these containers.
excludes.namespace- Specifies the pattern of namespaces to ignore when collecting the logs.
excludes.containerSpecifies the set of containers to ignore when collecting the logs.
NoteThe
excludesfield takes precedence over theincludesfield.
Apply the
ClusterLogForwarderCR by running the following command:$ oc apply -f <filename>.yaml
7.3. Filter application logs by label Copy linkLink copied to clipboard!
You can include the application logs based on the label expressions or a matching label key and its values by using the input selector.
Procedure
Add a configuration for a filter to the
inputspec in theClusterLogForwarderCR.apiVersion: observability.openshift.io/v1 kind: ClusterLogForwarder # ... spec: serviceAccount: name: <service_account_name> inputs: - name: mylogs application: selector: matchExpressions: - key: env operator: In values: ["prod", "qa"] - key: zone operator: NotIn values: ["east", "west"] matchLabels: app: one name: app1 type: application # ...matchExpressions.key- Specifies the label key to match.
matchExpressions.operator-
Specifies the operator. Valid values include:
In,NotIn,Exists, andDoesNotExist. matchExpressions.values-
Specifies an array of string values. If the
operatorvalue is eitherExistsorDoesNotExist, the value array must be empty. matchLabels- Specifies an exact key and value mapping.
Apply the
ClusterLogForwarderCR by running the following command:$ oc apply -f <filename>.yaml
7.4. Filter audit and infrastructure logs by source Copy linkLink copied to clipboard!
You can define the list of audit and infrastructure sources to collect the logs by using the input selector.
Procedure
Add a configuration to define the
auditandinfrastructuresources in theClusterLogForwarderCR.apiVersion: observability.openshift.io/v1 kind: ClusterLogForwarder # ... spec: serviceAccount: name: <service_account_name> inputs: - name: mylogs1 type: infrastructure infrastructure: sources: - node - name: mylogs2 type: audit audit: sources: - kubeAPI - openshiftAPI - ovn # ...infrastructure.sourcesSpecifies the list of infrastructure sources to collect. The valid sources include:
-
node: Journal log from the node -
container: Logs from the workloads deployed in the namespaces
-
audit.sourcesSpecifies the list of audit sources to collect. The valid sources include:
-
kubeAPI: Logs from the Kubernetes API servers -
openshiftAPI: Logs from the OpenShift API servers -
auditd: Logs from a node auditd service -
ovn: Logs from an open virtual network service
-
Apply the
ClusterLogForwarderCR by running the following command:$ oc apply -f <filename>.yaml
7.5. Add audit log collection Copy linkLink copied to clipboard!
To add audit log collection to an existing pipeline, you must explicitly grant the collect-audit-logs permission to the collector service account. You must complete this step before updating your ClusterLogForwarder custom resource (CR). Failure to grant this permission before adding the audit log type will trigger a protective shutdown of the entire collector DaemonSet, stopping all log collection across the cluster.
If you add audit to the inputRefs list of the ClusterLogForwarder`CR without first granting the `collect-audit-logs permission to the ClusterRoleBinding, the Operator removes the entire collector DaemonSet. All log collection stops until you grant the missing permission.
Logging is not a Security Information and Event Monitoring (SIEM) system and does not provide secure storage for audit logs. Audit logs are not stored by default. You must configure a ClusterLogForwarder CR to forward audit logs to a storage solution. For more information about Logging capabilities and limitations, see "OpenShift Logging support".
Procedure
Grant the collector service account permission to collect audit logs:
$ oc adm policy add-cluster-role-to-user collect-audit-logs \ system:serviceaccount:<namespace>:<service_account_name>Replace
<namespace>with the namespace where yourClusterLogForwarderCR is deployed and<service_account_name>with the name you specified in theClusterLogForwarderCRspec.serviceAccount.namefield.Add
auditto theinputRefslist in yourClusterLogForwarderpipeline by editing the CR:$ oc edit clusterlogforwarder <clusterlogforwarder_name> -n <namespace>Replace
<clusterlogforwarder_name>with the name of yourClusterLogForwarderCR and<namespace>with the namespace where it is deployed.NoteIn Logging versions before 5.9, the
ClusterLogForwarderCR was a singleton namedinstancein theopenshift-loggingnamespace. Starting in Logging 5.9, you can create multipleClusterLogForwarderCRs with different names in different namespaces.Example configuration
spec: # ... other configuration outputs: - name: default-loki # ... output configuration pipelines: - name: my-pipeline inputRefs: - application - infrastructure - audit outputRefs: - default-lokiVerify that the
ClusterLogForwarderstill showsReadyby running the following command:$ oc get clusterlogforwarder <clusterlogforwarder_name> -n <namespace> \ -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}'Replace
<clusterlogforwarder_name>with the name of yourClusterLogForwarderCR and<namespace>with the namespace where it is deployed.The output should show
True.Verify that the
Authorizedcondition listsauditby running the following command:$ oc get clusterlogforwarder <clusterlogforwarder_name> -n <namespace> \ -o jsonpath='{.status.conditions[?(@.type=~"Authorized")].message}'Example output
permitted to collect log types: [application audit infrastructure]