After updating the OpenShift Container Platform cluster from 4.3 to 4.4, you must then upgrade cluster logging from 4.3 to 4.4.

Updating cluster logging

After updating the OpenShift Container Platform cluster, you can update cluster logging from 4.3 to 4.4 by updating the subscription for the Elasticsearch Operator and the Cluster Logging Operator.

Changes introduced by the new log forward feature modified the support for out_forward starting with the OpenShift Container Platform 4.3 release. You create a ConfigMap to configure out_forward. Any updates to the secure-forward.conf section of the Fluentd ConfigMap are removed.

If you use the out_forward plug-in, before updating, you can copy your current secure-forward.conf section from the Fluentd ConfigMap and use the copied data when you create the secure-forward ConfigMap.

Prerequisites
  • Update the cluster from 4.3 to 4.4.

  • Make sure the cluster logging status is healthy:

    • All Pods are ready.

    • Elasticsearch cluster is healthy.

  • Optionally, copy your current secure-forward.conf section from the Fluentd ConfigMap for use if you want to create the secure-forward ConfigMap. See the note above.

Procedure
  1. Update the Elasticsearch Operator:

    1. From the web console, click OperatorsInstalled Operators.

    2. Select the openshift-operators-redhat project.

    3. Click the Elasticsearch Operator.

    4. Click SubscriptionChannel.

    5. In the Change Subscription Update Channel window, select 4.4 and click Save.

    6. Wait for a few seconds, then click OperatorsInstalled Operators.

      The Elasticsearch Operator is shown as 4.4. For example:

      Elasticsearch Operator
      4.4.0-201909201915 provided
      by Red Hat, Inc
  2. Update the Cluster Logging Operator:

    1. From the web console, click OperatorsInstalled Operators.

    2. Select the openshift-logging Project.

    3. Click the Cluster Logging Operator.

    4. Click SubscriptionChannel.

    5. In the Change Subscription Update Channel window, select 4.4 and click Save.

    6. Wait for a few seconds, then click OperatorsInstalled Operators.

      The Cluster Logging Operator is shown as 4.4. For example:

      Cluster Logging
      4.4.0-201909201915 provided
      by Red Hat, Inc
  3. Check the logging components:

    1. Ensure that the Elasticsearch Pods are using a 4.4 image:

      $ oc get pod -o yaml -n openshift-logging --selector component=elasticsearch |grep 'image:'
      image: registry.redhat.io/openshift4/ose-logging-elasticsearch5@sha256:4e081ff048c3a56113bc672af5dfb8d29ea2ddca1fd79a3332a4446a461944f5
      image: registry.redhat.io/openshift4/ose-oauth-proxy@sha256:5fe478210770b21c1eb26c1570bcbda40bc5a79011580ff5ebd4c701a5b04eb2
      image: registry.redhat.io/openshift4/ose-logging-elasticsearch5@sha256:4e081ff048c3a56113bc672af5dfb8d29ea2ddca1fd79a3332a4446a461944f5
      image: registry.redhat.io/openshift4/ose-oauth-proxy@sha256:5fe478210770b21c1eb26c1570bcbda40bc5a79011580ff5ebd4c701a5b04eb2
      image: registry.redhat.io/openshift4/ose-logging-elasticsearch5@sha256:4e081ff048c3a56113bc672af5dfb8d29ea2ddca1fd79a3332a4446a461944f5
      image: registry.redhat.io/openshift4/ose-oauth-proxy@sha256:5fe478210770b21c1eb26c1570bcbda40bc5a79011580ff5ebd4c701a5b04eb2
      image: registry.redhat.io/openshift4/ose-logging-elasticsearch5@sha256:4e081ff048c3a56113bc672af5dfb8d29ea2ddca1fd79a3332a4446a461944f5
      image: registry.redhat.io/openshift4/ose-oauth-proxy@sha256:5fe478210770b21c1eb26c1570bcbda40bc5a79011580ff5ebd4c701a5b04eb2
      image: registry.redhat.io/openshift4/ose-logging-elasticsearch5@sha256:4e081ff048c3a56113bc672af5dfb8d29ea2ddca1fd79a3332a4446a461944f5
      image: registry.redhat.io/openshift4/ose-oauth-proxy@sha256:5fe478210770b21c1eb26c1570bcbda40bc5a79011580ff5ebd4c701a5b04eb2
      image: registry.redhat.io/openshift4/ose-logging-elasticsearch5@sha256:4e081ff048c3a56113bc672af5dfb8d29ea2ddca1fd79a3332a4446a461944f5
      image: registry.redhat.io/openshift4/ose-oauth-proxy@sha256:5fe478210770b21c1eb26c1570bcbda40bc5a79011580ff5ebd4c701a5b04eb2
    2. Ensure that all Elasticsearch Pods are in the Ready status:

      $ oc get pod -n openshift-logging --selector component=elasticsearch
      
      NAME                                            READY   STATUS    RESTARTS   AGE
      elasticsearch-cdm-1pbrl44l-1-55b7546f4c-mshhk   2/2     Running   0          31m
      elasticsearch-cdm-1pbrl44l-2-5c6d87589f-gx5hk   2/2     Running   0          30m
      elasticsearch-cdm-1pbrl44l-3-88df5d47-m45jc     2/2     Running   0          29m
    3. Ensure that the Elasticsearch cluster is healthy:

      oc exec -n openshift-logging -c elasticsearch elasticsearch-cdm-1pbrl44l-1-55b7546f4c-mshhk -- es_cluster_health
      
      {
        "cluster_name" : "elasticsearch",
        "status" : "green",
      
      ....
    4. Ensure that the logging collector Pods are using a 4.4 image:

      $ oc get pod -n openshift-logging --selector logging-infra=fluentd -o yaml |grep 'image:'
      image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483
      image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483
      image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483
      image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483
      image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483
      image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483
      image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483
      image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483
      image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483
      image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483
      image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483
      image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483
    5. Ensure that the Kibana Pods are using a 4.4 image:

      $ oc get pod -n openshift-logging --selector logging-infra=kibana -o yaml |grep 'image:'
      image: registry.redhat.io/openshift4/ose-logging-kibana5@sha256:3d657e3b90fae604a8351b1923250f93c04529b36e6ada0aba7c0a038ffef56e
      image: registry.redhat.io/openshift4/ose-oauth-proxy@sha256:5fe478210770b21c1eb26c1570bcbda40bc5a79011580ff5ebd4c701a5b04eb2
      image: registry.redhat.io/openshift4/ose-logging-kibana5@sha256:3d657e3b90fae604a8351b1923250f93c04529b36e6ada0aba7c0a038ffef56e
      image: registry.redhat.io/openshift4/ose-oauth-proxy@sha256:5fe478210770b21c1eb26c1570bcbda40bc5a79011580ff5ebd4c701a5b04eb2
    6. Ensure that the Curator CronJob is using a 4.4 image:

      $ oc get CronJob curator -n openshift-logging -o yaml |grep 'image:'
      image: registry.redhat.io/openshift4/ose-logging-curator5@sha256:330c3499e790d0e184414125a4843cd48849c601eb9f19ff82f30794c858b0bc