Troubleshoot a stale cluster

Restore a cluster showing Stale status in OpenShift Cluster Manager to Ready by checking Telemetry endpoint connectivity and restarting the telemeter-client pod.

Before you begin

  • A Red Hat login
  • An OpenShift Container Platform cluster
  • You have access to the cluster as a user with the cluster-admin role.
  • You have installed the OpenShift CLI (oc).

About this task

If your cluster is reporting a Stale status, you can restore it to Ready and resume monitoring cluster health with the following steps.

Procedure

  1. Check that the OpenShift Container Platform cluster can connect to the Telemetry endpoints. If applicable, ensure that the cluster proxy configuration allows access to the Telemetry endpoints. The required Telemetry endpoints can be found in the Configuring your firewall for OpenShift Container Platform in the OpenShift Container Platform Installing documentation.
  2. Review the logs of the telemeter-client pod running in the openshift-monitoring namespace for any errors: 
    $ oc get pods -n openshift-monitoring -l app.kubernetes.io/name=telemeter-client
  3. Restart the telemeter-client pod in the openshift-monitoring namespace and the insights-operator pod in the openshift-insights namespace: 
    $ oc delete pod -n openshift-monitoring -l app.kubernetes.io/name=telemeter-client
$ oc delete pod -n openshift-insights -l app=insights-operator
  4. Wait a few minutes after restarting the pods, then locate your cluster in the Clusters list in OpenShift Cluster Manager and check that the Status column shows Ready. In a few minutes, you will also see information appear about the cluster in the OpenShift Cluster Manager Monitoring tab.
    Note

    If you can view the cluster in OpenShift Cluster Manager but the status is still Stale, occasionally this might be because the cluster is owned by an inactive user in your Red Hat organization. If this is the case, you can restore the cluster to Ready by transferring cluster ownership to an active user in your organization, as described in the remaining steps.
  5. To determine the cluster owner, check the cluster pull secret using the following command: 
    $ oc get secret pull-secret -n openshift-config -o jsonpath='{.data.\.dockerconfigjson}' | base64 -d | jq
    Note

    The jq JSON processor is needed to execute the above command and make the output more readable. You can also run the above command without using jq by removing | jq from the command, but the output is more difficult to read.
  6. Check that the user account in the secret is still active in the Red Hat Customer Portal.
  7. If the user is no longer active, transfer your cluster to another owner using the steps in Transferring cluster ownership. When the ownership transfer is complete, the cluster will show in OpenShift Cluster Manager as Ready.