This topic reviews how to check cluster certificate expiration dates, back up the certificates, and redeploy them using Ansible playbooks. This method fixes common certificate errors. Possible use cases include:

  • The installer detected the wrong host names and the issue was identified too late.

  • The certificates are expired and you need to update them.

  • You have a new CA and would like to create certificates using it instead

Checking Certificate Expirations

You can use the installer to warn you about any certificates expiring within a configurable window of days and notify you about any certificates that have already expired. Certificate expiry playbooks use the Ansible role openshift_certificate_expiry.

Certificates examined by the role include:

  • Master and node service certificates

  • Router and registry service certificates from etcd secrets

  • Master, node, router, registry, and kubeconfig files for cluster-admin users

  • etcd certificates (including embedded)

Role Variables

The openshift_certificate_expiry role uses the following variables:

Table 1. Core Variables
Variable Name Default Value Description



Base OpenShift Container Platform configuration directory.



Flag certificates that will expire in this many days from now.



Include healthy (non-expired and non-warning) certificates in results.

Table 2. Optional Variables
Variable Name Default Value Description



Generate an HTML report of the expiry check results.



The full path for saving the HTML report.



Save expiry check results as a JSON file.



The full path for saving the JSON report.

Running Certificate Expiration Playbooks

The OpenShift Container Platform installer provides a set of example certificate expiration playbooks, using different sets of configuration for the openshift_certificate_expiry role.

Just like the redeploying certificates playbook, these playbooks must be used with an inventory that is representative of the cluster. For best results, run ansible-playbook with the -v option.

Using the easy-mode.yaml example playbook, you can try the role out before tweaking it to your specifications as needed. This playbook:

  • Produces JSON and stylized HTML reports in /tmp/.

  • Sets the warning window very large, so you will almost always get results back.

  • Includes all certificates (healthy or not) in the results.

easy-mode.yaml Playbook
- name: Check cert expirys
  hosts: nodes:masters:etcd
  become: yes
  gather_facts: no
    openshift_certificate_expiry_warning_days: 1500
    openshift_certificate_expiry_save_json_results: yes
    openshift_certificate_expiry_generate_html_report: yes
    openshift_certificate_expiry_show_all: yes
    - role: openshift_certificate_expiry

To run the easy-mode.yaml playbook:

$ ansible-playbook -v -i <inventory_file> \

Other Example Playbooks

The other example playbooks are also available to run directly out of the /usr/share/ansible/openshift-ansible/playbooks/certificate_expiry/ directory.

Table 3. Other Example Playbooks
File Name Usage


Produces the default behavior of the openshift_certificate_expiry role.


Generates HTML and JSON artifacts in their default paths.


Changes the expiration warning window to 1500 days.


Changes the expiration warning window to 1500 days and saves the results as a JSON file.

To run any of these example playbooks:

$ ansible-playbook -v -i <inventory_file> \

Output Formats

As noted above, there are two ways to format your check report. In JSON format for machine parsing, or as a stylized HTML page for easy skimming.

HTML Report

An example of an HTML report is provided with the installer. You can open the following file in your browser to view it:


JSON Report

There are two top-level keys in the saved JSON results: data and summary.

The data key is a hash where the keys are the names of each host examined and the values are the check results for the certificates identified on each respective host.

The summary key is a hash that summarizes the total number of certificates:

  • examined on the entire cluster

  • that are OK

  • expiring within the configured warning window

  • already expired

For an example of the full JSON report, see /usr/share/ansible/openshift-ansible/roles/openshift_certificate_expiry/examples/cert-expiry-report.json.

The summary from the JSON data can be easily checked for warnings or expirations using a variety of command-line tools. For example, using grep you can look for the word summary and print out the two lines after the match (-A2):

$ grep -A2 summary /tmp/cert-expiry-report.json
    "summary": {
        "warning": 16,
        "expired": 0

If available, the jq tool can also be used to pick out specific values. The first two examples below show how to select just one value, either warning or expired. The third example shows how to select both values at once:

$ jq '.summary.warning' /tmp/cert-expiry-report.json

$ jq '.summary.expired' /tmp/cert-expiry-report.json

$ jq '.summary.warning,.summary.expired' /tmp/cert-expiry-report.json

Running the Certificate Redeploy Playbook

Running the certificate redeploy playbook will redeploy OpenShift Container Platform certificates that exist on systems (master, node, etcd):

$ ansible-playbook -i <inventory> playbooks/byo/openshift-cluster/redeploy-certificates.yml

This playbook must be run with an inventory that is representative of the cluster. The inventory must specify or override all host names and IP addresses set via openshift_hostname, openshift_public_hostname, openshift_ip, openshift_public_ip, openshift_master_cluster_hostname, or openshift_master_cluster_public_hostname such that they match the current cluster configuration.

By default, the redeploy playbook does not redeploy the OpenShift Container Platform CA. New certificates are created using the original OpenShift Container Platform CA.

To redeploy all certificates including the OpenShift Container Platform CA, specify openshift_certificates_redeploy_ca=true.

For example:

$ ansible-playbook -i <inventory> playbooks/byo/openshift-cluster/redeploy-certificates.yml \
--extra-vars "openshift_certificates_redeploy_ca=true"

This also adds a custom CA certificate:

# Configure custom ca certificate
# NOTE: CA certificate will not be replaced with existing clusters.
# This option may only be specified when creating a new cluster or
# when redeploying cluster certificates with the redeploy-certificates
# playbook.
#openshift_master_ca_certificate={'certfile': '/path/to/ca.crt', 'keyfile': '/path/to/ca.key'}

All pods using service accounts to communicate with the OpenShift Container Platform API must be redeployed when the OpenShift Container Platform CA is replaced so the certificate redeploy playbook will serially evacuate all nodes in the cluster when this variable is set.

Manually Update Certificates for the Registry and Router

When nodes are evacuated, routers are restarted and outages to data plans are possible. This results in the routers not being able to reach the masters because they have old certificates. To manually update certificates:

  1. Add registry certificates to a secret named registry-certificates. To update the certificate, create a new certificate, then update it within the registry-certificates secret:

    REGISTRY_IP=`oc get service docker-registry -o jsonpath='{.spec.clusterIP}'`
    REGISTRY_HOSTNAME=`oc get route/docker-registry -o jsonpath='{.spec.host}'`
    $ oc adm ca create-server-cert \
      --signer-cert=/etc/origin/master/ca.crt \
      --signer-key=/etc/origin/master/ca.key \
      --hostnames=$REGISTRY_IP,docker-registry.default.svc.cluster.local,$REGISTRY_HOSTNAME \
      --cert=/etc/origin/master/registry.crt \
      --key=/etc/origin/master/registry.key \
    $ oc secret new registry-certificates \
      /etc/origin/master/registry.crt \
      /etc/origin/master/registry.key \
      -o json | oc replace -f -
  2. Redeploy the registry:

    $ oc deploy dc/docker-registry --latest
  3. The router is secured using a service serving certificate secret, which was automatically created after adding an annotation to the router service. That service serving certificate can be triggered to be recreated by deleting the service, then clearing or re-adding annotations to the router service:

$ oc delete secret router-certs

$ oc annotate service router \
  service.alpha.openshift.io/serving-cert-secret-name- \

$ oc annotate service router \

$ oc deploy dc/router --latest