Chapter 5. Avoiding downtime with rolling updates
Avoid downtime when changing themes, providers, or configurations in optimized images.
By default, the Red Hat build of Keycloak Operator will perform rolling updates on configuration changes without downtime, and recreate updates with downtime when the image name or tag changes.
This chapter describes how to minimize downtimes by configuring the Red Hat build of Keycloak Operator to perform rolling updates of Red Hat build of Keycloak automatically where possible, and how to override automatic detection for rolling updates.
Use it, for example, avoid downtimes when rolling out
- an update to a theme,
- provider or build time configuration in a custom or optimized image, or
- patch releases of Red Hat build of Keycloak.
5.1. Prerequisites
5.1.1. Verify the graceful shutdown
In order for rolling updates to work as expected, check if Red Hat build of Keycloak nodes perform a graceful shutdown when receiving a TERM signal.
Check the log file of the node shutting down for the following log entry. If it shows, the shutdown was graceful.
INFO [io.quarkus] (Shutdown thread) Keycloak stopped in ...sSee the {links_server_container_name} chapter for details for possible causes when it does not show.
5.1.2. Verify the readiness probe in the load balancer
Your load balancer should honor the Red Hat build of Keycloak readiness probe to stop forwarding traffic to nodes shutting down.
See Tracking instance status with health checks and Configuring a reverse proxy chapters for details.
5.2. Supported Update Strategies
The Operator supports the following update strategies:
- Rolling Updates
- Update the StatefulSet in a rolling fashion, avoiding a downtime when at least two replicas are running.
- Recreate Updates
- Scale down the StatefulSet before applying updates, causing temporary downtime.
5.3. Configuring the Update Strategy
Specify the update strategy within the spec section of the Keycloak CR YAML definition:
apiVersion: k8s.keycloak.org/v2beta1
kind: Keycloak
metadata:
name: example-kc
spec:
update:
strategy: RecreateOnImageChange|Auto|Explicit
revision: "abc" | Value | Downtime? | Description |
|---|---|---|
|
| On image name or tag change | Mimics Red Hat build of Keycloak 26.1 or older behavior. When the image field changes, the Operator scales down the StatefulSet before applying the new image. |
|
| On incompatible changes | The Red Hat build of Keycloak Operator detects if a rolling or recreate update is possible.
Red Hat build of Keycloak supports rolling updates for patch releases. It performs a rolling update if the Red Hat build of Keycloak version is the same or when upgrading to a newer patch version in the same |
|
|
Only the |
The Red Hat build of Keycloak Operator checks the |
5.3.1. Understanding Auto and Explicit Update Strategies
When using the Auto update strategy, the Red Hat build of Keycloak Operator automatically starts a Job to assess the feasibility of a rolling update. Read more about the process in the Checking if rolling updates are possible chapter. This process consumes cluster resources for the time of the check and introduces a slight delay before the StatefulSet update begins.
If the Keycloak CR configured a podTemplate as part of the unsupported configuration parameters, the Keycloak Operator will do its best to use those settings for the started Job. Still it might miss some settings due to the flexibility of the podTemplate feature and its unsupported nature.
As a consequence, the Operator might draw the wrong conclusions if a rolling update is possible from changes to the podTemplate or information pulled in from Secrets, ConfigMaps or Volumes in the podTemplate.
Therefore, if you are using the unsupported podTemplate, you may need to use one of the other update strategies.
The Explicit update strategy delegates the update decision to the user. The revision field acts as a user-controlled trigger. While the Red Hat build of Keycloak Operator does not interpret the revision value itself, any change to the Custom Resource (CR) while the revision remains unchanged will prompt a rolling update.
Exercise caution when using this with automatic Operator upgrades. The Operator Lifecycle Manager (OLM) may upgrade the Red Hat build of Keycloak Operator, and if the Explicit update strategy is in use, this could lead to unexpected behavior or deployment failures as the Operator would attempt a rolling update when this is actually not supported. If you are using the Explicit update strategy, thorough testing in a non-production environment is highly recommended before upgrading.
5.3.2. CR Statuses
The Keycloak CR status of RecreateUpdateUsed indicates the update strategy employed during the last update operation. The lastTransitionTime field indicates when the last update occurred. Use this information to observe actions and decisions taken by the Operator.
| Status | Description |
|---|---|
|
| The initial state. It means no update has taken place. |
|
| The Operator applied the rolling update strategy in the last update. |
|
|
The Operator applied the recreate update strategy in the last update. The |
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}