Upgrading ROSA with HCP clusters | Upgrading | Red Hat OpenShift Service on AWS

Upgrade options for ROSA with HCP clusters
Life cycle policies and planning
Upgrading the hosted control plane with the ROSA CLI
Upgrading machine pools with the ROSA CLI
Upgrading the whole cluster with the ROSA CLI
- Upgrading the hosted control plane
- Upgrading machine pools
Upgrading with the ROSA CLI
Upgrading with the OpenShift Cluster Manager console
Deleting an upgrade with the OpenShift Cluster Manager console

Upgrade options for ROSA with HCP clusters

In OpenShift, upgrading means provisioning a new component with updated software and using it to replace an existing component that has outdated software.

You can control the impact of upgrades to your workload by controlling which parts of the cluster are upgraded, for example:

Upgrade only the hosted control plane: This initiates upgrade of the hosted control plane. It does not impact your worker nodes.
Upgrade nodes in a machine pool: This initiates a rolling replacement of nodes in the specified machine pool, and temporarily impacts the worker nodes on that machine pool. You can also upgrade multiple machine pools concurrently.

You cannot upgrade the hosted control plane at the same time as any machine pool upgrade.

To maintain compatibility between nodes in the cluster, nodes in machine pools cannot use a newer version than the hosted control plane. This means that the hosted control plane should always be upgraded to a given version before any machine pools are upgraded to the same version.

You can further control the time required for a machine pool upgrade, and the impact of an upgrade to your workload, by editing the --max-surge and --max-unavailable values for each machine pool. These options control the number of nodes that can be upgraded simultaneously on a machine pool, and whether an upgrade provisions excess nodes or makes some existing nodes unavailable or both, for example:

To prioritize high workload availability, you can provision excess nodes instead of making existing nodes unavailable by setting a higher value for --max-surge and setting --max-unavailable to 0.
To prioritize lower infrastructure costs, you can make some existing nodes unavailable and avoid provisioning excess nodes by setting a higher value for --max-unavailable and setting --max-surge to 0.
To prioritize upgrade speed by upgrading multiple nodes simultaneously, you can provision excess nodes and allow some existing nodes to be made unavailable by configuring moderate values for both --max-surge and --max-unavailable.

For more information about these parameters and their usage, see the ROSA CLI reference for rosa edit machinepool.

edit

Life cycle policies and planning

To plan an upgrade, review the Red Hat OpenShift Service on AWS update life cycle.

The life cycle page includes release definitions, support and upgrade requirements, installation policy information and life cycle dates.

Upgrades are manually initiated or automatically scheduled. Red Hat Site Reliability Engineers (SREs) monitor upgrade progress and remedy any issues encountered.

If your control plane is not currently multi-architecture enabled, the upgrade process will first migrate the cluster to a multi-architecture image and then apply the version upgrade. Multi-architecture clusters are capable of running both x86-based and Arm-based workloads. Clusters created after 25 July, 2024 are multi-architecture enabled by default.

edit

Upgrading the hosted control plane with the ROSA CLI

You can manually upgrade the hosted control plane of a ROSA with HCP cluster by using the ROSA CLI. This method schedules the control plane for an upgrade if a more recent version is available, either immediately, or at a specified future time.

Your control plane only supports machine pools within two minor Y-stream versions. For example, a ROSA with HCP cluster with a control plane using version 4.15.z supports machine pools with version 4.13.z and 4.14.z, but the control plane does not support machine pools using version 4.12.z.

Prerequisites

You have installed and configured the latest version of the ROSA CLI.
No machine pool upgrades are in progress or scheduled to take place at the same time as the hosted control plane upgrade.

Procedure

Verify the current version of your cluster by running the following command:
```
$ rosa describe cluster --cluster=<cluster_name_or_id> (1)
```
1 Replace <cluster_name_or_id> with the cluster name or the cluster ID.
List the versions that you can upgrade your control plane to by running the following command:
```
$ rosa list upgrade --cluster=<cluster_name_or_id>
```
The command returns a list of available updates, including the recommended version.
Example output
```
VERSION  NOTES
4.14.8   recommended
4.14.7
4.14.6
```
Upgrade the cluster’s hosted control plane by running the following command:
```
$ rosa upgrade cluster -c <cluster_name_or_id> --control-plane [--schedule-date=<yyyy-mm-dd> --schedule-time=<HH:mm>] --version <version_number>
```
- To schedule an immediate upgrade to the specified version, run the following command:
  $ rosa upgrade cluster -c <cluster_name_or_id> --control-plane --version <version_number>
  Your hosted control plane is scheduled for an immediate upgrade.
- To schedule an upgrade to the specified version at a future date, run the following command:
  $ rosa upgrade cluster -c <cluster_name_or_id> --control-plane --schedule-date=<yyyy-mm-dd> --schedule-time=<HH:mm> --version=<version_number>
  Your hosted control plane is scheduled for an upgrade at the specified time in Coordinated Universal Time (UTC).

Troubleshooting

Sometimes a scheduled upgrade does not initiate. See Upgrade maintenance canceled for more information.

edit

Upgrading machine pools with the ROSA CLI

You can manually upgrade one or more machine pools in a ROSA with HCP cluster by using the ROSA CLI. This method schedules the specified machine pool for an upgrade if a more recent version is available, either immediately, or at a specified future time.

Prerequisites

You have installed and configured the latest version of the ROSA CLI.
No upgrades for the hosted control plane are in progress on the cluster, or scheduled to occur at the same time as the machine pool upgrade.

Machine pool configurations such as node drain timeout, max-unavailable, and max-surge can affect the timing and success of upgrades.

Procedure

Verify the current version of your cluster by running the following command:
```
$ rosa describe cluster --cluster=<cluster_name_or_id> (1)
```
1 Replace <cluster_name_or_id> with the cluster name or the cluster ID.
Example output
```
OpenShift Version:     4.14.0
```
List the versions that you can upgrade your machine pools to by running the following command:
```
$ rosa list upgrade --cluster <cluster-name> --machinepool <machinepool_name>
```
The command returns a list of available updates, including the recommended version.
Example output
```
VERSION  NOTES
4.14.5   recommended
4.14.4
4.14.3
```
Do not upgrade your machine pool to a version higher than your control plane. If you want to move to a higher version, upgrade the control plane to that version first.
Verify the upgrade behavior of the machine pools you intend to upgrade by running the following command:
```
$ rosa describe machinepool --cluster=<cluster_name_or_id> <machinepool_name>
```
Example output
```
Replicas: 5
Node drain grace period:   30 minutes

Management upgrade:
- Type: Replace
- Max surge: 20%
- Max unavailable: 20%
```
In the example, these settings allow the machine pool to provision one excess node (max-surge of 20% of replicas) and to have up to one node unavailable (max-unavailable of 20% of replicas) during an upgrade. This machine pool can therefore upgrade two nodes at a time, by provisioning one new node in excess of the replica count, and by making one node unavailable and replacing it. Node upgrades may be delayed by up to 30 minutes (node-drain-grace-period of 30 minutes) if necessary to protect workloads that have a pod disruption budget.
Upgrade a machine pool by running the following command:
```
$ rosa upgrade machinepool -c <cluster_name> <machinepool_name> [--schedule-date=<yyyy-mm-dd> --schedule-time=<HH:mm>] --version <version_number>
```
You can upgrade multiple machine pools concurrently by running this command for each machine pool you want to upgrade.
- To schedule the immediate upgrade of a machine pool, run the following command:
  $ rosa upgrade machinepool -c <cluster_name> <machinepool_name> --version <version_number>
  The machine pool is scheduled for immediate upgrade, which initiates a rolling replacement of all nodes in the specified machine pool.
- To schedule an upgrade to start at a future time, run the following command:
  $ rosa upgrade machinepool -c <cluster_name> <machinepool_name> --schedule-date=<yyyy-mm-dd> --schedule-time=<HH:mm> --version <version_number>
  The machine pool is scheduled to begin an upgrade at the specified time and date in Coordinated Universal Time (UTC). This will initiate a rolling replacement of all nodes in the specified machine pool, beginning at the specified time.

Upgrading the whole cluster with the ROSA CLI

Upgrading the entire cluster involves upgrading both the hosted control plane and nodes in the machine pools. However, these components cannot be upgraded at the same time. They must be upgraded in sequence. This can be done in any order. However, to maintain compatibility between nodes in the cluster, nodes in machine pools cannot use a newer version than the hosted control plane. Therefore, if both the hosted control plane and the nodes in your machine pools require upgrade to the same OpenShift version, you must upgrade the hosted control plane first, followed by the machine pools.

Prerequisites

You have installed and configured the latest version of the ROSA CLI.
No other upgrades are in progress or scheduled to take place at the same time as this upgrade.

Upgrading the hosted control plane

When you need to upgrade the whole cluster, upgrade the hosted control plane first.

Prerequisites

You have installed and configured the latest version of the ROSA CLI.
No machine pool upgrades are in progress or scheduled to take place at the same time as the hosted control plane upgrade.

Procedure

Verify the current version of your cluster by running the following command:
```
$ rosa describe cluster --cluster=<cluster_name_or_id> (1)
```
1 Replace <cluster_name_or_id> with the cluster name or the cluster ID.
List the versions that you can upgrade your control plane to by running the following command:
```
$ rosa list upgrade --cluster=<cluster_name_or_id>
```
The command returns a list of available updates, including the recommended version.
Example output
```
VERSION  NOTES
4.14.8   recommended
4.14.7
4.14.6
```
Upgrade the cluster’s hosted control plane by running the following command:
```
$ rosa upgrade cluster -c <cluster_name_or_id> --control-plane [--schedule-date=<yyyy-mm-dd> --schedule-time=<HH:mm>] --version <version_number>
```
- To schedule an immediate upgrade to the specified version, run the following command:
  $ rosa upgrade cluster -c <cluster_name_or_id> --control-plane --version <version_number>
  Your hosted control plane is scheduled for an immediate upgrade.
- To schedule an upgrade to the specified version at a future date, run the following command:
  $ rosa upgrade cluster -c <cluster_name_or_id> --control-plane --schedule-date=<yyyy-mm-dd> --schedule-time=<HH:mm> --version=<version_number>
  Your hosted control plane is scheduled for an upgrade at the specified time in Coordinated Universal Time (UTC).

Upgrading machine pools

When your hosted control plane upgrade is complete, you can upgrade one or more machine pools.

Machine pool configurations such as node drain timeout, max-unavailable, and max-surge can affect the timing and success of upgrades.

Procedure

Verify the current version of your cluster by running the following command:
```
$ rosa describe cluster --cluster=<cluster_name_or_id> (1)
```
1 Replace <cluster_name_or_id> with the cluster name or the cluster ID.
Example output
```
OpenShift Version:     4.14.8
```
List the versions that you can upgrade your machine pools to by running the following command:
```
$ rosa list upgrade --cluster <cluster-name> --machinepool <machinepool_name>
```
The command returns a list of available updates, including the recommended version.
Example output
```
VERSION  NOTES
4.14.5   recommended
4.14.4
4.14.3
```
Do not upgrade your machine pool to a version higher than your control plane. If you want to move to a higher version, upgrade the control plane to that version first.
Verify the upgrade behavior of the machine pools you intend to upgrade by running the following command:
```
$ rosa describe machinepool --cluster=<cluster_name_or_id> <machinepool_name>
```
Example output
```
Replicas: 5
Node drain grace period:   30 minutes

Management upgrade:
- Type: Replace
- Max surge: 20%
- Max unavailable: 20%
```
In the example, these settings allow the machine pool to provision one excess node (max-surge of 20% of replicas) and to have up to one node unavailable (max-unavailable of 20% of replicas) during an upgrade. This machine pool can therefore upgrade two nodes at a time, by provisioning one new node in excess of the replica count, and by making one node unavailable and replacing it. Node upgrades may be delayed by up to 30 minutes (node-drain-grace-period of 30 minutes) if necessary to protect workloads that have a pod disruption budget.
Upgrade a machine pool by running the following command:
```
$ rosa upgrade machinepool -c <cluster_name> <machinepool_name> [--schedule-date=<yyyy-mm-dd> --schedule-time=<HH:mm>] --version <version_number>
```
You can upgrade multiple machine pools concurrently by running this command for each machine pool you want to upgrade.
- To schedule the immediate upgrade of a machine pool, run the following command:
  $ rosa upgrade machinepool -c <cluster_name> <machinepool_name> --version <version_number>
  The machine pool is scheduled for immediate upgrade, which initiates a rolling replacement of all nodes in the specified machine pool.
- To schedule an upgrade to start at a future time, run the following command:
  $ rosa upgrade machinepool -c <cluster_name> <machinepool_name> --schedule-date=<yyyy-mm-dd> --schedule-time=<HH:mm> --version <version_number>
  The machine pool is scheduled to begin an upgrade at the specified time and date in Coordinated Universal Time (UTC). This will initiate a rolling replacement of all nodes in the specified machine pool, beginning at the specified time.

edit

Upgrading with the ROSA CLI

You can manually upgrade a ROSA with HCP cluster by using the ROSA CLI. This method schedules the cluster for an immediate upgrade if a more recent version is available.

Prerequisites

You have installed and configured the latest version of the ROSA CLI.

Procedure

Verify the current version of your cluster by running the following command:
```
$ rosa describe cluster --cluster=<cluster_name_or_id> (1)
```
1 Replace <cluster_name_or_id> with the cluster name or the cluster ID.
List the versions that you can upgrade your control plane and machine pools to by running the following commands:
1. For the control plane versions, run the following command:
  $ rosa list upgrade --cluster=<cluster_name|cluster_id>
  The command returns a list of available updates, including the recommended version.
  Example output
  VERSION NOTES 4.14.8 recommended 4.14.7 4.14.6
2. For the machine pool versions, run the following command:
  $ rosa list upgrade --cluster <cluster-name> --machinepool <machinepool_name>
  The command returns a list of available updates, including the recommended version.
  Example output
  VERSION NOTES 4.14.5 recommended 4.14.4 4.14.3
  The latest available update for machine pools is limited to the current current version of the control plane. Ensure your control plane is up to date first.
Upgrade your cluster with one of the following options:
- Upgrade the cluster’s hosted control plane by running the following command:
  $ rosa upgrade cluster -c <cluster_name> --control-plane [--schedule-date=XX --schedule-time=XX] [--version <version_number>]
  Your hosted control plane is now scheduled for an upgrade.
- Upgrade a specific machine pool on your cluster by running the following command:
  $ rosa upgrade machinepool -c <cluster_name> <machinepool_name> [--schedule-date=XX --schedule-time=XX] [--version <version_number>]
  Your machine pool is now scheduled for an upgrade.

edit

Upgrading with the OpenShift Cluster Manager console

You can schedule upgrades for a ROSA cluster manually either one time or on a recurring schedule by using OpenShift Cluster Manager console.

Procedure

Log in to OpenShift Cluster Manager.
Select a cluster to upgrade.
Click the Settings tab.
In the Update strategy pane, select which type of update you want:
- For individual updates, you can request the upgrade either immediately (to start within an hour) or at a future time.
- For recurring updates, select a recurring date and time to start the upgrade automatically to the latest x.y.Z (z-stream) version available.
  
  Recurring updates are applicable only for z-stream updates. Minor version or y-stream updates need to be done manually. You will be notified when a new y-stream update is available.
In the Update strategy pane, click Save to apply your update strategy.
In the Update status pane, review the Update available information and click Update.

The Update button is enabled only when an upgrade is available.
The Update cluster dialog opens. Recommended cluster upgrades appear in the Select version pane. Select the version you want to upgrade your cluster to, and click Next.
Optional: For ROSA clusters that use AWS Security Token Service (STS), the account-level and cluster-specific Operator roles might need to be updated, depending on the selected target version.
1. In the ROSA CLI, run the rosa list account-roles command to list and verify that the account roles are compatible with the target minor version chosen for the upgrade. If the roles are not compatible, run the rosa upgrade account-roles command to upgrade the account roles to the latest OpenShift version.
2. In the ROSA CLI, run the rosa list operator-roles command to list and verify that Operator roles associated with the cluster are compatible with the target minor version chosen for the upgrade. If not, run the rosa upgrade operators-roles command to upgrade the cluster’s Operator roles to the latest OpenShift version.
3. If you select an update version that requires approval, provide an administrator’s acknowledgment by typing Acknowledge into the field provided, and click Next.
In the Schedule update dialog, schedule your cluster upgrade.
- To upgrade within an hour, select Update now and click Next.
- To upgrade at a later time, select Schedule a different time and set a time and date for your upgrade. Click Next to proceed to the confirmation dialog.
After reviewing the version and schedule summary, select Confirm update.
Click Close to exit out of the Update cluster dialog.

The cluster is scheduled for an upgrade to the target version. This action can take up to an hour, depending on the selected upgrade schedule and your workload configuration, such as pod disruption budgets.

The status is displayed in the Update status pane.

Troubleshooting

Sometimes a scheduled upgrade does not trigger. See Upgrade maintenance cancelled for more information.

edit

Deleting an upgrade with the OpenShift Cluster Manager console

You can use the OpenShift Cluster Manager console to delete a scheduled upgrade.

Procedure

Log in to OpenShift Cluster Manager.
Select the cluster with the scheduled upgrade.
Click the Settings tab.
In the Update status pane, click Cancel this update.
Review the update details in the Cancel update dialog and click Cancel this update.

You will receive an email notification confirming that the scheduled upgrade has been canceled.