Reboot updated Compute nodes

Check which updated Compute nodes require a reboot. You can reboot the nodes any time after completing the minor update. For data plane development kit (DPDK) implementations, the old kernel and Open vSwitch (OVS) remain active until reboot. Migrate instances to ensure minimal downtime.

Before you begin

  • You have decided whether to migrate instances to another Compute node before you start the reboot. Note
    • If you have a Multi-RHEL environment, and you want to migrate virtual machines from a Compute node that is running RHEL 9.4 or 9.6 to a Compute node that is running RHEL 9.2, only cold migration is supported.
    • If you cannot migrate the instances, you can set the shutdown_timeout configuration option to control the state of the instances after the Compute node reboots. This option determines the number of seconds to wait for an instance to perform a clean shutdown. The default value is 60.

About this task

Note
If you enabled kernel live patching (kpatch) on a set of Compute nodes, you are not required to reboot those nodes. For more information about enabling kpatch, see Enabling kernel live patching on Compute nodes.

Procedure

  1. Confirm which updated nodes need a reboot: 
    $ oc logs jobs/reboot-os-<deployment_name>-<nodeSet_name>
    • Replace <deployment_name> with the name of the deployment that includes your Compute nodes.
    • Replace <nodeSet_name> with the names of the node sets that you need to check.

      The command shows the following output if a reboot is required: Reboot is required but was not started. Edpm_reboot_strategy is set to never or this is already deployed machine. Reboot has to be planned. To start reboot set edpm_reboot_strategy to force.

  2. Open a remote shell connection to the OpenStackClient pod: 
    $ oc rsh -n openstack openstackclient
  3. Retrieve a list of your Compute nodes to identify the hostname of the nodes that require a reboot: 
    $ openstack compute service list
  4. Disable the Compute service on the Compute node that you need to reboot: 
    $ openstack compute service set <hostname> nova-compute --disable
    • Replace <hostname> with the hostname of the Compute node on which you are disabling the service.
  5. If Red Hat Ceph Storage is also hosted on the Compute node, stop Ceph services but do not shut down hosts.
    • In Red Hat Ceph Storage 7 and 8, perform steps 1 to 14 in the procedure "Powering down and rebooting the cluster using Ceph Orchestrator" in the Red Hat Ceph Storage Administration Guide. Do not reboot the cluster as instructed in the Administration Guide. The cluster is rebooted later in this procedure.
    • In Red Hat Ceph Storage 9, perform steps 1 to 12 in the procedure "Powering down the Red Hat Ceph Storage cluster" in the Red Hat Ceph Storage Administration Guide. Do not perform steps 13-15, which shut down the OSD hosts, monitor hosts, and remaining standalone hosts. The Compute node is rebooted later in this procedure.
  6. List all instances on the Compute node: 
    $ openstack server list --host <hostname> --all-projects
  7. Optional: If you decide to migrate the instances to another Compute node, for example, if you plan to reboot nodes that include running workloads, run the following command: 
    $ openstack server migrate --live-migration --host <target_host>  --wait <instance_id>
    • Replace <instance_id> with your instance ID.
    • Replace <target_host> with the host that you are migrating the instance to.
      1. Wait until migration completes.
      2. Confirm that the migration was successful: 
        $ openstack server list --host <hostname> --all-projects
      3. Continue to migrate instances until none remain on the Compute node.
  8. Exit the OpenStackClient pod: 
    $ exit
  9. Create an OpenStackDataPlaneDeployment CR file to reboot the nodes: 
    apiVersion: dataplane.openstack.org/v1beta1
kind: OpenStackDataPlaneDeployment
metadata:
  name: openstack-edpm-ipam-reboot
  namespace: openstack
spec:
  nodeSets:
  - <nodeSet_name>
  servicesOverride:
  - reboot-os
  ansibleExtraVars:
    edpm_reboot_strategy: force
  ansibleLimit: <node_hostname>,...,<node_hostname>
    • spec.nodeSets lists the OpenStackDataPlaneNodeSet CRs that contain the nodes that you are rebooting.
    • spec.servicesOverride specifies the reboot-os as the only service to run.
    • spec.ansibleExtraVars reboots all the nodes in the node set at the same time.
    • Optional: spec.ansibleLimit lists the individual nodes in the node set to reboot. If not set, all the nodes in the node set are rebooted at the same time.
    • <node_hostname> specifies the hostname of the node to reboot.
  10. Save the OpenStackDataPlaneDeployment CR file.
  11. Apply the OpenStackDataPlaneDeployment CR to your environment: 
    $ oc apply -f <openstack_dataplane_deployment_CR_file> -n openstack

    where:

    <openstack_dataplane_deployment_CR_file>
    Specifies the name of the OpenStackDataPlaneDeployment CR file.
  12. Verify that the openstack-edpm-ipam-reboot deployment completed: 
    $ oc get openstackdataplanedeployment
NAME                                    STATUS   MESSAGE
openstack-edpm-deployment-ipam-reboot   True     Setup complete

    If the deployment fails, see Troubleshoot data plane creation and deployment.

  13. Re-enable the Compute node: 
    $ oc rsh openstackclient -n openstack
$ openstack compute service set <hostname>  nova-compute --enable
  14. If Red Hat Ceph Storage is also hosted on the Compute node, perform steps 1 to 7 in the procedure "Powering down and rebooting the cluster using Ceph Orchestrator" in the Red Hat Ceph Storage Administration Guide:
  15. Check that the Compute node is enabled: 
    $ openstack compute service list