OpenShift docs are moving and will soon only be available at docs.redhat.com, the home of all Red Hat product documentation. Explore the new docs experience today.
  1. Documentation
    2. history bug_report picture_as_pdf
×

OpenShift Virtualization release notes

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright’s message.

Providing documentation feedback

To report an error or to improve our documentation, log in to your Red Hat Jira account and submit a Jira issue.

About Red Hat OpenShift Virtualization

Red Hat OpenShift Virtualization enables you to bring traditional virtual machines (VMs) into OpenShift Container Platform where they run alongside containers, and are managed as native Kubernetes objects.

OpenShift Virtualization is represented by the OpenShift Virtualization icon.

You can use OpenShift Virtualization with either the OVN-Kubernetes or the OpenShiftSDN default Container Network Interface (CNI) network provider.

Learn more about what you can do with OpenShift Virtualization.

Learn more about OpenShift Virtualization architecture and deployments.

Prepare your cluster for OpenShift Virtualization.

OpenShift Virtualization supported cluster version

OpenShift Virtualization 4.14 is supported for use on OpenShift Container Platform 4.14 clusters. To use the latest z-stream release of OpenShift Virtualization, you must first upgrade to the latest version of OpenShift Container Platform.

Supported guest operating systems

To view the supported guest operating systems for OpenShift Virtualization, see Certified Guest Operating Systems in Red Hat OpenStack Platform, Red Hat Virtualization, OpenShift Virtualization and Red Hat Enterprise Linux with KVM.

New and changed features

  • OpenShift Virtualization is certified in Microsoft’s Windows Server Virtualization Validation Program (SVVP) to run Windows Server workloads.

    The SVVP Certification applies to:

    • Red Hat Enterprise Linux CoreOS workers. In the Microsoft SVVP Catalog, they are named Red Hat OpenShift Container Platform 4 on RHEL CoreOS 9.

    • Intel and AMD CPUs.

  • Using the NVIDIA GPU Operator to provision worker nodes for GPU-enabled VMs was previously Technology Preview and is now generally available. For more information, see Configuring the NVIDIA GPU Operator.

  • You can add a static authorized SSH key to a project by using the web console. The key is then added to all VMs that you create in the project.

  • OpenShift Virtualization now supports persisting the virtual Trusted Platform Module (vTPM) device state by using Persistent Volume Claims (PVCs) for VMs. You must specify the storage class to be used by the PVC by setting the vmStateStorageClass attribute in the HyperConverged custom resource (CR).

  • The access mode and volume mode fields in storage profiles are populated automatically with their optimal values for the following additional Containerized Storage Interface provisioners:

    • Dell PowerFlex

    • Dell PowerMax

    • Dell PowerScale

    • Dell Unity

    • Dell PowerStore

    • Hitachi Virtual Storage Platform

    • IBM Fusion Hyper-Converged Infrastructure

    • IBM Fusion HCI with Fusion Data Foundation or Fusion Global Data Platform

    • IBM Fusion Software-Defined Storage

    • IBM FlashSystems

    • Hewlett Packard Enterprise 3PAR

    • Hewlett Packard Enterprise Nimble

    • Hewlett Packard Enterprise Alletra

    • Hewlett Packard Enterprise Primera

  • Garbage collection for data volumes is disabled by default.

  • You can add a static authorized SSH key to a project by using the web console. The key is then added to all VMs that you create in the project.

Quick starts

  • Quick start tours are available for several OpenShift Virtualization features. To view the tours, click the Help icon ? in the menu bar on the header of the OpenShift Virtualization console and then select Quick Starts. You can filter the available tours by entering the virtualization keyword in the Filter field.

Networking

Web console

  • Cluster administrators can now enable automatic subscription for Red Hat Enterprise Linux (RHEL) virtual machines in the OpenShift Virtualization web console.

  • You can now force stop an unresponsive VM from the action menu. To force stop a VM, select Stop and then Force stop from the action menu.

  • The DataSources and the Bootable volumes pages have been merged into the Bootable volumes page so that you can manage these similar resources in a single location.

  • Cluster administrators can enable or disable Technology Preview features on the Settings tab on the Virtualization → Overview page.

Deprecated and removed features

Deprecated features

Deprecated features are included in the current release and supported. However, they will be removed in a future release and are not recommended for new deployments.

  • The tekton-tasks-operator is deprecated and Tekton tasks and example pipelines are now deployed by the ssp-operator.

  • The copy-template, modify-vm-template, and create-vm-from-template tasks are deprecated.

  • Support for Windows Server 2012 R2 templates is deprecated.

Removed features

Removed features are not supported in the current release.

  • Support for the legacy HPP custom resource, and the associated storage class, has been removed for all new deployments. In OpenShift Virtualization 4.14, the HPP Operator uses the Kubernetes Container Storage Interface (CSI) driver to configure local storage. A legacy HPP custom resource is supported only if it had been installed on a previous version of OpenShift Virtualization.

  • Installing the virtctl client as an RPM is no longer supported for Red Hat Enterprise Linux (RHEL) 7 and RHEL 9.

  • CentOS 7 and CentOS Stream 8 are now in the End of Life phase. As a consequence, the container images for these operating systems have been removed from OpenShift Virtualization and are no longer community supported.

Technology Preview features

Some features in this release are currently in Technology Preview. These experimental features are not intended for production use. Note the following scope of support on the Red Hat Customer Portal for these features:

Technology Preview Features Support Scope

  • You can now install and edit customized instance types and preferences to create a VM from a volume or PersistentVolumeClaim (PVC).

  • You can hot plug a bridge network interface to a running virtual machine (VM). Hot plugging and hot unplugging is supported only for VMs created with OpenShift Virtualization 4.14 or later.

Bug fixes

  • The mediated devices configuration API in the HyperConverged custom resource (CR) has been updated to improve consistency. The field that was previously named mediatedDevicesTypes is now named mediatedDeviceTypes to align with the naming convention used for the nodeMediatedDeviceTypes field. (BZ#2054863)

  • Virtual machines created from common templates on a Single Node OpenShift (SNO) cluster no longer display a VMCannotBeEvicted alert when the cluster-level eviction strategy is None for SNO. (BZ#2092412)

  • Windows 11 virtual machines now boot on clusters running in FIPS mode. (BZ#2089301)

  • When you use two pods with different SELinux contexts, VMs with the ocs-storagecluster-cephfs storage class no longer fail to migrate. (BZ#2092271)

  • If you stop a node on a cluster and then use the Node Health Check Operator to bring the node back up, connectivity to Multus is retained. (OCPBUGS-8398)

  • When restoring a VM snapshot for storage whose binding mode is WaitForFirstConsumer, the restored PVCs no longer remain in the Pending state and the restore operation proceeds. (BZ#2149654)

Known issues

Monitoring

  • The Pod Disruption Budget (PDB) prevents pod disruptions for migratable virtual machine images. If the PDB detects pod disruption, then openshift-monitoring sends a PodDisruptionBudgetAtLimit alert every 60 minutes for virtual machine images that use the LiveMigrate eviction strategy. (BZ#2026733)

Networking

  • If your OpenShift Container Platform cluster uses OVN-Kubernetes as the default Container Network Interface (CNI) provider, you cannot attach a Linux bridge or bonding device to a host’s default interface because of a change in the host network topology of OVN-Kubernetes. (BZ#1885605)

    • As a workaround, you can use a secondary network interface connected to your host, or switch to the OpenShift SDN default CNI provider.

  • You cannot SSH into a VM when using the networkType: OVNKubernetes option in your install-config.yaml file. (BZ#2165895)

  • You cannot run OpenShift Virtualization on a single-stack IPv6 cluster. (BZ#2193267)

Nodes

  • Uninstalling OpenShift Virtualization does not remove the feature.node.kubevirt.io node labels created by OpenShift Virtualization. You must remove the labels manually. (CNV-22036)

  • In a heterogeneous cluster with different compute nodes, virtual machines that have HyperV reenlightenment enabled cannot be scheduled on nodes that do not support timestamp-counter scaling (TSC) or have the appropriate TSC frequency. (BZ#2151169)

Storage

  • In some instances, multiple virtual machines can mount the same PVC in read-write mode, which might result in data corruption. (BZ#1992753)

    • As a workaround, avoid using a single PVC in read-write mode with multiple VMs.

  • If you clone more than 100 VMs using the csi-clone cloning strategy, then the Ceph CSI might not purge the clones. Manually deleting the clones might also fail. (BZ#2055595)

    • As a workaround, you can restart the ceph-mgr to purge the VM clones.

  • If you use Portworx as your storage solution on AWS and create a VM disk image, the created image might be smaller than expected due to the filesystem overhead being accounted for twice. (BZ#2237287)

    • As a workaround, you can manually expand the Persistent Volume Claim (PVC) to increase the available space after the initial provisioning process completes.

  • If you simultaneously clone more than 1000 VMs using the provided DataSources in the openshift-virtualization-os-images namespace, it is possible that not all of the VMs will move to a running state. (BZ#2216038)

    • As a workaround, deploy VMs in smaller batches.

  • Live migration cannot be enabled for a virtual machine instance (VMI) after a hotplug volume has been added and removed. (BZ#2247593)

Virtualization

  • OpenShift Virtualization links a service account token in use by a pod to that specific pod. OpenShift Virtualization implements a service account volume by creating a disk image that contains a token. If you migrate a VM, then the service account volume becomes invalid. (BZ#2037611)

    • As a workaround, use user accounts rather than service accounts because user account tokens are not bound to a specific pod.

  • With the release of the RHSA-2023:3722 advisory, the TLS Extended Master Secret (EMS) extension (RFC 7627) is mandatory for TLS 1.2 connections on FIPS-enabled RHEL 9 systems. This is in accordance with FIPS-140-3 requirements. TLS 1.3 is not affected. (BZ#2157951)

    Legacy OpenSSL clients that do not support EMS or TLS 1.3 now cannot connect to FIPS servers running on RHEL 9. Similarly, RHEL 9 clients in FIPS mode cannot connect to servers that only support TLS 1.2 without EMS. This in practice means that these clients cannot connect to servers on RHEL 6, RHEL 7 and non-RHEL legacy operating systems. This is because the legacy 1.0.x versions of OpenSSL do not support EMS or TLS 1.3. For more information, see TLS Extension "Extended Master Secret" enforced with Red Hat Enterprise Linux 9.2.

    As a workaround, upgrade legacy OpenSSL clients to a version that supports TLS 1.3 and configure OpenShift Virtualization to use TLS 1.3, with the Modern TLS security profile type, for FIPS mode.

Web console

  • If you upgrade OpenShift Container Platform 4.13 to 4.14 without upgrading OpenShift Virtualization, the Virtualization pages of the web console crash. (OCPBUGS-22853)

    You must upgrade the OpenShift Virtualization Operator to 4.14 manually or set your subscription approval strategy to "Automatic."