Chapter 1. Virtualization
Manage Red Hat OpenShift Virtualization virtual machine resources across all the clusters that Red Hat Advanced Cluster Management manages. To use virtualization capabilities in Red Hat Advanced Cluster Management, ensure that a supported version of OpenShift Container Platform and the Red Hat OpenShift Virtualization operator is installed on the clusters that you want to manage. For instance, for Red Hat Advanced Cluster Management 2.16, you need to use Red Hat OpenShift Virtualization 4.21.1 or later.
To view VirtualMachine resources across all the clusters that Red Hat Advanced Cluster Management manages, use the Search service to list and filter the VirtualMachine resources that are created with the Red Hat OpenShift Virtualization. You can also enable the following actions from the Red Hat Advanced Cluster Management console on your VirtualMachine resources:
- Start
- Stop
- Restart
- Pause
- Unpause
- Snapshot
Learn more about virtual machine capability:
- Viewing virtual machine metrics through observability fields
- Configuring backup policies for Red Hat OpenShift Virtualization
- Backing up and restoring a virtual machine
- Virtualization policies installed on the hub cluster
- Defining policies with ConfigMap resources
- Virtualization policy ConfigMap overview
- Preparing to migrate virtual machines between clusters
- Migrating virtual machines between clusters
1.1. Viewing virtual machine metrics through Observability fields Copy linkLink copied to clipboard!
If you have the Observability service installed on your hub cluster, you can access the Observability fields on the Red Hat Advanced Cluster Management console to view your virtual machine metrics.
Procedure
-
On the Red Hat Advanced Cluster Management console, go to the
Virtual Machinepage. Select the
Observability dashboardslink to launch theGrafanadashboard.-
Optional: Select the
Observability metricslink in theLaunch linkstable cell to launch theGrafanadashboard,
-
Optional: Select the
-
From the
Grafanadashboard, view the virtual machines and their individual metrics.
1.2. Configuring backup policies for Red Hat OpenShift Virtualization Copy linkLink copied to clipboard!
Use OpenShift APIs for Data Protection to backup and restore virtual machines. After you back up the virtual machines, you can restore it on the hub or managed clusters. To configure the policies, and either enable a backup schedule or run a restore operation for a virtual machine.
Red Hat Advanced Cluster Management policies for OpenShift Virtualization support the following backup and restore storage options:
- Container Storage Interface (CSI) backups
- Container Storage Interface (CSI) backups with DataMover
Red Hat Advanced Cluster Management policies do not support the following options:
- File system backup and restore
- Volume snapshot backups and restores
Procedure
- Back up and restore a virtual machine. See Backing up and restoring a virtual machine.
- Review the virtualization policies that are installed on the hub cluster. See Virtualization policies installed on the hub cluster.
-
Define the policies with
ConfigMaps. See Defining policies with ConfigMap resources. -
Review the virtualization policy
ConfigMaps. See Virtualization policy ConfigMap overview.
1.2.1. Backing up and restoring a virtual machine Copy linkLink copied to clipboard!
You can back up virtual machines that run on a cluster, or restore a virtual machine on a cluster. Complete the following steps.
Procedure
-
Enable the backup component on the hub cluster by setting the
cluster-backupparameter in theMultiClusterHubresource totrue. To place the policies on the
cluster-namecluster, add theacm-virt-configlabel to theManagedClusterresource by applying the following YAML sample:apiVersion: cluster.open-cluster-management.io/v1 kind: ManagedCluster metadata: name: cluster-name labels: acm-virt-config: acm-dr-virt-config-
The
acm-dr-virt-configthat passes through the label annotation is aacm-virt-configConfigMap. Read theacm-dr-virt-configConfigMapsection to understand how to configure OpenShift APIs for Data Protection and backup storage location.
-
The
Complete the following steps, depending on whether you want to backup or restore your virtual machine:
-
Back up your virtual machine by adding the
cluster.open-cluster-management.io/backup-vmto thekubevirt.io.VirtualMachineresource. Yourkubevirt.io.VirtualMachineresource might resemble the following YAML sample:
apiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: name: vm-name labels: cluster.open-cluster-management.io/backup-vm: daily_8am-
Restore your virtual machine by updating the
acm-dr-restore-configConfigMapas described in Virtualization policy ConfigMap overview.
-
Back up your virtual machine by adding the
1.2.2. Virtualization policies installed on the hub cluster Copy linkLink copied to clipboard!
When you enable the backup operator, a set of Red Hat Advanced Cluster Management policies and config maps are installed on the hub cluster. When you disable the policy or remove it from the cluster, all the resources that the policy created is removed. Learn more about the policies and config maps that are installed on the hub cluster.
Installation policy
The
acm-dr-virt-installinstallation policy installs OpenShift APIs for Data Protection and configures theoadp.openshift.io.DataProtectionApplicationresource on the cluster where this policy is placed.For the hub cluster, the policy verifies that OpenShift APIs for Data Protection is installed in the
open-cluster-management-backupnamespace, and that theDataProtectionApplicationresource exists and has the required configuration. The policy does not attempt to install OpenShift APIs for Data Protection, or create theDataProtectionApplicationresource on the hub cluster.When you enable the
cluster-backupoption on theMultiClusterHubresource, the backup chart installs the OpenShift APIs for Data Protection. When you enable thecluster-backup-option, you create theDataProtectionApplicationon the hub cluster.Table 1.1. Installation policy templates Template
Description
check-config-fileVerifies that the
ConfigMapthat is defined by theacm-virt-configlabel set on theManagedClusterresource exists on the hub cluster, in theopen-cluster-management-backupnamespace.check-oadp-channelVerifies if there is an OpenShift APIs for Data Protection version installed on the cluster, and that it matches the version installed by the policy.
check-dpa-configVerifies that the
DataProtectionApplicationresource exists, has the expected configuration, and has a status ofReconciled. Verifies that theBackupStorageLocationexists and is in theAvailablestatus phase.install-oadp-copy-configCreates the namespace where OpenShift APIs for Data Protection gets installed, if this namespace is not found. Copies the OpenShift APIs for Data Protection secret and installs OpenShift APIs for Data Protection, but only for managed clusters. Creates the
DataProtectionApplicationresource as defined by theacm-virt-configlabel, but only for the managed clusters. The user must manually create the OpenShift APIs for Data Protection andDataProtectionApplicationon the hub cluster.Backup policy
The
acm-dr-virt-backupbackup policy backs up thekubevirt.io.VirtualMachineresources that has thecluster.open-cluster-management.io/backup-vm: schedule_cron_namelabel.Table 1.2. Backup policy templates Template
Description
check-cron-schedule-validShows a violation error if there is any cron schedule name used by a virtual machine
cluster.open-cluster-management.io/backup-vmlabel which is not found in theConfigMapfor the cron schedules. The name of theConfigMapfor the cron schedules is defined by theschedule_hub_config_nameproperty and is available within theopen-cluster-management-backup namespace.check-backup-status-completedIf the schedule is not paused, it verifies that the status is
Enabled. Finds the latest backup generated by this schedule and verifies that the status isCompleted. If anyDataUploadhas been created for this backup, the status isCompleted. Shows a violation if any of these conditions are notTrue.create-virt-backupCreates Velero schedules for all virtual machine resources with a
cluster.open-cluster-management.io/backup-vmlabel. Creates one schedule for each cron job schedule found with the virtual machines label. Backs up the virtual machine sharing the same cron job schedule within the same backup. Creates the Velero schedule resource only if theacm-dr-virt-installpolicy is compliant.Restore policy
The
acm-dr-virt-restorerestore policy restores thekubevirt.io.VirtualMachineresources by UID. The policy creates Velero restore resources on clusters using the information from theConfigMapthat is identified by therestore_hub_config_nameproperty from theacm-dr-virt-configConfigMap.Table 1.3. Restore policy templates Template
Description
check-velero-restore-statusVerifies that the restore resource with the name defined by the
clusterID_restoreNameproperty exists and has a status ofCompleted.create-velero-restoreIf the
clusterID_restoreNameproperty value is not empty, and theclusterIDmatches this cluster UID, it creates a Velero restore resources using the properties defined with theConfigMapthat is identified by therestore_hub_config_name. Creates the Velero restore resource only if theacm-dr-virt-installpolicy is compliant.
1.2.3. Defining policies with ConfigMap resources Copy linkLink copied to clipboard!
The acm-dr-virt-config config map defines the OpenShift APIs for Data Protection configuration and other settings that are related to the backup or restore operation. You can use the name of this resource as the value for the acm-virt-config label, which is set on the managed cluster where the policies are placed. You can also create a new ConfigMap resource that uses the acm-dr-virt-config as a sample, then use your new ConfigMap resource to place the virtualization policies on the cluster.
Procedure
Use the
acm-dr-virt-configor create a newConfigMapresource on the hub cluster in theopen-cluster-management-backupnamespace by using theacm-dr-virt-config-newConfigMapas an example.-
Update the
dpa_specsection to match the storage location where you want to store the backups that are created by OpenShift APIs for Data Protection. -
Optional: Update other properties that are available with the
acm-dr-virt-config-newConfigMap, such as OpenShift APIs for Data Protection version or channel.
-
Update the
Add the
acm-virt-config=acm-dr-virt-config-newlabel to theManagedClusterresource. The value of theacm-virt-configlabel is the name of the config map that you created. For an example of aManagedClusterresource with a virtualization backup label set to useacm-dr-virt-config-new, see the following YAML sample:apiVersion: cluster.open-cluster-management.io/v1 kind: ManagedCluster metadata: name: managed-cluster-name labels: acm-virt-config: acm-dr-virt-config-new
Next steps
- See Virtualization policy ConfigMap overview for ConfigMap properties, schedule configuration, and restore ConfigMap examples.
1.2.4. Virtualization policy ConfigMap overview Copy linkLink copied to clipboard!
When you install the policies on the hub cluster, the following ConfigMap resources are created on the hub cluster, in the open-cluster-management-backup namespace. When you place the virtualization policies on the hub or managed cluster, these ConfigMap resources define the backup and restore configuration for the virtualization policies.
| Name | Description | Type | Default value | Optional |
|
| The channel defaults to the supported OpenShift APIs for Data Protection version, which is based on the version of the OpenShift Container Platform cluster. Use this property to overwrite the default value. | String | None | Yes |
|
|
OpenShift APIs for Data Protection channel name. Set this property for custom installation, for example: | String |
| Yes |
|
|
OpenShift APIs for Data Protection subscription. Set this property for custom installation, for example: | String |
| Yes |
|
|
OpenShift APIs for Data Protection subscription source. Set this property for custom installation, for example: | String |
| Yes |
|
| OpenShift APIs for Data Protection subscription startingCSV. | String | None | Yes |
|
| OpenShift APIs for Data Protection install plan. | String | Automatic | Yes |
|
| Namespace where OpenShift APIs for Data Protection gets installed on the managed cluster. | String | None | No |
|
|
Name of the OpenShift APIs for Data Protection secret that you use on the managed cluster. A Secret with this name must exist on the hub cluster in the | String | None | No |
|
|
Name of the secret used by the OpenShift APIs for Data Protection | JSON | None | No |
|
|
Name of the | String | None | No |
|
|
Valid JSON defining the | JSON | None | No |
| Name | Description | Type | Default value | Optional |
|
| Backup expiration time. | String |
| Yes |
|
|
Set to | String | False | Yes |
|
|
Name of the | String | None | Yes |
| Name | Description | Type | Default value | Optional |
|
|
Name of the ConfigMap that is used to define the restore operation. This name must exist on the hub cluster in the | String | None | No |
Example of a
acm-dr-virt-schedule-cronConfigMapUse the
acm-dr-virt-schedule-cronConfigMapto define the valid cron job schedules to use when you are scheduling a virtual machine backup. Theacm-dr-virt-schedule-cronConfigMapis created by the hub cluster backup component in theopen-cluster-management-backupnamespace and includes the following YAML:apiVersion: v1 kind: ConfigMap metadata: name: acm-dr-virt-schedule-cron namespace: open-cluster-management-backup data: hourly: "0 */1 * * *" every_2_hours: "0 */2 * * *" every_3_hours: "0 */3 * * *" every_4_hours: "0 */4 * * *" every_5_hours: "0 */5 * * *" every_6_hours: "0 */6 * * *" twice_a_day: "0 */12 * * *" daily_8am: "0 8 * * *" every_sunday: "0 0 * * 0"Example
VirtualMachinewith a daily backup scheduleYou can use any of the schedules created by the
acm-dr-virt-schedule-cronConfigMapto set the backup schedule for akubevirt.io.VirtualMachine. For example, if you want to backup a virtual machine every day at 8am, add the following label to the virtual machine resource:apiVersion: kubevirt.io/v1 kind: VirtualMachine metadata: name: vm-name labels: cluster.open-cluster-management.io/backup-vm: daily_8amSince this
acm-dr-virt-schedule-cronConfigMapis created and managed by the hub cluster backup component, you cannot change any of the existing cron properties or delete them. The properties are recreated when the content is reconciled. You can only add new properties to the existing ones. For example, if you want to create a cron job to run every Saturday, add the following label to the existingacm-dr-virt-schedule-cronproperties:every_sunday: "0 0 * * SAT".Example of complete restore operation in the
acm-dr-virt-restore-configConfigMapUse the
acm-dr-virt-restore-configConfigMapto define the restore operations to be implemented by theacm-dr-virt-restorepolicy. Use the hub cluster backup component in theopen-cluster-management-backupnamespace to create theConfigMap. For an example of a complete restore operation, see the following YAML sample:kind: ConfigMap apiVersion: v1 metadata: name: acm-dr-virt-restore-config namespace: open-cluster-management-backup data: 2a054d24-3235-4249-9c81-f58ebc6110c7_backupName: acm-rho-virt-schedule-every-10-hours-20250120230438 2a054d24-3235-4249-9c81-f58ebc6110c7_restoreName: restore-20250120230438 2a054d24-3235-4249-9c81-f58ebc6110c7_vmsUID: 2a054d24-3235-4249-9c81-f58ebc6110c7 2a054d24-3235-4249-9c81-f58ebc6110c7_namespaceMapping: backup-ns-1=restore-ns-1{{ backup-ns-2=restore-ns-2}}-
2a054d24-3235-4249-9c81-f58ebc6110c7is the cluster ID where the restore operation is complete.acm-rho-virt-schedule-every-10-hours-20250120230438is the name of the backup that is used for the restore operation example. -
restore-20250120230438is the name of theRestoreresource that is created as a result of this restore operation. -
2a054d24-3235-4249-9c81-f58ebc6110c7is the UID of the virtual machine backed up with theacm-rho-virt-schedule-every-10-hours-20250120230438backup to be restored. -
namespaceMappingis an optional property and is used to restore resources in a different namespace than the initial resource. Any source namespaces not included in the map is restored into namespaces of the same name.backup-ns-1=restore-ns-1 backup-ns-2=restore-ns-2is a space separated list of namespace mapping. In this sample, backup resources created within thebackup-ns-1namespace are restored within therestore-ns-1namespace. Therestore-ns-1is created on the restore cluster if it does not exist. Similarly, resources created within thebackup-ns-2are restored within therestore-ns-2namespace.
-
Virtualization policies for your restore operations scenarios
-
If you set the
clusterID_restoreNameproperty from theacm-dr-virt-restore-configConfigMapto an empty value, then the restore resource created by theacm-dr-virt-restorepolicy on the cluster withclusterIDgets deleted. -
If you set the
clusterID_restoreNameproperty from theacm-dr-virt-restore-configConfigMapto a non-empty value, then theacm-dr-virt-restorepolicy creates a VeleroRestoreresource . The name of the Velero restore resource matches theclusterID_restoreNamevalue. If the restore resource with this name is deleted from the restore cluster, theacm-dr-virt-restorepolicy recreates the resource, which results in the restore operation running again.
-
If you set the
Important considerations for your restore operations scenarios
- Do not delete the restore resource if you do not want to run the restore again.
-
Remove the value for the
clusterID_restoreNameparameter on theacm-dr-virt-restore-configConfigMapwhen the restore operation gets verified to start. Doing this prevents you from having to rerun the restore operation if theacm-dr-virt-restorepolicy is disabled and enabled again. When you disable theacm-dr-virt-restorepolicy, all resources created by it on the restore cluster get cleaned up and recreated when you enable the policy again.
Additional resources
1.3. Preparing to migrate virtual machines between clusters Copy linkLink copied to clipboard!
Migrating virtual machines helps you move resources during cluster upgrades, times when you need to clear a node for maintenance, or during configuration changes. When you enable migration, the Red Hat OpenShift Virtualization is automatically installed on your managed clusters with the acm/cnv-operator-install label. The migration toolkit for virtualization is also installed on your hub cluster.
Required access: Cluster administrator
Before you perform migration operations or access the Fleet Virtualization console, ensure that the following requirements are met:
| Role | Location | Purpose |
|
| Hub cluster | Perform migration operations and access the Fleet Virtualization console. |
|
| Source and target clusters | Manage virtual machine extended and related resources. |
|
| Target cluster | Perform mandatory cluster version and Red Hat OpenShift Virtualization operator readiness checks. |
Procedure
- Enable fine-grained RBAC on your clusters to assign and use the required roles. See Enabling fine-grained role-based access control for virtualization.
-
Ensure you have access to the
openshift-cnvnamespace. -
Ensure you have the
kubevirt.io:adminrole on both the source and target clusters. - Install the latest supported version of OpenShift Container Platform. See the OpenShift Container Platform overview.
- Understand the required roles for migration operations and access to the Fleet Virtualization console. See the following table for more information about the required roles. For additional details about fine-grained roles and permissions, see the Fine-grained RBAC roles and descriptions documentation.
- See Migrating virtual machines between clusters to complete the migration procedure.
Additional resources
1.3.1. Migrating virtual machines between clusters Copy linkLink copied to clipboard!
Complete the following steps to enable virtual machine migration between clusters and verify that the required operators are available.
Procedure
Enable
cnv-mtv-integrationscomponent for the integration of Red Hat OpenShift Virtualization.-
Run the following command to edit your
multiclusterhubinstance:
oc edit MultiClusterHub multiclusterhub -n open-cluster-management-
From the
componentsspecification, set theenabledparameter totruefor thecnv-mtv-integrationscomponent.
-
Run the following command to edit your
Verify that the migration toolkit for virtualization is available on your cluster. Run the following command:
oc get clustermanagementaddon mtv-operator -o jsonpath='{.status.installProgressions[*].conditions[*].reason}'Verify that the
kubevirt-hyperconvergedresource is available on your cluster with the following command:oc get clustermanagementaddon kubevirt-hyperconverged -o jsonpath='{.status.installProgressions[*].conditions[*].reason}Verify that the
kubevirt-hyperconverged-operatoris available on your cluster with the following command:oc get clustermanagementaddon kubevirt-hyperconverged-operator -o jsonpath='{.status.installProgressions[*].conditions[*].reason}'-
To designate a cluster as a source or target for the virtual machine migration, apply the following label to your clusters:
acm/cnv-operator-install: "true". Optional: If you don’t see the Virtualization console, run the following command:
oc -n openshift-cnv patch hco kubevirt-hyperconverged --type='merge' -p '{"spec":{"featureGates":{"decentralizedLiveMigration":true}}}'-
Optional: If you want to further configure your environment in the managed cluster, edit the
HyperConvergedresource.