OpenShift docs are moving and will soon only be available at docs.redhat.com, the home of all Red Hat product documentation. Explore the new docs experience today.
  1. Documentation
    2. history bug_report picture_as_pdf
×

Hot plugging secondary network interfaces

You can add or remove secondary network interfaces without stopping your virtual machine (VM). OpenShift Virtualization supports hot plugging and hot unplugging for secondary interfaces that use bridge binding and the VirtIO device driver. OpenShift Virtualization also supports hot plugging secondary interfaces that use SR-IOV binding.

Hot unplugging is not supported for Single Root I/O Virtualization (SR-IOV) interfaces.

VirtIO limitations

Each VirtIO interface uses one of the limited Peripheral Connect Interface (PCI) slots in the VM. There are a total of 32 slots available. The PCI slots are also used by other devices and must be reserved in advance, therefore slots might not be available on demand. OpenShift Virtualization reserves up to four slots for hot plugging interfaces. This includes any existing plugged network interfaces. For example, if your VM has two existing plugged interfaces, you can hot plug two more network interfaces.

The actual number of slots available for hot plugging also depends on the machine type. For example, the default PCI topology for the q35 machine type supports hot plugging one additional PCIe device. For more information on PCI topology and hot plug support, see the libvirt documentation.

If you restart the VM after hot plugging an interface, that interface becomes part of the standard network interfaces.

Hot plugging a secondary network interface by using the CLI

Hot plug a secondary network interface to a virtual machine (VM) while the VM is running.

Prerequisites

  • A network attachment definition is configured in the same namespace as your VM.

  • You have installed the virtctl tool.

  • You have installed the OpenShift CLI (oc).

Procedure

  1. If the VM to which you want to hot plug the network interface is not running, start it by using the following command:

    $ virtctl start <vm_name> -n <namespace>

  2. Use the following command to add the new network interface to the running VM. Editing the VM specification adds the new network interface to the VM and virtual machine instance (VMI) configuration but does not attach it to the running VM.

    $ oc edit vm <vm_name>
    Example VM configuration
    apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: vm-fedora
template:
  spec:
    domain:
      devices:
        interfaces:
        - name: defaultnetwork
          masquerade: {}
        # new interface
        - name: <secondary_nic> (1)
          bridge: {}
    networks:
    - name: defaultnetwork
      pod: {}
    # new network
    - name: <secondary_nic> (2)
      multus:
        networkName: <nad_name> (3)
# ...
    1 Specifies the name of the new network interface.
    2 Specifies the name of the network. This must be the same as the name of the new network interface that you defined in the template.spec.domain.devices.interfaces list.
    3 Specifies the name of the NetworkAttachmentDefinition object.

  3. To attach the network interface to the running VM, live migrate the VM by running the following command:

    $ virtctl migrate <vm_name>
Verification

  1. Verify that the VM live migration is successful by using the following command:

    $ oc get VirtualMachineInstanceMigration -w
    Example output
    NAME                        PHASE             VMI
kubevirt-migrate-vm-lj62q   Scheduling        vm-fedora
kubevirt-migrate-vm-lj62q   Scheduled         vm-fedora
kubevirt-migrate-vm-lj62q   PreparingTarget   vm-fedora
kubevirt-migrate-vm-lj62q   TargetReady       vm-fedora
kubevirt-migrate-vm-lj62q   Running           vm-fedora
kubevirt-migrate-vm-lj62q   Succeeded         vm-fedora

  2. Verify that the new interface is added to the VM by checking the VMI status:

    $ oc get vmi vm-fedora -ojsonpath="{ @.status.interfaces }"
    Example output
    [
  {
    "infoSource": "domain, guest-agent",
    "interfaceName": "eth0",
    "ipAddress": "10.130.0.195",
    "ipAddresses": [
      "10.130.0.195",
      "fd02:0:0:3::43c"
    ],
    "mac": "52:54:00:0e:ab:25",
    "name": "default",
    "queueCount": 1
  },
  {
    "infoSource": "domain, guest-agent, multus-status",
    "interfaceName": "eth1",
    "mac": "02:d8:b8:00:00:2a",
    "name": "bridge-interface", (1)
    "queueCount": 1
  }
]
    1 The hot plugged interface appears in the VMI status.

Hot unplugging a secondary network interface by using the CLI

You can remove a secondary network interface from a running virtual machine (VM).

Hot unplugging is not supported for Single Root I/O Virtualization (SR-IOV) interfaces.
Prerequisites

  • Your VM must be running.

  • The VM must be created on a cluster running OpenShift Virtualization 4.14 or later.

  • The VM must have a bridge network interface attached.

Procedure

  1. Edit the VM specification to hot unplug a secondary network interface. Setting the interface state to absent detaches the network interface from the guest, but the interface still exists in the pod.

    $ oc edit vm <vm_name>
    Example VM configuration
    apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: vm-fedora
template:
  spec:
    domain:
      devices:
        interfaces:
          - name: defaultnetwork
            masquerade: {}
          # set the interface state to absent
          - name: <secondary_nic>
            state: absent (1)
            bridge: {}
    networks:
      - name: defaultnetwork
        pod: {}
      - name: <secondary_nic>
        multus:
          networkName: <nad_name>
# ...
    1 Set the interface state to absent to detach it from the running VM. Removing the interface details from the VM specification does not hot unplug the secondary network interface.

  2. Remove the interface from the pod by migrating the VM:

    $ virtctl migrate <vm_name>

Additional resources