NFV performance factors

The Red Hat OpenStack Services on OpenShift (RHOSO) network functions virtualization (NFV) solution exploits the Kernel-based Virtual Machine (KVM) hypervisor to match or exceed the performance of physical implementations, especially with regard to throughput, latency, and jitter.

You configure RHOSO Compute nodes to enforce resource partitioning and fine tuning to achieve line rate performance for the guest virtual network functions (VNFs).

You can enable high-performance packet switching between physical NICs and virtual machines by using data plane development kit (DPDK) accelerated virtual machines. Open vSwitch (OVS) embeds support for Data Plane Development Kit (DPDK) and includes support for vhost-user multiqueue, allowing scalable performance. OVS-DPDK provides line-rate performance for guest VNFs.

Single root I/O virtualization (SR-IOV) networking provides enhanced performance, including improved throughput for specific networks and virtual machines.

Other important features for performance tuning include huge pages, NUMA alignment, host isolation, and CPU pinning. VNF flavors require huge pages and emulator thread isolation for better performance. Host isolation and CPU pinning improve NFV performance and prevent spurious packet loss.

CPU and NUMA node allocation for NFV performance
Copy link

Previously, all memory on x86 systems was equally accessible to all CPUs in the system. This resulted in memory access times that were the same regardless of which CPU in the system was performing the operation and was referred to as Uniform Memory Access (UMA).

In Non-Uniform Memory Access (NUMA), system memory is divided into zones called nodes, which are allocated to particular CPUs or sockets. Access to memory that is local to a CPU is faster than memory connected to remote CPUs on that system. Normally, each socket on a NUMA system has a local memory node whose contents can be accessed faster than the memory in the node local to another CPU or the memory on a bus shared by all CPUs.

Similarly, physical NICs are placed in PCI slots on the Compute node hardware. These slots connect to specific CPU sockets that are associated with a particular NUMA node. For optimum performance, connect your data path NICs to the same NUMA nodes in your CPU configuration (SR-IOV or OVS-DPDK).

The performance impact of NUMA misses are significant, generally starting at a 10% performance hit or higher. Each CPU socket can have multiple CPU cores which are treated as individual CPUs for virtualization purposes.

NUMA node layout example
Copy link

A two-node system can demonstrate a NUMA topology and the way CPU cores and memory pages are made available.

Figure 1. Example: two-node NUMA system

Example: two-node NUMA system

Note

Remote memory available via Interconnect is accessed only if VM1 from NUMA node 0 has a CPU core in NUMA node 1. In this case, the memory of NUMA node 1 acts as local for the third CPU core of VM1 (for example, if VM1 is allocated with CPU 4 in the diagram above), but at the same time, it acts as remote memory for the other CPU cores of the same VM.

NUMA-aware instance placement for optimal performance
Copy link

You can configure an OpenStack environment to use NUMA topology awareness on systems with a NUMA architecture.

When running a guest operating system in a virtual machine (VM) there are two NUMA topologies involved:

  • NUMA topology of the physical hardware of the host
  • NUMA topology of the virtual hardware exposed to the guest operating system

You can optimize the performance of guest operating systems by aligning the virtual hardware with the physical hardware NUMA topology.