Create a custom SR-IOV Compute service

You must create a custom SR-IOV Compute service for NFV in a RHOSO environment.

Before you begin

  • You have the oc command line tool installed on your workstation.
  • You are logged on to a workstation that has access to the RHOSO control plane as a user with cluster-admin privileges.

About this task

This SR-IOV Compute service is an Ansible service that runs on the data plane. This custom service performs the following tasks on the SR-IOV Compute nodes:

  • Applies CPU pinning parameters.
  • Performs PCI passthrough.

To create the SR-IOV custom service, you must perform these actions:

  • Create a ConfigMap for CPU pinning that maps a CPU pinning configuration to a specified set of SR-IOV Compute nodes.
  • Create a ConfigMap for PCI passthrough that maps a PCI passthrough configuration to a specified set of SR-IOV Compute nodes.
  • Create the actual SR-IOV custom service that will implement the configMaps on your data plane.

Procedure

  1. Create a ConfigMap CR that defines configurations for CPU pinning and PCI passthrough, and save it to a YAML file on your workstation, for example, pinning-passthrough.yaml.

    Change the values (in boldface) as appropriate for your environment:

    ---
apiVersion: v1
kind: ConfigMap
metadata:
  name: cpu-pinning-nova
data:
  25-cpu-pinning-nova.conf: |
    [DEFAULT]
    reserved_host_memory_mb = 4096
    [compute]
    cpu_shared_set = 0-3,24-27
    cpu_dedicated_set = 8-23,32-47
    [neutron]
    physnets = <network_name1>, <network_name2>
    [neutron_physnet_<network_name1>]
    numa_nodes = <ID list>
    [neutron_physnet_<network_name2>]
    numa_nodes = <ID list>
    [neutron_tunnel]
    numa_nodes = <ID list>
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: sriov-nova
data:
  26-sriov-nova.conf: |
    [libvirt]
    cpu_power_management=false
    [pci]
    passthrough_whitelist = {"address": "0000:05:00.2", "physical_network":"sriov-1", "trusted":"true"}
    passthrough_whitelist = {"address": "0000:05:00.3", "physical_network":"sriov-2", "trusted":"true"}
---
    • cpu_shared_set: enter a comma-separated list or range of physical host CPU numbers used to provide vCPU inventory, determine the host CPUs that unpinned instances can be scheduled to, and determine the host CPUs that instance emulator threads should be offloaded to for instances configured with the share emulator thread policy.
    • cpu_dedicated_set: enter a comma-separated list or range of physical host CPU numbers to which processes for pinned instance CPUs can be scheduled. For example, 4-12,^8,15 reserves cores from 4-12 and 15, excluding 8.
    • <network_name_n_>: replace <network_name1> and <network_name2> with the names of the physical networks that your gateways are on. (This network is set in the neutron network provider:*name field.)
    • numa_nodes = <ID list>: replace <ID list> with a comma-separated list of IDs of the NUMA nodes associated with this physnet. For example: 0,1. For example: 
      [neutron]
physnets = foo,bar

[neutron_physnet_foo]
numa_nodes = 0

[neutron_physnet_bar]
numa_nodes = 2, 3

      This configuration ensures that instances that use one or more L2-type networks with provider:physical_network=foo must be scheduled on host cores from NUMA node 0, while instances that use one or more networks with provider:physical_network=bar must be scheduled on host cores from both NUMA nodes 2 and 3. For the latter case, it will be necessary to split the guest across two or more host NUMA nodes by using the hw:numa_nodes extra spec.

    • passthrough_whitelist: specify valid NIC addresses and names for "address" and "physical_network".
  2. Create the ConfigMap object, using the ConfigMap CR file:
    Example
    $ oc create -f sriov-pinning-passthru.yaml -n openstack
  3. Create an OpenStackDataPlaneService CR that defines the SR-IOV custom service, and save it to a YAML file on your workstation, for example nova-custom-sriov.yaml: 
    apiVersion: dataplane.openstack.org/v1beta1
kind: OpenStackDataPlaneService
metadata:
  name: nova-custom-sriov
  4. Add the ConfigMap CRs to the custom service, and specify the Secret CR for the cell that the node set that runs this service connects to: 
    apiVersion: dataplane.openstack.org/v1beta1
kind: OpenStackDataPlaneService
metadata:
  name: nova-custom-sriov
spec:
  label: dataplane-deployment-nova-custom-sriov
  dataSources:
    - configMapRef:
        name: cpu-pinning-nova
    - configMapRef:
        name: sriov-nova
    - secretRef:
        name: nova-cell1-compute-config
    - secretRef:
        name: nova-migration-ssh-key
  tlsCerts:
    default:
      contents:
        - dnsnames
        - ips
      networks:
        - ctlplane
      issuer: osp-rootca-issuer-internal
  caCerts: combined-ca-bundle
  5. Specify the Ansible commands to create the custom service, by referencing an Ansible Playbook or by including the Ansible play in the playbookContents field: 
    apiVersion: dataplane.openstack.org/v1beta1
kind: OpenStackDataPlaneService
metadata:
  name: nova-custom-sriov
spec:
  label: dataplane-deployment-nova-custom-sriov
  edpmServiceType: nova
  dataSources:
    - configMapRef:
        name: cpu-pinning-nova
    - configMapRef:
        name: sriov-nova
    - secretRef:
        name: nova-cell1-compute-config
    - secretRef:
        name: nova-migration-ssh-key
  playbook: osp.edpm.nova
  tlsCerts:
    default:
      contents:
        - dnsnames
        - ips
      networks:
        - ctlplane
      issuer: osp-rootca-issuer-internal
  caCerts: combined-ca-bundle
  6. Create the custom-nova-sriov service: 
    $ oc apply -f nova-custom-sriov.yaml -n openstack
  7. Verify that the custom service is created: 
    $ oc get openstackdataplaneservice nova-custom-sriov -o yaml -n openstack