You can configure a network attachment for an Single Root I/O Virtualization (SR-IOV) device in the cluster.

Configuring SR-IOV additional network

You can configure an additional network that uses SR-IOV hardware by creating a SriovNetwork Custom Resource (CR). When you create a SriovNetwork CR, the SR-IOV Operator automatically creates a NetworkAttachmentDefinition CR.

Do not modify or delete a SriovNetwork Custom Resource (CR) if it is attached to any Pods in the running state.

Prerequisites
  • Install the OpenShift Command-line Interface (CLI), commonly known as oc.

  • Log in as a user with cluster-admin privileges.

Procedure
  1. Create the following SriovNetwork CR, and then save the YAML in the <name>-sriov-network.yaml file. Replace <name> with a name for this additional network.

apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetwork
metadata:
  name: <name> (1)
  namespace: openshift-sriov-network-operator (2)
spec:
  networkNamespace: <target_namespace> (3)
  ipam: <ipam> (4)
  vlan: <vlan> (5)
  resourceName: <sriov_resource_name> (6)
  linkState: <link_state> (7)
  maxTxRate: <max_tx_rate> (8)
  minTxRate: <min_rx_rate> (9)
  vlanQoS: <vlan_qos> (10)
  spoofChk: "<spoof_check>" (11)
  trust: "<trust_vf>" (12)
  capabilities: <capabilities> (13)
1 Replace <name> with a name for the CR. The Operator will create a NetworkAttachmentDefinition CR with same name.
2 Specify the namespace where the SR-IOV Operator is installed.
3 Optional: Replace <target_namespace> with the namespace where the NetworkAttachmentDefinition CR will be created. The default value is openshift-sriov-network-operator.
4 Optional: Replace <ipam> a configuration object for the ipam CNI plug-in as a YAML block scalar. The plug-in manages IP address assignment for the attachment definition.
5 Optional: Replace <vlan> with a Virtual LAN (VLAN) ID for the additional network. The integer value must be from 0 to 4095. The default value is 0.
6 Replace <sriov_resource_name> with the value for the .spec.resourceName parameter from the SriovNetworkNodePolicy CR that defines the SR-IOV hardware for this additional network.
7 Optional: Replace <link_state> with the link state of Virtual Function (VF). Allowed value are enable, disable and auto.
8 Optional: Replace <max_tx_rate> with a maximum transmission rate, in Mbps, for the VF.
9 Optional: Replace <min_tx_rate> with a minimum transmission rate, in Mbps, for the VF. This value should always be less than or equal to Maximum transmission rate.

Intel NICs do not support the minTxRate parameter. For more information, see BZ#1772847.

10 Optional: Replace <vlan_qos> with an IEEE 802.1p priority level for the VF. The default value is 0.
11 Optional: Replace <spoof_check> with the spoof check mode of the VF. The allowed values are the strings "on" and "off".

You must enclose the value you specify in quotes or the CR will be rejected by the SR-IOV Network Operator.

12 Optional: Replace <trust_vf> with the trust mode of the VF. The allowed values are the strings "on" and "off".

You must enclose the value you specify in quotes or the CR will be rejected by the SR-IOV Network Operator.

13 Optional: Replace <capabilities> with the capabilities to configure for this network. You can specify "{ "ips": true }" to enable IP address support or "{ "mac": true }" to enable MAC address support.
  1. Create the CR by running the following command:

    $ oc create -f <filename> (1)
    1 Replace <filename> with the name of the file you created in the previous step.
  2. Optional: Confirm that the NetworkAttachmentDefinition CR associated with the SriovNetwork CR that you created in the previous step exists by running the following command. Replace <namespace> with the namespace you specified in the SriovNetwork CR.

    oc get net-attach-def -n <namespace>

Configuration for ipam CNI plug-in

The IP address management (IPAM) CNI plug-in manages IP address assignment for other CNI plug-ins. You can configure ipam for either static IP address assignment or dynamic IP address assignment by using DHCP. The DHCP server you specify must be reachable from the additional network.

The following JSON configuration object describes the parameters that you can set.

If you set the type parameter to the DHCP value, you cannot set any other parameters.
ipam CNI plug-in JSON configuration object
{
  "ipam": {
    "type": "<type>", (1)
    "addresses": [ (2)
      {
        "address": "<address>", (3)
        "gateway": "<gateway>" (4)
      }
    ],
    "routes": [ (5)
      {
        "dst": "<dst>" (6)
        "gw": "<gw>" (7)
      }
    ],
    "dns": { (8)
      "nameservers": ["<nameserver>"], (9)
      "domain": "<domain>", (10)
      "search": ["<search_domain>"] (11)
    }
  }
}
1 Specify static to configure the plug-in to manage IP address assignment. Specify DHCP to allow a DHCP server to manage IP address assignment. You cannot specify any additional parameters if you specify a value of DHCP.
2 An array describing IP addresses to assign to the virtual interface. Both IPv4 and IPv6 IP addresses are supported.
3 A block of IP addresses that you specify in CIDR format to assign to Pods on a worker node, such as 10.1.1.0/24.
4 The default gateway to route egress network traffic to.
5 An array describing routes to configure inside the Pod.
6 The IP address range in CIDR format.
7 The gateway to use to route network traffic to.
8 The DNS configuration. Optional.
9 An of array of one or more IP addresses for to send DNS queries to.
10 The default domain to append to a host name. For example, if the domain is set to example.com, a DNS lookup query for example-host will be rewritten as example-host.example.com.
11 An array of domain names to append to an unqualified host name, such as example-host, during a DNS lookup query.

Static IP address assignment configuration example

You can configure ipam for static IP address assignment:

{
  "ipam": {
    "type": "static",
      "addresses": [
        {
          "address": "191.168.1.1/24"
        }
      ]
  }
}

Dynamic IP address assignment configuration example

You can configure ipam for DHCP:

{
  "ipam": {
    "type": "DHCP"
  }
}

Configuring static MAC and IP addresses on additional SR-IOV networks

You can configure static MAC and IP addresses on additional an SR-IOV network by specifying CNI runtimeConfig data in a pod annotation.

Prerequisites
  • Install the OpenShift Command-line Interface (CLI), commonly known as oc.

  • Log in as a user with cluster-admin privileges when creating the SriovNetwork CR.

Procedure
  1. Create the following SriovNetwork CR, and then save the YAML in the <name>-sriov-network.yaml file. Replace <name> with a name for this additional network.

    apiVersion: sriovnetwork.openshift.io/v1
    kind: SriovNetwork
    metadata:
      name: <name> (1)
      namespace: sriov-network-operator (2)
    spec:
      networkNamespace: <target_namespace> (3)
      ipam: '{"type": "static"}' (4)
      capabilities: '{"mac": true, "ips": true}' (5)
      resourceName: <sriov_resource_name> (6)
    1 Replace <name> with a name for the CR. The Operator will create a NetworkAttachmentDefinition CR with same name.
    2 Specify the namespace where the SR-IOV Operator is installed.
    3 Replace <target_namespace> with the namespace where the NetworkAttachmentDefinition CR will be created.
    4 Specify static type for the ipam CNI plug-in as a YAML block scalar.
    5 Specify mac and ips capabilities to true.
    6 Replace <sriov_resource_name> with the value for the .spec.resourceName parameter from the SriovNetworkNodePolicy CR that defines the SR-IOV hardware for this additional network.
  2. Create the CR by running the following command:

    $ oc create -f <filename> (1)
    1 Replace <filename> with the name of the file you created in the previous step.
  3. Optional: Confirm that the NetworkAttachmentDefinition CR associated with the SriovNetwork CR that you created in the previous step exists by running the following command. Replace <namespace> with the namespace you specified in the SriovNetwork CR.

    oc get net-attach-def -n <namespace>

Do not modify or delete a SriovNetwork Custom Resource (CR) if it is attached to any Pods in the running state.

  1. Create the following SR-IOV pod spec, and then save the YAML in the <name>-sriov-pod.yaml file. Replace <name> with a name for this pod.

    apiVersion: v1
    kind: Pod
    metadata:
      name: sample-pod
      annotations:
        k8s.v1.cni.cncf.io/networks: '[
            {
                    "name": "<name>", (1)
                    "mac": "20:04:0f:f1:88:01", (2)
                    "ips": ["192.168.10.1/24", "2001::1/64"] (3)
            }
    ]'
    spec:
      containers:
      - name: sample-container
        image: <image>
        imagePullPolicy: IfNotPresent
        command: ["sleep", "infinity"]
    1 Replace <name> with then name of the SR-IOV network attachment definition CR.
    2 Specify the mac address for the SR-IOV device which is allocated from the resource type defined in the SR-IOV network attachment definition CR.
    3 Specify the IPv4 and/or IPv6 addresses for the SR-IOV device which is allocated from the resource type defined in the SR-IOV network attachment definition CR.
  2. Create the sample SR-IOV pod by running the following command:

    $ oc create -f <filename> (1)
    1 Replace <filename> with the name of the file you created in the previous step.
  3. Optional: Confirm that mac and ips addresses are applied to the SR-IOV device by running the following command. Replace <namespace> with the namespace you specified in the SriovNetwork CR.

    oc exec sample-pod -n <namespace> -- ip addr show