Adding Pod to an SR-IOV network - Hardware networks | Networking

Adding a Pod to an additional network
Creating a non-uniform memory access (NUMA) aligned SR-IOV pod

You can add a Pod to an existing Single Root I/O Virtualization (SR-IOV) network.

Adding a Pod to an additional network

You can add a Pod to an additional network. The Pod continues to send normal cluster related network traffic over the default network.

The Network Resources Injector will inject the resource parameter into the Pod CR automatically if a NetworkAttachmentDefinition CR associated with the SR-IOV CNI plug-in is specified.

Prerequisites

The Pod must be in the same namespace as the additional network.
Install the OpenShift Command-line Interface (CLI), commonly known as oc.
You must log in to the cluster.
You must have the SR-IOV Operator installed and a SriovNetwork CR defined.

Procedure

To add a Pod with additional networks, complete the following steps:

Create the Pod resource definition and add the k8s.v1.cni.cncf.io/networks parameter to the Pod metadata mapping. The k8s.v1.cni.cncf.io/networks accepts a comma separated string of one or more NetworkAttachmentDefinition Custom Resource (CR) names:

metadata:
  annotations:
    k8s.v1.cni.cncf.io/networks: <network>[,<network>,...] (1)

1 Replace <network> with the name of the additional network to associate with the Pod. To specify more than one additional network, separate each network with a comma. Do not include whitespace between the comma. If you specify the same additional network multiple times, that Pod will have multiple network interfaces attached to that network.

In the following example, two additional networks are attached to the Pod:

apiVersion: v1
kind: Pod
metadata:
  name: example-pod
  annotations:
    k8s.v1.cni.cncf.io/networks: net1,net2
spec:
  containers:
  - name: example-pod
    command: ["/bin/bash", "-c", "sleep 2000000000000"]
    image: centos/tools

Create the Pod by running the following command:
```
$ oc create -f pod.yaml
```

Optional: Confirm that the annotation exists in the Pod CR by running the following command. Replace <name> with the name of the Pod.

$ oc get pod <name> -o yaml

In the following example, the example-pod Pod is attached to the net1 additional network:

$ oc get pod example-pod -o yaml
apiVersion: v1
kind: Pod
metadata:
  annotations:
    k8s.v1.cni.cncf.io/networks: macvlan-bridge
    k8s.v1.cni.cncf.io/networks-status: |- (1)
      [{
          "name": "openshift-sdn",
          "interface": "eth0",
          "ips": [
              "10.128.2.14"
          ],
          "default": true,
          "dns": {}
      },{
          "name": "macvlan-bridge",
          "interface": "net1",
          "ips": [
              "20.2.2.100"
          ],
          "mac": "22:2f:60:a5:f8:00",
          "dns": {}
      }]
  name: example-pod
  namespace: default
spec:
  ...
status:
  ...

1	The `k8s.v1.cni.cncf.io/networks-status` parameter is a JSON array of objects. Each object describes the status of an additional network attached to the Pod. The annotation value is stored as a plain text value.

Creating a non-uniform memory access (NUMA) aligned SR-IOV pod

You can create a NUMA aligned SR-IOV pod by restricting SR-IOV and the CPU resources allocated from the same NUMA node with restricted or single-numa-node Topology Manager polices.

Prerequisites

Install the OpenShift Command-line Interface (CLI), commonly known as oc.
Enable a LatencySensitive profile and configure the CPU Manager policy to static.

Procedure

Create the following SR-IOV pod spec, and then save the YAML in the <name>-sriov-pod.yaml file. Replace <name> with a name for this pod.

The following example shows an SR-IOV pod spec:

apiVersion: v1
kind: Pod
metadata:
  name: sample-pod
  annotations:
    k8s.v1.cni.cncf.io/networks: <name> (1)
spec:
  containers:
  - name: sample-container
    image: <image> (2)
    command: ["sleep", "infinity"]
    resources:
      limits:
        memory: "1Gi" (3)
        cpu: "2" (4)
      requests:
        memory: "1Gi"
        cpu: "2"

1	Replace `<name>` with the name of the SR-IOV network attachment definition CR.
2	Replace `<image>` with the name of the `sample-pod` image.
3	To create the SR-IOV pod with guaranteed QoS, set `memory limits` equal to `memory requests`.
4	To create the SR-IOV pod with guaranteed QoS, set `cpu limits` equals to `cpu requests`.

Create the sample SR-IOV pod by running the following command:
```
$ oc create -f <filename> (1)
```
1 Replace <filename> with the name of the file you created in the previous step.
Confirm that the sample-pod is configured with guaranteed QoS.
```
$ oc describe pod sample-pod
```

Confirm that the sample-pod is allocated with exclusive CPUs.

$ oc exec sample-pod -- cat /sys/fs/cgroup/cpuset/cpuset.cpus

Confirm that the SR-IOV device and CPUs that are allocated for the sample-pod are on the same NUMA node.
```
$ oc exec sample-pod -- cat /sys/fs/cgroup/cpuset/cpuset.cpus
```

Adding a Pod to an SR-IOV additional network

Adding a Pod to an additional network

Creating a non-uniform memory access (NUMA) aligned SR-IOV pod