As a cluster administrator, you can configure an additional network for your cluster using the bridge Container Network Interface (CNI) plug-in. When configured, all Pods on a node are connected to a virtual switch. Each Pod is assigned an IP address on the additional network.

Creating an additional network attachment with the bridge CNI plug-in

The Cluster Network Operator (CNO) manages additional network definitions. When you specify an additional network to create, the CNO creates the NetworkAttachmentDefinition Custom Resource (CR) automatically.

Do not edit the NetworkAttachmentDefinition CRs that the Cluster Network Operator manages. Doing so might disrupt network traffic on your additional network.

Prerequisites
  • Install the OpenShift Command-line Interface (CLI), commonly known as oc.

  • Log in as a user with cluster-admin privileges.

Procedure

To create an additional network for your cluster, complete the following steps:

  1. Edit the CNO CR by running the following command:

    $ oc edit networks.operator.openshift.io cluster
  2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

    The following YAML configures the bridge CNI plug-in:

    apiVersion: operator.openshift.io/v1
    kind: Network
    metadata:
      name: cluster
    spec:
      additionalNetworks: (1)
      - name: test-network-1
        type: Raw
        rawCNIConfig: '{
          "cniVersion": "0.3.1",
          "type": "bridge",
          "master": "eth1",
          "ipam": {
            "type": "static",
            "addresses": [
              {
                "address": "191.168.1.1/24"
              }
            ]
          }
        }'
    1 Specify the configuration for the additional network attachment definition.
  3. Save your changes and quit the text editor to commit your changes.

  4. Optional: Confirm that the CNO created the NetworkAttachmentDefinition CR by running the following command. There might be a delay before the CNO creates the CR.

    $ oc get network-attachment-definitions
    NAME                 AGE
    example-network      14m
    example-macvlan      21m

Configuration for bridge

The configuration for an additional network attachment that uses the bridge Container Network Interface (CNI) plug-in is provided in two parts:

  • Cluster Network Operator (CNO) configuration

  • CNI plug-in configuration

The CNO configuration specifies the name for the additional network attachment and the namespace to create the attachment in. The plug-in is configured by a JSON object specified by the rawCNIConfig parameter in the CNO configuration.

The following YAML describes the configuration parameters for the CNO:

Cluster Network Operator YAML configuration
name: <name> (1)
namespace: <namespace> (2)
rawCNIConfig: '{ (3)
  ...
}'
type: Raw
1 Specify a name for the additional network attachment that you are creating. The name must be unique within the specified namespace.
2 Specify the namespace to create the network attachment in. If you do not specify a value, then the default namespace is used.
3 Specify the CNI plug-in configuration in JSON format, which is based on the following template.

The following object describes the configuration parameters for the bridge CNI plug-in:

bridge CNI plug-in JSON configuration object
{
  "cniVersion": "0.3.1",
  "name": "<name>", (1)
  "type": "bridge",
  "bridge": "<bridge>", (2)
  "ipam": { (3)
    ...
  },
  "ipMasq": false, (4)
  "isGateway": false, (5)
  "isDefaultGateway": false, (6)
  "forceAddress": false, (7)
  "hairpinMode": false, (8)
  "promiscMode": false, (9)
  "vlan": <vlan>, (10)
  "mtu": <mtu> (11)
}
1 Specify the value for the name parameter you provided previously for the CNO configuration.
2 Specify the name of the virtual bridge to use. If the bridge interface does not exist on the host, it is created. The default value is cni0.
3 Specify a configuration object for the ipam CNI plug-in. The plug-in manages IP address assignment for the network attachment definition.
4 Set to true to enable IP masquerading for traffic that leaves the virtual network. The source IP address for all traffic is rewritten to the bridge’s IP address. If the bridge does not have an IP address, this setting has no effect. The default value is false.
5 Set to true to assign an IP address to the bridge. The default value is false.
6 Set to true to configure the bridge as the default gateway for the virtual network. The default value is false. If isDefaultGateway is set to true, then isGateway is also set to true automatically.
7 Set to true to allow assignment of a previously assigned IP address to the virtual bridge. When set to false, if an IPv4 address or an IPv6 address from overlapping subsets is assigned to the virtual bridge, an error occurs. The default value is false.
8 Set to true to allow the virtual bridge to send an ethernet frame back through the virtual port it was received on. This mode is also known as reflective relay. The default value is false.
9 Set to true to enable promiscuous mode on the bridge. The default value is false.
10 Specify a virtual LAN (VLAN) tag as an integer value. By default, no VLAN tag is assigned.
11 Set the maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel.

bridge configuration example

The following example configures an additional network named bridge-net:

name: bridge-net
type: Raw
rawCNIConfig: '{ (1)
  "cniVersion": "0.3.1",
  "type": "bridge",
  "master": "eth1",
  "isGateway": true,
  "vlan": 2,
  "ipam": {
    "type": "dhcp"
    }
}'
1 The CNI configuration object is specified as a YAML string.

Configuration for ipam CNI plug-in

The IP address management (IPAM) CNI plug-in manages IP address assignment for other CNI plug-ins. You can configure ipam for either static IP address assignment or dynamic IP address assignment by using DHCP. The DHCP server you specify must be reachable from the additional network.

In OpenShift Container Platform 4.2.0, if you attach a Pod to an additional network that uses DHCP for IP address management, the Pod will fail to start. This is fixed in OpenShift Container Platform 4.2.1. For more information, see BZ#1754686.

The following JSON configuration object describes the parameters that you can set.

If you set the type parameter to the DHCP value, you cannot set any other parameters.
ipam CNI plug-in JSON configuration object
{
  "ipam": {
    "type": "<type>", (1)
    "addresses": [ (2)
      {
        "address": "<address>", (3)
        "gateway": "<gateway>" (4)
      }
    ],
    "routes": [ (5)
      {
        "dst": "<dst>" (6)
        "gw": "<gw>" (7)
      }
    ],
    "dns": { (8)
      "nameservers": ["<nameserver>"], (9)
      "domain": "<domain>", (10)
      "search": ["<search_domain>"] (11)
    }
  }
}
1 Specify static to configure the plug-in to manage IP address assignment. Specify DHCP to allow a DHCP server to manage IP address assignment. You cannot specify any additional parameters if you specify a value of DHCP.
2 An array of that define IP addresses to assign to the virtual interface. Both IPv4 and IPv6 IP addresses are supported.
3 A block of IP addresses that you specify in CIDR format to assign to Pods on a worker node, such as 10.1.1.0/24.
4 The default gateway to route egress network traffic to.
5 An array describing routes to configure inside the Pod.
6 The IP address range in CIDR format.
7 The gateway to use to route network traffic to.
8 The DNS configuration. Optional.
9 An of array of one or more IP addresses for to send DNS queries to.
10 The default domain to append to a host name. For example, if the domain is set to example.com, a DNS lookup query for example-host will be rewritten as example-host.example.com.
11 An array of domain names to append to an unqualified host name, such as example-host, during a DNS lookup query.

Static IP address assignment configuration example

You can configure ipam for static IP address assignment:

{
  "ipam": {
    "type": "static",
      "addresses": [
        {
          "address": "191.168.1.1/24"
        }
      ]
  }
}

Dynamic IP address assignment configuration example

You can configure ipam for DHCP:

{
  "ipam": {
    "type": "DHCP"
  }
}