Understanding multiple networks - Multiple networks | Networking

Usage scenarios for an additional network
Additional networks in OpenShift Container Platform
UserDefinedNetwork and NetworkAttachmentDefinition support matrix

By default, OVN-Kubernetes serves as the Container Network Interface (CNI) of an OpenShift Container Platform cluster. With OVN-Kubernetes as the default CNI of a cluster, OpenShift Container Platform administrators or users can leverage user-defined networks (UDNs) or NetworkAttachmentDefinition (NADs) to create one, or multiple, default networks that handle all ordinary network traffic of the cluster. Both user-defined networks and Network Attachment Definitions can serve as the following network types:

Primary networks: Act as the primary network for the pod. By default, all traffic passes through the primary network unless a pod route is configured to send traffic through other networks.
Secondary networks: Act as additional, non-default networks for a pod. Secondary networks provide separate interfaces dedicated to specific traffic types or purposes. Only pod traffic that is explicitly configured to use a secondary network is routed through its interface.

However, during cluster installation, OpenShift Container Platform administrators can configure alternative default secondary pod networks by leveraging the Multus CNI plugin. With Multus, multiple CNI plugins such as ipvlan, macvlan, or Network Attachment Definitions can be used together to serve as secondary networks for pods.

User-defined networks are only available when OVN-Kubernetes is used as the CNI. They are not supported for use with other CNIs.

You can define an additional network based on the available CNI plugins and attach one or more of these networks to your pods. You can define more than one additional network for your cluster depending on your needs. This gives you flexibility when you configure pods that deliver network functionality, such as switching or routing.

For a complete list of supported CNI plugins, see "Additional networks in OpenShift Container Platform".

For information about user-defined networks, see About user-defined networks (UDNs).

For information about Network Attachment Definitions, see Creating primary networks using a NetworkAttachmentDefinition.

Usage scenarios for an additional network

You can use an additional network in situations where network isolation is needed, including data plane and control plane separation. Isolating network traffic is useful for the following performance and security reasons:

Performance

Traffic management: You can send traffic on two different planes to manage how much traffic is along each plane.
Security

Network isolation: You can send sensitive traffic onto a network plane that is managed specifically for security considerations, and you can separate private data that must not be shared between tenants or customers.

All of the pods in the cluster still use the cluster-wide default network to maintain connectivity across the cluster. Every pod has an eth0 interface that is attached to the cluster-wide pod network. You can view the interfaces for a pod by using the oc exec -it <pod_name> -- ip a command. If you add additional network interfaces that use Multus CNI, they are named net1, net2, …, netN.

To attach additional network interfaces to a pod, you must create configurations that define how the interfaces are attached. You specify each interface by using either a UserDefinedNetwork custom resource (CR) or a NetworkAttachmentDefinition CR. A CNI configuration inside each of these CRs defines how that interface is created.

For more information about creating a UserDefinedNetwork CR, see About user-defined networks.

For more information about creating a NetworkAttachmentDefinition CR, see Creating primary networks using a NetworkAttachmentDefinition.

Additional networks in OpenShift Container Platform

OpenShift Container Platform provides the following CNI plugins for creating additional networks in your cluster:

bridge: Configure a bridge-based additional network to allow pods on the same host to communicate with each other and the host.
host-device: Configure a host-device additional network to allow pods access to a physical Ethernet network device on the host system.
ipvlan: Configure an ipvlan-based additional network to allow pods on a host to communicate with other hosts and pods on those hosts, similar to a macvlan-based additional network. Unlike a macvlan-based additional network, each pod shares the same MAC address as the parent physical network interface.
vlan: Configure a VLAN-based additional network to allow VLAN-based network isolation and connectivity for pods.
macvlan: Configure a macvlan-based additional network to allow pods on a host to communicate with other hosts and pods on those hosts by using a physical network interface. Each pod that is attached to a macvlan-based additional network is provided a unique MAC address.
TAP: Configure a TAP-based additional network to create a tap device inside the container namespace. A TAP device enables user space programs to send and receive network packets.
SR-IOV: Configure an SR-IOV based additional network to allow pods to attach to a virtual function (VF) interface on SR-IOV capable hardware on the host system.

UserDefinedNetwork and NetworkAttachmentDefinition support matrix

The UserDefinedNetwork and NetworkAttachmentDefinition custom resources (CRs) provide cluster administrators and users the ability to create customizable network configurations and define their own network topologies, ensure network isolation, manage IP addressing for workloads, and configure advanced network features. A third CR, ClusterUserDefinedNetwork, is also available, which allows administrators the ability to create and define additional networks spanning multiple namespaces at the cluster level.

User-defined networks and network attachment definitions can serve as both the primary and secondary network interface, and each support layer2 and layer3 topologies; a third network topology, Localnet, is also supported with network attachment definitions with secondary networks.

As of OpenShift Container Platform 4.18, the Localnet topology is unavailable for use with the UserDefinedNetwork and ClusterUserDefinedNetwork CRs. It is only available for NetworkAttachmentDefinition CRs that leverage secondary networks.

The following section highlights the supported features of the UserDefinedNetwork and NetworkAttachmentDefinition CRs when they are used as either the primary or secondary network. A separate table for the ClusterUserDefinedNetwork CR is also included.

Table 1. Primary network support matrix for `UserDefinedNetwork` and `NetworkAttachmentDefinition` CRs
Network feature	Layer2 topology	Layer3 topology
east-west traffic	✓	✓
north-south traffic	✓	✓
Persistent IPs	✓	X
Services	✓	✓
`EgressIP` resource	✓	✓
Multicast ^[1]	X	✓
`NetworkPolicy` resource ^[2]	✓	✓
`MultinetworkPolicy` resource	X	X

Multicast must be enabled in the namespace, and it is only available between OVN-Kubernetes network pods. For more information about multicast, see "Enabling multicast for a project".
When creating a UserDefinedNetwork CR with a primary network type, network policies must be created after the UserDefinedNetwork CR.

Table 2. Secondary network support matrix for `UserDefinedNetwork` and `NetworkAttachmentDefinition` CRs
Network feature	Layer2 topology	Layer3 topology	Localnet topology ^[1]
east-west traffic	✓	✓	✓ (`NetworkAttachmentDefinition` CR only)
north-south traffic	X	X	✓
Persistent IPs	✓	X	✓ (`NetworkAttachmentDefinition` CR only)
Services	X	X	X
`EgressIP` resource	X	X	X
Multicast	X	X	X
`NetworkPolicy` resource	X	X	X
`MultinetworkPolicy` resource	✓	✓	✓ (`NetworkAttachmentDefinition` CR only)

The Localnet topology is unavailable for use with the UserDefinedNetwork CR. It is only supported on secondary networks for NetworkAttachmentDefinition CRs.

Table 3. Support matrix for `ClusterUserDefinedNetwork` CRs
Network feature	Layer2 topology	Layer3 topology
east-west traffic	✓	✓
north-south traffic	✓	✓
Persistent IPs	✓	X
Services	✓	✓
`EgressIP` resource	✓	✓
Multicast ^[1]	X	✓
`MultinetworkPolicy` resource	X	X
`NetworkPolicy` resource ^[2]	✓	✓

Multicast must be enabled in the namespace, and it is only available between OVN-Kubernetes network pods. For more information, see "About multicast".
When creating a ClusterUserDefinedNetwork CR with a primary network type, network policies must be created after the UserDefinedNetwork CR.

Additional resources

Enabling multicast for a project