Configuring multi-network policies - Multiple networks | Networking

Differences between multi-network policy and network policy
Enabling multi-network policy for the cluster
Supporting multi-network policies in IPv6 networks
Working with multi-network policy
Additional resources

Administrators can use the MultiNetworkPolicy API to create multiple network policies that manage traffic for pods attached to secondary networks. For example, you can create policies that allow or deny traffic based on specific ports, IPs/ranges, or labels.

Multi-network policies can be used to manage traffic on secondary networks in the cluster. They cannot managed the cluster’s default network or primary network of user-defined networks.

As a cluster administrator, you can configure a multi-network policy for any of the following network types:

Single-Root I/O Virtualization (SR-IOV)
MAC Virtual Local Area Network (MacVLAN)
IP Virtual Local Area Network (IPVLAN)
Bond Container Network Interface (CNI) over SR-IOV
OVN-Kubernetes additional networks

Support for configuring multi-network policies for SR-IOV additional networks is only supported with kernel network interface controllers (NICs). SR-IOV is not supported for Data Plane Development Kit (DPDK) applications.

Differences between multi-network policy and network policy

Although the MultiNetworkPolicy API implements the NetworkPolicy API, there are several important differences:

You must use the MultiNetworkPolicy API:

apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy

You must use the multi-networkpolicy resource name when using the CLI to interact with multi-network policies. For example, you can view a multi-network policy object with the oc get multi-networkpolicy <name> command where <name> is the name of a multi-network policy.
You must specify an annotation with the name of the network attachment definition that defines the macvlan or SR-IOV additional network:
```
apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy
metadata:
  annotations:
    k8s.v1.cni.cncf.io/policy-for: <network_name>
```
where:

<network_name>

Specifies the name of a network attachment definition.

Enabling multi-network policy for the cluster

As a cluster administrator, you can enable multi-network policy support on your cluster.

Prerequisites

Install the OpenShift CLI (oc).
Log in to the cluster with a user with cluster-admin privileges.

Procedure

Create the multinetwork-enable-patch.yaml file with the following YAML:

apiVersion: operator.openshift.io/v1
kind: Network
metadata:
  name: cluster
spec:
  useMultiNetworkPolicy: true

Configure the cluster to enable multi-network policy:

$ oc patch network.operator.openshift.io cluster --type=merge --patch-file=multinetwork-enable-patch.yaml

Example output

network.operator.openshift.io/cluster patched

Supporting multi-network policies in IPv6 networks

The ICMPv6 Neighbor Discovery Protocol (NDP) is a set of messages and processes that enable devices to discover and maintain information about neighboring nodes. NDP plays a crucial role in IPv6 networks, facilitating the interaction between devices on the same link.

The Cluster Network Operator (CNO) deploys the iptables implementation of multi-network policy when the useMultiNetworkPolicy parameter is set to true.

To support multi-network policies in IPv6 networks the Cluster Network Operator deploys the following set of rules in every pod affected by a multi-network policy:

Multi-network policy custom rules

kind: ConfigMap
apiVersion: v1
metadata:
  name: multi-networkpolicy-custom-rules
  namespace: openshift-multus
data:

  custom-v6-rules.txt: |
    # accept NDP
    -p icmpv6 --icmpv6-type neighbor-solicitation -j ACCEPT (1)
    -p icmpv6 --icmpv6-type neighbor-advertisement -j ACCEPT (2)
    # accept RA/RS
    -p icmpv6 --icmpv6-type router-solicitation -j ACCEPT (3)
    -p icmpv6 --icmpv6-type router-advertisement -j ACCEPT (4)

1	This rule allows incoming ICMPv6 neighbor solicitation messages, which are part of the neighbor discovery protocol (NDP). These messages help determine the link-layer addresses of neighboring nodes.
2	This rule allows incoming ICMPv6 neighbor advertisement messages, which are part of NDP and provide information about the link-layer address of the sender.
3	This rule permits incoming ICMPv6 router solicitation messages. Hosts use these messages to request router configuration information.
4	This rule allows incoming ICMPv6 router advertisement messages, which give configuration information to hosts.

You cannot edit these predefined rules.

These rules collectively enable essential ICMPv6 traffic for correct network functioning, including address resolution and router communication in an IPv6 environment. With these rules in place and a multi-network policy denying traffic, applications are not expected to experience connectivity issues.

Working with multi-network policy

As a cluster administrator, you can create, edit, view, and delete multi-network policies.

Prerequisites

You have enabled multi-network policy support for your cluster.

Creating a multi-network policy using the CLI

To define granular rules describing ingress or egress network traffic allowed for namespaces in your cluster, you can create a multi-network policy.

Prerequisites

Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin, with mode: NetworkPolicy set.
You installed the OpenShift CLI (oc).
You are logged in to the cluster with a user with cluster-admin privileges.
You are working in the namespace that the multi-network policy applies to.

Procedure

Create a policy rule:

Create a <policy_name>.yaml file:
```
$ touch <policy_name>.yaml
```
where:

<policy_name>

Specifies the multi-network policy file name.

Define a multi-network policy in the file that you just created, such as in the following examples:

Deny ingress from all pods in all namespaces

This is a fundamental policy, blocking all cross-pod networking other than cross-pod traffic allowed by the configuration of other Network Policies.

apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy
metadata:
  name: deny-by-default
  annotations:
    k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name>
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  ingress: []

where:

<network_name>: Specifies the name of a network attachment definition.

Allow ingress from all pods in the same namespace

apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy
metadata:
  name: allow-same-namespace
  annotations:
    k8s.v1.cni.cncf.io/policy-for: <network_name>
spec:
  podSelector:
  ingress:
  - from:
    - podSelector: {}

where:

<network_name>: Specifies the name of a network attachment definition.

Allow ingress traffic to one pod from a particular namespace

This policy allows traffic to pods labelled pod-a from pods running in namespace-y.

apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy
metadata:
  name: allow-traffic-pod
  annotations:
    k8s.v1.cni.cncf.io/policy-for: <network_name>
spec:
  podSelector:
   matchLabels:
      pod: pod-a
  policyTypes:
  - Ingress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
           kubernetes.io/metadata.name: namespace-y

where:

<network_name>: Specifies the name of a network attachment definition.

Restrict traffic to a service

This policy when applied ensures every pod with both labels app=bookstore and role=api can only be accessed by pods with label app=bookstore. In this example the application could be a REST API server, marked with labels app=bookstore and role=api.

This example addresses the following use cases:

Restricting the traffic to a service to only the other microservices that need to use it.

Restricting the connections to a database to only permit the application using it.

apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy
metadata:
  name: api-allow
  annotations:
    k8s.v1.cni.cncf.io/policy-for: <network_name>
spec:
  podSelector:
    matchLabels:
      app: bookstore
      role: api
  ingress:
  - from:
      - podSelector:
          matchLabels:
            app: bookstore

where:

<network_name>: Specifies the name of a network attachment definition.

To create the multi-network policy object, enter the following command:
```
$ oc apply -f <policy_name>.yaml -n <namespace>
```
where:

<policy_name>

Specifies the multi-network policy file name.

<namespace>

Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.
Example output
```
multinetworkpolicy.k8s.cni.cncf.io/deny-by-default created
```

If you log in to the web console with cluster-admin privileges, you have a choice of creating a network policy in any namespace in the cluster directly in YAML or from a form in the web console.

Editing a multi-network policy

You can edit a multi-network policy in a namespace.

Prerequisites

Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin, with mode: NetworkPolicy set.
You installed the OpenShift CLI (oc).
You are logged in to the cluster with a user with cluster-admin privileges.
You are working in the namespace where the multi-network policy exists.

Procedure

Optional: To list the multi-network policy objects in a namespace, enter the following command:
```
$ oc get multi-networkpolicy
```
where:

<namespace>

Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.
Edit the multi-network policy object.
- If you saved the multi-network policy definition in a file, edit the file and make any necessary changes, and then enter the following command.
  $ oc apply -n <namespace> -f <policy_file>.yaml
  where:
  
  <namespace>
  
  Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.
  
  <policy_file>
  
  Specifies the name of the file containing the network policy.
- If you need to update the multi-network policy object directly, enter the following command:
  $ oc edit multi-networkpolicy <policy_name> -n <namespace>
  where:
  
  <policy_name>
  
  Specifies the name of the network policy.
  
  <namespace>
  
  Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.
Confirm that the multi-network policy object is updated.
```
$ oc describe multi-networkpolicy <policy_name> -n <namespace>
```
where:

<policy_name>

Specifies the name of the multi-network policy.

<namespace>

Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.

If you log in to the web console with cluster-admin privileges, you have a choice of editing a network policy in any namespace in the cluster directly in YAML or from the policy in the web console through the Actions menu.

Viewing multi-network policies using the CLI

You can examine the multi-network policies in a namespace.

Prerequisites

You installed the OpenShift CLI (oc).
You are logged in to the cluster with a user with cluster-admin privileges.
You are working in the namespace where the multi-network policy exists.

Procedure

List multi-network policies in a namespace:
- To view multi-network policy objects defined in a namespace, enter the following command:
  $ oc get multi-networkpolicy
- Optional: To examine a specific multi-network policy, enter the following command:
  $ oc describe multi-networkpolicy <policy_name> -n <namespace>
  where:
  
  <policy_name>
  
  Specifies the name of the multi-network policy to inspect.
  
  <namespace>
  
  Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.

If you log in to the web console with cluster-admin privileges, you have a choice of viewing a network policy in any namespace in the cluster directly in YAML or from a form in the web console.

Deleting a multi-network policy using the CLI

You can delete a multi-network policy in a namespace.

Prerequisites

Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin, with mode: NetworkPolicy set.
You installed the OpenShift CLI (oc).
You are logged in to the cluster with a user with cluster-admin privileges.
You are working in the namespace where the multi-network policy exists.

Procedure

To delete a multi-network policy object, enter the following command:
```
$ oc delete multi-networkpolicy <policy_name> -n <namespace>
```
where:

<policy_name>

Specifies the name of the multi-network policy.

<namespace>

Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.
Example output
```
multinetworkpolicy.k8s.cni.cncf.io/default-deny deleted
```

If you log in to the web console with cluster-admin privileges, you have a choice of deleting a network policy in any namespace in the cluster directly in YAML or from the policy in the web console through the Actions menu.

Creating a default deny all multi-network policy

This is a fundamental policy, blocking all cross-pod networking other than network traffic allowed by the configuration of other deployed network policies. This procedure enforces a default deny-by-default policy.

If you log in with a user with the cluster-admin role, then you can create a network policy in any namespace in the cluster.

Prerequisites

Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin, with mode: NetworkPolicy set.
You installed the OpenShift CLI (oc).
You are logged in to the cluster with a user with cluster-admin privileges.
You are working in the namespace that the multi-network policy applies to.

Procedure

Create the following YAML that defines a deny-by-default policy to deny ingress from all pods in all namespaces. Save the YAML in the deny-by-default.yaml file:

apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy
metadata:
  name: deny-by-default
  namespace: default (1)
  annotations:
    k8s.v1.cni.cncf.io/policy-for: <namespace_name>/<network_name> (2)
spec:
  podSelector: {} (3)
  policyTypes: (4)
  - Ingress (5)
  ingress: [] (6)

1	`namespace: default` deploys this policy to the `default` namespace.
2	`network_name`: specifies the name of a network attachment definition.
3	`podSelector:` is empty, this means it matches all the pods. Therefore, the policy applies to all pods in the default namespace.
4	`policyTypes:` a list of rule types that the `NetworkPolicy` relates to.
5	Specifies as `Ingress` only `policyType`.
6	There are no `ingress` rules specified. This causes incoming traffic to be dropped to all pods.

Apply the policy by entering the following command:

$ oc apply -f deny-by-default.yaml

Example output

multinetworkpolicy.k8s.cni.cncf.io/deny-by-default created

Creating a multi-network policy to allow traffic from external clients

With the deny-by-default policy in place you can proceed to configure a policy that allows traffic from external clients to a pod with the label app=web.

If you log in with a user with the cluster-admin role, then you can create a network policy in any namespace in the cluster.

Follow this procedure to configure a policy that allows external service from the public Internet directly or by using a Load Balancer to access the pod. Traffic is only allowed to a pod with the label app=web.

Prerequisites

Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin, with mode: NetworkPolicy set.
You installed the OpenShift CLI (oc).
You are logged in to the cluster with a user with cluster-admin privileges.
You are working in the namespace that the multi-network policy applies to.

Procedure

Create a policy that allows traffic from the public Internet directly or by using a load balancer to access the pod. Save the YAML in the web-allow-external.yaml file:

apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy
metadata:
  name: web-allow-external
  namespace: default
  annotations:
    k8s.v1.cni.cncf.io/policy-for: <network_name>
spec:
  policyTypes:
  - Ingress
  podSelector:
    matchLabels:
      app: web
  ingress:
    - {}

Apply the policy by entering the following command:
```
$ oc apply -f web-allow-external.yaml
```
Example output
```
multinetworkpolicy.k8s.cni.cncf.io/web-allow-external created
```
This policy allows traffic from all resources, including external traffic as illustrated in the following diagram:

Creating a multi-network policy allowing traffic to an application from all namespaces

If you log in with a user with the cluster-admin role, then you can create a network policy in any namespace in the cluster.

Follow this procedure to configure a policy that allows traffic from all pods in all namespaces to a particular application.

Prerequisites

Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin, with mode: NetworkPolicy set.
You installed the OpenShift CLI (oc).
You are logged in to the cluster with a user with cluster-admin privileges.
You are working in the namespace that the multi-network policy applies to.

Procedure

Create a policy that allows traffic from all pods in all namespaces to a particular application. Save the YAML in the web-allow-all-namespaces.yaml file:
```
apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy
metadata:
  name: web-allow-all-namespaces
  namespace: default
  annotations:
    k8s.v1.cni.cncf.io/policy-for: <network_name>
spec:
  podSelector:
    matchLabels:
      app: web (1)
  policyTypes:
  - Ingress
  ingress:
  - from:
    - namespaceSelector: {} (2)
```
1 Applies the policy only to app:web pods in default namespace.

2 Selects all pods in all namespaces.

By default, if you omit specifying a namespaceSelector it does not select any namespaces, which means the policy allows traffic only from the namespace the network policy is deployed to.

Apply the policy by entering the following command:

$ oc apply -f web-allow-all-namespaces.yaml

Example output

multinetworkpolicy.k8s.cni.cncf.io/web-allow-all-namespaces created

Verification

Start a web service in the default namespace by entering the following command:

$ oc run web --namespace=default --image=nginx --labels="app=web" --expose --port=80

Run the following command to deploy an alpine image in the secondary namespace and to start a shell:
```
$ oc run test-$RANDOM --namespace=secondary --rm -i -t --image=alpine -- sh
```

Run the following command in the shell and observe that the request is allowed:

# wget -qO- --timeout=2 http://web.default

Expected output

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Creating a multi-network policy allowing traffic to an application from a namespace

If you log in with a user with the cluster-admin role, then you can create a network policy in any namespace in the cluster.

Follow this procedure to configure a policy that allows traffic to a pod with the label app=web from a particular namespace. You might want to do this to:

Restrict traffic to a production database only to namespaces where production workloads are deployed.
Enable monitoring tools deployed to a particular namespace to scrape metrics from the current namespace.

Prerequisites

Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin, with mode: NetworkPolicy set.
You installed the OpenShift CLI (oc).
You are logged in to the cluster with a user with cluster-admin privileges.
You are working in the namespace that the multi-network policy applies to.

Procedure

Create a policy that allows traffic from all pods in a particular namespaces with a label purpose=production. Save the YAML in the web-allow-prod.yaml file:

apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy
metadata:
  name: web-allow-prod
  namespace: default
  annotations:
    k8s.v1.cni.cncf.io/policy-for: <network_name>
spec:
  podSelector:
    matchLabels:
      app: web (1)
  policyTypes:
  - Ingress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          purpose: production (2)

1	Applies the policy only to `app:web` pods in the default namespace.
2	Restricts traffic to only pods in namespaces that have the label `purpose=production`.

Apply the policy by entering the following command:

$ oc apply -f web-allow-prod.yaml

Example output

multinetworkpolicy.k8s.cni.cncf.io/web-allow-prod created

Verification

Start a web service in the default namespace by entering the following command:

$ oc run web --namespace=default --image=nginx --labels="app=web" --expose --port=80

Run the following command to create the prod namespace:
```
$ oc create namespace prod
```
Run the following command to label the prod namespace:
```
$ oc label namespace/prod purpose=production
```
Run the following command to create the dev namespace:
```
$ oc create namespace dev
```
Run the following command to label the dev namespace:
```
$ oc label namespace/dev purpose=testing
```
Run the following command to deploy an alpine image in the dev namespace and to start a shell:
```
$ oc run test-$RANDOM --namespace=dev --rm -i -t --image=alpine -- sh
```
Run the following command in the shell and observe that the request is blocked:
```
# wget -qO- --timeout=2 http://web.default
```
Expected output
```
wget: download timed out
```
Run the following command to deploy an alpine image in the prod namespace and start a shell:
```
$ oc run test-$RANDOM --namespace=prod --rm -i -t --image=alpine -- sh
```

Run the following command in the shell and observe that the request is allowed:

# wget -qO- --timeout=2 http://web.default

Expected output

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

1	Applies the policy only to `app:web` pods in default namespace.
2	Selects all pods in all namespaces.

Configuring multi-network policy

Differences between multi-network policy and network policy

Enabling multi-network policy for the cluster

Supporting multi-network policies in IPv6 networks

Working with multi-network policy

Prerequisites

Creating a multi-network policy using the CLI

Editing a multi-network policy

Viewing multi-network policies using the CLI

Deleting a multi-network policy using the CLI

Creating a default deny all multi-network policy

Creating a multi-network policy to allow traffic from external clients

Creating a multi-network policy allowing traffic to an application from all namespaces

Creating a multi-network policy allowing traffic to an application from a namespace

Additional resources