OpenShift docs are moving and will soon only be available at docs.redhat.com, the home of all Red Hat product documentation. Explore the new docs experience today.
  1. Documentation
    2. history
×

Updating node network configuration

You can update the node network configuration, such as adding or removing interfaces from nodes, by applying NodeNetworkConfigurationPolicy manifests to the cluster.

About nmstate

Container-native virtualization uses nmstate to report on and configure the state of the node network. This makes it possible to modify network policy configuration, such as by creating a Linux bridge on all nodes, by applying a single configuration manifest to the cluster.

Node networking is monitored and updated by the following objects:

NodeNetworkState

Reports the state of the network on that node.

NodeNetworkConfigurationPolicy

Describes the requested network configuration on nodes. You update the node network configuration, including adding and removing interfaces, by applying a NodeNetworkConfigurationPolicy manifest to the cluster.

NodeNetworkConfigurationEnactment

Reports the network policies enacted upon each node.

Creating an interface on nodes

Create an interface on nodes in the cluster by applying a NodeNetworkConfigurationPolicy manifest to the cluster. The manifest details the requested configuration for the interface.

By default, the manifest applies to all nodes in the cluster. To add the interface to specific nodes, add the spec: nodeSelector parameter and the appropriate <key>:<value> for your node selector.

Procedure

  1. Create the NodeNetworkConfigurationPolicy manifest. The following example configures a Linux bridge on all worker nodes:

    apiVersion: nmstate.io/v1alpha1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: <br1-eth1-policy> (1)
spec:
  nodeSelector: (2)
    node-role.kubernetes.io/worker: "" (3)
  desiredState:
    interfaces:
      - name: br1
        description: Linux bridge with eth1 as a port (4)
        type: linux-bridge
        state: up
        ipv4:
          dhcp: true
          enabled: true
        bridge:
          options:
            stp:
              enabled: false
          port:
            - name: eth1
    1 Name of the Policy.
    2 Optional. If you do not include the nodeSelector, the Policy applies to all nodes in the cluster.
    3 This example uses the node-role.kubernetes.io/worker: "" node selector to select all worker nodes in the cluster.
    4 Optional. Human-readable description for the interface.

  2. Create the Policy:

    $ oc apply -f <br1-eth1-policy.yaml> (1)
    1 File name of the Policy manifest.

Confirming Policy updates on nodes

A NodeNetworkConfigurationPolicy manifest describes your requested network configuration for nodes in the cluster. The Policy object includes your requestd network configuration and the status of execution of the Policy on the cluster as a whole.

When you apply a Policy, a NodeNetworkConfigurationEnactment is created for every node in the cluster. The Enactment is a read-only object that represents the status of execution of the Policy on that node. If the Policy fails to be applied on the node, the Enactment for that node includes a traceback for troubleshooting.

Procedure

  1. To confirm that a Policy has been applied to the cluster, list the Policies and their status:

    $ oc get nncp

  2. (Optional) If a Policy is taking longer than expected to successfully configure, you can inspect the requested state and status conditions of a particular Policy:

    $ oc get nncp <policy> -o yaml

  3. (Optional) If a policy is taking longer than expected to successfully configure on all nodes, you can list the status of the Enactments on the cluster:

    $ oc get nnce

  4. (Optional) To view the configuration of a particular Enactment, including any error reporting for a failed configuration:

    $ oc get nnce <node>.<policy> -o yaml

Removing an interface from nodes

Remove an interface from nodes by editing the NodeNetworkConfigurationPolicy object and set the state of the interface to absent.

Deleting the Policy that added an interface does not change the configuration of the network policy on the node. Although a NodeNetworkConfigurationPolicy is an object in the cluster, it only represents the requested configuration.
Similarly, removing an interface does not delete the Policy.
Procedure

  1. Update the NodeNetworkConfigurationPolicy manifest used to create the interface. The following example removes a Linux bridge:

    apiVersion: nmstate.io/v1alpha1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: <br1-eth1-policy> (1)
spec:
  nodeSelector: (2)
    node-role.kubernetes.io/worker: "" (3)
  desiredState:
    interfaces:
      - name: br1
        type: linux-bridge
        state: absent (4)
    1 Name of the Policy.
    2 Optional. If you do not include the nodeSelector, the Policy applies to all nodes in the cluster.
    3 This example uses the node-role.kubernetes.io/worker: "" node selector to select all worker nodes in the cluster.
    4 Changing the state to absent removes the interface.

  2. Update the Policy on the node and remove the interface:

    $ oc apply -f <br1-eth1-policy.yaml> (1)
    1 File name of the Policy manifest.

Restoring node network configuration after removing an interface

Removing an interface from a node does not automatically restore the node network configuration to a previous state. After you remove an interface, any of the node NICs throughout the cluster that were previously attached or subordinate to the interface are placed in a down state. Restore the NICs by applying a new NodeNetworkConfigurationPolicy manifest to the cluster.

Procedure

  1. Create a NodeNetworkConfigurationPolicy manifest that specifies the NIC and the desired state of up:

    apiVersion: nmstate.io/v1alpha1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: eth1
spec:
  desiredState:
    interfaces:
    - name: eth1
      type: ethernet
      state: up
      ipv4:
        dhcp: true
        enabled: true

  2. Apply the manifest to the cluster:

    $ oc apply -f <eth1.yaml> (1)
    1 File name of the Policy manifest.

Example: Linux bridge interface NodeNetworkConfigurationPolicy

Create a Linux bridge interface on nodes in the cluster by applying a NodeNetworkConfigurationPolicy manifest to the cluster.

The following YAML file is an example of a manifest for a Linux bridge interface. It includes samples values that you must replace with your own information.

apiVersion: nmstate.io/v1alpha1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: br1-eth1-policy (1)
spec:
  nodeSelector: (2)
    kubernetes.io/hostname: <node01> (3)
  desiredState:
    interfaces:
      - name: br1 (4)
        description: Linux bridge with eth1 as a port (5)
        type: linux-bridge (6)
        state: up (7)
        ipv4:
          dhcp: true (8)
          enabled: true (9)
        bridge:
          options:
            stp:
              enabled: false (10)
          port:
            - name: eth1 (11)
1 Name of the Policy.
2 Optional. If you do not include the nodeSelector, the Policy applies to all nodes in the cluster.
3 This example uses a hostname node selector.
4 Name of the interface.
5 Optional. Human-readable description of the interface.
6 The type of interface. This example creates a bridge.
7 The requested state for the interface after creation.
8 Optional. If you do not use dhcp, you can either set a static IP or leave the interface without an IP address.
9 Enables ipv4 in this example.
10 Disables stp in this example.
11 The node NIC to which the bridge attaches.

Example: VLAN interface NodeNetworkConfigurationPolicy

Create a VLAN interface on nodes in the cluster by applying a NodeNetworkConfigurationPolicy manifest to the cluster.

The following YAML file is an example of a manifest for a VLAN interface. It includes samples values that you must replace with your own information.

apiVersion: nmstate.io/v1alpha1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: vlan-eth1-policy (1)
spec:
  nodeSelector: (2)
    kubernetes.io/hostname: <node01> (3)
  desiredState:
    interfaces:
    - name: eth1.102 (4)
      description: VLAN using eth1 (5)
      type: vlan (6)
      state: up (7)
      vlan:
        base-iface: eth1 (8)
        id: 102 (9)
1 Name of the Policy.
2 Optional. If you do not include the nodeSelector, the Policy applies to all nodes in the cluster.
3 This example uses a hostname node selector.
4 Name of the interface.
5 Optional. Human-readable description of the interface.
6 The type of interface. This example creates a VLAN.
7 The requested state for the interface after creation.
8 The node NIC to which the VLAN is attached.
9 The VLAN tag.

Example: Bond interface NodeNetworkConfigurationPolicy

Create a bond interface on nodes in the cluster by applying a NodeNetworkConfigurationPolicy manifest to the cluster.

Container-native virtualization only supports the following bond modes:

  • mode=1 active-backup

  • mode=5 balance-tlb

  • mode=6 balance-alb

The following YAML file is an example of a manifest for a bond interface. It includes samples values that you must replace with your own information.

apiVersion: nmstate.io/v1alpha1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: bond0-eth1-eth2-policy (1)
spec:
  nodeSelector: (2)
    kubernetes.io/hostname: <node01> (3)
  desiredState:
    interfaces:
    - name: bond0 (4)
      description: Bond enslaving eth1 and eth2 (5)
      type: bond (6)
      state: up (7)
      ipv4:
        dhcp: true (8)
        enabled: true (9)
      link-aggregation:
        mode: active-backup (10)
        options:
          miimon: '140' (11)
        slaves: (12)
        - eth1
        - eth2
      mtu: 1450 (13)
1 Name of the Policy.
2 Optional. If you do not include the nodeSelector, the Policy applies to all nodes in the cluster.
3 This example uses a hostname node selector.
4 Name of the interface.
5 Optional. Human-readable description of the interface.
6 The type of interface. This example creates a bond.
7 The requested state for the interface after creation.
8 Optional. If you do not use dhcp, you can either set a static IP or leave the interface without an IP address.
9 Enables ipv4 in this example.
10 The driver mode for the bond. This example uses an active backup mode.
11 Optional. This example uses miimon to inspect the bond link every 140ms.
12 The subordinate node NICs in the bond.
13 Optional. The maximum transmission unit (MTU) for the bond. If not specified, this value is set to 1500 by default.