Customize RHOCP network routes

To optimize the network configuration of your Red Hat OpenShift Container Platform (RHOCP) cluster, add routes to the Red Hat OpenShift Container Platform (RHOCP) network configuration you prepared for your Red Hat OpenStack Services on OpenShift (RHOSO) environment on your RHOCP cluster.

Extend RHOCP network connectivity with additional routes
Copy link

To add routes to the Red Hat OpenShift Container Platform (RHOCP) networks, you add the routes field to the NodeNetworkConfigurationPolicy (nncp), NetworkAttachmentDefinition (net-attach-def), and NetConfig custom resources.

Before you begin
Copy link

Procedure
Copy link

  1. Open the NodeNetworkConfigurationPolicy (nncp) CR file on your workstation, for example, openstack-nncp.yaml.
  2. Add the routes field to the nncp CR, and add configuration for each route. The following example adds routes to the nncp CR that configures the enp6s0 interface for worker node 1, osp-enp6s0-worker-1: 
    apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: osp-enp6s0-worker-1
spec:
  desiredState:
    interfaces:
      ...
    routes:
      config:
      - destination: 192.168.123.0/24
        metric: 150
        next-hop-address: 192.168.122.1
        next-hop-interface: ospbr
      - destination: 192.168.124.0/24
        metric: 150
        next-hop-address: 192.168.122.1
        next-hop-interface: ospbr
      - destination: 172.17.1.0/24
        metric: 150
        next-hop-address: 172.17.0.1
        next-hop-interface: internalapi
      - destination: 172.17.2.0/24
        metric: 150
        next-hop-address: 172.17.0.1
        next-hop-interface: internalapi
      - destination: 172.18.1.0/24
  3. Update the nncp CR in the cluster: 
    $ oc apply -f openstack-nncp.yaml
  4. Open the NetworkAttachmentDefinition (net-attach-def) CR file on your workstation, for example, o`penstack-net-attach-def.yaml`.
  5. Add the routes field to the config field for each isolated network, and add configuration for each route. The following example adds routes to the net-attach-def CR for the tenant network: 
    apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: tenant
  namespace: openstack
spec:
  config: |
    {
      "cniVersion": "0.3.1",
      "name": "tenant",
      "type": "macvlan",
      "master": "tenant",
      "ipam": {
        "type": "whereabouts",
        "range": "172.19.0.0/24",
        "range_start": "172.19.0.30",
        "range_end": "172.19.0.70",
        "routes": [
          {
            "dst": "172.19.1.0/24",
            "gw": "172.19.0.1"
          },
          {
            "dst": "172.19.2.0/24",
            "gw": "172.19.0.1"
          }
        ]
      }
    }
  6. Update the NetworkAttachmentDefinition CR in the cluster: 
    $ oc apply -f openstack-net-attach-def.yaml
  7. Open the NetConfig CR file on your workstation, for example, openstack_netconfig.yaml.
  8. Add the routes field to the networks field for each isolated network. The following example adds routes to the three subnets of the ctlplane network: 
    apiVersion: network.openstack.org/v1beta1
kind: NetConfig
metadata:
  name: openstacknetconfig
  namespace: openstack
spec:
  networks:
  - name: ctlplane
    dnsDomain: ctlplane.openstack.lab
    subnets:
    - name: subnet1
      allocationRanges:
      - end: 192.168.122.120
        start: 192.168.122.100
      - end: 192.168.122.200
        start: 192.168.122.150
      cidr: 192.168.122.0/24
      gateway: 192.168.122.1
      routes:
      - destination: 192.168.123.0/24
        nexthop: 192.168.122.1
      - destination: 192.168.124.0/24
        nexthop: 192.168.122.1
    - name: subnet2
      allocationRanges:
      - end: 192.168.123.120
        start: 192.168.123.100
      - end: 192.168.123.200
        start: 192.168.123.150
      cidr: 192.168.123.0/24
      gateway: 192.168.123.1
      routes:
      - destination: 192.168.122.0/24
        nexthop: 192.168.123.1
      - destination: 192.168.124.0/24
        nexthop: 192.168.123.1
    - name: subnet3
      allocationRanges:
      - end: 192.168.124.120
        start: 192.168.124.100
      - end: 192.168.124.200
        start: 192.168.124.150
      cidr: 192.168.124.0/24
      gateway: 192.168.124.1
      routes:
      - destination: 192.168.122.0/24
        nexthop: 192.168.124.1
      - destination: 192.168.123.0/24
        nexthop: 192.168.124.1
  9. Update the data plane network: 
    $ oc apply -f openstack_netconfig.yaml -n openstack

Update network configuration on deployed data plane nodes
Copy link

To change the gateway IP in a 'NetConfig" subnet, update the node network configuration for a deployed data plane node set and re-deploy the node sets.

About this task
Copy link

Note
Not all network configuration parameters can be updated after the data plane network is initially created. If you try to modify a parameter that can only be set on initial network creation, then an error is returned when you try to apply your changes.

Procedure
Copy link

  1. Open the NetConfig custom resource (CR) file on your workstation, for example, openstack_netconfig.yaml.
  2. Update the node network configuration in your NetConfig CR as required.
    Tip
    If you are changing the gateway IP in a NetConfig subnet, then ensure that the new gateway is reachable before updating the NetConfig CR. Also, verify that switching to the new gateway does not disrupt any running services on the nodes.
  3. Update the data plane network: 
    $ oc apply -f openstack_netconfig.yaml -n openstack
  4. Open the OpenStackDataPlaneNodeSet CR file for the node set you want to update, for example, openstack_data_plane.yaml.
  5. Set the edpm_network_config_update field to true: 
    spec:
 ...
 nodeTemplate:
   ...
   ansible:
     ...
     ansibleVars:
       ...
       edpm_network_config_update: true
  6. Apply the updated OpenStackDataPlaneNodeSet CR configuration: 
    $ oc apply -f openstack_data_plane.yaml
  7. Create an OpenStackDataPlaneDeployment CR to apply the updated network configuration to the nodes: 
    apiVersion: dataplane.openstack.org/v1beta1
kind: OpenStackDataPlaneDeployment
metadata:
  name: <node_set_deployment_name>
  namespace: openstack
spec:
  nodeSets:
  - <nodeSet_name>
  servicesOverride:
  - configure-network
    • metadata.name: The name of the OpenStackDataPlaneDeployment CR. The name must be unique, must consist of lower case alphanumeric characters, - (hyphen) or . (period), and must start and end with an alphanumeric character.
    • spec.nodeSets: Lists the OpenStackDataPlaneNodeSet CRs that contain the nodes that you are updating.
    • spec.servicesOverride: Specifies the names of the only services to run. Set to the configure-network service to update only the network configuration for the node set.
  8. Deploy the modified OpenStackDataPlaneNodeSet CR: 
    $ oc create -f openstack_data_plane_deploy.yaml -n openstack
  9. Verify that the modified OpenStackDataPlaneNodeSet CR is deployed: 
    $ oc -n openstack wait openstackdataplanedeployment data-plane-node-set --for condition=Ready --timeout=20m
  10. Log in to a data plane node to confirm that the network configuration has been applied. For example, use the following commands if you changed the gateway IP in a NetConfig subnet: 
    $ ssh <data_plane_node>
[root@data_plane_node ~]# ip r
default via 192.168.122.1 dev br-ex proto static
  11. Open the OpenStackDataPlaneNodeSet CR file for the node set you updated and reset the edpm_network_config_update field to false.
    Important
    If you do not reset the edpm_network_config_update field to false, the nodes might lose network access. Whenever edpm_network_config_update is true, the updated network configuration is reapplied every time an OpenStackDataPlaneDeployment CR is created that includes the configure-network service that is a member of the servicesOverride list.
  12. Apply the updated OpenStackDataPlaneNodeSet CR configuration: 
    $ oc apply -f openstack_data_plane.yaml