IP address pool types

Copy link

A RHOSO deployment uses four types of IP address pools that must not overlap.

MetalLB IPAddressPool for VIPs

The MetalLB Operator uses these IP addresses to expose internal service endpoints on the isolated networks. Each RHOSO service that requires network access gets a VIP from the appropriate network’s IPAddressPool. You configure these pools in the IPAddressPool custom resource (CR).

A typical deployment requires approximately 10-15 VIP addresses per isolated network, depending on which services you deploy. Services such as keystone, nova, neutron, cinder, glance, placement, and heat each require VIPs on the Internal API network.

Note

These services can share VIPs if their target ports are different.

The dnsmasq service typically requires a VIP on the ctlplane network. Additional services such as designate might require VIPs on other networks.

NetworkAttachmentDefinition IPAM ranges

The Whereabouts CNI IPAM plugin uses these IP addresses to assign to control plane service pods that are attached to the isolated networks. You configure these ranges in the NetworkAttachmentDefinition (net-attach-def) CR for each network.

The number of IP addresses required depends on the number of replicas for each service and the pod churn rate in your environment. A typical deployment allocates 40-50 IP addresses per network to accommodate multiple pod replicas and allow for pod recreation during updates or failures. Also, networks can have unique requirements that require pods to connect to them, which require additional IP addresses.

NetConfig allocation range for data plane nodes

The data plane network configuration uses these IP addresses to assign static IP addresses to data plane Compute, Networker, and Storage nodes. You configure these ranges in the NetConfig CR for each network.

The number of IP addresses required equals the number of data plane nodes that connect to each network. For example, if you have 100 compute nodes, you need at least 100 IP addresses per network attached to the nodes or node sets.

NodeNetworkConfigurationPolicy (NNCP) ranges for RHOCP worker nodes

The NMState Operator uses these IP addresses to assign static IP addresses to the Red Hat OpenShift Container Platform (RHOCP) worker node interfaces for the isolated networks. You configure these IP addresses in the NodeNetworkConfigurationPolicy (nncp) CR for each worker node.

The number of IP addresses required equals the number of RHOCP worker nodes in your cluster. Each worker node requires one IP address per isolated network that requires worker node connectivity. For example, if you have 3 worker nodes, you need at least 3 IP addresses from the NNCP range for each applicable network, such as internalapi, storage, and tenant. A typical deployment allocates 10-15 IP addresses per network to allow for additional worker nodes.

Note

Not all isolated networks require NNCP configuration. Networks such as external and octavia typically do not require static IP addresses on the RHOCP worker nodes.

Sizing considerations

Copy link

When planning your IP address ranges, consider these factors:

• Minimum subnet size: Use at least a /24 (256 addresses) subnet for each isolated network to provide sufficient addresses for all three pool types and allow for growth.
• Service pod replicas: High-availability deployments run multiple replicas of each service, increasing the number of IP addresses required in the NetworkAttachmentDefinition IPAM ranges.
• Pod recreation: During updates or failures, new pods might be created before old pods are terminated, temporarily requiring additional IP addresses from the NetworkAttachmentDefinition IPAM ranges.
• Future growth: Plan for additional compute nodes, storage nodes, and services that you might deploy later.
• Non-overlapping ranges: Ensure that the pool types do not overlap. A common practice is to allocate:
  • Data plane node addresses at the upper end of the subnet (for example, .100-.250)
  • VIP addresses in the higher range (for example, .80-.90)
  • Control plane pod addresses in the middle range (for example, .30-.70)
  • Node network configuration policy (NNCPs) addresses at the lower end (for example, .10-.20)

Example IP address allocation

Copy link

The following example shows how to allocate IP addresses from a 172.17.0.0/24 subnet for the internalapi network:

• Data plane nodes (NetConfig): 172.17.0.100 - 172.17.0.250 (151 addresses)
• VIPs (IPAddressPool): 172.17.0.80 - 172.17.0.90 (11 addresses)
• Control plane pods (net-attach-def): 172.17.0.30 - 172.17.0.70 (41 addresses)
• RHOCP worker nodes (nncp): 172.17.0.10 - 172.17.0.20 (11 addresses)

NICs and OVN gateways

Copy link

A deployment can have OVN gateways for external network traffic. You can deploy OVN gateways on the control plane or the data plane.

If deploy gateways on the control plane, gateway requires a dedicated NIC on a control plane node.

You can also deploy OVN gateways on dedicated Networker nodes on the data plane. Each Networker node has at least one NIC to connect the Networker node to the control plane and the external network.

When you run gateway services on the control plane, OVN version updates might cause some disruption of gateway traffic, because OpenShift requires service restarts for OVN updates. Data plane nodes, controlled by RHEL, do not require service restart after an OVN version updates.

NICs and scaling considerations

Copy link

Network requirements vary based on environment and business requirements. For example, you might require the following networking capabilities:

• Dedicated NICs on control plane nodes for particular RHOSP isolated networks.
• Port switches with VLANs for the required isolated networks.

Consult with your RHOCP and network administrators about whether these are requirements in your deployment. Each Compute node requires at least one NIC. You can scale up to provide connections to the isolated networks.