$ sudo openstack quota set --secgroups 250 --secgroup-rules 1000 --ports 1500 --subnets 250 --networks 250 <project>
install-config.yaml
file for RHOSP with KuryrIn OpenShift Container Platform version 4.11, you can install a cluster on Red Hat OpenStack Platform (RHOSP) that runs on user-provisioned infrastructure.
Using your own infrastructure allows you to integrate your cluster with existing infrastructure and modifications. The process requires more labor on your part than installer-provisioned installations, because you must create all RHOSP resources, like Nova servers, Neutron ports, and security groups. However, Red Hat provides Ansible playbooks to help you in the deployment process.
You reviewed details about the OpenShift Container Platform installation and update processes.
You read the documentation on selecting a cluster installation method and preparing it for users.
You verified that OpenShift Container Platform 4.11 is compatible with your RHOSP version by using the Supported platforms for OpenShift clusters section. You can also compare platform support across different versions by viewing the OpenShift Container Platform on RHOSP support matrix.
You have an RHOSP account where you want to install OpenShift Container Platform.
On the machine from which you run the installation program, you have:
A single directory in which you can keep the files you create during the installation process
Python 3
Kuryr is a container network interface (CNI) plugin solution that uses the Neutron and Octavia Red Hat OpenStack Platform (RHOSP) services to provide networking for pods and Services.
Kuryr and OpenShift Container Platform integration is primarily designed for OpenShift Container Platform clusters running on RHOSP VMs. Kuryr improves the network performance by plugging OpenShift Container Platform pods into RHOSP SDN. In addition, it provides interconnectivity between pods and RHOSP virtual instances.
Kuryr components are installed as pods in OpenShift Container Platform using the
openshift-kuryr
namespace:
kuryr-controller
- a single service instance installed on a master
node.
This is modeled in OpenShift Container Platform as a Deployment
object.
kuryr-cni
- a container installing and configuring Kuryr as a CNI driver on
each OpenShift Container Platform node. This is modeled in OpenShift Container Platform as a DaemonSet
object.
The Kuryr controller watches the OpenShift Container Platform API server for pod, service, and namespace create, update, and delete events. It maps the OpenShift Container Platform API calls to corresponding objects in Neutron and Octavia. This means that every network solution that implements the Neutron trunk port functionality can be used to back OpenShift Container Platform via Kuryr. This includes open source solutions such as Open vSwitch (OVS) and Open Virtual Network (OVN) as well as Neutron-compatible commercial SDNs.
Kuryr is recommended for OpenShift Container Platform deployments on encapsulated RHOSP tenant networks to avoid double encapsulation, such as running an encapsulated OpenShift Container Platform SDN over an RHOSP network.
If you use provider networks or tenant VLANs, you do not need to use Kuryr to avoid double encapsulation. The performance benefit is negligible. Depending on your configuration, though, using Kuryr to avoid having two overlays might still be beneficial.
Kuryr is not recommended in deployments where all of the following criteria are true:
The RHOSP version is less than 16.
The deployment uses UDP services, or a large number of TCP services on few hypervisors.
or
The ovn-octavia
Octavia driver is disabled.
The deployment uses a large number of TCP services on few hypervisors.
When using Kuryr SDN, the pods, services, namespaces, and network policies are using resources from the RHOSP quota; this increases the minimum requirements. Kuryr also has some additional requirements on top of what a default install requires.
Use the following quota to satisfy a default cluster’s minimum requirements:
Resource | Value |
---|---|
Floating IP addresses |
3 - plus the expected number of Services of LoadBalancer type |
Ports |
1500 - 1 needed per Pod |
Routers |
1 |
Subnets |
250 - 1 needed per Namespace/Project |
Networks |
250 - 1 needed per Namespace/Project |
RAM |
112 GB |
vCPUs |
28 |
Volume storage |
275 GB |
Instances |
7 |
Security groups |
250 - 1 needed per Service and per NetworkPolicy |
Security group rules |
1000 |
Server groups |
2 - plus 1 for each additional availability zone in each machine pool |
Load balancers |
100 - 1 needed per Service |
Load balancer listeners |
500 - 1 needed per Service-exposed port |
Load balancer pools |
500 - 1 needed per Service-exposed port |
A cluster might function with fewer than recommended resources, but its performance is not guaranteed.
If RHOSP object storage (Swift) is available and operated by a user account with the |
If you are using Red Hat OpenStack Platform (RHOSP) version 16 with the Amphora driver rather than the OVN Octavia driver, security groups are associated with service accounts instead of user projects. |
Take the following notes into consideration when setting resources:
The number of ports that are required is larger than the number of pods. Kuryr uses ports pools to have pre-created ports ready to be used by pods and speed up the pods' booting time.
Each network policy is mapped into an RHOSP security group, and
depending on the NetworkPolicy
spec, one or more rules are added to the
security group.
Each service is mapped to an RHOSP load balancer. Consider this requirement when estimating the number of security groups required for the quota.
If you are using
RHOSP version 15 or earlier, or the ovn-octavia driver
, each load balancer
has a security group with the user project.
The quota does not account for load balancer resources (such as VM resources), but you must consider these resources when you decide the RHOSP deployment’s size. The default installation will have more than 50 load balancers; the clusters must be able to accommodate them.
If you are using RHOSP version 16 with the OVN Octavia driver enabled, only one load balancer VM is generated; services are load balanced through OVN flows.
An OpenShift Container Platform deployment comprises control plane machines, compute machines, and a bootstrap machine.
To enable Kuryr SDN, your environment must meet the following requirements:
Run RHOSP 13+.
Have Overcloud with Octavia.
Use Neutron Trunk ports extension.
Use openvswitch
firewall driver if ML2/OVS Neutron driver is used instead
of ovs-hybrid
.
When using Kuryr SDN, you must increase quotas to satisfy the Red Hat OpenStack Platform (RHOSP) resources used by pods, services, namespaces, and network policies.
Increase the quotas for a project by running the following command:
$ sudo openstack quota set --secgroups 250 --secgroup-rules 1000 --ports 1500 --subnets 250 --networks 250 <project>
Kuryr CNI leverages the Neutron Trunks extension to plug containers into the
Red Hat OpenStack Platform (RHOSP) SDN, so you must use the trunks
extension for Kuryr to properly work.
In addition, if you leverage the default ML2/OVS Neutron driver, the firewall
must be set to openvswitch
instead of ovs_hybrid
so that security groups are
enforced on trunk subports and Kuryr can properly handle network policies.
Kuryr SDN uses Red Hat OpenStack Platform (RHOSP)'s Octavia LBaaS to implement OpenShift Container Platform services. Thus, you must install and configure Octavia components in RHOSP to use Kuryr SDN.
To enable Octavia, you must include the Octavia service during the installation of the RHOSP Overcloud, or upgrade the Octavia service if the Overcloud already exists. The following steps for enabling Octavia apply to both a clean install of the Overcloud or an Overcloud update.
The following steps only capture the key pieces required during the deployment of RHOSP when dealing with Octavia. It is also important to note that registry methods vary. This example uses the local registry method. |
If you are using the local registry, create a template to upload the images to the registry. For example:
(undercloud) $ openstack overcloud container image prepare \
-e /usr/share/openstack-tripleo-heat-templates/environments/services-docker/octavia.yaml \
--namespace=registry.access.redhat.com/rhosp13 \
--push-destination=<local-ip-from-undercloud.conf>:8787 \
--prefix=openstack- \
--tag-from-label {version}-{product-version} \
--output-env-file=/home/stack/templates/overcloud_images.yaml \
--output-images-file /home/stack/local_registry_images.yaml
Verify that the local_registry_images.yaml
file contains the Octavia images.
For example:
...
- imagename: registry.access.redhat.com/rhosp13/openstack-octavia-api:13.0-43
push_destination: <local-ip-from-undercloud.conf>:8787
- imagename: registry.access.redhat.com/rhosp13/openstack-octavia-health-manager:13.0-45
push_destination: <local-ip-from-undercloud.conf>:8787
- imagename: registry.access.redhat.com/rhosp13/openstack-octavia-housekeeping:13.0-45
push_destination: <local-ip-from-undercloud.conf>:8787
- imagename: registry.access.redhat.com/rhosp13/openstack-octavia-worker:13.0-44
push_destination: <local-ip-from-undercloud.conf>:8787