$ openstack role add --user <user> --project <project> swiftoperator
In OpenShift Container Platform 4.10, you can install a cluster on Red Hat OpenStack Platform (RHOSP) in a restricted network by creating an internal mirror of the installation release content.
You reviewed details about the OpenShift Container Platform installation and update processes.
You read the documentation on selecting a cluster installation method and preparing it for users.
You verified that OpenShift Container Platform 4.10 is compatible with your RHOSP version by using the Supported platforms for OpenShift clusters section. You can also compare platform support across different versions by viewing the OpenShift Container Platform on RHOSP support matrix.
You created a registry on your mirror host and obtained the imageContentSources
data for your version of OpenShift Container Platform.
Because the installation media is on the mirror host, you can use that computer to complete all installation steps. |
You have the metadata service enabled in RHOSP.
In OpenShift Container Platform 4.10, you can perform an installation that does not require an active connection to the internet to obtain software components. Restricted network installations can be completed using installer-provisioned infrastructure or user-provisioned infrastructure, depending on the cloud platform to which you are installing the cluster.
If you choose to perform a restricted network installation on a cloud platform, you still require access to its cloud APIs. Some cloud functions, like Amazon Web Service’s Route 53 DNS and IAM services, require internet access. Depending on your network, you might require less internet access for an installation on bare metal hardware or on VMware vSphere.
To complete a restricted network installation, you must create a registry that mirrors the contents of the OpenShift Container Platform registry and contains the installation media. You can create this registry on a mirror host, which can access both the internet and your closed network, or by using other methods that meet your restrictions.
Clusters in restricted networks have the following additional limitations and restrictions:
The ClusterVersion
status includes an Unable to retrieve available updates
error.
By default, you cannot use the contents of the Developer Catalog because you cannot access the required image stream tags.
To support an OpenShift Container Platform installation, your Red Hat OpenStack Platform (RHOSP) quota must meet the following requirements:
Resource | Value |
---|---|
Floating IP addresses |
3 |
Ports |
15 |
Routers |
1 |
Subnets |
1 |
RAM |
88 GB |
vCPUs |
22 |
Volume storage |
275 GB |
Instances |
7 |
Security groups |
3 |
Security group rules |
60 |
Server groups |
2 - plus 1 for each additional availability zone in each machine pool |
A cluster might function with fewer than recommended resources, but its performance is not guaranteed.
If RHOSP object storage (Swift) is available and operated by a user account with the |
By default, your security group and security group rule quotas might be low. If you encounter problems, run openstack quota set --secgroups 3 --secgroup-rules 60 <project> as an administrator to increase them.
|
An OpenShift Container Platform deployment comprises control plane machines, compute machines, and a bootstrap machine.
By default, the OpenShift Container Platform installation process creates three control plane machines.
Each machine requires:
An instance from the RHOSP quota
A port from the RHOSP quota
A flavor with at least 16 GB memory and 4 vCPUs
At least 100 GB storage space from the RHOSP quota
By default, the OpenShift Container Platform installation process creates three compute machines.
Each machine requires:
An instance from the RHOSP quota
A port from the RHOSP quota
A flavor with at least 8 GB memory and 2 vCPUs
At least 100 GB storage space from the RHOSP quota
Compute machines host the applications that you run on OpenShift Container Platform; aim to run as many as you can. |
During installation, a bootstrap machine is temporarily provisioned to stand up the control plane. After the production control plane is ready, the bootstrap machine is deprovisioned.
The bootstrap machine requires:
An instance from the RHOSP quota
A port from the RHOSP quota
A flavor with at least 16 GB memory and 4 vCPUs
At least 100 GB storage space from the RHOSP quota
In OpenShift Container Platform 4.10, you require access to the internet to obtain the images that are necessary to install your cluster.
You must have internet access to:
Access OpenShift Cluster Manager to download the installation program and perform subscription management. If the cluster has internet access and you do not disable Telemetry, that service automatically entitles your cluster.
Access Quay.io to obtain the packages that are required to install your cluster.
Obtain the packages that are required to perform cluster updates.
If your cluster cannot have direct internet access, you can perform a restricted network installation on some types of infrastructure that you provision. During that process, you download the required content and use it to populate a mirror registry with the installation packages. With some installation types, the environment that you install your cluster in will not require internet access. Before you update the cluster, you update the content of the mirror registry. |
Swift is operated by a user account with the swiftoperator
role. Add the role to an account before you run the installation program.
If the Red Hat OpenStack Platform (RHOSP) object storage service, commonly known as Swift, is available, OpenShift Container Platform uses it as the image registry storage. If it is unavailable, the installation program relies on the RHOSP block storage service, commonly known as Cinder. If Swift is present and you want to use it, you must enable access to it. If it is not present, or if you do not want to use it, skip this section. |
You have a RHOSP administrator account on the target environment.
The Swift service is installed.
On Ceph RGW, the account in url
option is enabled.
To enable Swift on RHOSP:
As an administrator in the RHOSP CLI, add the swiftoperator
role to the account that will access Swift:
$ openstack role add --user <user> --project <project> swiftoperator
Your RHOSP deployment can now use Swift for the image registry.
The OpenShift Container Platform installation program relies on a file that is called clouds.yaml
. The file describes Red Hat OpenStack Platform (RHOSP) configuration parameters, including the project name, log in information, and authorization service URLs.
Create the clouds.yaml
file:
If your RHOSP distribution includes the Horizon web UI, generate a clouds.yaml
file in it.
Remember to add a password to the |
If your RHOSP distribution does not include the Horizon web UI, or you do not want to use Horizon, create the file yourself. For detailed information about clouds.yaml
, see Config files in the RHOSP documentation.
clouds:
shiftstack:
auth:
auth_url: http://10.10.14.42:5000/v3
project_name: shiftstack
username: shiftstack_user
password: XXX
user_domain_name: Default
project_domain_name: Default
dev-env:
region_name: RegionOne
auth:
username: 'devuser'
password: XXX
project_name: 'devonly'
auth_url: 'https://10.10.14.22:5001/v2.0'
If your RHOSP installation uses self-signed certificate authority (CA) certificates for endpoint authentication:
Copy the certificate authority file to your machine.
Add the cacerts
key to the clouds.yaml
file. The value must be an absolute, non-root-accessible path to the CA certificate:
clouds:
shiftstack:
...
cacert: "/etc/pki/ca-trust/source/anchors/ca.crt.pem"
After you run the installer with a custom CA certificate, you can update the certificate by editing the value of the
|
Place the clouds.yaml
file in one of the following locations:
The value of the OS_CLIENT_CONFIG_FILE
environment variable
The current directory
A Unix-specific user configuration directory, for example ~/.config/openstack/clouds.yaml
A Unix-specific site configuration directory, for example /etc/openstack/clouds.yaml
The installation program searches for clouds.yaml
in that order.
Optionally, you can edit the cloud provider configuration for your cluster. The cloud provider configuration controls how OpenShift Container Platform interacts with Red Hat OpenStack Platform (RHOSP).
For a complete list of cloud provider configuration parameters, see the cloud.conf
specification.
If you have not already generated manifest files for your cluster, generate them by running the following command:
$ openshift-install --dir <destination_directory> create manifests
In a text editor, open the cloud-provider configuration manifest file. For example:
$ vi openshift/manifests/cloud-provider-config.yaml
Modify the options based on the cloud.conf
specification.
Configuring Octavia for load balancing is a common case for clusters that do not use Kuryr. For example:
#...
[LoadBalancer]
use-octavia=true (1)
lb-provider = "amphora" (2)
floating-network-id="d3deb660-4190-40a3-91f1-37326fe6ec4a" (3)
create-monitor = True (4)
monitor-delay = 10s (5)
monitor-timeout = 10s (6)
monitor-max-retries = 1 (7)
#...
1 | This property enables Octavia integration. |
2 | This property sets the Octavia provider that your load balancer uses. It accepts "ovn" or "amphora" as values. If you choose to use OVN, you must also set lb-method to SOURCE_IP_PORT . |
3 | This property is required if you want to use multiple external networks with your cluster. The cloud provider creates floating IP addresses on the network that is specified here. |
4 | This property controls whether the cloud provider creates health monitors for Octavia load balancers. Set the value to True to create health monitors. As of RHOSP 16.1 and 16.2, this feature is only available for the Amphora provider. |
5 | This property sets the frequency with which endpoints are monitored. The value must be in the time.ParseDuration() format. This property is required if the value of the create-monitor property is True . |
6 | This property sets the time that monitoring requests are open before timing out. The value must be in the time.ParseDuration() format. This property is required if the value of the create-monitor property is True . |
7 | This property defines how many successful monitoring requests are required before a load balancer is marked as online. The value must be an integer. This property is required if the value of the create-monitor property is True . |
Prior to saving your changes, verify that the file is structured correctly. Clusters might fail if properties are not placed in the appropriate section. |
You must set the value of the |
For installations that use Kuryr, Kuryr handles relevant services. There is no need to configure Octavia load balancing in the cloud provider. |
Save the changes to the file and proceed with installation.
You can update your cloud provider configuration after you run the installer. On a command line, run:
After you save your changes, your cluster will take some time to reconfigure itself. The process is complete if none of your nodes have a |
Commonly, Red Hat OpenStack Platform (RHOSP) deployments disallow non-administrator users from creating specific floating IP addresses. If such a policy is in place and you use a floating IP address in your service specification, the cloud provider will fail to handle IP address assignment to load balancers.
If you use an external cloud provider, you can avoid this problem by pre-creating a floating IP address and specifying it in your service specification. The in-tree cloud provider does not support this method.
Download the Red Hat Enterprise Linux CoreOS (RHCOS) image to install OpenShift Container Platform on a restricted network Red Hat OpenStack Platform (RHOSP) environment.
Obtain the OpenShift Container Platform installation program. For a restricted network installation, the program is on your mirror registry host.
Log in to the Red Hat Customer Portal’s Product Downloads page.
Under Version, select the most recent release of OpenShift Container Platform 4.10 for RHEL 8.
The RHCOS images might not change with every release of OpenShift Container Platform. You must download images with the highest version that is less than or equal to the OpenShift Container Platform version that you install. Use the image versions that match your OpenShift Container Platform version if they are available. |
Download the Red Hat Enterprise Linux CoreOS (RHCOS) - OpenStack Image (QCOW) image.
Decompress the image.
You must decompress the image before the cluster can use it. The name of the downloaded file might not contain a compression extension, like $ file <name_of_downloaded_file> |
Upload the image that you decompressed to a location that is accessible from the bastion server, like Glance. For example:
$ openstack image create --file rhcos-44.81.202003110027-0-openstack.x86_64.qcow2 --disk-format qcow2 rhcos-${RHCOS_VERSION}
Depending on your RHOSP environment, you might be able to upload the image in either |
If the installation program finds multiple images with the same name, it chooses one of them at random. To avoid this behavior, create unique names for resources in RHOSP. |
The image is now available for a restricted installation. Note the image name or location for use in OpenShift Container Platform deployment.
You can customize the OpenShift Container Platform cluster you install on Red Hat OpenStack Platform (RHOSP).
Obtain the OpenShift Container Platform installation program and the pull secret for your cluster. For a restricted network installation, these files are on your mirror host.
Have the imageContentSources
values that were generated during mirror registry creation.
Obtain the contents of the certificate for your mirror registry.
Retrieve a Red Hat Enterprise Linux CoreOS (RHCOS) image and upload it to an accessible location.
Obtain service principal permissions at the subscription level.
Create the install-config.yaml
file.
Change to the directory that contains the installation program and run the following command:
$ ./openshift-install create install-config --dir <installation_directory> (1)
1 | For <installation_directory> , specify the directory name to store the
files that the installation program creates. |
Specify an empty directory. Some installation assets, like bootstrap X.509 certificates have short expiration intervals, so you must not reuse an installation directory. If you want to reuse individual files from another cluster installation, you can copy them into your directory. However, the file names for the installation assets might change between releases. Use caution when copying installation files from an earlier OpenShift Container Platform version. |
At the prompts, provide the configuration details for your cloud:
Optional: Select an SSH key to use to access your cluster machines.
For production OpenShift Container Platform clusters on which you want to perform installation debugging or disaster recovery, specify an SSH key that your |
Select openstack as the platform to target.
Specify the Red Hat OpenStack Platform (RHOSP) external network name to use for installing the cluster.
Specify the floating IP address to use for external access to the OpenShift API.
Specify a RHOSP flavor with at least 16 GB RAM to use for control plane nodes and 8 GB RAM for compute nodes.
Select the base domain to deploy the cluster to. All DNS records will be sub-domains of this base and will also include the cluster name.
Enter a name for your cluster. The name must be 14 or fewer characters long.
Paste the pull secret from the Red Hat OpenShift Cluster Manager.
In the install-config.yaml
file, set the value of platform.openstack.clusterOSImage
to the image location or name. For example:
platform:
openstack:
clusterOSImage: http://mirror.example.com/images/rhcos-43.81.201912131630.0-openstack.x86_64.qcow2.gz?sha256=ffebbd68e8a1f2a245ca19522c16c86f67f9ac8e4e0c1f0a812b068b16f7265d
Edit the install-config.yaml
file to give the additional information that
is required for an installation in a restricted network.
Update the pullSecret
value to contain the authentication information for
your registry:
pullSecret: '{"auths":{"<mirror_host_name>:5000": {"auth": "<credentials>","email": "you@example.com"}}}'
For <mirror_host_name>
, specify the registry domain name
that you specified in the certificate for your mirror registry, and for
<credentials>
, specify the base64-encoded user name and password for
your mirror registry.
Add the additionalTrustBundle
parameter and value.
additionalTrustBundle: |
-----BEGIN CERTIFICATE-----
ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ
-----END CERTIFICATE-----
The value must be the contents of the certificate file that you used for your mirror registry. The certificate file can be an existing, trusted certificate authority, or the self-signed certificate that you generated for the mirror registry.
Add the image content resources, which resemble the following YAML excerpt:
imageContentSources:
- mirrors:
- <mirror_host_name>:5000/<repo_name>/release
source: quay.example.com/openshift-release-dev/ocp-release
- mirrors:
- <mirror_host_name>:5000/<repo_name>/release
source: registry.example.com/ocp/release
For these values, use the imageContentSources
that you recorded during mirror registry creation.
Make any other modifications to the install-config.yaml
file that you require. You can find more information about
the available parameters in the Installation configuration parameters section.
Back up the install-config.yaml
file so that you can use
it to install multiple clusters.
The |
Production environments can deny direct access to the internet and instead have
an HTTP or HTTPS proxy available. You can configure a new OpenShift Container Platform
cluster to use a proxy by configuring the proxy settings in the
install-config.yaml
file.
Kuryr installations default to HTTP proxies. |
For Kuryr installations on restricted networks that use the Proxy
object, the proxy must be able to reply to the router that the cluster uses. To add a static route for the proxy configuration, from a command line as the root user, enter:
$ ip route add <cluster_network_cidr> via <installer_subnet_gateway>
The restricted subnet must have a gateway that is defined and available to be linked to the Router
resource that Kuryr creates.
You have an existing install-config.yaml
file.
You reviewed the sites that your cluster requires access to and determined whether any of them need to bypass the proxy. By default, all cluster egress traffic is proxied, including calls to hosting cloud provider APIs. You added sites to the Proxy
object’s spec.noProxy
field to bypass the proxy if necessary.
The For installations on Amazon Web Services (AWS), Google Cloud Platform (GCP), Microsoft Azure, and Red Hat OpenStack Platform (RHOSP), the |
Edit your install-config.yaml
file and add the proxy settings. For example:
apiVersion: v1
baseDomain: my.domain.com
proxy:
httpProxy: http://<username>:<pswd>@<ip>:<port> (1)
httpsProxy: https://<username>:<pswd>@<ip>:<port> (2)
noProxy: example.com (3)
additionalTrustBundle: | (4)
-----BEGIN CERTIFICATE-----
<MY_TRUSTED_CA_CERT>
-----END CERTIFICATE-----
...
1 | A proxy URL to use for creating HTTP connections outside the cluster. The
URL scheme must be http . |
2 | A proxy URL to use for creating HTTPS connections outside the cluster. |
3 | A comma-separated list of destination domain names, IP addresses, or
other network CIDRs to exclude from proxying. Preface a domain with . to match subdomains only. For example, .y.com matches x.y.com , but not y.com . Use * to bypass the proxy for all destinations. |
4 | If provided, the installation program generates a config map that is named user-ca-bundle in
the openshift-config namespace that contains one or more additional CA
certificates that are required for proxying HTTPS connections. The Cluster Network
Operator then creates a trusted-ca-bundle config map that merges these contents
with the Red Hat Enterprise Linux CoreOS (RHCOS) trust bundle, and this config map is referenced in the trustedCA field of the Proxy object. The additionalTrustBundle field is required unless
the proxy’s identity certificate is signed by an authority from the RHCOS trust
bundle. |
The installation program does not support the proxy |
Save the file and reference it when installing OpenShift Container Platform.
The installation program creates a cluster-wide proxy that is named cluster
that uses the proxy
settings in the provided install-config.yaml
file. If no proxy settings are
provided, a cluster
Proxy
object is still created, but it will have a nil
spec
.
Only the |
Before you deploy an OpenShift Container Platform cluster, you provide parameter values to describe your account on the cloud platform that hosts your cluster and optionally customize your cluster’s platform. When you create the install-config.yaml
installation configuration file, you provide values for the required parameters through the command line. If you customize your cluster, you can modify the install-config.yaml
file to provide more details about the platform.
After installation, you cannot modify these parameters in the |