OpenShift Container Platform 4.14 release notes

See Identifying pod security violations for information about how to find which workloads are causing pod security violations.

See Security context constraint synchronization with pod security standards to understand when pod security admission label synchronization is performed. Pod security admission labels are not synchronized in certain situations, such as the following situations:

If necessary, you can set a custom admission profile on the namespace or pod by setting the pod-security.kubernetes.io/enforce label.

Updating Go-based Operator projects

Updating Ansible-based Operator projects

Updating Helm-based Operator projects

Updating Hybrid Helm-based Operator projects

Updating Java-based Operator projects

General Availability

Deprecated

Removed

Previously, when creating a pod controller with a pod spec that would be mutated by security context constraints, users might get a warning that the pod did not meet the given namespace’s pod security level. With this release, you no longer get a warning about pod security violations if the pod controller will create pods that do not violate pod security in that namespace. (OCPBUGS-7267)

A user:check-access scoped token grants sufficient permissions to send a SelfSubjectAccessReview request. Previously, the cluster did not grant sufficient permissions to perform the access review unless the token also had the user:full scope or a role scope. With this release, the cluster authorizes a SelfSubjectAccessReview request as if it has either the full user’s permissions or the permissions of the user’s role set on the request in order to be able to perform the access review. (OCPBUGS-7415)

Previously, the pod security admission controller required the RoleBinding object’s .subject[].namespace field to be set when .subjects[].kind is set to ServiceAccount in order to successfully bind the service account to a role. With this release, the pod security admission controller uses the namespace of the RoleBinding object if the .subject[].namespace is not specified. (OCPBUGS-160)

Previously, the clientConfig of all the webhooks of ValidatingWebhookConfiguration and MutatingWebhookConfiguration objects did not get a properly injected caBundle with the service-ca trust bundle. With this release, the clientConfig of all the webhooks of ValidatingWebhookConfiguration and MutatingWebhookConfiguration objects now get a properly injected caBundle with the service-ca trust bundle. (OCPBUGS-19318)

Previously, kube-apiserver did not change to Degraded=True when an invalid secret name was specified for servingCertificate in namedCertificates. With this release, kube-apiserver now switches to Degraded=True and shows why the certificate was not accepted to allow for easier troubleshooting. (OCPBUGS-8404)

Previously, observability dashboards used large queries to show data which caused frequent timeouts on clusters with a large number of nodes. With this release, observability dashboards use recording rules that are precalculated to ensure reliability on clusters with a large number of nodes. (OCPBUGS-3986)

Previously, if the hostname of a bare-metal machine was not provided by either reverse DNS or DHCP, it would default to localhost during bare-metal cluster provisioning on installer-provisioned infrastructure. This issue caused Kubernetes node name conflicts and prevented the cluster from being deployed. Now, if the hostname is detected to be localhost, the provisioning agent sets the persistent hostname to the name of the BareMetalHost object. (OCPBUGS-9072)

Previously, the Machine API controller could not determine the zone of machines in vSphere clusters that use multiple zones. With this release, the zone lookup logic is based on the host of a VM and, as a result, machine objects indicate proper zones. (OCPBUGS-7249)

Previously, after the rotation of cloud credentials in the clouds.yaml file, the OpenStack machine API provider would need to be restarted in order to pick up the new cloud credentials. As a result, the ability of a machine set to scale to zero could be affected. With this change, cloud credentials are no longer cached, and the provider reads the corresponding secret freshly as needed. (OCPBUGS-8687)

Previously, some conditions during the startup process of the Cluster Autoscaler Operator caused a lock that prevented the Operator from successfully starting and marking itself as available. As a result, the cluster became degraded. The issue is resolved with this release. (OCPBUGS-20038)

Previously, the bootstrap credentials used to request client credentials for control plane nodes did not include the generic, all service accounts group. As a result, the cluster machine approver ignored certificate signing requests (CSRs) created during this phase. In certain conditions, this prevented approval of CSRs during bootstrap and caused the installation to fail. With this release, the bootstrap credential includes the groups that the cluster machine approver expects for a service account. This change allows the machine approver to take over from the bootstrap CSR approver earlier in the cluster lifecycle and should reduce bootstrap failures related to CSR approval. (OCPBUGS-8349)

Previously, if scaling the machines on a Nutanix cluster exceeded the available memory to complete the operation, machines would get stuck in the Provisioning state and could not be scaled up or down. The issue is resolved in this release. (OCPBUGS-19731)

Previously, for clusters on which the Control Plane Machine Set Operator is configured to use the OnDelete update strategy, cached information about machines caused the Operator to balance machines incorrectly and place them in an unexpected failure domain during reconciliation. With this release, the Operator refreshes this information immediately before creating new machines so that it correctly identifies the failure domains to place machines in. (OCPBUGS-15338)

Previously, the Control Plane Machine Set Operator used the Infrastructure object specification to determine the platform type for the cluster. For clusters upgraded from OpenShift Container Platform version 4.5 and earlier, this practice meant that the Operator could not correctly determine that a cluster was running on AWS, and therefore did not generate the ControlPlaneMachineSet custom resource (CR) as expected. With this release, the Operator uses the status platform type, which is populated on all clusters independent of when they were created and is now able to generate the ControlPlaneMachineSet CR for all clusters. (OCPBUGS-11389)

Previously, machines created by a control plane machine set were considered ready once the underlying Machine API machine was running. With this release, the machine is not considered ready until the node linked to that machine is also ready. (OCPBUGS-7989)

Previously, the Control Plane Machine Set Operator prioritized failure domains alphabetically and moved machines from alphabetically later failure domains to alphabetically earlier failure domains, even if doing so did not improve the availability of the machines across the failure domains. With this release, the Operator is updated to prioritize failure domains that are present in the existing machines and to respect existing failure domains that provide better availability. (OCPBUGS-7921)

Previously, when a control plane machine on a vSphere cluster that uses a control plane machine set was deleted, sometimes two replacement machines were created. With this release, the control plane machine set no longer causes an extra machine to be created. (OCPBUGS-7516)

Previously, when the availability zone and subnet ID in a machine set were mismatched, a machine was created successfully by using the machine set specification with no indication to the user of the mismatch. Because the mismatched values can cause problems with some configurations, this occurrence might be visible as a warning message. With this release, a warning about the mismatch is logged. (OCPBUGS-6882)

Previously, when creating an OpenShift Container Platform cluster on Nutanix that uses Dynamic Host Configuration Protocol (DHCP) instead of an IP address management (IPAM) network configuration, the hostname of the VM was not set by DHCP. With this release, the VM hostname is set with values from the ignition configuration files. As a result, the issue is resolved for DHCP as well as other network configuration types. (OCPBUGS-6727)

Previously, multiple clusters could be created in the openshift-cluster-api namespace. This namespace must contain only one cluster. With this release, additional clusters cannot be created in this namespace. (OCPBUGS-4147)

Previously, clearing some parameters from the providerSpec field of a control plane machine set custom resource caused a loop of control plane machine deletion and creation. With this release, these parameters receive a default value if they are cleared or left empty, which resolves the issue. (OCPBUGS-2960)

Previously, the Cloud Credential Operator utility (ccoctl) used an incorrect Amazon Resource Names (ARN) prefix for AWS GovCloud (US) and AWS China regions. The incorrect ARN prefix caused the ccoctl aws create-all command that is used to create AWS resources during installation to fail. This release updates the ARN prefixes to the correct values. (OCPBUGS-13549)

Previously, security changes to Amazon S3 buckets caused the Cloud Credential Operator utility (ccoctl) command that is used to create AWS resources during installation (ccoctl aws create-all) to fail. With this release, the ccoctl utility is updated to reflect the Amazon S3 security changes. (OCPBUGS-11671)

Previously, the Cluster Version Operator (CVO) did not reconcile SecurityContextConstraints resources as expected. The CVO now properly reconciles SecurityContextConstraints resources towards the state defined in the release image, reverting any unsupported modifications to them.

Previously, the Cluster Version Operator did not prioritize likely targets when determining which conditional update risks to evaluate first. Now for conditional updates to which risks do not apply, these updates are available faster after Cluster Version Operator detection. . (OCPBUGS-5469)

Previously, if you tried to edit a Helm chart repository in the Developer console by navigating to Helm, clicking the Repositories tab, then selecting Edit HelmChartRepository through the menu for your Helm chart repository, an Error page displayed a 404: Page Not Found error. This was caused by a component path that was not up to date. This issue is now fixed. (OCPBUGS-14660)

Previously, distinguishing between the types of samples listed in the Samples page was difficult. With this fix, you can easily identify the sample type from the badges displayed on the Samples page. (OCPBUGS-7446)

Previously on the Pipeline Metrics page, only four legends were visible for TaskRun duration charts. With this update, you can see all the legends present for the TaskRun duration charts. (OCPBUGS-19878)

Previously, an issue occurred when creating an application by using the Import JAR form in a disconnected cluster with the Cluster Samples Operator not installed. With this update, the Import JAR form from the Add page and the Topology page is hidden when the Java Builder Image is absent. (OCPBUGS-15011)

Previously, the Operator backed catalog did not show any catalog items if cluster service version (CSV) copies were disabled. With this fix, Operator backed catalogs are shown in every namespace even if CSV copies are disabled. (OCPBUGS-14907)

Previously, in the Import from Git and Deploy Image flows, the Resource Type section was moved to Advanced section. As a result, it was difficult to identify the type of resource created. With this fix, the Resource Type section is moved to the General section. (OCPBUGS-7395)

Previously, the etcdctl binary was cached on the local machine indefinitely, making updates to the binary impossible. The binary is now properly updated on every invocation of the cluster-backup.sh script. (OCPBUGS-19499)

Previously, if you did not specify a custom Red Hat Enterprise Linux CoreOS (RHCOS) Amazon Machine Image (AMI) when installing an AWS cluster to a supported secret partition, the installation failed. With this update, the installation program validates that you have specified the ID of an RHCOS AMI in the installation configuration file before deploying the cluster. (OCPBUGS-13636)

Previously, the OpenShift Container Platform installation program did not find private hosted zones in the host project during installations on Google Cloud Platform (GCP) by using a shared VPC. With this update, the installation program checks for an existing private hosted zone in the host project and uses the private hosted zone if it exists. (OCPBUGS-11736)

Previously, if you configured user-defined outbound routing when installing a private Azure cluster, the cluster was incorrectly deployed with the default public load balancer. This behavior occurred when using the installer-provisioned infrastructure to install the cluster. With this update, the installation program no longer creates the public load balancer when user-defined routing is configured. (OCPBUGS-9404)

Previously, for clusters that run on RHOSP, in the deprovisioning phase of installation, the installer deleted object storage containers sequentially. This behavior caused slow and inefficient deletion of objects, especially with large containers. This problem occurred in part because image streams that use Swift containers accumulated objects over time. Now, bulk object deletion occurs concurrently with up to 3 calls to the RHOSP API, improving efficiency by handling a higher object count per call. This optimization speeds up resource cleanup during deprovisioning. (OCPBUGS-9081)

Previously, SSH access to bootstrap and cluster nodes failed when the bastion host ran in the same VPC network as the cluster nodes. Additionally, this configuration caused SSH access from the temporary bootstrap node to the cluster nodes to fail. These issues are now fixed by updating the IBM Cloud SecurityGroupRules to support SSH traffic between the temporary bootstrap node and cluster nodes, and to support SSH traffic from a bastion host to cluster nodes on the same VPC network. Log and debug information can be accurately collected for analysis during installer-provisioned infrastructure failure.(OCPBUGS-8035)

Previously, DNS records that the installation program created were not removed when uninstalling a private cluster. With this update, the installation program now correctly removes these DNS records. (OCPBUGS-7973)

Previously, a script provided in the documentation for checking invalid HTTPS certificates in the RHOSP API assumed a recent version of the RHOSP client. For users who did not have a recent version of the client, this script failed. Now, manual instructions are added to the documentation that users can follow to perform the check with any version of the client. (OCPBUGS-7954)

Previously, when defining static IP addresses in the agent-config.yaml or nmstateconfig.yaml files for the configuration of an Agent-based install, the configured static IP addresses might not have been configured during bootstrap. As a result, the host interfaces would choose an address through DHCP. With this update, timing issues are fixed to ensure that the configured static IP address is correctly applied to the host interface. (OCPBUGS-16219)

Previously, during an Agent-based installation, the certificates in the AdditionalTrustBundle field of the install-config.yaml file were only propagated to the final image when the ImageContentSources field was also set for mirroring. If mirroring was not set, the additional certificates were on the bootstrap but not the final image. This situation can cause issues when you have set up a proxy and want to add additional certificates as described in Configuring the cluster-wide proxy during installation. With this update, these additional certificates are propagated to the final image whether or not the ImageContentSources field is also set. (OCPBUGS-13535)

Previously, the openshift-install agent create command did not return the help output when running an invalid command. With this update, the help output is now shown when you run an invalid openshift-install agent create command. (OCPBUGS-10638)

Previously, primary networks were not correctly set for generated machines that used Technology Preview failure domains. As a consequence, port targets with the ID control-plane were not set as the primary network on machines, which could cause installations that use Kuryr to function improperly. The field is now set to use the proper port target, if set. The primary network for generated machines is now set correctly, allowing installations that use Kuryr to complete. (OCPBUGS-10570)

Previously, when running the openshift-install agent create image command while using a releaseImage that contained a digest, the command produced the following warning message: WARNING The ImageContentSources configuration in install-config.yaml should have at-least one source field matching the releaseImage. This message was produced every time, regardless of how ImageContentSources was configured, and could cause confusion. With this update, the warning message is only produced when ImageContentSources is legitimately not set to have at least one source field matching the release image. (OCPBUGS-10207)

Previously, when running the openshift-install agent create image command to generate a bootable ISO image, the command output provided a message indicating a successful generated image. This output message existed even if the Agent-based installer could not extract a base ISO image from the release image. With this update, the command output now produces an error message if the Agent-based Installer cannot locate the base ISO image, which might be indicative of an issue with releaseImage. (OCPBUGS-9949)

Previously, shared VPC installations on GCP that used passthrough credentials mode could fail because the installation program used credentials from the default service account. With this update, you can specify another service account to use for node creation instead of the default. (OCPBUGS-15421)

Previously, if you defined more control plane nodes than compute nodes in either the agent-config.yaml or the nmstateconfig.yaml configuration file, you received a warning message. Now, if you specify this configuration in either file, you receive an error message, which indicates that compute nodes cannot exceed control plane nodes in either file. (OCPBUGS-14877)

Previously, an Agent-based installation would fail if a non-canonical IPv6 address was used for the RendezvousIP field in the agent-config.yaml file. Non-canonical IPv6 addresses contain leading zeros, for example, 2001:0db8:0000:0000:0000:0000:0000:0000. With this update, these valid addresses can now be used for the RendezvousIP. (OCPBUGS-14121)

Previously, the Operator cached the cloud credentials, which resulted in authentication issues when these credentials were rotated. Now, the Operator always uses the latest credentials. The Manila CSI Driver Operator now automatically creates an OpenShift storage class for each available Manila share type. As part of this operation, the Operator queries the Manila API. (OCPBUGS-14049)

Previously, when configuring the install-config.yaml file for use during an Agent-based installation, changing the cpuPartitioning field to a non-default value did not produce a warning to alert users that the field is ignored for Agent-based installations. With this update, changing the cpuPartitioning field causes a warning to users that the configuration does not impact the install. (OCPBUGS-13662)

Previously, installing an Azure cluster into an existing Azure Virtual Network (VNet) could fail because the installation program created a default network security group, which allowed traffic from 0.0.0.0. The failure occurred when the existing VNet had the following rule enabled in the tenant: Rule: Network Security Groups shall not allow rule with 0.0.0.0/Any Source/Destination IP Addresses - Custom Deny. With this fix, the installation program no longer creates the default network security group when installing a cluster into an existing VNet, and the installation succeeds. (OCPBUGS-11796)

During an installation, when the cluster status is installing-pending-user-action, the installation does not complete until the status is resolved. Previously, if you ran the openshift-install agent wait-for bootstrap-complete command, no indication existed of how to resolve the problem that caused this status. With this update, the command output provides a message indicating which actions must be taken to resolve the issue. (OCPBUGS-4998)

"level=info msg=Cluster has hosts requiring user input
level=debug msg=Host master-1 Expected the host to boot from disk, but it booted the installation image - please reboot and fix boot order to boot from disk QEMU_HARDDISK drive-scsi0-0-0-0 (sda, /dev/disk/by-path/pci-0000:03:00.0-scsi-0:0:0:0)
level=debug msg=Host master-2 Expected the host to boot from disk, but it booted the installation image - please reboot and fix boot order to boot from disk QEMU_HARDDISK drive-scsi0-0-0-0 (sda, /dev/disk/by-path/pci-0000:03:00.0-scsi-0:0:0:0)
level=info msg=cluster has stopped installing... working to recover installation"

Previously, the assisted-installer-controller on the installed cluster would run continuously even after the cluster had completed installation. Because assisted-service runs on the bootstrap node and not on the cloud, and because the assisted-service goes offline after the bootstrap node reboots to join the cluster, the assisted-installer-controller was unable to communicate with assisted-service to post updates and upload logs and loops. Now, the assisted-installer-controller checks the cluster installation without using assisted-service, and exits when the cluster installation is complete. (OCPBUGS-4240)

Previously, installing a cluster to the AWS Commercial Cloud Services (C2S) us-iso-east-1 region failed with an error message stating an UnsupportedOperation. With this fix, installing to this region now succeeds. (OCPBUGS-2324)

Previously, installations on AWS could fail because the installation program did not create the cloud.conf file with the necessary service endpoints in it. This led to the machine config operator creating an empty cloud.conf file that lacked the service endpoints, leading to an error. With this update, the installation program always creates the cloud.conf file so that the installation succeeds. (OCPBUGS-20401)

Previously, if you installed a cluster using the Agent-based installer and your pull secret had a null auth or email field, the installation would fail without providing a useful error. With this update, the openshift-install agent wait-for install-complete command validates your pull secret and notifies you if there are null fields. (OCPBUGS-14405)

Previously, the create agent-config-template command printed a line with INFO only, but no details about whether the command was successful and where the template file was written to. Now, if the command is successful, the command will print INFO Created Agent Config Template in <path> directory. (OCPBUGS-13408)

Previously, when a user specified the vendor hint in the agent-config.yaml file, the value was checked against the wrong field so that the hint would not match. With this update, the use of the vendor hint correctly selects a disk. (OCPBUGS-13356)

Previously, setting the metadataService.authentication field to Required when installing a cluster on AWS did not configure the bootstrap VM to use IMDSv2 authentication. This could result in installations failing if you configured your AWS account to block IMDSv1 authentication. With this update, the metadataService.authentication field correctly configures the bootstrap VM to use IMDSv2 authentication when set to Required. (OCPBUGS-12964)

Previously, if you configured user-defined outbound routing when installing a private Azure cluster, the cluster was incorrectly deployed with the default public load balancer. This behavior occurred when using the installer-provisioned infrastructure to install the cluster. With this update, the installation program no longer creates the public load balancer when user-defined routing is configured. (OCPBUGS-9404)

Previously, the vSphere Terraform vsphere_virtual_machine resource did not include the firmware parameter. This issue caused the firmware of the VM to be set to bios by default instead of efi. Now, the resource includes the firmware parameter and sets efi as the default value for the parameter, so that the VM runs the Extensible Firmware Interface (EFI) instead of the basic input/output system (BIOS) interface. (OCPBUGS-9378)

Previously, for clusters that run on RHOSP, in the deprovisioning phase of installation, the installer deleted object storage containers sequentially. This behavior caused slow and inefficient deletion of objects, especially with large containers. This problem occurred in part because image streams that use Swift containers accumulated objects over time. Now, bulk object deletion now occurs concurrently with up to 3 calls to the RHOSP API, improving efficiency by handling a higher object count per call. This optimization speeds up resource cleanup during deprovisioning. (OCPBUGS-9081)

Previously, the installation program did not exit with an error if you installed a cluster on Azure by using disk encryption without providing a subscription ID. This caused the installation to begin, and then only to fail later on. With this update, the installation program requires you to specify a subscription ID for encrypted Azure installations and exits with an error if you do not provide one. (OCPBUGS-8449)

Previously, the Agent-based installer showed the results of secondary checks such as ping and nslookup, which can harmlessly fail even when the installation succeeds. This could result in errors being displayed despite the cluster installing successfully. With this update, secondary checks only display results if the primary installation checks fail, so that you can use the secondary checks to troubleshoot the failed installation. (OCPBUGS-8390)

Using an IPI install-config with the Agent-based Installer results in warning log messages showing the contents of any unused fields. Previously, these warnings printed sensitive information such as passwords. With this update, the warning messages for the credentials fields in the vsphere and baremetal platform sections have been changed to avoid logging any sensitive information. (OCPBUGS-8203)

Previously, clusters on Azure Stack Hub could not create new control plane nodes unless the nodes had custom disk sizes, because the default disk size could not be validated. With this update, the default disk size has been set to 128 GB and the installation program enforces user-specified disk size values between 128 and 1023 GB. (OCPBUGS-6759)

Previously, the installation program used port 80 to provide images to the Baseboard Management Controller (BMC) and the deployment agent when installing on bare metal with installer-provisioned infrastructure. This could present security concerns because many types of public traffic use port 80. With this update, the installation program uses port 6180 for this purpose. (OCPBUGS-8509)

Previously, OpenShift Container Platform clusters that were installed on AWS used 4.1 boot images that were not able to scale up. This issue occurred because two systemd units, configured from Ignition then rendered and launched by the MCO during the initial boot of a new machine, have a dependency on the application Afterburn. Because OpenShift Container Platform 4.1 boot images do not contain Afterburn, this issue prevented new nodes from being able to join the cluster. Now, systemd units contain an additional check for Afterburn along with fallback code that does not rely on the presence of Afterburn. (OCPBUGS-7559)

Previously, alerts loaded from non-Prometheus datasources such as logs. This caused the source of all alerts to be displayed always as Prometheus. With this update, alert sources are displayed correctly. (OCPBUGS-9907)

Previously, there was an issue with Patternfly 4 where you could not select or change the log component under the logs section of master node once a selection was already made. With this update, when you change to the log component from the log section of the master node, refresh the page to reload the default options. (OCPBUGS-18727)

Previously, an empty page was displayed when viewing route details on the Metrics tab of the alertmanager-main page. With this update, user privileges were updated so you can view the route details on the Metrics tab. (OCPBUGS-15021)

Previously, Red Hat OpenShift Service on AWS used custom branding and the favicon would disappear so no specific branding appeared when custom branding was being used. With this update, Red Hat OpenShift Service on AWS branding is now part of the branding API. (OCPBUGS-14716)

Previously, the OpenShift Container Platform web console did not render the monitoring Dashboard page when a proxy was expected. As a result, the websocket connection failed. With this update, the web console also detects proxy settings from environment variables. (OCPBUGS-14550)

Previously, if the console.openshift.io/disable-operand delete: "true" and operator.openshift.io/uninstall-message: "some message" annotations were used on an operator CSV, the uninstall instructions did not show up in the web console. With this update, the instructions to opt out of the installment are available. (OCPBUGS-13782)

Previously, the size on the PersistentVolumeClaims namespace Details page was incorrect. With this update, the Prometheus query on PersistentVolumeClaims namespace Details page includes the namespace label and the size is now correct. (OCPBUGS-13208)

Previously, after customizing the routes for console and downloads, the downloads route did not update in the ConsoleCLIDownloads link and pointed to the default downloads route. With this update, the ConsoleCLIDownloads link updates when the custom downloads route is set. (OCPBUGS-12990)

Previously, the print preview displayed incomplete topology information from the list view. With this update, a full list of resources is printed when they are longer than one page. (OCPBUGS-11219)

Previously, dynamic plugins that proxy to services with longer response times timed out at 30 seconds with a 504 error message. With this update, a 5-minute HAProxy timeout annotation was added to the console route to match the maximum timeout of most browsers. (OCPBUGS-9917)

Previously, the provided API page used the displayName of the provided API, but this value was not always set. As a result, the list was empty but you could still click all instances to get to the YAML of a new instance. With this update, if the displayName is not set, the list displays text. (OCPBUGS-8682)

Previously, the CronJobs table and details view did not have a suspend indication. With this update, spec.suspend was added to the list and details view for CronJobs. (OCPBUGS-8299)

Previously, when enabling a single plugin in the configuration of the console operator, the redeployed console fails. With this update, the list of plugins is now unique and pods run as expected. (OCPBUGS-5059)

Previously, after upgrading a plugin image, old plugin files were still requested. With this update, the ?cacheBuster=${getRandomChars()} query string was added when plugin-entry.js resources are requested. (OCPBUGS-3495)

Before this update, large amounts of CPU resources might be consumed during metrics scraping as a result of the way the node-exporter collected network interface information. This release fixes this issue by improving the performance of node-exporter when collecting network interface information, thereby resolving the issue with excessive CPU usage during metrics scraping. (OCPBUGS-12714)

Before this update, Thanos Querier failed to de-duplicate metrics by node roles. This update fixes the issue so that Thanos Querier now properly de-duplicates metrics by node roles. (OCPBUGS-12525)

Before this update, the btrfs collector of node-exporter was always enabled, which caused increased CPU usage because Red Hat Enterprise Linux (RHEL) does not support the btrfs storage format. With this update, the btrfs collector is now disabled, thereby resolving the issue. (OCPBUGS-11434)

Before this update, for the cluster:capacity_cpu_cores:sum metric, nodes with the infra role but not master role were not assigned a value of infra for the label_node_role_kubernetes_io label. With this update, nodes with the infra role, but not master role, are now correctly labeled as infra for this metric. (OCPBUGS-10387)

Before this update, the lack of a startup probe prevented the Prometheus Adapter pods from starting when the Kubernetes API had many custom resource definitions installed because the program initialization would take longer than what was allowed by the liveness probe. With this update, the Prometheus Adapter pods are now configured with a startup probe that waits five minutes before failing, thereby resolving the issue. (OCPBUGS-7694)

The node_exporter collector is meant to collect network interface metrics for physical interfaces only, but before this update, the node-exporter collector did not exclude Calico Virtual network interface controllers (NICs) when collecting these metrics. This update adds the cali[a-f0-9]* value to the collector.netclass.ignored-devices list to ensure that metrics are not collected for Calico Virtual NICs. (OCPBUGS-7282)

With this release, as a security measure, Cross Origin Resource Sharing (CORS) headers are now disabled by default for Thanos Querier. If you still need to use CORS headers, you can enable them by setting the value of the enableCORS parameter to true for the ThanosQuerierConfig resource. (OCPBUGS-11889)

Previously, when a client mutual TLS (mTLS) was configured on an ingress controller, and the certificate authority (CA) certificates in the CA bundle required more than 1 MB of certificate revocation lists (CRL) to be downloaded, the CRL config map could not be updated due to size limitations. Because of the missing CRLs, connections with valid client certificates might have been rejected with the following error: unknown ca.

Previously, a non-compliant upstream DNS server that provided a UDP response larger than OpenShift Container Platform’s specified buffer size of 512 bytes caused CoreDNS to throw an overflow error. Consequently, it would not provide a response to a DNS query.

Previously, if cluster administrators configured an infra node using a taint with the NoExecute effect, the Ingress Operator’s canary pods would not be scheduled on these infra nodes. After some time, the DaemonSet configuration would get overridden, and the pods would be terminated on the infra nodes.

Previously, when a client mutual TLS (mTLS) was configured on an ingress controller, if any of the client certificate authority (CA) certificates included a certificate revocation list (CRL) distribution point for a CRL issued by a different CA and that CRL expired, the mismatch between the distributing CA and the issuing CA caused the incorrect CRL to be downloaded. Consequently, the CRL bundle would be updated to contain an extra copy of the erroneously downloaded CRL, and the CRL that needed to be updated would be missing. Because of the missing CRL, connections with valid client certificates might have been rejected with the following error: unknown ca.

Previously, when the Gateway API was enabled for Red Hat OpenShift Service Mesh, the Ingress Operator would fail to configure and would return the following error: the spec.techPreview.controlPlaneMode field is not supported in version 2.4+; use spec.mode. With this release, the Service Mesh spec.techPreview.controlPlaneMode API field in the ServiceMeshControlPlane custom resource (CR) has been replaced with spec.mode. As a result, the Ingress Operator is able to create a ServiceMeshControlPlane custom resource, and the Gateway API works properly. (OCPBUGS-10714)

Previously, when configuring DNS for Gateway API gateways, the Ingress Operator would attempt to create a DNS record for a gateway listener, even if the listener specified a hostname with a domain that was outside of the cluster’s base domain. Consequently, the Ingress Operator attempted, and failed, to publish DNS records, and would return the following error: failed to publish DNS record to zone.

Previously, the oc explain route.spec.tls.insecureEdgeTerminationPolicy command documented the incorrect possible options that could be confusing to some users. With this release, the API documentation has been updated so that it shows the correct possible options for the insecureEdgeTerminationPolicy field. This is an API documentation fix only. (OCPBUGS-11393)

Previously, a Cluster Network Operator controller monitored a broader set of resources than necessary, which resulted in its reconciler being triggered too often. Consequently, this increased the loads on both the Cluster Network Operator and the kube-apiserver.

segfault failures that were related to HaProxy have been resolved. Users should no longer receive these errors. (OCPBUGS-11595)

Previously, CoreDNS terminated unexpectedly if a user created an EndpointSlice port without a port number. With this update, validation was added to CoreDNS to prevent it from unexpectedly terminated. (OCPBUGS-19805)

Previously, the OpenShift router directed traffic to a route with a weight of 0 when it had only one back-end service. With this update, the router no longer sends traffic to routes with a single backend with weight 0. (OCPBUGS-16623)

Previously, the Ingress Operator created its canary route without specifying the spec.subdomain or the spec.host parameter on the route. Usually, this caused the API server to use the cluster’s Ingress domain, which matches the domain of the default Ingress Controller, to set a default value for the spec.host parameter. However, if you configured the cluster by using the appsDomain option to set an alternative Ingress domain, the route host would have the alternative domain. Further, if you deleted the canary route, the route would be recreated with a domain that did not match the default Ingress Controller’s domain, which would cause canary checks to fail. Now, the Ingress Controller specifies the spec.subdomain parameter when it creates the canary route. If you use the appsDomain option to configure your cluster and then delete the canary route, the canary checks do not fail. (OCPBUGS-16089)

Previously, the Ingress Operator did not check status of DNS records in public hosted zones when updating the Operator status. This caused the Ingress Operator to report the DNS status as Ready when there could be errors in DNS records in public hosted zones. Now, the Ingress Operator checks the status of both public and private hosted zones, which fixes the issue. (OCPBUGS-15978)

Previously, the CoreDNS bufsize setting was configured as 512 bytes. Now, the maximum size of the buffer for OpenShift Container Platform CoreDNS is 1232 bytes. This modification enhances DNS performance by reducing the occurrence of DNS truncations and retries. (OCPBUGS-15605)

Previously, the Ingress Operator would specify the spec.template.spec.hostNetwork: true parameter on a router deployment without specifying the spec.template.spec.containers[].ports[].hostPort. This caused the API server to set a default value for each port’s hostPort field, which the Ingress Operator would then detect as an external update and attempt to revert it. Now, the Ingress Operator no longer incorrectly performs these updates. (OCPBUGS-14995)

Previously, the DNS Operator logged the cluster-dns-operator startup has an error message: [controller-runtime] log.SetLogger(…​) was never called, logs will not be displayed: error message on startup, which could mislead users. Now, the error message is not displayed on startup. (OCPBUGS-14395)

Previously, the Ingress Operator was leaving the spec.internalTrafficPolicy, spec.ipFamilies, and spec.ipFamilyPolicy fields unspecified for NodePort and ClusterIP type services. The API would then set default values for these fields, which the Ingress Operator would try to revert. With this update, the Ingress Operator specifies an initial value and fixes the error caused by API default values. (OCPBUGS-13190)

Previously, transmission control protocol (TCP) connections were load balanced for all DNS. With this update, TCP connections are enabled to prefer local DNS endpoints. (OCPBUGS-9985)

Previously, for Intel E810 NICs, resetting a MAC address on an SR-IOV with a virtual function (VF) when a pod was deleted caused a failure. This resulted in a long delay when creating a pod with SR-IOV VF. With this update, the container network interface (CNI) does not fail fixing this issue. (OCPBUGS-5892)

Previously, an issue was observed in OpenShift Container Platform with some pods getting stuck in the terminating state. This affected the reconciliation loop of the allowlist controller, which resulted in unwanted retries that caused the creation of multiple pods. With this update, the allowlist controller only inspects pods that belong to the current daemon set. As a result, retries no longer occur when one or more pods are not ready. (OCPBUGS-16019)

Previously, container image references that have both tag and digest were not correctly interpreted by the oc-mirror plug-in and resulted in the following error:

"localhost:6000/cp/cpd/postgresql:13.7@sha256" is not a valid image reference: invalid reference format

Previously, you were receiving 401 - Unauthorized error for registries where the number of path components exceeded the expected maximum path components. This issue is fixed by ensuring that the oc-mirror fails when the number of path components exceeds maximum path components. You can now set the maximum path components by using the flag --max-nested-paths, which accepts an integer value. By default, there is no limit to the maximum path components and is set to 0. The generated ImageContentSourcePolicy will contain source and mirror references up to the repository level. (OCPBUGS-8111, OCPBUGS-11910, OCPBUGS-11922)

Previously, the oc-mirror flags --short, -v, and --verbose provided incorrect version information. You can now use the oc mirror version flag to know the correct version of oc-mirror. The oc-mirror flags --short, -v, and --verbose have been deprecated and will no longer be supported. (OCPBUGS-7845)

Previously, mirroring from registry to disk would fail when several digests of an image were specified in the imageSetConfig without tags. The oc-mirror would add the default tag latest to the images. The issue is now fixed by using a truncated digest as the tag. (OCPBUGS-2633)

Previously, oc-mirror would incorrectly add the Operator catalog to ImageContentSourcePolicy specification. This is an unexpected behavior because the Operator catalog is directly used from the destination registry through CatalogSource resource. This bug is fixed by ensuring that the oc-mirror does not add the Operator catalog as an entry to ImageContentSourcePolicy. (OCPBUGS-10051)

Previously, mirroring images for Operators would fail when the registry domain name was not a part of the image reference. With this fix, the images are downloaded from docker.io if the registry domain name is not specified.(OCPBUGS-10348)

Previously, when both tag and digest were included in container image references, oc-mirror would incorrectly interpret it resulting in an invalid reference format error. This issue has been fixed and the images are successfully mirrored. (OCPBUGS-11840)

Previously, you could not create a CatalogSource resource if the name started with a number. With this fix, by default, the CatalogSource resource name is generated with the cs- prefix and is compliant with RFC 1035. (OCPBUGS-13332)

Previously, when using the registries.conf file, some images were not included in the mapping. With this bug fix, you can now see the images included in the mapping without any errors. (OCPBUGS-13962)

Previously, while using the insecure mirrors in the registries.conf file that is referenced in --oci-registries-config flag, oc-mirror tried to establish an HTTPS connection with the mirror registry. With this fix, you can configure oc-mirror to not use an HTTPS connection by specifying either --source-skip-tls or --source-use-http in the command line. (OCPBUGS-14402)

Previously, image mirroring would fail when you attempted to mirror OCI indexes by using oc-mirror plugins. With this fix, you can mirror OCI indexes by using oc-mirror plugins. (OCPBUGS-15329)

Previously, when mirroring several large catalogs on a low-bandwidth network, mirroring would be interrupted due to an expired authentication token resulting in an HTTP 401 unauthorized error. This issue is now fixed by refreshing the authentication tokens before starting the mirroring process of each catalog. (OCPBUGS-20137)

Before this update, Operator Lifecycle Manager (OLM) could cause failed installations due to initialization errors when the API server was busy. This update fixes the issue by adding a one-minute-retry interval for initialization errors. (OCPBUGS-13128)

Before this update, a race condition occurred if custom catalogs used the same names as the default Red Hat catalogs in a disconnected environment. If the default Red Hat catalogs were disabled, the catalogs were created at start and deleted after the OperatorHub custom resource (CR) was reconciled. As a result, the custom catalogs were deleted along with the default Red Hat catalogs. With this update, the OperatorHub CR is reconciled before any catalogs are deleted, preventing the race condition. (OCPBUGS-9357)

Before this update, the channels of some Operators were displayed on OperatorHub in a random order. With this update, Operator channels are displayed in lexicographical order. (OCPBUGS-7910)

Before this update, registry pods were not drained gracefully by the autoscaler if the controller flag was not set to true in the owner references file. With this update, the controller flag is set to true and draining nodes no longer requires a forceful shutdown. (OCPBUGS-7431)

Before this update, collect-profiles pods caused regular spikes of CPU usage due to the way certificates were generated. With this update, certificates are generated daily, the loading of the certificate is optimized, and CPU usage is lower. (OCPBUGS-1684)

Previously, the metadata.namespace field would be automatically populated in update and patch requests to the projects resource. As a result, the affected requests would generate spurious validation errors. With this release, the projects resource is no longer automatically populated. (OCPBUGS-8232)

Previously, pods in OpenShift Container Platform that access block persistent volume claims (PVC) storage with logical volume manager (LVM) metadata could get stuck when terminating. This is because the same LVM devices were active both inside the container and on the host. An example of this occurred when running a virtual machine inside a pod that used OpenShift Virtualization that in turn used LVM for the virtual machine. With this update, RHCOS by default only attempts to setup and access devices that are in the /etc/lvm/devices/system.devices file. This prevents contentious access to the LVM devices inside the virtual machine guests. (OCPBUGS-5223)

Previously, pods were stuck in the ContainerCreating state on Google Cloud Platform (GCP) Confidential Computing instances, which caused a volume mount failure. This fix adds support for the Persistent Disk storage type for Confidential Computing instances in Google Cloud Platform, which can be used as persistent volumes in OpenShift Container Platform. As a result, pods are able to enter a Running state and volumes can be mounted. (OCPBUGS-7582)

Previously, when the cluster-wide proxy is enabled on IBM Cloud® clusters, there was a failure to provision volumes. (OCPBUGS-18142)

The vsphereStorageDriver field of the Storage Operator object has been deprecated. This field was used to opt in to CSI migration on OpenShift Container Platform 4.13 vSphere clusters, but it has no effect on OpenShift Container Platform 4.14 and newer clusters. (OCPBUGS-13914)

Technology Preview

General Availability

Not Available

Deprecated