OpenShift Container Platform 4.16 release notes

"" (Default)

Standard

Slower

General Availability

Deprecated

Removed

Previously, ephemeral and csi volumes were not properly added to security context constraints (SCCs) on upgraded clusters. With this release, SCCs on upgraded clusters are properly updated to have ephemeral and csi volumes. (OCPBUGS-33522)

Previously, the ServiceAccounts resource could not be used with OAuth clients for a cluster with the ImageRegistry capability enabled. With this release, this issue is fixed. (OCPBUGS-30319)

Previously, when you created a pod with an empty security context and you have access to all security context constraints (SCCs), the pod would receive the anyuid SCC. After the ovn-controller component added a label to the pod, the pod would be re-admitted for SCC selection, where the pod received an escalated SCC, such as privileged. With this release, this issue is resolved so the pod is not re-admitted for SCC selection. (OCPBUGS-11933)

Previously, the hostmount-anyuid security context constraints (SCC) did not have a built-in cluster role because the name of the SCC was incorrectly named hostmount in the cluster role. With this release, the SCC name in the cluster role was updated properly to hostmount-anyuid, so the hostmount-anyuid SCC now has a functioning cluster role. (OCPBUGS-33184)

Previously, clusters that were created before OpenShift Container Platform 4.7 had several secrets of type SecretTypeTLS. Upon upgrading to OpenShift Container Platform 4.16, these secrets are deleted and re-created with the type kubernetes.io/tls. This removal could cause a race condition and the contents of the secrets could be lost. With this release, the secret type change now happens automatically and clusters created before OpenShift Container Platform 4.7 can upgrade to 4.16 without risking losing the contents of these secrets. (OCPBUGS-31384)

Previously, some Kubernetes API server events did not have the correct timestamps. With this release, Kubernetes API server events now have the correct timestamps. (OCPBUGS-27074)

Previously, the Kubernetes API Server Operator attempted to delete a Prometheus rule that was removed in OpenShift Container Platform 4.13 to ensure it was deleted. This resulted in failed deletion messages in the audit logs every few minutes. With this release, the Kubernetes API Server Operator no longer attempts to remove this nonexistent rule and there are no more failed deletion messages in the audit logs. (OCPBUGS-25894)

Previously, newer versions of Redfish used Manager resources to deprecate the Uniform Resource Identifier (URI) for the RedFish Virtual Media API. This caused any hardware that used the newer Redfish URI for Virtual Media to not be provisioned. With this release, the Ironic API identifies the correct Redfish URI to deploy for the RedFish Virtual Media API so that hardware relying on either the deprecated or newer URI could be provisioned. (OCPBUGS-30171)

Previously, the Bare Metal Operator (BMO) was not using a leader lock to control incoming and outgoing Operator pod traffic. After an OpenShift Deployment object included a new Operator pod, the new pod competed with system resources, such as the ClusterOperator status, and this terminated any outgoing Operator pods. This issue also impacted clusters that do not include any bare-metal nodes. With this release, the BMO includes a leader lock to manage new pod traffic, and this fix resolves the competing pod issue. (OCPBUGS-25766)

Previously, when you attempted to delete a BareMetalHost object before the installation started, the metal3 Operator attempted to create a PreprovImage image. The process of creating this image caused the BareMetalHost object to still exist in certain processes. With this release, an exception is added for this situation so that the BareMetalHost object is deleted without impacting running processes. (OCPBUGS-33048)

Previously, a Redfish virtual media in the context of Hewlett Packard Enterprise (HPE) Lights Out (iLO) 5 had its bare-metal machine compression forcibly disabled to work around other unrelated issues in different hardware models. This caused the FirmwareSchema resource to be missing from each iLO 5 bare-metal machine. Each machine needs compression to fetch message registries from their Redfish Baseboard Management Controller (BMC) endpoints. With this release, each iLO 5 bare-metal machine that needs the FirmwareSchema resource does not have compression forcibly disabled. (OCPBUGS-31104)

Previously, the inspector.ipxe configuration file used the IRONIC_IP variable, which did not account for IPv6 addresses because they have brackets. Consequently, when the user supplied an incorrect boot_mac_address, iPXE fell back to the inspector.ipxe configuration file, which supplied a malformed IPv6 host header since it did not contain brackets. With this release, the inspector.ipxe configuration file has been updated to use the IRONIC_URL_HOST variable, which accounts for IPv6 addresses and resolves the issue. (OCPBUGS-22699)

Previously, Ironic Python Agent assumed all server disks to have a 512 byte sector size when trying to wipe disks. This caused the disk wipe to fail. With this release, Ironic Python Agent checks the disk sector size and has separate values for disk wiping so that the disk wipe succeeds. (OCPBUGS-31549)

Previously, clusters that updated from earlier versions to 4.16 continued to allow builds to be triggered by unauthenticated webhooks. With this release, new clusters require build webhooks to be authenticated. Builds are not triggered by unauthenticated webhooks unless a cluster administrator allows unauthenticated webhooks in the namespace or cluster. (OCPBUGS-33378)

Previously, if the developer or cluster administrator used lowercase environment variable names for proxy information, these environment variables were carried into the build output container image. At runtime, the proxy settings were active and had to be unset. With this release, lowercase versions of the *_PROXY environment variables are prevented from leaking into built container images. Now, buildDefaults are only kept during the build and settings created for the build process only are removed before pushing the image in the registry. (OCPBUGS-34825)

Previously, the Cloud Controller Manager (CCM) Operator used predefined roles on Google Cloud Platform (GCP) instead of granular permissions. With this release, the CCM Operator is updated to use granular permissions on GCP clusters. (OCPBUGS-26479)

Previously, the installation program populated the network.devices, template and workspace fields in the spec.template.spec.providerSpec.value section of the VMware vSphere control plane machine set custom resource (CR). These fields should be set in the vSphere failure domain, and the installation program populating them caused unintended behaviors. Updating these fields did not trigger an update to the control plane machines, and these fields were cleared when the control plane machine set was deleted.

Previously, a node associated with a rebooting machine briefly having a status of Ready=Unknown triggered the UnavailableReplicas condition in the Control Plane Machine Set Operator. This condition caused the Operator to enter the Available=False state and trigger alerts because that state indicates a nonfunctional component that requires immediate administrator intervention. This alert should not have been triggered for the brief and expected unavailabilty while rebooting. With this release, a grace period for node unreadiness is added to avoid triggering unnecessary alerts. (OCPBUGS-34970)

Previously, a transient failure to fetch bootstrap data during machine creation, such as a transient failure to connect to the API server, caused the machine to enter a terminal failed state. With this release, failure to fetch bootstrap data during machine creation is retried indefinitely until it eventually succeeds. (OCPBUGS-34158)

Previously, the Machine API Operator operator panicked when deleting a server in an error state because it was not passed a port list. With this release, deleting a machine stuck in an ERROR state does not crash the controller. (OCPBUGS-34155)

Previously, an optional internal function of the cluster autoscaler caused repeated log entries when it was not implemented. The issue is resolved in this release. (OCPBUGS-33932)

Previously, if the control plane machine set was created with a template without a path during installation on a VMware vSphere cluster, the Control Plane Machine Set Operator rejected modification or deletion of the control plane machine set custom resource (CR). With this release, the Operator allows template names for vSphere in the control plane machine set definition. (OCPBUGS-32295)

Previously, the Control Plane Machine Set Operator crashed when attempting to update a VMware vSphere cluster because the infrastructure resource was not configured. With this release, the Operator can handle this scenario so that the cluster update is able to proceed. (OCPBUGS-31808)

Previously, when a user created a compute machine set with taints, they could choose to not specify the Value field. Failure to specify this field caused the cluster autoscaler to crash. With this release, the cluster autoscaler is updated to handle an empty Value field. (OCPBUGS-31421)

Previously, IPv6 services were wrongly marked as internal on the RHOSP cloud provider, making it impossible to share IPv6 load balancers between OpenShift Container Platform services. With this release, IPv6 services are not marked as internal, allowing IPv6 load balancers to be shared between services that use stateful IPv6 addresses. This fix allows load balancers to use stateful IPv6 addresses that are defined in the loadBalancerIP property of the service. (OCPBUGS-29605)

Previously, when a control plane machine was marked as unready and a change was initiated by the modifying the control plane machine set, the unready machine was removed prematurely. This premature action caused multiple indexes to be replaced simultaneously. With this release, the control plane machine set no longer deletes a machine when only a single machine exists within the index. This change prevents premature roll-out of changes and prevents more than one index from being replaced at a time. (OCPBUGS-29249)

Previously, connections to the Azure API sometimes hung for up to 16 minutes. With this release, a timeout is introduced to prevent hanging API calls. (OCPBUGS-29012)

Previously, the Machine API IBM Cloud controller did not integrate the full logging options from the klogr package. As a result, the controller crashed in Kubernetes version 1.29 and later. With this release, the missing options are included and the issue is resolved. (OCPBUGS-28965)

Previously, the Cluster API IBM Power Virtual Server controller pod would start on the unsupported IBM Cloud platform. This caused the controller pod to get stuck in the creation phase. With this release, the cluster detects the difference between IBM Power Virtual Server and IBM Cloud. The cluster then only starts on the supported platform. (OCPBUGS-28539)

Previously, the machine autoscaler could not account for any taint set directly on the compute machine set spec due to a parsing error. This could cause undesired scaling behavior when relying on a compute machine set taint to scale from zero. The issue is resolved in this release and the machine autoscaler can now scale up correctly and identify taints that prevent workloads from scheduling. (OCPBUGS-27509)

Previously, machine sets that ran on Microsoft Azure regions with no availability zone support always created AvailabilitySets objects for Spot instances. This operation caused Spot instances to fail because the instances did not support availability sets. With this release, machine sets do not create AvailabilitySets objects for Spot instances that operate in non-zonal configured regions. (OCPBUGS-25940)

Previously, the removal of code that provided image credentials from the kubelet in OpenShift Container Platform 4.14 caused pulling images from the Amazon Elastic Container Registry (ECR) to fail without a specified pull secret. This release includes a separate credential provider that provides ECR credentials for the kubelet. (OCPBUGS-25662)

Previously, the default VM type for the Azure load balancer was changed from Standard to VMSS, but the service type load balancer code could not attach standard VMs to load balancers. With this release, the default VM type is reverted to remain compatible with OpenShift Container Platform deployments. (OCPBUGS-25483)

Previously, OpenShift Container Platform did not include the cluster name in the names of the RHOSP load balancer resources that were created by the OpenStack Cloud Controller Manager. This behavior caused issues when LoadBalancer services had the same name in multiple clusters that ran in a single RHOSP project. With this release, the cluster name is included in the names of Octavia resources. When upgrading from a previous cluster version, the load balancers are renamed. The new names follow the pattern kube_service_<cluster-name>_<namespace>_<service-name> instead of kube_service_kubernetes_<namespace>_<service-name>. (OCPBUGS-13680)

Previously, when you created or deleted large volumes of service objects simultaneously, service controller ability to process each service sequentially would slow down. This caused short timeout issues for the service controller and backlog issues for the objects. With this release, the service controller can now process up to 10 service objects simultaneously to reduce the backlog and timeout issues. (OCPBUGS-13106)

Previously, the logic that fetches the name of a node did not account for the possibility of multiple values for the returned hostname from the AWS metadata service. When multiple domains are configured for a VPC Dynamic Host Configuration Protocol (DHCP) option, this hostname might return multiple values. The space between multiple values caused the logic to crash. With this release, the logic is updated to use only the first returned hostname as the node name. (OCPBUGS-10498)

Previously, the Machine API Operator requested unnecessary virtualMachines/extensions permissions on Microsoft Azure clusters. The unnecessary credentials request is removed in this release. (OCPBUGS-29956)

Previously, the Cloud Credential Operator (CCO) was missing some permissions required to create a private cluster on Microsoft Azure. These missing permissions prevented installation of an Azure private cluster using Microsoft Entra Workload ID. This release includes the missing permissions and enables installation of an Azure private cluster using Workload ID. (OCPBUGS-25193)

Previously, a bug caused the Cloud Credential Operator (CCO) to report an incorrect mode in the metrics. Even though the cluster was in the default mode, the metrics reported that it was in the credentials removed mode. This update uses a live client in place of a cached client so that it is able to obtain the root credentials, and the CCO no longer reports an incorrect mode in the metrics. (OCPBUGS-26488)

Previously, the Cloud Credential Operator credentials mode metric on an OpenShift Container Platform cluster that uses Microsoft Entra Workload ID reported using manual mode. With this release, clusters that use Workload ID are updated to report that they are using manual mode with pod identity. (OCPBUGS-27446)

Previously, creating an Amazon Web Services (AWS) root secret on a bare metal cluster caused the Cloud Credential Operator (CCO) pod to crash. The issue is resolved in this release. (OCPBUGS-28535)

Previously, removing the root credential from a Google Cloud Platform (GCP) cluster that used the Cloud Credential Operator (CCO) in mint mode caused the CCO to become degraded after approximately one hour. In a degraded state, the CCO cannot manage the component credential secrets on a cluster. The issue is resolved in this release. (OCPBUGS-28787)

Previously, the Cloud Credential Operator (CCO) checked for a nonexistent s3:HeadBucket permission during installation on Amazon Web Services (AWS). When the CCO failed to find this permission, it considered the provided credentials insufficient for mint mode. With this release, the CCO no longer checks for the nonexistent permission. (OCPBUGS-31678)

This release expands the ClusterOperatorDown and ClusterOperatorDegraded alerts to cover ClusterVersion conditions and send alerts for Available=False (ClusterOperatorDown) and Failing=True (ClusterOperatorDegraded). In previous releases, those alerts only covered ClusterOperator conditions. (OCPBUGS-9133)

Previously, Cluster Version Operator (CVO) changes that were introduced in OpenShift Container Platform 4.15.0, 4.14.0, 4.13.17, and 4.12.43 caused failing risk evaluations to block the CVO from fetching new update recommendations. When the risk evaluations failed, the bug caused the CVO to overlook the update recommendation service. With this release, the CVO continues to poll the update recommendation service, regardless of whether update risks are being successfully evaluated and the issue has been resolved. (OCPBUGS-25708)

Previously, when a serverless function was created in the create serverless form, BuilldConfig was not created. With this update, if the Pipelines Operator is not installed, the pipeline resource is not created for particular resource, or the pipeline is not added while creating a serverless function, it will create BuildConfig as expected. (OCPBUGS-34143)

Previously, after installing the Pipelines Operator, Pipeline templates took some time to become available in the cluster, but users were still able to create the deployment. With this update, the Create button on the Import from Git page is disabled if there is no pipeline template present for the resource selected. (OCPBUGS-34142)

Previously, the maximum number of nodes was set to 100 on the Topology page. A persistent alert, "Loading is taking longer than expected." was provided. With this update, the limit of nodes is increased to 300. (OCPBUGS-32307)

With this update, an alert message to notify you that Service Bindings are deprecated with OpenShift Container Platform 4.15 was added to the ServiceBinding list, ServiceBinding details, Add, and Topology pages when creating a ServiceBinding, binding a component, or a ServiceBinding was found in the current namespace. (OCPBUGS-32222)

Previously, the Helm Plugin index view did not display the same number of charts as the Helm CLI if the chart names varied. With this release, the Helm catalog now looks for charts.openshift.io/name and charts.openshift.io/provider so that all versions are grouped together in a single catalog title. (OCPBUGS-32059)

Previously, the TaskRun status was not displayed near the TaskRun name on the TaskRun details page. With this update, the TaskRun status is located beside the name of the TaskRun in the page heading. (OCPBUGS-31745)

Previously, there is an error when adding parameters to the Pipeline when the resources field was added to the payload, and as resources are deprecated. With this update, the resources fields have been removed from the payload, and you can add parameters to the Pipeline. (OCPBUGS-31082)

This release updates the OpenShift Pipelines plugin to support the latest Pipeline Trigger API version for the custom resource definitions (CRDs) ClusterTriggerBinding, TriggerTemplate and EventListener. (OCPBUGS-30958)

Previously, CustomTasks were not recognized or remained in a Pending state. With this update, CustomTasks can be easily identified from the Pipelines List and Details pages. (OCPBUGS-29513)

Previously, if there was a build output image with an Image tag then the Output Image link would not redirect to the correct ImageStream page. With this update, this has been fixed by generating a URL for the ImageStream page without adding the tag in the link. (OCPBUGS-29355)

Previously, BuildRun logs were not visible in the Logs tab of the BuildRun page due to a recent update in the API version of the specified resources. With this update, the logs of the TaskRuns were added back into the Logs tab of the BuildRun page for both v1alpha1 and v1beta1 versions of the Builds Operator. (OCPBUGS-27473)

Previously, the annotations to set scale bound values were setting to autoscaling.knative.dev/maxScale and autoscaling.knative.dev/minScale. With this update, the annotations to set scale bound values are updated to autoscaling.knative.dev/min-scale and autoscaling.knative.dev/max-scale to determine the minimum and maximum numbers of replicas that can serve an application at any given time. You can set scale bounds for an application to help prevent cold starts or control computing costs. (OCPBUGS-27469)

Previously, the Log tab for PipelineRuns from the Tekton Results API never finished loading. With this release, this tab loads fully complete for PipelineRuns loaded from the Kubernetes API or the Tekton Results API. (OCPBUGS-25612)

Previously, there was no indicator shown to differentiate between PipelineRuns that are loaded from the Kubernetes API or the Tekton Results API. With this update, a small archived icon in the PipelineRun list and details page to differentiate between PipelineRuns that are loaded from the Kubernetes API or the Tekton Results API. (OCPBUGS-25396)

Previously, on the PipelineRun list page, all TaskRuns were fetched and separated based on pipelineRun name. With this update, TaskRuns are fetched only for Failed and Cancelled PipelineRun. A caching mechanism was also added to fetch PipelineRuns and TaskRuns associated to the Failed and Cancelled PipelineRuns. (OCPBUGS-23480)

Previously, the visual connector was not present between the VMs node and other non-VMs nodes in the Topology view. With this update, the visual connector is located between VMs nodes and non-VMs nodes. (OCPBUGS-13114)

Previously, the wait-for-ceo command that was used during bootstrap to verify etcd rollout did not report errors for some failure modes. With this release, those error messages now are visible on the bootkube script if the cmd exits in an error case. (OCPBUGS-33495)

Previously, the etcd Cluster Operator entered a state of panic during pod health checks and this caused requests to an etcd cluster to fail. With this release, the issue is fixed so that these panic situations no longer occur. (OCPBUGS-27959)

Previously, the etcd Cluster Operator wrongly identified non-running controllers as deadlocked and this caused an unnecessary pod restart. With this release, this issue is now fixed so that the Operator marks a non-running controller as an unhealthy etcd member without restarting a pod. (OCPBUGS-30873)

Previously, Multus Container Network Interface (CNI) required certificate signing requests (CSRs) to be approved when you used the Other network type in hosted clusters. The proper role-based access control (RBAC) rules were set only when the network type was Other and was set to Calico. As a consequence, the CSRs were not approved when the network type was Other and set to Cilium. With this update, the correct RBAC rules are set for all valid network types, and RBACs are now properly configured when you use the Other network type. (OCPBUGS-26977)

Previously, an Amazon Web Services (AWS) policy issue prevented the Cluster API Provider AWS from retrieving the necessary domain information. As a consequence, installing an AWS hosted cluster with a custom domain failed. With this update, the policy issue is resolved. (OCPBUGS-29391)

Previously, in disconnected environments, the HyperShift Operator ignored registry overrides. As a consequence, changes to node pools were ignored, and node pools encountered errors. With this update, the metadata inspector works as expected during the HyperShift Operator reconciliation, and the override images are properly populated. (OCPBUGS-34773)

Previously, the HyperShift Operator was not using the RegistryOverrides mechanism to inspect the image from the internal registry. With this release, the metadata inspector works as expected during the HyperShift Operator reconciliation, and the OverrideImages are properly populated. (OCPBUGS-32220)

Previously, the Red Hat OpenShift Cluster Manager container did not have the correct Transport Layer Security (TLS) certificates. As a result, image streams could not be used in disconnected deployments. With this update, the TLS certificates are added as projected volumes. (OCPBUGS-34390)

Previously, the azure-kms-provider-active container in the KAS pod used an entrypoint statement in shell form in the Dockerfile. As a consequence, the container failed. To resolve this issue, use the exec form for the entrypoint statement. (OCPBUGS-33940)

Previously, the konnectivity-agent daemon set used the ClusterIP DNS policy. As a result, when CoreDNS was down, the konnectivity-agent pods on the data plane could not resolve the proxy server URL, and they could fail to konnectivity-server in the control plane. With this update, the konnectivity-agent daemon set was modified to use dnsPolicy: Default. The konnectivity-agent uses the host system DNS service to look up the proxy server address, and it does not depend on CoreDNS anymore. (OCPBUGS-31444)

Previously, the inability to find a resource caused re-creation attempts to fail. As a consequence, many 409 response codes were logged in Hosted Cluster Config Operator logs. With this update, specific resources were added to the cache so that the Hosted Cluster Config Operator does not try to re-create existing resources. (OCPBUGS-23228)

Previously, the pod security violation alert was missing in hosted clusters. With this update, the alert is added to hosted clusters. (OCPBUGS-31263)

Previously, the recycler-pod template in hosted clusters in disconnected environments pointed to quay.io/openshift/origin-tools:latest. As a consequence, the recycler pods failed to start. With this update, the recycler pod image now points to the OpenShift Container Platform payload reference. (OCPBUGS-31398)

With this update, in disconnected deployments, the HyperShift Operator receives the new ImageContentSourcePolicy (ICSP) or ImageDigestMirrorSet (IDMS) from the management cluster and adds them to the HyperShift Operator and the Control Plane Operator in every reconciliation loop. The changes to the ICSP or IDMS cause the control-plane-operator pod to be restarted. (OCPBUGS-29110)

With this update, the ControllerAvailabilityPolicy setting becomes immutable after it is set. Changing between SingleReplica and HighAvailability is not supported. (OCPBUGS-27282)

With this update, the machine-config-operator custom resource definitions (CRDs) are renamed to ensure that resources are being omitted properly in hosted control planes. (OCPBUGS-34575)

With this update, the size is reduced for audit log files that are stored in the kube-apiserver, openshift-apiserver, and oauth-apiserver pods for hosted control planes. (OCPBUGS-31106)

Previously, the Hypershift Operator was not using the RegistryOverrides mechanism to inspect the image from the internal registry. With this release, the metadata inspector works as expected during the Hypershift Operator reconciliation, and the OverrideImages are properly populated. (OCPBUGS-29494)

Previously, after you imported image streams tags, the ImageContentSourcePolicy (ICSP) custom resource (CR) could not co-exist with the ImageDigestMirrorSet (IDMS) or ImageTagMirrorSet (ITMS) CR. OpenShift Container Platform chose ICSP instead of the other CR types. With this release, these custom resources can co-exist, so after you import image stream tags, OpenShift Container Platform can choose the required CR. (OCPBUGS-30279)

Previously, the oc tag command did not validate tag names when the command created new tags. After images were created from tags with invalid names, the podman pull command would fail. With this release, a validation step checks new tags for invalid names and you can now delete existing tags that have invalid names, so that this issue no longer exists. (OCPBUGS-25703)

Previously, the Image Registry Operator had maintained its own list of IBM Power® Virtual Server regions, so any new regions were not added to the list. With this release, the Operator relies on an external library for accessing regions so that it can support new regions. (OCPBUGS-26767)

Previously, the image registry Microsoft Azure path-fix job incorrectly required the presence of AZURE_CLIENT_ID and TENANT_CLIENT_ID parameters to function. This caused a valid configuration to throw an error message. With this release, a check is added to the Identity and Access Management (IAM) service account key to validate if these parameters are needed, so that a cluster upgrade operation no longer fails. (OCPBUGS-32328)

Previously, the image registry did not support Amazon Web Services (AWS) region ca-west-1. With this release, the image registry can now be deployed in this region. (OCPBUGS-29233)

Previously, when the virtualHostedStyle parameter was set to regionEndpoint in the Image Registry Operator configuration, the image registry ignored the virtual hosted style configuration. With this release, the issue is resolved so that a new upstream distribution configuration, force path style, is used instead of the downstream only version, virtual hosted style. (OCPBUGS-34166)

Previously, when running an OpenShift Container Platform cluster on IBM Power® Virtual Server where service-endpoint override was enabled, the Cloud Credential Operator (CCO) Operator would ignore the overriding service endpoints. With this release, the CCO Operator no longer ignores overriding service endpoints. (OCPBUGS-32491)

Previously, the Image Registry Operator ignored endpoint service cluster-level overrides, making configuring your cluster in an IBM Cloud® disconnected environment difficult. This issue only existed on installer-provisioned infrastructure. With this release, the Image Registry Operator no longer ignores these cluster-level overrides. (OCPBUGS-26064)

Previously, installation of a three-node cluster with an invalid configuration on Google Cloud Platform (GCP) failed with a panic error that did not report the reason for the failure. With this release, the installation program validates the installation configuration to successfully install a three-node cluster on GCP. (OCPBUGS-35103)

Previously, installations with the Assisted Installer failed if the pull secret contained a colon in the password. With this release, pull secrets containing a colon in the password do not cause the Assisted Installer to fail. (OCPBUGS-34400)

Previously, the monitor-add-nodes command, which is used to monitor the process of adding nodes to an Agent-based cluster, failed to run due to a permission error. With this release, the command operates in the correct directory where it has permissions. (OCPBUGS-34388)

Previously, long cluster names were trimmed without warning the user. With this release, the installation program warns the user when trimming long cluster names. (OCPBUGS-33840)

Previously, when installing a cluster, the Ingress capability was enabled even if it was disabled in install-config.yaml because it is required. With this release, the installation program fails if the Ingress capability is disabled in install-config.yaml. (OCPBUGS-33794)

Previously, OpenShift Container Platform did not perform quota checking for clusters installed in the ca-west-1 an Amazon Web Services (AWS) region. With this release, quotas are properly enforced in this region. (OCPBUGS-33649)

Previously, the installation program could sometimes fail to detect that the OpenShift Container Platform API is unavailable. An additional error was resolved by increasing the disk size of the bootstrap node in Microsoft Azure installations. With this release, the installation program correctly detects if the API is unavailable. (OCPBUGS-33610)

Previously, control plane nodes on Microsoft Azure clusters were using Read-only caches. With this release, Microsoft Azure control plane nodes use ReadWrite caches. (OCPBUGS-33470)

Previously, when installing an Agent-based cluster with a proxy configured, the installation failed if the proxy configuration contained a string starting with a percent sign (%). With this release, the installation program correctly validates this configuration text. (OCPBUGS-33024)

Previously, installations on GCP could fail because the installation program attempted to create a bucket twice. With this release, the installation program no longer attempts to create the bucket twice. (OCPBUGS-32133)

Previously, a rare timing issue could prevent all control plane nodes from being added to an Agent-based cluster during installation. With this release, all control plane nodes are successfully rebooted and added to the cluster during installation. (OCPBUGS-32105)

Previously, when using the Agent-based installation program in a disconnected environment, unnecessary certificates were added to the Certificate Authority (CA) trust bundle. With this release, the CA bundle ConfigMap only contains CAs explicitly specified by the user. (OCPBUGS-32042)

Previously, the installation program required a non-existent permission s3:HeadBucket when installing a cluster on Amazon Web Services (AWS). With this release, the installation program correctly requires the permission s3:ListBucket instead. (OCPBUGS-31813)

Previously, if the installation program failed to gather logs from the bootstrap due to an SSH connection issue, it would also not provide virtual machine (VM) serial console logs even if they were collected. With this release, the installation program provides VM serial console logs even if the SSH connection to the bootstrap machine fails. (OCPBUGS-30774)

Previously, when installing a cluster on VMware vSphere with static IP addresses, the cluster could create control plane machines without static IP addresses due to a conflict with other Technology Preview features. With this release, the Control Plane Machine Set Operator correctly manages the static IP assignment for control plane machines. (OCPBUGS-29114)

Previously, when installing a cluster on GCP with user-provided DNS, the installation program still attempted to validate DNS within the GCP DNS network. With this release, the installation program does not perform this validation for user-provided DNS. (OCPBUGS-29068)

Previously, when deleting a private cluster on IBM Cloud® that used the same domain name as a non-private IBM Cloud® cluster, some resources were not deleted. With this release, all private cluster resources are deleted when the cluster is removed. (OCPBUGS-28870)

Previously, when installing a cluster using a proxy with a character string that used the percent sign (%) in the configuration string, the cluster installation failed. With this release, the installation program correctly validates proxy configuration strings containing "%". (OCPBUGS-27965)

Previously, the installation program still allowed the use of the OpenShiftSDN network plugin even though it was removed. With this release, the installation program correctly prevents installing a cluster with this network plugin. (OCPBUGS-27813)

Previously, when installing a cluster on Amazon Web Services (AWS) Wavelengths or Local Zones into a region that supports either Wavelengths or Local Zones, but not both, the installation failed. With this release, installations into regions that support either Wavelengths or Local Zones can succeed. (OCPBUGS-27737)

Previously, when a cluster installation was attempted that used the same cluster name and base domain as an existing cluster and the installation failed due to DNS record set conflicts, removal of the second cluster would also remove the DNS record sets in the original cluster. With this release, the stored metadata contains the private zone name rather than the cluster domain, so only the correct DNS records are deleted from a removed cluster. (OCPBUGS-27156)

Previously, platform specific passwords that were configured in the installation configuration file of an Agent-based installation could be present in the output of the agent-gather command. With this release, passwords are redacted from the agent-gather output. (OCPBUGS-26434)

Previously, a OpenShift Container Platform cluster installed with version 4.15 or 4.16 showed a default upgrade channel of version 4.14. With this release, clusters have the correct upgrade channel after installation. (OCPBUGS-26048)

Previously, when deleting a VMware vSphere cluster, some TagCategory objects failed to be deleted. With this release, all cluster-related objects are correctly deleted when the cluster is removed. (OCPBUGS-25841)

Previously, when specifying the baremetal platform type but disabling the baremetal capability in install-config.yaml, the installation failed after a long timeout without a helpful error. With this release, the installation program provides a descriptive error and does not attempt a bare metal installation if the baremetal capability is disabled. (OCPBUGS-25835)

Previously, installations on VMware vSphere using the Assisted Installer could fail by preventing VMware vSphere from initializing nodes correctly. With this release, Assisted Installer installations on VMware vSphere successfully complete with all nodes initialized. (OCPBUGS-25718)

Previously, if a VM type was selected that did not match the architecture specified in the install-config.yaml file, the installation would fail. With this release, a validation check ensures that the architectures match before the installation begins. (OCPBUGS-25600)

Previously, agent-based installations could fail if an invalid number of control plane replicas was specified, such as 2. With this release, the installation program enforces the requirement of specifying either 1 or 3 control plane replicas for agent-based installations. (OCPBUGS-25462)

Previously, when installing a cluster on VMware vSphere using the control plane machine set Technology Preview feature, the resulting control plane machine sets had duplicate failure domains in their configuration. With this release, the installation program creates the control plane machine sets with the correct failure domains. (OCPBUGS-25453)

Previously, the required iam:TagInstanceProfile permission was not validated before an installer-provisioned installation, causing an installation to fail if the Identity and Access Management (IAM) permission was missing. With this release, a validation check ensures that the permission is included before the installation begins. (OCPBUGS-25440)

Previously, the installation program did not prevent users from installing a cluster on non-bare-metal platforms with the Cloud Credential capability disabled, although it is required. With this release, the installation program produces an error and prevents installation with the Cloud Credential capability disabled, except for on the bare-metal platform. (OCPBUGS-24956)

Previously, setting an architecture different from the one supported by the instance type resulted in the installation failing mid-process, after some resources were created. With this release, a validation check verifies that the instance type is compatible with the specified architecture. If the architecture is not compatible, the process fails before the installation begins. (OCPBUGS-24575)

Previously, the installation program did not prevent a user from installing a cluster on a cloud provider with the Cloud Controller Manager disabled, which failed without a helpful error message. With this release, the installation program produces an error stating that the Cloud Controller Manager capability is required for installations on cloud platforms. (OCPBUGS-24415)

Previously, the installation program could fail to remove a cluster installed on IBM Cloud® due to unexpected results from the IBM Cloud® API. With this release, clusters installed on IBM Cloud® can reliably be deleted by the installation program. (OCPBUGS-20085)

Previously, the installation program did not enforce the requirement that FIPS-enabled clusters were installed from FIPS-enabled Red Hat Enterprise Linux (RHEL) hosts. With this release, the installation program enforces the FIPS requirement. (OCPBUGS-15845)

Previously, proxy information that was set in the install-config.yaml file was not applied to the bootstrap process. With this release, proxy information is applied to bootstrap ignition data, which is then applied to the bootstrap machine. (OCPBUGS-12890)

Previously, when the IBM Power® Virtual Server platform had no Dynamic Host Configuration Protocol (DHCP) network name, the DHCP resource was not deleted. With this release, a check looks for any DHCP resources with an ERROR state and deletes them so that this issue no longer occurs. (OCPBUGS-35224)

Previously, when creating a IBM Power® Virtual Server cluster on installer-provisioned infrastructure by using the Cluster API, the load balancer would become busy and stall. With this release, you can use the 'AddIPToLoadBalancerPool' command in a PollUntilContextCancel loop to restart the load balancer. (OCPBUGS-35088)

Previously, an installer-provisioned installation on a bare-metal platform with FIPS-enabled nodes caused installation failures. With this release, the issue is resolved. (OCPBUGS-34985)

Previously, when creating an install configuration for an installer-provisioned installation on IBM Power® Virtual Server, the survey stopped if the administrator did not enter a command on the OpenShift CLI (oc). The survey stopped because no default region was set in the install-config survey. With this release, the issue is resolved. (OCPBUGS-34728)

Previously, solid state drives (SSD) that used SATA hardware were identified as removable. The Assisted Installer for OpenShift Container Platform reported that no eligible disks were found and the installation stopped. With this release, removable disks are eligible for installation. (OCPBUGS-34652)

Previously, Agent-based installations with dual-stack networking failed due to IPv6 connectivity check failures, even though IPv6 connectivity could be established between nodes. With this release, the issue has been resolved. (OCPBUGS-31631)

Previously, due to a programming error, a script created compute server groups with the policy set for control planes. As a consequence, the serverGroupPolicy property of install-config.yaml files was ignored for compute groups. With this fix, the server group policy set in the install-config.yaml file for compute machine pools is applied at installation in the script flow. (OCPBUGS-31050)

Previously, when configuring an Agent-based installation that uses the openshift-baremetal-install binary, the Agent-based installer erroneously attempted to verify the libvirt network interfaces. This might cause the following error:

Platform.BareMetal.externalBridge: Invalid value: "baremetal": could not find interface "baremetal"

Previously, using network types with dual-stack networking other than Open vSwitch-based software-defined networking (SDN) or Open Virtual Network (OVN) caused a validation error. With this release, the issue is resolved. (OCPBUGS-30232)

Previously, a closed IPv6 port range for nodePort services in user-provisioned-infrastructure installations on RHOSP caused traffic through certain node ports to be blocked. With this release, appropriate security group rules have been added to the security-group.yaml playbook, resolving the issue. (OCPBUGS-30154)

Previously, manifests that were generated by using the command openshift-install agent create cluster-manifests command were not directly applied to an OpenShift Container Platform cluster because the manifests did not include type data. With this release, type data has been added to the manifests. Administrators can now apply the manifests to initiate a Zero Touch Provisioning (ZTP) installation that uses the same settings as the Agent-based installation. (OCPBUGS-29968)

Previously, a file required for the aarch64 architecture was renamed by mistake while generating the aarch64 agent ISO. With this release, the specified file does not get renamed. (OCPBUGS-28827)

Previously, when installing a cluster on VMware vSphere, the installation would fail if an ESXi host was in maintenance mode due to the installation program failing to retrieve version information from the host. With this release, the installation program does not attempt to retrieve version information from ESXi hosts that are in maintenance mode, allowing the installation to proceed. (OCPBUGS-27848)

Previously, the IBM Cloud® Terraform Plugin incorrectly prevented the use of non-private service endpoints during cluster installation. With this release, the IBM Cloud® Terraform Plugin supports non-private service endpoints during installation. (OCPBUGS-24473)

Previously, installing a cluster on VMware vSphere required specifying the full path to the datastore. With this release, the installation program accepts full paths and relative paths for the datastore. (OCPBUGS-22410)

Previously, when you installed an OpenShift Container Platform cluster by using the Agent-based installation program, a large number of manifests before installation could fill the Ignition storage causing the installation to fail. With this release, the Ignition storage has been increased to allow for a much greater amount of installation manifests. (OCPBUGS-14478)

Previously, when the coreos-installer iso kargs show <iso> command was used on Agent ISO files, the output would not properly show the kernel arguments embedded in the specified ISO. With this release, the command output displays the information correctly. (OCPBUGS-14257)

Previously, Agent-based installations created ImageContentSource objects instead of ImageDigestSources even though the former object is deprecated. With this release, the Agent-based installation program creates ImageDigestSource objects. (OCPBUGS-11665)

Previously, there was an issue with the destroy functionality of the Power VS where not all resources were deleted as expected. With this release, the issue has been resolved. (OCPBUGS-29425)

The Insights Operator now collects instances outside of the openshift-monitoring of the following custom resources:

Previously, when deleting a ClusterResourceQuota resource using the foreground deletion cascading strategy, the removal failed to complete. With this release, ClusterResourceQuota resources are deleted properly when using the foreground cascading strategy. (OCPBUGS-22301)

Previously, the MachineConfigNode object was not created with a proper owner. As a result, the MachineConfigNode object could not be garbage collected, meaning that previously generated, but no longer useful, objects were not removed. With this release, the proper owner is set upon the creation of the MachineConfigNode object and objects that become obsolete are available for garbage collection. (OCPBUGS-30090)

Previously, the default value of the nodeStatusUpdateFrequency parameter was changed from 0s to 10s. This change inadvertently caused the nodeStatusReportFrequency to increase significantly, because the value was linked to the nodeStatusReportFrequency value. This resulted in high CPU usage on control plane operators and the API server. This fix manually sets the nodeStatusReportFrequency value to 5m, which prevents this high CPU usage. (OCPBUGS-29713)

Previously, a typographical error in an environment variable prevented a script from detecting if the node.env file was present. Because of this, the node.env file would be overwritten on every restart, preventing the kubelet hostname from being fixed. With this fix the typographical error is corrected. As a result, edits to the node.env are now persist across reboots. (OCPBUGS-27261)

Previously, when the kube-apiserver server Certificate Authority (CA) certificate was rotated, the Machine Config Operator (MCO) did not properly react and update the on-disk kubelet kubeconfig. This meant that the kubelet and some pods on the node were eventually unable to communicate with the APIserver, causing the node to enter the NotReady state. With this release, the MCO properly reacts to the change, and updates the on-disk kubeconfig such that authenticated communication with the APIServer can continue when this rotates, and also restarts kubelet/MCDaemon pod. The certificate authority has 10-year validity, so this rotation should happen rarely and is generally non-disruptive. (OCPBUGS-25821)

Previously, when a new node was added to or removed from a cluster, the MachineConfigNode (MCN) objects did not react. As a result, extraneous MCN objects existed. With this release, the Machine Config Operator removes and adds MCN objects as appropriate when nodes are added or removed. (OCPBUGS-24416)

Previously, the nodeip-configuration service did not send logs to the serial console, which made it difficult to debug problems when networking is not available and there is no access to the node. With this release, the nodeip-configuration service logs output to the serial console for easier debugging when there is no network access to the node. (OCPBUGS-19628)

Previously, when a MachineConfigPool had the OnClusterBuild functionality enabled and the configmap was updated with an invalid imageBuilderType, the machine-config ClusterOperator was not degraded. With this release, the Machine Config Operator (MCO) ClusterOperator status now validates the OnClusterBuild inputs each time it syncs, ensuring that if those are invalid, the ClusterOperator is degraded. (OCPBUGS-18955)

Previously, when the machine config not found error was reported, there was not enough information to troubleshoot and correct the problem. With this release, an alert and metric have been added to the Machine Config Operator. As a result, you have more information to troubleshoot and remediate the machine config not found error. (OCPBUGS-17788)

Previously, the Afterburn service used to set the hostname on nodes timed out while waiting for the metadata service to become available, causing issues when deploying with OVN-Kubernetes. Now, the Afterburn service waits longer for the metadata service to become available, resolving these timeouts. (OCPBUGS-11936)

Previously, when a node was removed from a MachineConfigPool, the Machine Config Operator (MCO) did not report an error or the removal of the node. The MCO does not support managing nodes when they are not in a pool and there was no indication that node management ceased after the node was removed. With this release, if a node is removed from all pools, the MCO now logs an error. (OCPBUGS-5452)

Previously, the Debug container link was not shown for pods with a Completed status. With this release, the link shows as expected. (OCPBUGS-34711)

Previously, due to an issue in PatternFly 5, text boxes in the web console were no longer resizable. With this release, text boxes are again resizable. (OCPBUGS-34393)

Previously, French and Spanish were not available in the web console. With this release, translations for French and Spanish are now available. (OCPBUGS-33965)

Previously, the masthead logo was not restricted to a max-height of 60px. As a result, logos that are larger than 60px high display at their native size and cause the masthead size too to be too large. With this release, the masthead logo is restricted to a max-height of 60px. (OCPBUGS-33523)

Previously, there was a missing return statement in the HealthCheck controller causing it to panic under certain circumstances. With this release, the proper return statement was added to the HealthCheck controller so it no longer panics. (OCPBUGS-33505)

Previously, an incorrect field was sent to the API server that was not noticeable. With the implementation of Admission Webhook display warning the same action would return a warning notification. A fix was provided to resolve the issue. (OCPBUGS-33222)

Previously, the message text of a StatusItem might have been vertically misaligned with the icon when a timestamp was not present. With this release, the message text is correctly aligned. (OCPBUGS-33219)

Previously, the creator field was autopopulated and not mandatory. Updates to the API made the field empty from OpenShift Container Platform 4.15 and higher. With this release, the field is marked as mandatory for correct validation. (OCPBUGS-31931)

Previously, the YAML editor in the web console did not have the Create button and samples did not show on the web console. With this release, you can now see the Create button and the samples. (OCPBUGS-31703)

Previously, changes to the bridge server flags on an external OpenID Connect (OIDC) feature caused the bridge server fail to start in local development. With this release, the flags usage are updated and the bridge server starts. (OCPBUGS-31695)

Previously, when editing a VMware vSphere connection, the form could be submitted even if no values were actually changed. This resulted in unnecessary node reboots. With this release, the console now detects the form changes, and does not allow submission if no value was changed. (OCPBUGS-31613)

Previously, the NetworkAttachmentDefinition was always created in the default namespace if the form method from the console was used. The selected name is also not honored, and creates the NetworkAttachmentDefinition object with the selected name and a random suffix. With this release, the NetworkAttachmentDefinition object is created in the current project. (OCPBUGS-31558)

Previously, when clicking the Configure button by the AlertmanagerRecieversNotConfigured alert, the Configuration page did not show. With this release, the link in the AlertmanagerRecieversNotConfigured alert is fixed and directs you to the Configuration page. (OCPBUGS-30805)

Previously, plugins using ListPageFilters were only using two filters: label and name. With this release, a parameter was added that enables plugins to configure multiple text-based search filters. (OCPBUGS-30077)

Previously, there was no response when clicking on quick start items. With this release, the quick start window shows when clicking on the quick start selections. (OCPBUGS-29992)

Previously, the OpenShift Container Platform web console terminated unexpectedly if authentication discovery failed on the first attempt. With this release, authentication initialization was updated to retry up to 5 minutes before failing. (OCPBUGS-29479)

Previously there was an issue causing an error message on the Image Manifest Vulnerability page after an Image Manifest Vulnerability (IMV) was created in the CLI. With this release, the error message no longer shows. (OCPBUGS-28967)

Previously, when using the modal dialog in a hook as part of the actions hook, an error occurred because the console framework passed null objects as part of the render cycle. With this release, getGroupVersionKindForResource is now null-safe and will return undefined if the apiVersion or kind are undefined. Additionally, the run time error for useDeleteModal no longer occurs, but note that it will not work with an undefined resource. (OCPBUGS-28856)

Previously, the Expand PersistentVolumeClaim modal assumes the pvc.spec.resources.requests.stroage value includes a unit. With this release, the size is updated to 2GiB and you can change the value of the persistent volume claim (PVC). (OCPBUGS-27779)

Previously, the value of image vulnerabilities reported in the OpenShift Container Platform web console were inconsistent. With this release, the image vulnerabilities on the Overview page were removed. (OCPBUGS-27455)

Previously, a certificate signing request (CSR) could show for a recently approved Node. With this release, the duplication is detected and does not show CSRs for approved Nodes. (OCPBUGS-27399)

Previously, the Type column was not first on the condition table on the MachineHealthCheck detail page. With this release, the Type is now listed first on the condition table. (OCPBUGS-27246)

Previously, the console plugin proxy was not copying the status code from plugin service responses. This caused all responses from the plugin service to have a 200 status, causing unexpected behavior, especially around browser caching. With this release, the console proxy logic was updated to forward the plugin service proxy response status code. Proxied plugin requests now behave as expected. (OCPBUGS-26933)

Previously, when cloning a persistent volume claim (PVC), the modal assumes pvc.spec.resources.requests.storage value includes a unit. With this release, pvc.spec.resources.requests.storage includes a unit suffix and the Clone PVC modal works as expected. (OCPBUGS-26772)

Previously, escaped strings were not handled properly when editing VMware vSphere connection, causing broken VMware vSphere configuration. With this release, the escape strings work as expected and the VMware vSphere configuration no longer breaks. (OCPBUGS-25942)

Previously, when configuring a VMware vSphere connection, the resourcepool-path key was not added to the VMware vSphere config map which might have caused issues connecting to VMware vSphere. With this release, there are no longer issues connecting to VMware vSphere. (OCPBUGS-25927)