×

About Windows Container Support for Red Hat OpenShift

Red Hat OpenShift support for Windows Containers enables running Windows compute nodes in an OpenShift Container Platform cluster. Running Windows workloads is possible by using the Red Hat Windows Machine Config Operator (WMCO) to install and manage Windows nodes. With Windows nodes available, you can run Windows container workloads in OpenShift Container Platform.

The release notes for Red Hat OpenShift support for Windows Containers tracks the development of the WMCO, which provides all Windows container workload capabilities in OpenShift Container Platform.

Getting support

Red Hat OpenShift support for Windows Containers is provided and available as an optional, installable component. Windows Container Support for Red Hat OpenShift is not part of the OpenShift Container Platform subscription. It requires an additional Red Hat subscription and is supported according to the Scope of coverage and Service level agreements.

You must have this separate subscription to receive support for Windows Container Support for Red Hat OpenShift. Without this additional Red Hat subscription, deploying Windows container workloads in production clusters is not supported. You can request support through the Red Hat Customer Portal.

For more information, see the Red Hat OpenShift Container Platform Life Cycle Policy document for Red Hat OpenShift support for Windows Containers.

If you do not have this additional Red Hat subscription, you can use the Community Windows Machine Config Operator, a distribution that lacks official support.

Release notes for Red Hat Windows Machine Config Operator 3.1.2

Issued: 2022-11-09

The WMCO 3.1.2 is now available with bug fixes. The components of the WMCO were released in RHBA-2022:7076.

Bug fixes

  • Previously, Windows containers on Windows nodes were assigned the wrong DNS Server IP if a non-default cluster DNS was specified during the cluster installation. Therefore, the DNS resolution did not work. This bug fix removes the hard-coded cluster DNS information and parametrizes the value as a command-line argument. (BZ#2030943)

Release notes for Red Hat Windows Machine Config Operator 3.1.1

Issued: 2021-12-07

The WMCO 3.1.1 is now available with bug fixes. The components of the WMCO were released in RHBA-2021:4710.

Bug fixes

  • Previously, the windows-exporter metrics endpoint object contained a reference to a deleted machine. This incorrect reference caused the WMCO to ignore deleted events for machines with invalid IP addresses. This bug fix removes the validation of the machine object from the event filtering, allowing the windows-exporter metrics endpoint object to correctly update when the machine is still in the Deleting phase. (BZ#2008994)

  • Previously, deleting the node associated with a Windows Machine object threw a reconciliation error upon restart of the Operator. This bug fix opts not to react or reconcile when the node referenced by a Windows machine in the Running state is not found within the cluster, preventing any error loop and standardizing functionality with Linux machine objects. (BZ#2009475)

  • Previously, the WMCO did not properly associate BYOH Windows VMs with their Node object when the VM was specified with a DNS object. This caused the WMCO to attempt to configure VMs that were already fully configured. The WMCO now correctly resolves VMs specified by a DNS address when looking for an associated node. (BZ#2020650)

  • Previously, encrypted usernames were being generated with extra tags, which caused them not to display correctly. This bug fix removes the extra tags, allowing the encrypted username to display correctly. (BZ#2023417)

  • Previously, certain commands being run by the WMCO in Windows VMs were not parsed correctly by PowerShell. This caused Windows VMs with PowerShell as its default SSH shell to be unable to join to a cluster as a node. The WMCO can now identify the default SSH shell of a Windows VM and run the associated commands accordingly. This new capability allows Windows VMs with PowerShell as the default SSH shell to be configured as nodes in a cluster. (BZ#2025730)

Release notes for Red Hat Windows Machine Config Operator 3.1.0

Issued: 2021-09-21

The WMCO 3.1.0 is now available with bug fixes and a new feature. The components of the WMCO were released in RHBA-2021:3215.

New features

Using Bring-Your-Own-Host (BYOH) Windows instances

You can now add an existing Windows instance to an OpenShift Container Platform cluster as a compute node. This requires creating a config map in the WMCO namespace.

BYOH Windows instances are supported with installer-provisioned infrastructure for the following platforms:

  • Amazon Web Services (AWS)

  • Microsoft Azure

  • VMware vSphere

BYOH Windows instances are supported with user-provisioned infrastructure, only when the platform: none field is set in the install-config.yaml file, for the following platforms:

  • VMware vSphere

  • bare metal

For more information on how to configure BYOH Windows instances, see Configuring BYOH Windows instance.

Bug fixes

  • For clusters installed on VMware vSphere, the WMCO ignored the Deleting phase notification event, leaving incorrect node information in the windows-exporter metrics endpoint. This resulted in an invalid mapping for the Prometheus metrics endpoint. This bug has been fixed; the WMCO now recognizes the Deleting phase notification event and maps the Prometheus metrics endpoint appropriately. (BZ#1995341)

Known issues

  • When installing a BYOH Windows instance using a DNS name entry in the config map, the WMCO configures the instance twice before marking it as a Ready node. This will be fixed in a future release of the WMCO. (BZ#2005360)

  • When the RunAsUser permission is set in the security context of a Linux-based pod, the projected files have the correct permissions set, including container user ownership. However, when the Windows equivalent RunAsUsername permission is set in a Windows pod, the kubelet is prevented from setting correct ownership on the files in the projected volume. This problem can get exacerbated when used in conjunction with a hostPath volume where best practices are not followed. For example, giving a pod access to the C:\var\lib\kubelet\pods\ folder results in that pod being able to access service account tokens from other pods.

    By default, the projected files will have the following ownership, as shown in this example Windows projected volume file:

    Path   : Microsoft.PowerShell.Core\FileSystem::C:\var\run\secrets\kubernetes.io\serviceaccount\..2021_08_31_22_22_18.318230061\ca.crt
    Owner  : BUILTIN\Administrators
    Group  : NT AUTHORITY\SYSTEM
    Access : NT AUTHORITY\SYSTEM Allow  FullControl
             BUILTIN\Administrators Allow  FullControl
             BUILTIN\Users Allow  ReadAndExecute, Synchronize
    Audit  :
    Sddl   : O:BAG:SYD:AI(A;ID;FA;;;SY)(A;ID;FA;;;BA)(A;ID;0x1200a9;;;BU)

    This indicates all administrator users, such as someone with the ContainerAdministrator role, have read, write, and execute access, while non-administrator users have read and execute access.

    OpenShift Container Platform applies the RunAsUser security context to all pods irrespective of its operating system. This means Windows pods automatically have the RunAsUser permission applied to its security context.

    In addition, if a Windows pod is created with a projected volume with the default RunAsUser permission set, the pod remains in the ContainerCreating phase.

    To handle these issues, OpenShift Container Platform forces the file permission handling in projected service account volumes set in the security context of the pod to not be honored for projected volumes on Windows. Note that this behavior for Windows pods is how file permission handling used to work for all pod types prior to OpenShift Container Platform 4.7. (BZ#1971745)

Release notes for Red Hat Windows Machine Config Operator 3.0.0

This release of the WMCO provides bug fixes and enhancements for running Windows compute nodes in an OpenShift Container Platform cluster. The components of the WMCO 3.0.0 were released in RHSA-2021:3001.

Users running WMCO 2.0.2 and below must uninstall the operator and reinstall WMCO 3.0.0 to work around a known issue that prevents users from upgrading from WMCO 2.0.2 to WMCO 3.0.0. (BZ#1983153)

Windows Machine Config Operator prerequisites

The following information details the supported platform versions, Windows Server versions, and networking configurations for the Windows Machine Config Operator. See the vSphere documentation for any information that is relevant to only that platform.

Supported platforms based on OpenShift Container Platform and WMCO versions

Platform Supported OpenShift Container Platform version Supported WMCO version Installer-provisioned infrastructure installation support User-provisioned infrastructure installation support

Amazon Web Services (AWS)

4.6+

WMCO 1.0+

GA

Tech Preview

Microsoft Azure

4.6+

WMCO 1.0+

GA

Tech Preview

VMware vSphere

4.7+

WMCO 2.0+

GA

Tech Preview

Supported platforms for Bring-Your-Own-Host (BYOH) instances based on OpenShift Container Platform and WMCO versions

Platform Supported OpenShift Container Platform version Supported WMCO version BYOH for installer-provisioned infrastructure installation support BYOH for user-provisioned infrastructure installation support

Amazon Web Services (AWS)

4.8+

WMCO 3.1+

GA

Tech Preview

Microsoft Azure

4.8+

WMCO 3.1+

GA

Tech Preview

VMware vSphere

4.8+

WMCO 3.1+

GA

GA[1]

bare metal

4.8+

WMCO 3.1+

GA[1]

  1. This installation type is only supported when the platform: none field is set in the install-config.yaml file during cluster installation.

Supported Windows Server versions

The following table lists the supported Windows Server version based on the applicable platform. Any unlisted Windows Server version is not supported and will cause errors. To prevent these errors, only use the appropriate version according to the platform in use.

Platform Supported Windows Server version

Amazon Web Services (AWS)

Windows Server 2019, version 1809

Microsoft Azure

Windows Server 2019, version 1809

VMware vSphere

Windows Server 2022, OS Build 20348.681 or later

Windows Server 2019 is unsupported, because the KB4565351 patch is not included.

bare metal

Windows Server 2019, version 1809

Supported networking

Hybrid networking with OVN-Kubernetes is the only supported networking configuration. See the additional resources below for more information on this functionality. The following tables outline the type of networking configuration and Windows Server versions to use based on your platform. You must specify the network configuration when you install the cluster. Be aware that OpenShift SDN networking is the default network for OpenShift Container Platform clusters. However, OpenShift SDN is not supported by WMCO.

Table 1. Platform networking support
Platform Supported networking

Amazon Web Services (AWS)

Hybrid networking with OVN-Kubernetes

Microsoft Azure

Hybrid networking with OVN-Kubernetes

VMware vSphere

Hybrid networking with OVN-Kubernetes with a custom VXLAN port

bare metal

Hybrid networking with OVN-Kubernetes

Table 2. Hybrid OVN-Kubernetes Windows Server support
Hybrid networking with OVN-Kubernetes Supported Windows Server version

Default VXLAN port

Windows Server 2019, version 1809

Custom VXLAN port

Windows Server 2022, OS Build 20348.681 or later

Running Windows container workloads is not supported for clusters in a restricted network or disconnected environment.

Version 3.x of the WMCO is only compatible with OpenShift Container Platform 4.8.

New features and improvements

This release adds the following new features and improvements.

Clarified limits on custom VXLAN port selection

Users must not select a custom VXLAN port when using the latest version of Windows server.

Bug fixes

  • Previously, the load balancer service would become unstable when the backing deployment had multiple pods scheduled on different Windows nodes. This issue has been fixed. (BZ#1905950)

  • Previously, WMCO added the public key annotation windowsmachineconfig.openshift.io/pub-key-hash to Linux nodes. Now, WMCO no longer adds an annotation to Linux nodes. (BZ#1930791)

  • Previously, when users provided an invalid private key, the WMCO would fail. With this update, the WMCO produces an error alerting the user of an invalid key. (BZ#1929579)

  • Previously, the kube-proxy service would crash upon the creation of a load balancer service when the backing development had multiple pods scheduled on different Windows nodes. This issue has now been fixed. (BZ#1939968)

  • Previously, the windows_exporter component reported various metrics as windows_*. This error caused some of the node-level metrics that provide insight into the nodes through telemetry services to go unreported. Now, the components export correctly showing all of the expected metrics. (BZ#1948037)

RHSA-2021:3001 - Windows Container support for OpenShift Container Platform security update

As part of the previously noted bug fix (BZ#1946538), an update for Windows kube-proxy is now available for Red Hat Windows Machine Config Operator 2.0.1. Details of the update are documented in the RHSA-2021:2130 advisory.

Known issues

  • The file system graphs available in the web console do not display for Windows nodes. This is caused by changes in the file system queries. This will be fixed in a future release of WMCO. (BZ#1930347)

  • When installing a BYOH Windows instance using a DNS name entry in the config map, the WMCO configures the instance twice before marking it as a Ready node. This will be fixed in a future release of the WMCO. (BZ#2005360)

Known limitations

Note the following limitations when working with Windows nodes managed by the WMCO (Windows nodes):

  • The following OpenShift Container Platform features are not supported on Windows nodes:

    • Red Hat OpenShift Developer CLI (odo)

    • Image builds

    • OpenShift Pipelines

    • OpenShift Service Mesh

    • OpenShift monitoring of user-defined projects

    • OpenShift Serverless

    • Horizontal Pod Autoscaling

    • Vertical Pod Autoscaling

  • The following Red Hat features are not supported on Windows nodes:

  • Windows nodes do not support pulling container images from private registries. You can use images from public registries or pre-pull the images.

  • Windows nodes do not support workloads created by using deployment configs. You can use a deployment or other method to deploy workloads.

  • Windows nodes are not supported in clusters that use a cluster-wide proxy. This is because the WMCO is not able to route traffic through the proxy connection for the workloads.

  • Windows nodes are not supported in clusters that are in a disconnected environment.

  • Red Hat OpenShift support for Windows Containers does not support adding Windows nodes to a cluster through a trunk port. The only supported networking configuration for adding Windows nodes is through an access port that carries traffic for the VLAN.

  • Red Hat OpenShift support for Windows Containers supports only in-tree storage drivers for all cloud providers.

  • Kubernetes has identified the following node feature limitations :

    • Huge pages are not supported for Windows containers.

    • Privileged containers are not supported for Windows containers.

    • Pod termination grace periods require the containerd container runtime to be installed on the Windows node.

  • Kubernetes has identified several API compatibility issues.