OpenShift docs are moving and will soon only be available at docs.redhat.com, the home of all Red Hat product documentation. Explore the new docs experience today.
  1. Documentation
    2. history bug_report picture_as_pdf
×

Control plane machine set configuration

Sample YAML for a control plane machine set custom resource

The base of the ControlPlaneMachineSet CR is structured the same way for all platforms.

Sample ControlPlaneMachineSet CR YAML file
apiVersion: machine.openshift.io/v1
kind: ControlPlaneMachineSet
metadata:
  name: cluster (1)
  namespace: openshift-machine-api
spec:
  replicas: 3 (2)
  selector:
    matchLabels:
      machine.openshift.io/cluster-api-cluster: <cluster_id> (3)
      machine.openshift.io/cluster-api-machine-role: master
      machine.openshift.io/cluster-api-machine-type: master
  state: Active (4)
  strategy:
    type: RollingUpdate (5)
  template:
    machineType: machines_v1beta1_machine_openshift_io
    machines_v1beta1_machine_openshift_io:
      failureDomains:
        platform: <platform> (6)
        <platform_failure_domains> (7)
      metadata:
        labels:
          machine.openshift.io/cluster-api-cluster: <cluster_id>
          machine.openshift.io/cluster-api-machine-role: master
          machine.openshift.io/cluster-api-machine-type: master
      spec:
        providerSpec:
          value:
            <platform_provider_spec> (8)
1 Specifies the name of the ControlPlaneMachineSet CR, which is cluster. Do not change this value.
2 Specifies the number of control plane machines. Only clusters with three control plane machines are supported, so the replicas value is 3. Horizontal scaling is not supported. Do not change this value.
3 Specifies the infrastructure ID that is based on the cluster ID that you set when you provisioned the cluster. You must specify this value when you create a ControlPlaneMachineSet CR. If you have the OpenShift CLI (oc) installed, you can obtain the infrastructure ID by running the following command: 
$ oc get -o jsonpath='{.status.infrastructureName}{"\n"}' infrastructure cluster
4 Specifies the state of the Operator. When the state is Inactive, the Operator is not operational. You can activate the Operator by setting the value to Active.

Before you activate the Operator, you must ensure that the ControlPlaneMachineSet CR configuration is correct for your cluster requirements. For more information about activating the Control Plane Machine Set Operator, see "Getting started with control plane machine sets".
5 Specifies the update strategy for the cluster. The allowed values are OnDelete and RollingUpdate. The default value is RollingUpdate. For more information about update strategies, see "Updating the control plane configuration".
6 Specifies the cloud provider platform name. Do not change this value.
7 Specifies the <platform_failure_domains> configuration for the cluster. The format and values of this section are provider-specific. For more information, see the sample failure domain configuration for your cloud provider.

VMware vSphere does not support failure domains.
8 Specifies the <platform_provider_spec> configuration for the cluster. The format and values of this section are provider-specific. For more information, see the sample provider specification for your cloud provider.
Additional resources

Provider-specific configuration

The <platform_provider_spec> and <platform_failure_domains> sections of the control plane machine set resources are provider-specific. Refer to the example YAML for your cluster:

Sample YAML for configuring Amazon Web Services clusters

Some sections of the control plane machine set CR are provider-specific. The example YAML in this section show failure domain and provider specification configurations for an Amazon Web Services (AWS) cluster.

Sample AWS failure domain configuration

The control plane machine set concept of a failure domain is analogous to existing AWS concept of an Availability Zone (AZ). The ControlPlaneMachineSet CR spreads control plane machines across multiple failure domains when possible.

When configuring AWS failure domains in the control plane machine set, you must specify the availability zone name and the subnet to use.

Sample AWS failure domain values
failureDomains:
  aws:
  - placement:
      availabilityZone: <aws_zone_a> (1)
    subnet: (2)
      filters:
      - name: tag:Name
        values:
        - <cluster_id>-private-<aws_zone_a> (3)
      type: Filters (4)
  - placement:
      availabilityZone: <aws_zone_b> (5)
    subnet:
      filters:
      - name: tag:Name
        values:
        - <cluster_id>-private-<aws_zone_b> (6)
      type: Filters
  platform: AWS (7)
1 Specifies an AWS availability zone for the first failure domain.
2 Specifies a subnet configuration. In this example, the subnet type is Filters, so there is a filters stanza.
3 Specifies the subnet name for the first failure domain, using the infrastructure ID and the AWS availability zone.
4 Specifies the subnet type. The allowed values are: ARN, Filters and ID. The default value is Filters.
5 Specifies the subnet name for an additional failure domain, using the infrastructure ID and the AWS availability zone.
6 Specifies the cluster’s infrastructure ID and the AWS availability zone for the additional failure domain.
7 Specifies the cloud provider platform name. Do not change this value.

Sample AWS provider specification

When you create a control plane machine set for an existing cluster, the provider specification must match the providerSpec configuration in the control plane machine CR that is created by the installation program. You can omit any field that is set in the failure domain section of the CR.

In the following example, <cluster_id> is the infrastructure ID that is based on the cluster ID that you set when you provisioned the cluster. If you have the OpenShift CLI installed, you can obtain the infrastructure ID by running the following command:

$ oc get -o jsonpath='{.status.infrastructureName}{"\n"}' infrastructure cluster
Sample AWS providerSpec values
providerSpec:
  value:
    ami:
      id: ami-<ami_id_string> (1)
    apiVersion: machine.openshift.io/v1beta1
    blockDevices:
    - ebs: (2)
        encrypted: true
        iops: 0
        kmsKey:
          arn: ""
        volumeSize: 120
        volumeType: gp3
    credentialsSecret:
      name: aws-cloud-credentials (3)
    deviceIndex: 0
    iamInstanceProfile:
      id: <cluster_id>-master-profile (4)
    instanceType: m6i.xlarge (5)
    kind: AWSMachineProviderConfig (6)
    loadBalancers: (7)
    - name: <cluster_id>-int
      type: network
    - name: <cluster_id>-ext
      type: network
    metadata:
      creationTimestamp: null
    metadataServiceOptions: {}
    placement: (8)
      region: <region> (9)
    securityGroups:
    - filters:
      - name: tag:Name
        values:
        - <cluster_id>-master-sg (10)
    subnet: {} (11)
    userDataSecret:
      name: master-user-data (12)
1 Specifies the Red Hat Enterprise Linux CoreOS (RHCOS) Amazon Machine Images (AMI) ID for the cluster. The AMI must belong to the same region as the cluster. If you want to use an AWS Marketplace image, you must complete the OpenShift Container Platform subscription from the AWS Marketplace to obtain an AMI ID for your region.
2 Specifies the configuration of an encrypted EBS volume.
3 Specifies the secret name for the cluster. Do not change this value.
4 Specifies the AWS Identity and Access Management (IAM) instance profile. Do not change this value.
5 Specifies the AWS instance type for the control plane.
6 Specifies the cloud provider platform type. Do not change this value.
7 Specifies the internal (int) and external (ext) load balancers for the cluster.

You can omit the external (ext) load balancer parameters on private OpenShift Container Platform clusters.
8 This parameter is configured in the failure domain, and is shown with an empty value here. If a value specified for this parameter differs from the value in the failure domain, the Operator overwrites it with the value in the failure domain.
9 Specifies the AWS region for the cluster.
10 Specifies the control plane machines security group.
11 This parameter is configured in the failure domain, and is shown with an empty value here. If a value specified for this parameter differs from the value in the failure domain, the Operator overwrites it with the value in the failure domain.
12 Specifies the control plane user data secret. Do not change this value.
Additional resources

Sample YAML for configuring Microsoft Azure clusters

Some sections of the control plane machine set CR are provider-specific. The example YAML in this section show failure domain and provider specification configurations for an Azure cluster.

Sample Azure failure domain configuration

The control plane machine set concept of a failure domain is analogous to existing Azure concept of an Azure availability zone. The ControlPlaneMachineSet CR spreads control plane machines across multiple failure domains when possible.

When configuring Azure failure domains in the control plane machine set, you must specify the availability zone name.

Sample Azure failure domain values
failureDomains:
  azure: (1)
  - zone: "1"
  - zone: "2"
  - zone: "3"
  platform: Azure (2)
1 Each instance of zone specifies an Azure availability zone for a failure domain.
2 Specifies the cloud provider platform name. Do not change this value.

Sample Azure provider specification

When you create a control plane machine set for an existing cluster, the provider specification must match the providerSpec configuration in the control plane Machine CR that is created by the installation program. You can omit any field that is set in the failure domain section of the CR.

In the following example, <cluster_id> is the infrastructure ID that is based on the cluster ID that you set when you provisioned the cluster. If you have the OpenShift CLI installed, you can obtain the infrastructure ID by running the following command:

$ oc get -o jsonpath='{.status.infrastructureName}{"\n"}' infrastructure cluster
Sample Azure providerSpec values
providerSpec:
  value:
    acceleratedNetworking: true
    apiVersion: machine.openshift.io/v1beta1
    credentialsSecret:
      name: azure-cloud-credentials (1)
      namespace: openshift-machine-api
    diagnostics: {}
    image: (2)
      offer: ""
      publisher: ""
      resourceID: /resourceGroups/<cluster_id>-rg/providers/Microsoft.Compute/galleries/gallery_<cluster_id>/images/<cluster_id>-gen2/versions/412.86.20220930 (3)
      sku: ""
      version: ""
    internalLoadBalancer: <cluster_id>-internal (4)
    kind: AzureMachineProviderSpec (5)
    location: <region> (6)
    managedIdentity: <cluster_id>-identity
    metadata:
      creationTimestamp: null
      name: <cluster_id>
    networkResourceGroup: <cluster_id>-rg
    osDisk: (7)
      diskSettings: {}
      diskSizeGB: 1024
      managedDisk:
        storageAccountType: Premium_LRS
      osType: Linux
    publicIP: false
    publicLoadBalancer: <cluster_id> (8)
    resourceGroup: <cluster_id>-rg
    subnet: <cluster_id>-master-subnet (9)
    userDataSecret:
      name: master-user-data (10)
    vmSize: Standard_D8s_v3
    vnet: <cluster_id>-vnet
    zone: "" (11)
1 Specifies the secret name for the cluster. Do not change this value.
2 Specifies the image details for your control plane machine set.
3 Specifies an image that is compatible with your instance type. The Hyper-V generation V2 images created by the installation program have a -gen2 suffix, while V1 images have the same name without the suffix.
4 Specifies the internal load balancer for the control plane. This field might not be preconfigured but is required in both the ControlPlaneMachineSet and control plane Machine CRs.
5 Specifies the cloud provider platform type. Do not change this value.
6 Specifies the region to place control plane machines on.
7 Specifies the disk configuration for the control plane.
8 Specifies the public load balancer for the control plane.

You can omit the publicLoadBalancer parameter on private OpenShift Container Platform clusters that have user-defined outbound routing.
9 Specifies the subnet for the control plane.
10 Specifies the control plane user data secret. Do not change this value.
11 This parameter is configured in the failure domain, and is shown with an empty value here. If a value specified for this parameter differs from the value in the failure domain, the Operator overwrites it with the value in the failure domain.
Additional resources

Sample YAML for configuring VMware vSphere clusters

Some sections of the control plane machine set CR are provider-specific. The example YAML in this section shows a provider specification configuration for a VMware vSphere cluster.

Sample vSphere provider specification

When you create a control plane machine set for an existing cluster, the provider specification must match the providerSpec configuration in the control plane machine CR that is created by the installation program.

Sample vSphere providerSpec values
providerSpec:
  value:
    apiVersion: machine.openshift.io/v1beta1
    credentialsSecret:
      name: vsphere-cloud-credentials (1)
    diskGiB: 120 (2)
    kind: VSphereMachineProviderSpec (3)
    memoryMiB: 16384 (4)
    metadata:
      creationTimestamp: null
    network: (5)
      devices:
      - networkName: <vm_network_name>
    numCPUs: 4 (6)
    numCoresPerSocket: 4 (7)
    snapshot: ""
    template: <vm_template_name> (8)
    userDataSecret:
      name: master-user-data (9)
    workspace:
      datacenter: <vcenter_datacenter_name> (10)
      datastore: <vcenter_datastore_name> (11)
      folder: <path_to_vcenter_vm_folder> (12)
      resourcePool: <vsphere_resource_pool> (13)
      server: <vcenter_server_ip> (14)
1 Specifies the secret name for the cluster. Do not change this value.
2 Specifies the VM disk size for the control plane machines.
3 Specifies the cloud provider platform type. Do not change this value.
4 Specifies the memory allocated for the control plane machines.
5 Specifies the network on which the control plane is deployed.
6 Specifies the number of CPUs allocated for the control plane machines.
7 Specifies the number of cores for each control plane CPU.
8 Specifies the vSphere VM template to use, such as user-5ddjd-rhcos.
9 Specifies the control plane user data secret. Do not change this value.
10 Specifies the vCenter Datacenter for the control plane.
11 Specifies the vCenter Datastore for the control plane.
12 Specifies the path to the vSphere VM folder in vCenter, such as /dc1/vm/user-inst-5ddjd.
13 Specifies the vSphere resource pool for your VMs.
14 Specifies the vCenter server IP or fully qualified domain name.