×

Managing machines with the Cluster API is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see https://access.redhat.com/support/offerings/techpreview/.

The Cluster API is an upstream project that is integrated into OpenShift Container Platform as a Technology Preview for Amazon Web Services (AWS) and Google Cloud Platform (GCP) clusters. You can use the Cluster API to create and manage machine sets and machines in your OpenShift Container Platform cluster. This capability is in addition or an alternative to managing machines with the Machine API.

For OpenShift Container Platform 4.11 clusters, you can use the Cluster API to perform node host provisioning management actions after the cluster installation finishes. This system enables an elastic, dynamic provisioning method on top of public or private cloud infrastructure.

With the Cluster API Technology Preview, you can create compute machines and machine sets on OpenShift Container Platform clusters for supported providers. You can also explore the features that are enabled by this implementation that might not be available with the Machine API.

Benefits

By using the Cluster API, OpenShift Container Platform users and developers are able to realize the following advantages:

  • The option to use upstream community Cluster API infrastructure providers which might not be supported by the Machine API.

  • The opportunity to collaborate with third parties who maintain machine controllers for infrastructure providers.

  • The ability to use the same set of Kubernetes tools for infrastructure management in OpenShift Container Platform.

  • The ability to create machine sets using the Cluster API that support features that are not available with the Machine API.

Limitations

Using the Cluster API to manage machines is a Technology Preview feature and has the following limitations:

  • Only AWS and GCP clusters are supported.

  • To use this feature, you must enable the ClusterAPIEnabled feature gate in the TechPreviewNoUpgrade feature set. Enabling this feature set cannot be undone and prevents minor version updates.

  • You must create the primary resources that the Cluster API requires manually.

  • Control plane machines cannot be managed by the Cluster API.

  • Migration of existing machine sets created by the Machine API to Cluster API machine sets is not supported.

  • Full feature parity with the Machine API is not available.

Cluster API architecture

The OpenShift Container Platform integration of the upstream Cluster API is implemented and managed by the Cluster CAPI Operator. The Cluster CAPI Operator and its operands are provisioned in the openshift-cluster-api namespace, in contrast to the Machine API, which uses the openshift-machine-api namespace.

The Cluster CAPI Operator

The Cluster CAPI Operator is an OpenShift Container Platform Operator that maintains the lifecycle of Cluster API resources. This Operator is responsible for all administrative tasks related to deploying the Cluster API project within an OpenShift Container Platform cluster.

If a cluster is configured correctly to allow the use of the Cluster API, the Cluster CAPI Operator installs the Cluster API Operator on the cluster.

The Cluster CAPI Operator is distinct from the upstream Cluster API Operator.

For more information, see the entry for the Cluster CAPI Operator in the Cluster Operators reference content.

Primary resources

The Cluster API is comprised of the following primary resources. For the Technology Preview of this feature, you must create these resources manually in the openshift-cluster-api namespace.

Cluster

A fundamental unit that represents a cluster that is managed by the Cluster API.

Infrastructure

A provider-specific resource that defines properties that are shared by all the machine sets in the cluster, such as the region and subnets.

Machine template

A provider-specific template that defines the properties of the machines that a machine set creates.

Machine set

A group of machines.

Machine sets are to machines as replica sets are to pods. If you need more machines or must scale them down, you change the replicas field on the machine set to meet your compute needs.

With the Cluster API, a machine set references a Cluster object and a provider-specific machine template.

Machine

A fundamental unit that describes the host for a node.

The Cluster API creates machines based on the configuration in the machine template.

Additional resources

Sample YAML files

For the Cluster API Technology Preview, you must create the primary resources that the Cluster API requires manually. The example YAML files in this section demonstrate how to make these resources work together and configure settings for the machines that they create that are appropriate for your environment.

Sample YAML for a Cluster API cluster resource

The cluster resource defines the name and infrastructure provider for the cluster and is managed by the Cluster API. This resource has the same structure for all providers.

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <cluster_name> (1)
  namespace: openshift-cluster-api
spec:
  infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    kind: <infrastructure_kind> (2)
    name: <cluster_name> (1)
    namespace: openshift-cluster-api
1 Specify the name of the cluster.
2 Specify the infrastructure kind for the cluster. Valid values are:
  • AWSCluster: The cluster is running on Amazon Web Services (AWS).

  • GCPCluster: The cluster is running on Google Cloud Platform (GCP).

The remaining Cluster API resources are provider-specific. Refer to the example YAML files for your cluster:

Sample YAML files for configuring Amazon Web Services clusters

Some Cluster API resources are provider-specific. The example YAML files in this section show configurations for an Amazon Web Services (AWS) cluster.

Sample YAML for a Cluster API infrastructure resource on Amazon Web Services

The infrastructure resource is provider-specific and defines properties that are shared by all the machine sets in the cluster, such as the region and subnets. The machine set references this resource when creating machines.

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: AWSCluster (1)
metadata:
  name: <cluster_name> (2)
  namespace: openshift-cluster-api
spec:
  region: <region> (3)
1 Specify the infrastructure kind for the cluster. This value must match the value for your platform.
2 Specify the name of the cluster.
3 Specify the AWS region.

Sample YAML for a Cluster API machine template resource on Amazon Web Services

The machine template resource is provider-specific and defines the basic properties of the machines that a machine set creates. The machine set references this template when creating machines.

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
kind: AWSMachineTemplate (1)
metadata:
  name: <template_name> (2)
  namespace: openshift-cluster-api
spec:
  template:
    spec: (3)
      uncompressedUserData: true
      iamInstanceProfile: ....
      instanceType: m5.large
      cloudInit:
        insecureSkipSecretsManager: true
      ami:
        id: ....
      subnet:
        filters:
        - name: tag:Name
          values:
          - ...
      additionalSecurityGroups:
      - filters:
        - name: tag:Name
          values:
          - ...
1 Specify the machine template kind. This value must match the value for your platform.
2 Specify a name for the machine template.
3 Specify the details for your environment. The values here are examples.

Sample YAML for a Cluster API machine set resource on Amazon Web Services

The machine set resource defines additional properties of the machines that it creates. The machine set also references the infrastructure resource and machine template when creating machines.

apiVersion: cluster.x-k8s.io/v1alpha4
kind: MachineSet
metadata:
  name: <machine_set_name> (1)
  namespace: openshift-cluster-api
spec:
  clusterName: <cluster_name> (2)
  replicas: 1
  selector:
    matchLabels:
      test: example
  template:
    metadata:
      labels:
        test: example
    spec:
      bootstrap:
         dataSecretName: worker-user-data (3)
      clusterName: <cluster_name> (2)
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1alpha4
        kind: AWSMachineTemplate (4)
        name: <cluster_name> (2)
1 Specify a name for the machine set.
2 Specify the name of the cluster.
3 For the Cluster API Technology Preview, the Operator can use the worker user data secret from openshift-machine-api namespace.
4 Specify the machine template kind. This value must match the value for your platform.

Sample YAML files for configuring Google Cloud Platform clusters

Some Cluster API resources are provider-specific. The example YAML files in this section show configurations for a Google Cloud Platform (GCP) cluster.

Sample YAML for a Cluster API infrastructure resource on Google Cloud Platform

The infrastructure resource is provider-specific and defines properties that are shared by all the machine sets in the cluster, such as the region and subnets. The machine set references this resource when creating machines.

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: GCPCluster (1)
metadata:
  name: <cluster_name> (2)
spec:
  network:
    name: <cluster_name>-network (2)
  project: <project> (3)
  region: <region> (4)
1 Specify the infrastructure kind for the cluster. This value must match the value for your platform.
2 Specify the name of the cluster.
3 Specify the GCP project name.
4 Specify the GCP region.

Sample YAML for a Cluster API machine template resource on Google Cloud Platform

The machine template resource is provider-specific and defines the basic properties of the machines that a machine set creates. The machine set references this template when creating machines.

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: GCPMachineTemplate (1)
metadata:
  name: <template_name> (2)
  namespace: openshift-cluster-api
spec:
  template:
    spec: (3)
      rootDeviceType: pd-ssd
      rootDeviceSize: 128
      instanceType: n1-standard-4
      image: projects/rhcos-cloud/global/images/rhcos-411-85-202203181601-0-gcp-x86-64
      subnet: <cluster_name>-worker-subnet
      serviceAccounts:
        email: <service_account_email_address>
        scopes:
          - https://www.googleapis.com/auth/cloud-platform
      additionalLabels:
        kubernetes-io-cluster-<cluster_name>: owned
      additionalNetworkTags:
        - <cluster_name>-worker
      ipForwarding: Disabled
1 Specify the machine template kind. This value must match the value for your platform.
2 Specify a name for the machine template.
3 Specify the details for your environment. The values here are examples.

Sample YAML for a Cluster API machine set resource on Google Cloud Platform

The machine set resource defines additional properties of the machines that it creates. The machine set also references the infrastructure resource and machine template when creating machines.

apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineSet
metadata:
  name: <machine_set_name> (1)
  namespace: openshift-cluster-api
spec:
  clusterName: <cluster_name> (2)
  replicas: 1
  selector:
    matchLabels:
      test: test
  template:
    metadata:
      labels:
        test: test
    spec:
      bootstrap:
         dataSecretName: worker-user-data (3)
      clusterName: <cluster_name> (2)
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
        kind: GCPMachineTemplate (4)
        name: <machine_set_name> (1)
      failureDomain: <failure_domain> (5)