After you have set up your environment, you can create an OpenShift cluster that uses the AWS Security Token Service (STS).

About IAM resources for clusters that use STS

To deploy a Red Hat OpenShift Service on AWS (ROSA) cluster that uses the AWS Security Token Service (STS), you must create the following AWS Identity Access Management (IAM) resources:

  • Specific account-wide IAM roles and policies that provide the STS permissions required for ROSA support, installation, control plane and compute functionality. This includes account-wide Operator policies.

  • Cluster-specific Operator IAM roles that permit the ROSA cluster Operators to carry out core OpenShift functionality.

  • An OpenID Connect (OIDC) provider that the cluster Operators use to authenticate.

This section provides an overview about each of the IAM resources that you must deploy when you create a ROSA cluster that uses STS. For detailed steps to create the resources, see the Creating your cluster using STS section.

Account-wide IAM roles and policies

The following account-wide IAM roles and policies are required for ROSA deployments that use STS.

The account-wide roles and policies are specific to an OpenShift version, for example OpenShift 4.8. You can minimize the required STS resources by reusing the account-wide roles and policies for multiple clusters of the same version.

If your use case requires it, you can deploy multiple sets of account-wide IAM roles and policies for a cluster version by specifying different prefixes for each set.

Resource Description

ManagedOpenShift-Installer-Role

An IAM role used by the ROSA installer.

ManagedOpenShift-Installer-Role-Policy

An inline IAM policy that provides the ROSA installer with the permissions required to complete cluster installation tasks.

ManagedOpenShift-Support-Role

An IAM role used by the Red Hat Site Reliability Engineering (SRE) support team.

ManagedOpenShift-Support-Role-Policy

An inline IAM policy that provides the Red Hat SRE support team with the permissions required to support ROSA clusters.

ManagedOpenShift-ControlPlane-Role

An IAM role used by the ROSA control plane.

ManagedOpenShift-ControlPlane-Role-Policy

An inline IAM policy that provides the ROSA control plane with the permissions required to manage its components.

ManagedOpenShift-Worker-Role

An IAM role used by the ROSA compute instances.

ManagedOpenShift-Worker-Role-Policy

An inline IAM policy that provides the ROSA compute instances with the permissions required to manage their components.

ManagedOpenShift-openshift-machine-api-aws-cloud-credentials

A managed IAM policy that provides the ROSA Machine Config Operator with the permissions required to perform core cluster functionality.

ManagedOpenShift-openshift-cloud-credential-operator-cloud-credentials

A managed IAM policy that provides the ROSA Cloud Credential Operator with the permissions required to manage cloud provider credentials.

ManagedOpenShift-openshift-image-registry-installer-cloud-credentials

A managed IAM policy that provides the ROSA Image Registry Operator with the permissions required to manage the internal registry storage in AWS S3 for a cluster.

ManagedOpenShift-openshift-ingress-operator-cloud-credentials

A managed IAM policy that provides the ROSA Ingress Operator with the permissions required to manage external access to a cluster.

ManagedOpenShift-openshift-cluster-csi-drivers-ebs-cloud-credentials

A managed IAM policy required by ROSA to manage back-end storage through the Container Storage Interface (CSI).

The IAM role and policy names include the role prefix that is specified when the STS resources are created. The resource names in the examples provided include the default prefix ManagedOpenShift.

Cluster-specific Operator IAM roles

The following cluster-specific IAM roles are required by the cluster Operators in ROSA deployments that use STS.

When you create the Operator roles by using the rosa CLI, the account-wide Operator policies for the matching cluster version are attached to the roles. The Operator policies are tagged with the Operator and version they are compatible with. The correct policy for an Operator role is determined by using the tags.

If more than one matching policy is available in your account for an Operator role, an interactive list of options is provided when you create the Operator.

Resource Description

ManagedOpenShift-openshift-machine-api-aws-cloud-credentials

An IAM role required by the ROSA Machine Config Operator to perform core cluster functionality.

ManagedOpenShift-openshift-cloud-credential-operator-cloud-credentials

An IAM role required by the ROSA Cloud Credential Operator to cloud provider credentials.

ManagedOpenShift-openshift-image-registry-installer-cloud-credentials

An IAM role required by the ROSA Image Registry Operator to manage the internal registry storage in AWS S3 for a cluster.

ManagedOpenShift-openshift-ingress-operator-cloud-credentials

An IAM role required by the ROSA Ingress Operator to manage external access to a cluster.

ManagedOpenShift-openshift-cluster-csi-drivers-ebs-cloud-credentials

An IAM role required by ROSA to manage back-end storage through the Container Storage Interface (CSI).

An OIDC provider for Operator authentication

For ROSA installations that use STS, you must create a cluster-specific OIDC provider that is used by the cluster Operators to authenticate.

Creating your cluster using STS

Use the Red Hat OpenShift Service on AWS CLI (rosa) to create an OpenShift cluster that uses the AWS Security Token Service (STS).

Only public and AWS PrivateLink clusters are supported with STS. Regular private clusters (non-PrivateLink) are not available for use with STS.

Prerequisites
  • You have completed the AWS prerequisites for ROSA with STS.

  • You have the required AWS service quotas.

  • You have enabled the ROSA service in the AWS Console.

  • You have installed and configured the latest AWS, ROSA, and oc CLIs on your installation host.

AWS Shared VPCs are not currently supported for ROSA installations.

Procedure
  1. Create the required account-wide roles and policies, including Operator policies:

    $ rosa create account-roles --version 4.8 --prefix ManagedOpenShift  (1) (2)
    1 You must specify an OpenShift release by using the --version option. The version that is installed when a ROSA cluster is created is determined by the roles and policies that are used.
    2 The --prefix argument is optional. The ManagedOpenShift prefix is the default. If you specify a different prefix, you must reference it when declaring the Amazon Resource Names (ARNs) when you create the cluster.

    The account-wide roles and policies are specific to a OpenShift version, for example OpenShift 4.8. If you are using multiple cluster versions in one account, you can specify a version-specific prefix to easily identify the roles for each cluster version.

    You can select from the following modes in the interactive prompt:

    • auto: The account roles and policies are created directly using the current AWS account.

    • manual: The role and policy JSON files are saved in the current directory. The command output includes the aws CLI commands required to create the roles. This mode enables you to review the commands before running them manually.

      You can optionally specify the mode by using the --mode auto -y or --mode manual CLI arguments.

  2. Create a cluster by using the interactive prompts, by specifying custom settings, or by using the default options. To view other options when creating a cluster, enter rosa create cluster --help.

    Creating a cluster can take up to 40 minutes.

    Multiple availability zones (Multi-AZ) are recommended for production workloads. The default is a single availability zone. Use --help for an example of how to set this option manually or use interactive mode to be prompted for this setting.

    • To create a cluster with STS using the defaults:

      $ rosa create cluster --cluster-name ${name} --sts
      Example output
      I: Using arn:aws:iam::<account_id>:role/ManagedOpenShift-Installer-Role for the Installer role
      I: Using arn:aws:iam::<account_id>:role/ManagedOpenShift-ControlPlane-Role for the ControlPlane role
      I: Using arn:aws:iam::<account_id>:role/ManagedOpenShift-Worker-Role for the Worker role
      I: Using arn:aws:iam::<account_id>:role/ManagedOpenShift-Support-Role for the Support role
      I: Creating cluster '<cluster_name>'
      I: To view a list of clusters and their status, run 'rosa list clusters'
      I: Cluster '<cluster_name>' has been created.
      I: Once the cluster is installed you will need to add an Identity Provider before you can login into the cluster. See 'rosa create idp --help' for more information.
      I: To determine when your cluster is Ready, run 'rosa describe cluster -c <cluster_name>'.
      I: To watch your cluster installation logs, run 'rosa logs install -c <cluster_name> --watch'.
      ...

      If more than one matching set of account-wide roles are available in your account for a cluster version, an interactive list of options is provided.

    • To create a cluster with STS using interactive prompts:

      $ rosa create cluster --interactive
      Example output
      I: Interactive mode enabled.
      Any optional fields can be left empty and a default will be selected.
      ? Cluster name: <cluster_name>
      ? Deploy cluster using AWS STS: Yes
      ? OpenShift version: 4.8.9
      I: Using arn:aws:iam::<account_id>:role/ManagedOpenShift-Installer-Role for the Installer role
      I: Using arn:aws:iam::<account_id>:role/ManagedOpenShift-ControlPlane-Role for the ControlPlane role
      I: Using arn:aws:iam::<account_id>:role/ManagedOpenShift-Worker-Role for the Worker role
      I: Using arn:aws:iam::<account_id>:role/ManagedOpenShift-Support-Role for the Support role
      ? External ID (optional):
      ? Operator roles prefix: <cluster_name>-z9y3
      ? Multiple availability zones (optional): No
      ? AWS region: us-east-1
      ? PrivateLink cluster (optional): No
      ? Install into an existing VPC (optional): No
      ? Enable Customer Managed key (optional): No
      ? Compute nodes instance type (optional):
      ? Enable autoscaling (optional): No
      ? Compute nodes: 2
      ? Machine CIDR: 10.0.0.0/16
      ? Service CIDR: 172.30.0.0/16
      ? Pod CIDR: 10.128.0.0/14
      ? Host prefix: 23
      I: Creating cluster '<cluster_name>'
      I: To create this cluster again in the future, you can run:
         rosa create cluster --cluster-name <cluster_name> --role-arn arn:aws:iam::<account_id>:role/ManagedOpenShift-Installer-Role --support-role-arn arn:aws:iam::<account_id>:role/ManagedOpenShift-Support-Role --master-iam-role arn:aws:iam::<account_id>:role/ManagedOpenShift-ControlPlane-Role --worker-iam-role arn:aws:iam::<account_id>:role/ManagedOpenShift-Worker-Role --operator-roles-prefix <cluster_name>-z9y3 --region us-east-1 --version 4.8.9 --compute-nodes 2 --machine-cidr 10.0.0.0/16 --service-cidr 172.30.0.0/16 --pod-cidr 10.128.0.0/14 --host-prefix 23
      I: To view a list of clusters and their status, run 'rosa list clusters'
      I: Cluster '<cluster_name>' has been created.
      I: Once the cluster is installed you will need to add an Identity Provider before you can login into the cluster. See 'rosa create idp --help' for more information.
      I: To determine when your cluster is Ready, run 'rosa describe cluster -c <cluster_name>'.
      I: To watch your cluster installation logs, run 'rosa logs install -c <cluster_name> --watch'.
    • To create a cluster with STS by specifying the installation options from the CLI:

      $ rosa create cluster \
        --cluster-name ${name} \
        --region ${region} \
        --version ${version} \
        --role-arn arn:aws:iam::${aws_account_id}:role/ManagedOpenShift-Installer-Role \ (1)
        --support-role-arn arn:aws:iam::${aws_account_id}:role/ManagedOpenShift-Support-Role \
        --master-iam-role arn:aws:iam::${aws_account_id}:role/ManagedOpenShift-ControlPlane-Role \
        --worker-iam-role arn:aws:iam::${aws_account_id}:role/ManagedOpenShift-Worker-Role \
        --operator-roles-prefix ManagedOpenShift (2)
      1 If you specified a custom prefix in the preceding command, you must replace the ManagedOpenShift prefix with the custom one in each ARN declaration. You must specify ARNs for STS account-wide roles that are created using rosa create account-roles.
      2 Declares the prefix for the cluster-specific Operator roles that are defined in the following step.
      Example output
      I: Creating cluster '<cluster_name>'
      I: To view a list of clusters and their status, run 'rosa list clusters'
      I: Cluster '<cluster_name>' has been created.
      I: Once the cluster is installed you will need to add an Identity Provider before you can login into the cluster. See 'rosa create idp --help' for more information.
      I: To determine when your cluster is Ready, run 'rosa describe cluster -c <cluster_name>'.
      I: To watch your cluster installation logs, run 'rosa logs install -c <cluster_name> --watch'.

      You can configure the following default network IP ranges:

      • Machine CIDR: 10.0.0.0/16

      • Service CIDR: 172.30.0.0/16

      • Pod CIDR: 10.128.0.0/14

      For the CIDR-related rosa CLI arguments, see rosa create cluster --help | grep cidr. In the interactive mode, you are prompted for the settings.

      The cluster state is Pending until the following steps are complete.

  3. Create the cluster-specific Operator IAM roles:

    $ rosa create operator-roles --cluster <cluster_name|cluster_id> (1)
    1 Replace <cluster_name|cluster_id> with the cluster name or the ID of the cluster.

    You can select from the following modes in the interactive prompt:

    • auto: The Operator roles are created directly using the current AWS account.

    • manual: The role JSON files are saved in the current directory. The command output includes the aws CLI commands required to create the roles. This mode enables you to review the commands before running them manually.

      You can optionally specify the mode by using the --mode auto -y or --mode manual CLI arguments.

  4. Create the OpenID Connect (OIDC) provider that the cluster Operators will use to authenticate:

    $ rosa create oidc-provider --cluster <cluster_name|cluster_id> (1)
    1 Replace <cluster_name|cluster_id> with the cluster name or the ID of the cluster.

    You can select from the following modes in the interactive prompt:

    • auto: The OIDC provider is created directly using the current AWS account.

    • manual: The command output includes the aws CLI commands required to create the OIDC provider, including the thumbprint. This mode enables you to review the commands before running them manually.

      You can optionally specify the mode by using the --mode auto -y or --mode manual CLI arguments.

  5. Check the status of your cluster and retrieve your cluster ID. The State field changes from pending to installing to ready:

    $ rosa describe cluster --cluster=<cluster_name|cluster_id> (1)
    1 Replace <cluster_name|cluster_id> with the cluster name or the ID of the cluster.
    Example output
    Name:                       <cluster_name>
    ID:                         <cluster_id>
    External ID:                <external_id>
    OpenShift Version:          <version>
    Channel Group:              stable
    DNS:                        *.openshiftapps.com
    AWS Account:                123456789012
    API URL:                    https://api.<cluster_name>.openshiftapps.com:6443
    Console URL:                https://console-openshift-console.apps.<cluster_name>.openshiftapps.com
    Region:                     <region>
    Multi-AZ:                   false
    Nodes:
     - Master:                  3
     - Infra:                   2
     - Compute:                 2
    Network:
     - Service CIDR:            172.30.0.0/16
     - Machine CIDR:            10.0.0.0/16
     - Pod CIDR:                10.128.0.0/14
     - Host Prefix:             /23
    State:                      pending (Waiting for OIDC configuration)
    Private:                    No
    Created:                    Jun 10 2021 15:47:56 UTC
    Details Page:               https://cloud.redhat.com/openshift/details/s/<subscription_id>
    OIDC Endpoint URL:          https://rh-oidc.s3.us-east-1.amazonaws.com/<cluster_id>

    If installation fails or the State field does not change to ready after 40 minutes, check the installation troubleshooting documentation for more details.

  6. Track the progress of the cluster creation by watching the OpenShift installer logs:

    $ rosa logs install --cluster=<cluster_name|cluster_id> --watch (1) (2)
    1 Replace <cluster_name|cluster_id> with the cluster name or the ID of the cluster.
    2 Specify the --watch flag to watch for new log messages as the installation progresses. This argument is optional.

Additional resources