SupportConsoleDevelopersStart a trialContact

Chapter 8. Create Red Hat OpenShift Service on AWS clusters with egress zero

You can create a Red Hat OpenShift Service on AWS cluster with egress zero to pull Red Hat container images from a regional Amazon Elastic Container Registry (ECR) instead of from the public internet. This enhances cluster stability and security.

Your cluster first tries to pull the images from Quay, and when they are not reached, it instead pulls the images from the image registry in the local region.

Clusters with egress zero pull Red Hat container images from an ECR in the cluster’s local region instead of from registries on the public internet. ECR provides storage for OpenShift release images as well as Red Hat Operators. All requests for ECR are kept within your AWS network by serving them over a VPC endpoint within your cluster.

Red Hat OpenShift Service on AWS clusters with egress zero use AWS ECR to provision your clusters without the need for public internet. Because necessary cluster lifecycle processes occur over AWS private networking, AWS ECR serves as a critical service for core cluster platform images.

You can create a fully operational cluster that does not require a public egress by configuring a virtual private cloud (VPC) and using the --properties zero_egress:true flag when creating your cluster.

Note

Clusters created in restricted network environments cannot use certain Red Hat OpenShift Service on AWS features, such as Red Hat Lightspeed and Telemetry, and might experience failures for workloads that require public access to registries such as quay.io. With egress zero, you can install Red Hat-owned Operators from the software catalog, but only the default Operator channel is mirrored. For a complete list of Red Hat-owned Operators, see the Red Hat Ecosystem Catalog.

Before you can create a Red Hat OpenShift Service on AWS cluster, you must complete several prerequisites, such as configuring a virtual private cloud (VPC) and creating the required account-wide IAM roles and resources.

Before you begin the egress zero install procedure, ensure you have the following:

  • An AWS account with sufficient permissions to create VPCs, subnets, and other required infrastructure
  • Terraform v1.4.0 or higher installed
  • ROSA command-line interface (CLI) (rosa) v1.2.45 or higher installed
  • AWS CLI installed and configured with the necessary credentials
  • git CLI installed
  • The necessary rosa CLI and Red Hat Hybrid Cloud Console firewall rules enabled
Important
  • You can use egress zero on all supported Red Hat OpenShift Service on AWS versions. Red Hat recommends using the latest available z-stream release for each OpenShift Container Platform version.
  • Due to an upstream issue with the internal image registry in disconnected environments, clusters with egress zero cannot fully use all platform components. You can restore these features by upgrading or installing your cluster with the latest OpenShift Container Platform version.

8.2. Glossary of network environment terms
Copy link

The Red Hat OpenShift Service on AWS documentation uses several terms to describe network environments with limited or no internet connectivity. These environments differ from a standard network and might require additional configuration.

The following table describes the different terms used to refer to environments without a full internet connection:

Table 8.1. Disconnected environment terms
TermDescription

Air-gapped network

An environment or network that is completely isolated from an external network.

This isolation depends on a physical separation, or an "air gap", between machines on the internal network and any part of an external network. Air-gapped environments are often used in industries with strict security or regulatory requirements.

Disconnected environment

An environment or network that has some level of isolation from an external network.

This isolation could be enabled by physical or logical separation between machines on the internal network and an external network. A cluster in a disconnected environment does not have access to Red Hat public services and requires additional setup to maintain full cluster functionality.

Restricted network

An environment or network with limited connection to an external network.

A physical connection might exist between machines on the internal network and an external network, but network traffic is limited by additional configurations, such as firewalls and proxies.

You can set environment variables to streamline resource creation for your Red Hat OpenShift Service on AWS cluster.

Procedure

  • Set your environment variable by running the following command: 

    $ export <variable_name>=<variable_value>
    Table 8.2. Suggested variables for disconnected Red Hat OpenShift Service on AWS clusters
    Variable nameVariable valueNotes

    AWS_ACCOUNT_ID

    $(aws sts get-caller-identity --query Account --output text)

    You must be logged in to your AWS account with rosa login.

    CLUSTER_NAME

    The name you want for your cluster.

    Your cluster name cannot exceed 26 characters.

    OIDC_ID

    The 32-digit ID for your OpenID Connect (OIDC) configuration.

    You generate this ID by running rosa create oidc-config.

    OPERATOR_ROLES_PREFIX

    The Operator role prefix.

    If you want to make your AWS account roles use the same prefix as your Operator roles, you can run ACCOUNT_ROLES_PREFIX=$OPERATOR_ROLES_PREFIX after setting your Operator role prefix variable.

    PRIVATE_SUBNET

    The ID of your private subnets.

    You must enclose this value in quotation marks (") and separate the subnet IDs with commas.

    REGION

    Your AWS region.

    -

    SUBNET_IDS

    The IDs of all your subnets.

    You must enclose this value in quotation marks (") and separate the subnet IDs with commas.

You must have a Virtual Private Cloud (VPC) to create a Red Hat OpenShift Service on AWS cluster. To pull images from the local Amazon Elastic Container Registry (ECR) mirror over your VPC endpoint, you must configure a privatelink service connection and modify the default security groups with specific tags.

You can create a VPC by using one of these methods:

  • Create a VPC by using the ROSA command-line interface (CLI)
  • Create a VPC by using a Terraform template
  • Create a VPC by using the AWS CLI
  • Create the VPC resources manually in the AWS console

8.4.1. Create an AWS VPC using the ROSA CLI
Copy link

The rosa create network command is available in v.1.2.48 or later of the ROSA command-line interface (CLI) (rosa). The command uses AWS CloudFormation to create a virtual private cloud (VPC) and associated networking components necessary to install a Red Hat OpenShift Service on AWS cluster. CloudFormation is a native AWS infrastructure-as-code tool and is compatible with the AWS CLI.

If you do not specify a template, CloudFormation uses a default template that creates resources with the following parameters:

Table 8.3. Default VPC parameters
VPC parameterValue

Availability zones

1

Region

us-east-1

VPC CIDR

10.0.0.0/16

You can create and customize CloudFormation templates to use with the rosa create network command. See the additional resources of this section for information on the default VPC template.

Prerequisites

  • You have configured your AWS account
  • You have configured your Red Hat accounts
  • You have installed the rosa CLI and configured it to the latest version

Procedure

  1. Create an AWS VPC using the default CloudFormations template by running the following command: 

    $ rosa create network

  2. Optional: Customize your VPC by specifying additional parameters.

    You can use the --param flag to specify changes to the default VPC template. The following example command specifies custom values for region, Name, AvailabilityZoneCount and VpcCidr. 

    $ rosa create network --param Region=us-east-2 --param Name=quickstart-stack --param AvailabilityZoneCount=3 --param VpcCidr=10.0.0.0/16

    The command takes about 5 minutes to run and provides regular status updates from AWS as resources are created. If there is an issue with CloudFormation, a rollback is attempted. For all other errors that are encountered, please follow the error message instructions or contact AWS support.

Verification

  • When completed, you receive a summary of the created resources: 

    INFO[0140] Resources created in stack:
INFO[0140] Resource: AttachGateway, Type: AWS::EC2::VPCGatewayAttachment, ID: <gateway_id>
INFO[0140] Resource: EC2VPCEndpoint, Type: AWS::EC2::VPCEndpoint, ID: <vpce_id>
INFO[0140] Resource: EcrApiVPCEndpoint, Type: AWS::EC2::VPCEndpoint, ID: <vpce_id>
INFO[0140] Resource: EcrDkrVPCEndpoint, Type: AWS::EC2::VPCEndpoint, ID: <vpce_id>
INFO[0140] Resource: ElasticIP1, Type: AWS::EC2::EIP, ID: <IP>
INFO[0140] Resource: ElasticIP2, Type: AWS::EC2::EIP, ID: <IP>
INFO[0140] Resource: InternetGateway, Type: AWS::EC2::InternetGateway, ID: igw-016e1a71b9812464e
INFO[0140] Resource: KMSVPCEndpoint, Type: AWS::EC2::VPCEndpoint, ID: <vpce_id>
INFO[0140] Resource: NATGateway1, Type: AWS::EC2::NatGateway, ID: <nat-gateway_id>
INFO[0140] Resource: PrivateRoute, Type: AWS::EC2::Route, ID: <route_id>
INFO[0140] Resource: PrivateRouteTable, Type: AWS::EC2::RouteTable, ID: <route_id>
INFO[0140] Resource: PrivateSubnetRouteTableAssociation1, Type: AWS::EC2::SubnetRouteTableAssociation, ID: <route_id>
INFO[0140] Resource: PublicRoute, Type: AWS::EC2::Route, ID: <route_id>
INFO[0140] Resource: PublicRouteTable, Type: AWS::EC2::RouteTable, ID: <route_id>
INFO[0140] Resource: PublicSubnetRouteTableAssociation1, Type: AWS::EC2::SubnetRouteTableAssociation, ID: <route_id>
INFO[0140] Resource: S3VPCEndpoint, Type: AWS::EC2::VPCEndpoint, ID: <vpce_id>
INFO[0140] Resource: STSVPCEndpoint, Type: AWS::EC2::VPCEndpoint, ID: <vpce_id>
INFO[0140] Resource: SecurityGroup, Type: AWS::EC2::SecurityGroup, ID: <security-group_id>
INFO[0140] Resource: SubnetPrivate1, Type: AWS::EC2::Subnet, ID: <private_subnet_id-1>
INFO[0140] Resource: SubnetPublic1, Type: AWS::EC2::Subnet, ID: <public_subnet_id-1>
INFO[0140] Resource: VPC, Type: AWS::EC2::VPC, ID: <vpc_id>
INFO[0140] Stack rosa-network-stack-5555 created
    • The <private_subnet_id-1> and <public_subnet_id-1> subnet IDs are used to create your cluster when using the rosa create cluster command.
    • The network stack name (rosa-network-stack-5555) is used to delete the resource later.

8.4.1.1. Tag your subnets
Copy link

Before you can use your virtual private cloud (VPC) to create a Red Hat OpenShift Service on AWS cluster, you must tag your VPC subnets. Automated service preflight checks verify that these resources are tagged correctly.

The following table shows how to tag your resources:

Table 8.4. Required subnet tags
ResourceKeyValue

Public subnet

kubernetes.io/role/elb

1 or no value

Private subnet

kubernetes.io/role/internal-elb

1 or no value

Note

You must tag at least one private subnet and one public subnet, if applicable.

Procedure

  • Tag your resources in your terminal:

    • For public subnets, run the following command: 

      $ aws ec2 create-tags --resources <public_subnet_id> --region <aws_region> --tags Key=kubernetes.io/role/elb,Value=1

    • For private subnets, run the following command: 

      $ aws ec2 create-tags --resources <private_subnet_id> --region <aws_region> --tags Key=kubernetes.io/role/internal-elb,Value=1

Verification

  • Verify that the tag is correct by running the following command: 

    $ aws ec2 describe-tags --filters "Name=resource-id,Values=<subnet_id>"

    Example output

    TAGS    Name                    <subnet_id>        subnet  <prefix>-subnet-public1-us-east-1a
TAGS    kubernetes.io/role/elb  <subnet_id>        subnet  1

Terraform is a tool that allows you to create various resources using an established template. You can use Terraform with default options to create a Virtual Private Cloud for your Red Hat OpenShift Service on AWS cluster.

Note

The Terraform instructions are for testing and demonstration purposes. Your own installation requires some modifications to the VPC for your own use. Use this Terraform script in the same region where you intend to install your cluster. These examples use us-east-2.

Prerequisites

  • You have installed Terraform version 1.4.0 or newer on your machine.
  • You have installed Git on your machine.

Procedure

  1. Open a shell prompt and clone the Terraform VPC repository by running the following command: 

    $ git clone https://github.com/openshift-cs/terraform-vpc-example

  2. Navigate to the created directory by running the following command: 

    $ cd terraform-vpc-example/zero-egress

  3. Initiate the Terraform file by running the following command: 

    $ terraform init

    A message confirming the initialization appears when this process completes.

  4. To build your VPC Terraform plan based on the existing Terraform template, run the plan command. You must include your AWS region, availability zones, CIDR blocks, and private subnets. You can choose to specify a cluster name. A rosa-zero-egress.tfplan file is added to the hypershift-tf directory after the terraform plan completes. For more detailed options, see the Terraform VPC repository’s README file. 

    $ terraform plan -out rosa-zero-egress.tfplan -var region=<aws_region> \
      -var 'availability_zones=<availability_zones>' \
      -var vpc_cidr_block=<vpc_cidr_block> \
      -var 'private_subnets=<private_subnets>'

    where:

    <aws_region>
    Enter your AWS region.
    <availability_zones>
    Enter the availability zones for the VPC. For example, for a VPC that uses ap-southeast-1, you would use the following as availability zones: ["ap-southeast-1a", "ap-southeast-1b", "ap-southeast-1c"].
    <vpc_cidr_block>
    Enter the CIDR block for your VPC. For example, 10.0.0.0/16.
    <private_subnets>
    Enter each of the subnets that are created for the VPC. For example, ["10.0.0.0/24", "10.0.1.0/24", "10.0.2.0/24"].

  5. Apply this plan file to build your VPC by running the following command: 

    $ terraform apply rosa-zero-egress.tfplan

8.4.2.1. Tag your subnets
Copy link

Before you can use your virtual private cloud (VPC) to create a Red Hat OpenShift Service on AWS cluster, you must tag your VPC subnets. Automated service preflight checks verify that these resources are tagged correctly.

The following table shows how to tag your resources:

Table 8.5. Required subnet tags
ResourceKeyValue

Public subnet

kubernetes.io/role/elb

1 or no value

Private subnet

kubernetes.io/role/internal-elb

1 or no value

Note

You must tag at least one private subnet and one public subnet, if applicable.

Procedure

  • Tag your resources in your terminal:

    • For public subnets, run the following command: 

      $ aws ec2 create-tags --resources <public_subnet_id> --region <aws_region> --tags Key=kubernetes.io/role/elb,Value=1

    • For private subnets, run the following command: 

      $ aws ec2 create-tags --resources <private_subnet_id> --region <aws_region> --tags Key=kubernetes.io/role/internal-elb,Value=1

Verification

  • Verify that the tag is correct by running the following command: 

    $ aws ec2 describe-tags --filters "Name=resource-id,Values=<subnet_id>"

    Example output

    TAGS    Name                    <subnet_id>        subnet  <prefix>-subnet-public1-us-east-1a
TAGS    kubernetes.io/role/elb  <subnet_id>        subnet  1

8.4.3. Create a VPC using the AWS CLI
Copy link

You can create a virtual private cloud (VPC) by using the AWS command-line interface (CLI).

If you do not use a managed infrastructure tool such as Terraform or AWS CloudFormation to create your Virtual Private Cloud (VPC), you can create it manually through the AWS console. A manually created VPC must meet specific requirements for use with Red Hat OpenShift Service on AWS.

Your VPC must meet the requirements shown in the following table.

Table 8.6. Requirements for your VPC
RequirementDetails

VPC name

You need to have the specific VPC name and ID when creating your cluster.

CIDR range

Your VPC CIDR range should match your machine CIDR.

Availability zone

You need one availability zone for a single zone, and you need three for availability zones for multi-zone.

Public subnet

You must have one public subnet with a NAT gateway for public clusters. Private clusters do not need a public subnet.

DNS hostname and resolution

You must ensure that the DNS hostname and resolution are enabled.

8.4.4.1. Tag your subnets
Copy link

If you created your own virtual private cloud (VPC) to create a Red Hat OpenShift Service on AWS cluster, you must tag your VPC subnets. Automated service preflight checks verify that these resources are tagged correctly before you can use these resources for a cluster.

Table 8.7. Required subnet tags
ResourceKeyValue

Public subnet

kubernetes.io/role/elb

1 (or no value)

Private subnet

kubernetes.io/role/internal-elb

1 (or no value)

Note

You must tag at least one private subnet and, if applicable, one public subnet.

Prerequisites

  • You have created a VPC.
  • You have installed the aws command-line interface (CLI).

Procedure

  • Tag your resources in your terminal by running the following commands:

    1. For public subnets, run: 

      $ aws ec2 create-tags --resources <public-subnet-id> --region <aws_region> --tags Key=kubernetes.io/role/elb,Value=1

    2. For private subnets, run: 

      $ aws ec2 create-tags --resources <private-subnet-id> --region <aws_region> --tags Key=kubernetes.io/role/internal-elb,Value=1

Verification

  • Verify that the tag is correctly applied by running the following command: 

    $ aws ec2 describe-tags --filters "Name=resource-id,Values=<subnet_id>"

    For example: 

    TAGS    Name                    <subnet-id>        subnet  <prefix>-subnet-public1-us-east-1a
TAGS    kubernetes.io/role/elb  <subnet-id>        subnet  1

8.4.5. Troubleshoot cluster creation
Copy link

If your cluster fails to install, common virtual private cloud (VPC) configuration issues might be the cause.

  • Ensure your DHCP option set includes a domain name, and ensure that the domain name does not include any spaces or capital letters.
  • If your VPC uses a custom DNS resolver (the domain name servers field is not AmazonProvideDNS), ensure that it can resolve the private hosted zones in Route53.

8.5. Create an ocm-role IAM role
Copy link

Create the ocm-role Identity and Access Management (IAM) roles by using the ROSA command-line interface (CLI) (rosa) to grant OpenShift Cluster Manager the permissions needed to install and manage your Red Hat OpenShift Service on AWS clusters.

If you want to create and manage clusters by using only the rosa CLI and the OpenShift CLI (oc), you can use the --no-console profile for the ocm-role IAM resource. For more information about the ocm-role IAM resource permissions profile, see the Additional resources.

Important

You must create the ocm-role IAM role before you can create your Red Hat OpenShift Service on AWS cluster.

Prerequisites

  • You have an AWS account.
  • You have Red Hat Organization Administrator privileges in the OpenShift Cluster Manager organization.
  • You have the permissions required to install AWS account-wide roles.
  • You have installed and configured the latest rosa CLI, rosa, on your installation host.

Procedure

  • Run one of the following commands to create the required ocm-role IAM resource:

    Important

    The process to change your ocm-role IAM resource profile requires you to unlink and delete the current ocm-role IAM resource and create a new one with the required profile.

    • To create an ocm-role IAM role with standard privileges, run the following command: 

      $ rosa create ocm-role

    • To create an ocm-role IAM role with admin privileges, run the following command:

      Important

      The admin profile supports "auto" mode configuration for Red Hat OpenShift Service on AWS clusters which provisions OIDC Configuration and Operator roles automatically. To achieve this automatic flow, the profile has a wider set of permissions than the standard profile. 

      $ rosa create ocm-role --admin

      This command allows you to create the role by specifying specific attributes. The following example output shows the "auto mode" selected, which lets the rosa CLI (rosa) create your Operator roles and policies. See "Methods of account-wide role creation" for more information. The following example shows what your creation flow might look like. 

      I: Creating ocm role
? Role prefix: ManagedOpenShift
? Enable admin capabilities for the OCM role (optional): No
? Permissions boundary ARN (optional):
? Role Path (optional):
? Role creation mode: auto
I: Creating role using 'arn:aws:iam::<ARN>:user/<UserName>'
? Create the 'ManagedOpenShift-OCM-Role-182' role? Yes
I: Created role 'ManagedOpenShift-OCM-Role-182' with ARN  'arn:aws:iam::<ARN>:role/ManagedOpenShift-OCM-Role-182'
I: Linking OCM role
? OCM Role ARN: arn:aws:iam::<ARN>:role/ManagedOpenShift-OCM-Role-182
? Link the 'arn:aws:iam::<ARN>:role/ManagedOpenShift-OCM-Role-182' role with organization '<AWS ARN>'? Yes
I: Successfully linked role-arn 'arn:aws:iam::<ARN>:role/ManagedOpenShift-OCM-Role-182' with organization account '<AWS ARN>'

      where:

      Role prefix
      A prefix value for all of the created AWS resources. In this example, ManagedOpenShift prepends all of the AWS resources.
      Enable admin capabilities for the OCM role (optional)

      Choose if you want this role to have the additional admin permissions.

      Note

      You do not see this prompt if you used the --admin option.

      Permissions boundary ARN (optional)
      The Amazon Resource Name (ARN) of the policy to set permission boundaries.
      Role Path (optional)
      Specify an IAM path for the user name.
      Role creation mode
      Choose the method to create your AWS roles. By using auto, the rosa CLI generates and links the roles and policies. In the auto mode, you receive some different prompts to create the AWS roles.
      Create the 'ManagedOpenShift-OCM-Role-182' role?
      The auto method asks if you want to create a specific ocm-role by using your prefix.
      OCM Role ARN
      Confirm that you want to associate your IAM role with your OpenShift Cluster Manager.
      Link the 'arn:aws:iam::<ARN>:role/ManagedOpenShift-OCM-Role-182' role with organization '<AWS ARN>'?
      Links the created role with your AWS organization.

    • To create an ocm-role IAM role with the minimum required privileges, run the following command:

      Note

      While the no-console profile offers the minimum permissions policy that can still create Red Hat OpenShift Service on AWS clusters, the permissions are insufficient if you want to use OpenShift Cluster Manager for cluster creation. 

      $ rosa create ocm-role --no-console

Before you create a Red Hat OpenShift Service on AWS cluster, you must create the required account-wide IAM roles and policies by using the ROSA command-line interface (CLI) (rosa).

Note

Specific AWS-managed policies for Red Hat OpenShift Service on AWS must be attached to each role. Customer-managed policies must not be used with these required account roles. For more information regarding AWS-managed policies for Red Hat OpenShift Service on AWS clusters, see AWS managed policies for Red Hat OpenShift Service on AWS.

Prerequisites

  • You have completed the AWS prerequisites for Red Hat OpenShift Service on AWS.
  • You have available AWS service quotas.
  • You have enabled the Red Hat OpenShift Service on AWS in the AWS Console.
  • You have installed and configured the latest ROSA command-line interface (CLI) (rosa) on your installation host.
  • You have logged in to your Red Hat account by using the rosa CLI.

Procedure

  1. If they do not exist in your AWS account, create the required account-wide STS roles and attach the policies by running the following command: 

    $ rosa create account-roles --hosted-cp

  2. Verify that your worker role has the correct AWS policy by running the following command: 

    $ aws iam attach-role-policy \
--role-name ManagedOpenShift-HCP-ROSA-Worker-Role \
--policy-arn "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"

    --role-name ManagedOpenShift-HCP-ROSA-Worker-Role::This role needs to include the prefix that was created in the previous step.

  3. Optional: Set your prefix as an environmental variable by running the following command: 

    $ export ACCOUNT_ROLES_PREFIX=<account_role_prefix>

    • View the value of the variable by running the following command: 

      $ echo $ACCOUNT_ROLES_PREFIX

      Example output

      ManagedOpenShift

      Note

      As an additional safeguard, after romodules/rosa-hcp-creating-account-wide-sts-roles-and-policies.adocle creation, you can manually update the trust policies of the Support and Installer account-wide roles to include an external ID. For more information, see About external ID.

8.7. Create an OpenID Connect configuration
Copy link

Create an OpenID Connect (OIDC) configuration before creating your Red Hat OpenShift Service on AWS cluster to authenticate Operator access to required AWS resources by using the AWS Security Token Service (STS). Each production cluster requires its own OIDC configuration.

Prerequisites

  • You have completed the AWS prerequisites for Red Hat OpenShift Service on AWS.
  • You have installed and configured the latest ROSA command-line interface (CLI) (rosa) on your installation host.

Procedure

  1. To create your OIDC configuration alongside the AWS resources, run the following command: 

    $ rosa create oidc-config --mode=auto --yes

    Example output

    ? Would you like to create a Managed (Red{nbsp}Hat hosted) OIDC Configuration Yes
I: Setting up managed OIDC configuration
I: To create Operator Roles for this OIDC Configuration, run the following command and remember to replace <user_defined> with a prefix of your choice:
	rosa create operator-roles --prefix <user_defined> --oidc-config-id 13cdr6b
If you are going to create a Hosted Control Plane cluster please include '--hosted-cp'
I: Creating OIDC provider using 'arn:aws:iam::4540112244:user/userName'
? Create the OIDC provider? Yes
I: Created OIDC provider with ARN 'arn:aws:iam::4540112244:oidc-provider/dvbwgdztaeq9o.cloudfront.net/13cdr6b'

    When creating your cluster, you must supply the OIDC config ID. The CLI output provides this value for --mode auto, otherwise you must determine these values based on aws CLI output for --mode manual.

  2. Optional: you can save the OIDC configuration ID as a variable to use later. Run the following command to save the variable: 

    $ export OIDC_ID=<oidc_config_id>
    <oidc_config_id>
    In this example output, the OIDC configuration ID is 13cdr6b.

    • View the value of the variable by running the following command: 

      $ echo $OIDC_ID

      Example output

      13cdr6b

Verification

  • You can list the possible OIDC configurations available for your clusters that are associated with your user organization. Run the following command: 

    $ rosa list oidc-config

    Example output

    ID                                MANAGED  ISSUER URL                                                             SECRET ARN
2330dbs0n8m3chkkr25gkkcd8pnj3lk2  true     https://dvbwgdztaeq9o.cloudfront.net/2330dbs0n8m3chkkr25gkkcd8pnj3lk2
233hvnrjoqu14jltk6lhbhf2tj11f8un  false    https://oidc-r7u1.s3.us-east-1.amazonaws.com                           aws:secretsmanager:us-east-1:242819244:secret:rosa-private-key-oidc-r7u1-tM3MDN

8.8. Create Operator roles and policies
Copy link

When you deploy a Red Hat OpenShift Service on AWS cluster, you must create the Operator Identity and Access Management (IAM) roles. The cluster Operators use the Operator roles and policies to obtain temporary permissions to perform cluster operations, such as managing storage and external access.

Prerequisites

  • You have completed the AWS prerequisites for Red Hat OpenShift Service on AWS.
  • You have installed and configured the latest ROSA command-line interface (CLI) (rosa) on your installation host.
  • You created the account-wide AWS roles.

Procedure

  1. To create your Operator roles, run the following command: 

    $ rosa create operator-roles --hosted-cp --prefix=$OPERATOR_ROLES_PREFIX --oidc-config-id=$OIDC_ID --installer-role-arn arn:aws:iam::$AWS_ACCOUNT_ID:role/${ACCOUNT_ROLES_PREFIX}-HCP-ROSA-Installer-Role

    The following breakdown provides options for the Operator role creation. 

    $ rosa create operator-roles --hosted-cp
	--prefix=$OPERATOR_ROLES_PREFIX
	--oidc-config-id=$OIDC_ID
	--installer-role-arn arn:aws:iam::$AWS_ACCOUNT_ID:role/$ACCOUNT_ROLES_PREFIX-HCP-ROSA-Installer-Role

    where:

    --prefix=
    You must supply a prefix when creating these Operator roles. Failing to do so produces an error. See the Additional resources of this section for information on the Operator prefix.
    --oidc-config-id=
    This value is the OpenID Connect (OIDC) configuration ID that you created for your Red Hat OpenShift Service on AWS cluster.
    --installer-role-arn
    This value is the installer role Amazon Resource Name (ARN) that you created when you created the Red Hat OpenShift Service on AWS account roles.

    You must include the --hosted-cp parameter to create the correct roles for Red Hat OpenShift Service on AWS clusters. This command returns the following information.

    Example output

    ? Role creation mode: auto
? Operator roles prefix: <pre-filled_prefix>
? OIDC Configuration ID: 23soa2bgvpek9kmes9s7os0a39i13qm4 | https://dvbwgdztaeq9o.cloudfront.net/23soa2bgvpek9kmes9s7os0a39i13qm4
? Create hosted control plane operator roles: Yes
W: More than one Installer role found
? Installer role ARN: arn:aws:iam::4540112244:role/<prefix>-HCP-ROSA-Installer-Role
? Permissions boundary ARN (optional):
I: Reusable OIDC Configuration detected. Validating trusted relationships to operator roles:
I: Creating roles using 'arn:aws:iam::4540112244:user/<userName>'
I: Created role '<prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials'
I: Created role '<prefix>-openshift-cloud-network-config-controller-cloud-credenti' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-cloud-network-config-controller-cloud-credenti'
I: Created role '<prefix>-kube-system-kube-controller-manager' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-kube-controller-manager'
I: Created role '<prefix>-kube-system-capa-controller-manager' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-capa-controller-manager'
I: Created role '<prefix>-kube-system-control-plane-operator' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-control-plane-operator'
I: Created role '<prefix>-kube-system-kms-provider' with ARN 'arn:aws:iam::4540112244:role/<prefix>-kube-system-kms-provider'
I: Created role '<prefix>-openshift-image-registry-installer-cloud-credentials' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-image-registry-installer-cloud-credentials'
I: Created role '<prefix>-openshift-ingress-operator-cloud-credentials' with ARN 'arn:aws:iam::4540112244:role/<prefix>-openshift-ingress-operator-cloud-credentials'
I: To create a cluster with these roles, run the following command:
	rosa create cluster --sts --oidc-config-id 23soa2bgvpek9kmes9s7os0a39i13qm4 --operator-roles-prefix <prefix> --hosted-cp

    where:

    Operator roles prefix
    This field is prepopulated with the prefix that you set in the initial creation command.
    OIDC Configuration ID
    This field requires you to select an OIDC configuration that you created for your Red Hat OpenShift Service on AWS cluster.

    The Operator roles are now created and ready to use for creating your Red Hat OpenShift Service on AWS cluster.

Verification

  • You can list the Operator roles associated with your Red Hat OpenShift Service on AWS account. Run the following command: 

    $ rosa list operator-roles

    Example output

    I: Fetching operator roles
ROLE PREFIX  AMOUNT IN BUNDLE
<prefix>      8
? Would you like to detail a specific prefix Yes
? Operator Role Prefix: <prefix>
ROLE NAME                                                         ROLE ARN                                                                                         VERSION  MANAGED
<prefix>-kube-system-capa-controller-manager                       arn:aws:iam::4540112244:role/<prefix>-kube-system-capa-controller-manager                       4.13     No
<prefix>-kube-system-control-plane-operator                        arn:aws:iam::4540112244:role/<prefix>-kube-system-control-plane-operator                        4.13     No
<prefix>-kube-system-kms-provider                                  arn:aws:iam::4540112244:role/<prefix>-kube-system-kms-provider                                  4.13     No
<prefix>-kube-system-kube-controller-manager                       arn:aws:iam::4540112244:role/<prefix>-kube-system-kube-controller-manager                       4.13     No
<prefix>-openshift-cloud-network-config-controller-cloud-credenti  arn:aws:iam::4540112244:role/<prefix>-openshift-cloud-network-config-controller-cloud-credenti  4.13     No
<prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials       arn:aws:iam::4540112244:role/<prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials       4.13     No
<prefix>-openshift-image-registry-installer-cloud-credentials      arn:aws:iam::4540112244:role/<prefix>-openshift-image-registry-installer-cloud-credentials      4.13     No
<prefix>-openshift-ingress-operator-cloud-credentials              arn:aws:iam::4540112244:role/<prefix>-openshift-ingress-operator-cloud-credentials              4.13     No

    After the command runs, it displays all the prefixes associated with your AWS account and notes how many roles are associated with this prefix. If you need to see all of these roles and their details, enter "Yes" on the detail prompt to have these roles listed out with specifics.

You can create a Red Hat OpenShift Service on AWS cluster with egress zero by using the ROSA command-line interface (CLI) (rosa).

Prerequisites

  • You have completed the AWS prerequisites for Red Hat OpenShift Service on AWS.
  • You have available AWS service quotas.
  • You have enabled the Red Hat OpenShift Service on AWS in the AWS Console.
  • You have installed and configured the latest ROSA CLI (rosa) on your installation host. Run rosa version to see your currently installed version of the ROSA CLI. If a newer version is available, the CLI provides a link to download this upgrade.
  • You have logged in to your Red Hat account by using the ROSA CLI.
  • You have created an OpenID Connect (OIDC) configuration.
  • You have verified that the AWS Elastic Load Balancing (ELB) service role exists in your AWS account.

Procedure

  • Use one of the following commands to create your Red Hat OpenShift Service on AWS cluster:

    Note

    When creating a Red Hat OpenShift Service on AWS cluster, the default machine Classless Inter-Domain Routing (CIDR) is 10.0.0.0/16. If this does not correspond to the CIDR range for your virtual private cloud (VPC) subnets, add --machine-cidr <address_block> to the following commands. To learn more about the default CIDR ranges for Red Hat OpenShift Service on AWS, see the CIDR range definitions.

    • If you did not set environment variables, run the following command: 

      $ rosa create cluster --cluster-name=<cluster_name> \
     --mode=auto --hosted-cp [--private] \
     --operator-roles-prefix <operator-role-prefix> \
     --oidc-config-id <id-of-oidc-configuration> \
     --subnet-ids=<private-subnet-id> --region <region> \
     --machine-cidr 10.0.0.0/16 --service-cidr 172.30.0.0/16 \
     --pod-cidr 10.128.0.0/14 --host-prefix 23 \
     --billing-account <root-acct-id> \
     --properties zero_egress:true

      where:

      <cluster_name>
      Specify the name of your cluster. If your cluster name is longer than 15 characters, it contains an autogenerated domain prefix as a subdomain for your provisioned cluster on openshiftapps.com. To customize the subdomain, use the --domain-prefix flag. The domain prefix cannot be longer than 15 characters, must be unique, and cannot be changed after cluster creation.
      <operator-role-prefix>

      By default, the cluster-specific Operator role names are prefixed with the cluster name and a random 4-digit hash. You can optionally specify a custom prefix to replace <cluster_name>-<hash> in the role names. The prefix is applied when you create the cluster-specific Operator Identity and Access Management (IAM) roles. For information about the prefix, see About custom Operator IAM role prefixes.

      Note

      If you specified custom Amazon Resource Name (ARN) paths when you created the associated account-wide roles, the custom path is automatically detected. The custom path is applied to the cluster-specific Operator roles when you create them in a later step.

      <root-acct-id>
      If your billing account is different from your user account, add this argument and specify the AWS account that is responsible for all billing.

    • If you set the environment variables, create a cluster with egress zero, a private API, and a private Ingress by running the following command: 

      $ rosa create cluster --private --cluster-name=$CLUSTER_NAME \
    --mode=auto --hosted-cp --operator-roles-prefix=$OPERATOR_ROLES_PREFIX \
    --oidc-config-id=$OIDC_ID --subnet-ids=$SUBNET_IDS \
    --region $REGION --machine-cidr 10.0.0.0/16 --service-cidr 172.30.0.0/16 \
    --pod-cidr 10.128.0.0/14 --host-prefix 23 \
    --private --properties zero_egress:true

Verification

  1. Check the status of your cluster by running the following command: 

    $ rosa describe cluster --cluster=<cluster_name>

    The following State field changes are listed in the output as cluster installation progresses:

    • pending (Preparing account)
    • installing (DNS setup in progress)
    • installing

    • ready

      Note

      If the installation fails or the State field does not change to ready after more than 10 minutes, check the installation troubleshooting documentation for details. For more information, see Troubleshooting installations. For steps to contact Red Hat Support for assistance, see Getting support for Red Hat OpenShift Service on AWS.

  2. Track the cluster creation progress by watching the Red Hat OpenShift Service on AWS installation program logs. To check the logs, run the following command: 

    $ rosa logs install --cluster=<cluster_name> [--watch]

    Optional: To watch for new log messages as the installation progresses, use the --watch argument.

Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Legal Notice

Theme

© 2026 Red HatBack to top