Creating a private cluster on ROSA with HCP | Install ROSA with HCP clusters

Creating a private ROSA with HCP cluster using the ROSA CLI
Configuring AWS security groups to access the API
Additional principals on your ROSA with HCP cluster
- Adding additional principals while creating your ROSA with HCP cluster
- Adding additional principals to your existing ROSA with HCP cluster
Next steps
Additional resources

For Red Hat OpenShift Service on AWS (ROSA) with hosted control planes (HCP) workloads that do not require public internet access, you can create a private cluster.

edit

Creating a private ROSA with HCP cluster using the ROSA CLI

You can create a private cluster with multiple availability zones (Multi-AZ) on ROSA with HCP using the ROSA command line interface (CLI), rosa.

Prerequisites

You have available AWS service quotas.
You have enabled the ROSA service in the AWS Console.
You have installed and configured the latest version of the ROSA CLI on your installation host.

Procedure

Creating a cluster with hosted control planes can take around 10 minutes.

Create a VPC with at least one private subnet. Ensure that your machine’s classless inter-domain routing (CIDR) matches your virtual private cloud’s CIDR. For more information, see Requirements for using your own VPC and VPC Validation.

If you use a firewall, you must configure it so that ROSA can access the sites that required to function.

For more information, see the "AWS PrivateLink firewall prerequisites" section.
Create the account-wide IAM roles by running the following command:
```
$ rosa create account-roles --hosted-cp
```

Create the OIDC configuration by running the following command:

$ rosa create oidc-config --mode=auto --yes

Save the OIDC configuration ID because you need it to create the Operator roles.

Example output

I: Setting up managed OIDC configuration
I: To create Operator Roles for this OIDC Configuration, run the following command and remember to replace <user-defined> with a prefix of your choice:
	rosa create operator-roles --prefix <user-defined> --oidc-config-id 28s4avcdt2l318r1jbk3ifmimkurk384
If you are going to create a Hosted Control Plane cluster please include '--hosted-cp'
I: Creating OIDC provider using 'arn:aws:iam::46545644412:user/user'
I: Created OIDC provider with ARN 'arn:aws:iam::46545644412:oidc-provider/oidc.op1.openshiftapps.com/28s4avcdt2l318r1jbk3ifmimkurk384'

Create the Operator roles by running the following command:

$ rosa create operator-roles --hosted-cp --prefix <operator_roles_prefix> --oidc-config-id <oidc_config_id> --installer-role-arn arn:aws:iam::$<account_roles_prefix>:role/$<account_roles_prefix>-HCP-ROSA-Installer-Role

Create a private ROSA with HCP cluster by running the following command:

$ rosa create cluster --private --cluster-name=<cluster-name> --sts --mode=auto --hosted-cp --operator-roles-prefix <operator_role_prefix> --oidc-config-id <oidc_config_id> [--machine-cidr=<VPC CIDR>/16] --subnet-ids=<private-subnet-id1>[,<private-subnet-id2>,<private-subnet-id3>]

Enter the following command to check the status of your cluster. During cluster creation, the State field from the output will transition from pending to installing, and finally, to ready.

$ rosa describe cluster --cluster=<cluster_name>

If installation fails or the State field does not change to ready after 10 minutes, see the "Troubleshooting Red Hat OpenShift Service on AWS installations" documentation in the Additional resources section.

Enter the following command to follow the OpenShift installer logs to track the progress of your cluster:
```
$ rosa logs install --cluster=<cluster_name> --watch
```

edit

Configuring AWS security groups to access the API

With ROSA with HCP private clusters, the AWS PrivateLink endpoint exposed in the customer’s VPC has a default security group. This security group has access to the PrivateLink endpoint that is limited to only those resources that exist within the VPC or resources that are present with an IP address associated with the VPC CIDR range. In order to grant access to any entities outside of the VPC, through VPC peering and transit gateway, you must create and attach another security group to the PrivateLink endpoint to grant the necessary access.

Prerequisites

Your corporate network or other VPC has connectivity.
You have permission to create and attach security groups within the VPC.

Procedure

Set your cluster name as an environmental variable by running the following command:
```
$ export CLUSTER_NAME=<cluster_name>
```
You can verify that the variable has been set by running the following command:
```
$ echo $CLUSTER_NAME
```
Example output
```
hcp-private
```

Find the VPC endpoint (VPCE) ID and VPC ID by running the following command:

$ read -r VPCE_ID VPC_ID <<< $(aws ec2 describe-vpc-endpoints --filters "Name=tag:api.openshift.com/id,Values=$(rosa describe cluster -c ${CLUSTER_NAME} -o yaml | grep '^id: ' | cut -d' ' -f2)" --query 'VpcEndpoints[].[VpcEndpointId,VpcId]' --output text)

Create your security group by running the following command:

$ export SG_ID=$(aws ec2 create-security-group --description "Granting API access to ${CLUSTER_NAME} from outside of VPC" --group-name "${CLUSTER_NAME}-api-sg" --vpc-id $VPC_ID --output text)

Add an ingress rule to the security group by running the following command:

$ aws ec2 authorize-security-group-ingress --group-id $SG_ID --ip-permissions FromPort=443,ToPort=443,IpProtocol=tcp,IpRanges=[{CidrIp=0.0.0.0/0}]

Add the new security group to the VPCE by running the following command:

$ aws ec2 modify-vpc-endpoint --vpc-endpoint-id $VPCE_ID --add-security-group-ids $SG_ID

You now can access the API with your ROSA with HCP private cluster.

edit

Additional principals on your ROSA with HCP cluster

You can allow AWS Identity and Access Management (IAM) roles as additional principals to connect to your cluster’s private API server endpoint.

You can access your ROSA with HCP cluster’s API Server endpoint from either the public internet or the interface endpoint that was created within the VPC private subnets. By default, you can privately access your ROSA with HCP API Server by using the -kube-system-kube-controller-manager Operator role. To be able to access ROSA with HCP API server from another account directly without using the primary account where cluster is installed, you must include cross-account IAM roles as additional principals. This feature allows you to simplify your network architecture and reduce data transfer costs by avoiding peering or attaching cross-account VPCs to cluster’s VPC.

In this diagram, the cluster creating account is designated as Account A. This account designates that another account, Account B, should have access to the API server.

After you have configured additional allowed principals, you must create the interface VPC endpoint in the VPC from where you want to access the cross-account ROSA with HCP API server. Then, create a private hosted zone in Route53 to route calls made to cross-account ROSA with HCP API server to pass through the created VPC endpoint.

edit

Adding additional principals while creating your ROSA with HCP cluster

Use the --additional-allowed-principals argument to permit access through other roles.

Procedure

Add the --additional-allowed-principals argument to the rosa create cluster command, similar to the following:
```
$ rosa create cluster [...] --additional-allowed-principals <arn_string>
```
You can use arn:aws:iam::account_id:role/role_name to approve a specific role.

When the cluster creation command runs, you receive a summary of your cluster with the --additional-allowed-principals specified:

Example output

Name:                       mycluster
Domain Prefix:              mycluster
Display Name:               mycluster
ID:                         <cluster-id>
External ID:                <cluster-id>
Control Plane:              ROSA Service Hosted
OpenShift Version:          4.15.17
Channel Group:              stable
DNS:                        Not ready
AWS Account:                <aws_id>
AWS Billing Account:        <aws_id>
API URL:
Console URL:
Region:                     us-east-2
Availability:
 - Control Plane:           MultiAZ
 - Data Plane:              SingleAZ

Nodes:
 - Compute (desired):       2
 - Compute (current):       0
Network:
 - Type:                    OVNKubernetes
 - Service CIDR:            172.30.0.0/16
 - Machine CIDR:            10.0.0.0/16
 - Pod CIDR:                10.128.0.0/14
 - Host Prefix:             /23
 - Subnets:                 subnet-453e99d40, subnet-666847ce827
EC2 Metadata Http Tokens:   optional
Role (STS) ARN:             arn:aws:iam::<aws_id>:role/mycluster-HCP-ROSA-Installer-Role
Support Role ARN:           arn:aws:iam::<aws_id>:role/mycluster-HCP-ROSA-Support-Role
Instance IAM Roles:
 - Worker:                  arn:aws:iam::<aws_id>:role/mycluster-HCP-ROSA-Worker-Role
Operator IAM Roles:
 - arn:aws:iam::<aws_id>:role/mycluster-kube-system-control-plane-operator
 - arn:aws:iam::<aws_id>:role/mycluster-openshift-cloud-network-config-controller-cloud-creden
 - arn:aws:iam::<aws_id>:role/mycluster-openshift-image-registry-installer-cloud-credentials
 - arn:aws:iam::<aws_id>:role/mycluster-openshift-ingress-operator-cloud-credentials
 - arn:aws:iam::<aws_id>:role/mycluster-openshift-cluster-csi-drivers-ebs-cloud-credentials
 - arn:aws:iam::<aws_id>:role/mycluster-kube-system-kms-provider
 - arn:aws:iam::<aws_id>:role/mycluster-kube-system-kube-controller-manager
 - arn:aws:iam::<aws_id>:role/mycluster-kube-system-capa-controller-manager
Managed Policies:           Yes
State:                      waiting (Waiting for user action)
Private:                    No
Delete Protection:          Disabled
Created:                    Jun 25 2024 13:36:37 UTC
User Workload Monitoring:   Enabled
Details Page:               https://console.redhat.com/openshift/details/s/Bvbok4O79q1Vg8
OIDC Endpoint URL:          https://oidc.op1.openshiftapps.com/vhufi5lap6vbl3jlq20e (Managed)
Audit Log Forwarding:       Disabled
External Authentication:    Disabled
Additional Principals:      arn:aws:iam::<aws_id>:role/additional-user-role

edit

Adding additional principals to your existing ROSA with HCP cluster

You can add additional principals to your cluster by using the command line interface (CLI).

Procedure

Run the following command to edit your cluster and add an additional principal who can access this cluster’s endpoint:
```
$ rosa edit cluster -c <cluster_name> --additional-allowed-principals <arn_string>
```
You can use arn:aws:iam::account_id:role/role_name to approve a specific role.

edit

Next steps

Configuring identity providers