1. Documentation
    2. history bug_report picture_as_pdf
×

Customer Cloud Subscriptions on GCP

Red Hat recommends the usage of a Google Cloud Platform (GCP) project, managed by the customer, to organize all of your GCP resources. A project consists of a set of users and APIs, as well as billing, authentication, and monitoring settings for those APIs.

It is a best practice for the OpenShift Dedicated CCS cluster to be hosted in a GCP project within a GCP organization. The Organization resource is the root node of the GCP resource hierarchy and all resources that belong to an organization are grouped under the organization node. An IAM service account with certain roles granted is created and applied to the GCP project. When you make calls to the API, you typically provide service account keys for authentication. Each service account is owned by a specific project, but service accounts can be provided roles to access resources for other projects.

Understanding Customer Cloud Subscriptions on GCP

Red Hat OpenShift Dedicated provides a Customer Cloud Subscription (CCS) model that allows Red Hat to deploy and manage OpenShift Dedicated into a customer’s existing Google Cloud Platform (GCP) account. Red Hat requires several prerequisites be met in order to provide this service.

Red Hat recommends the usage of GCP project, managed by the customer, to organize all of your GCP resources. A project consists of a set of users and APIs, as well as billing, authentication, and monitoring settings for those APIs.

It is recommended for the OpenShift Dedicated cluster using a CCS model to be hosted in a GCP project within a GCP organization. The Organization resource is the root node of the GCP resource hierarchy and all resources that belong to an organization are grouped under the organization node. An IAM service account with certain roles granted is created and applied to the GCP project. When you make calls to the API, you typically provide service account keys for authentication. Each service account is owned by a specific project, but service accounts can be provided roles to access resources for other projects.

Customer requirements

OpenShift Dedicated clusters using a Customer Cloud Subscription (CCS) model on Google Cloud Platform (GCP) must meet several prerequisites before they can be deployed.

Account

  • The customer ensures that Google Cloud limits are sufficient to support OpenShift Dedicated provisioned within the customer-provided GCP account.

  • The customer-provided GCP account should be in the customer’s Google Cloud Organization with the applicable Service Account applied.

  • The customer-provided GCP account must not be transferable to Red Hat.

  • The customer may not impose GCP usage restrictions on Red Hat activities. Imposing restrictions severely hinders Red Hat’s ability to respond to incidents.

  • Red Hat deploys monitoring into GCP to alert Red Hat when a highly privileged account, such as a root account, logs into the customer-provided GCP account.

  • The customer can deploy native GCP services within the same customer-provided GCP account.

    Customers are encouraged, but not mandated, to deploy resources in a Virtual Private Cloud (VPC) separate from the VPC hosting OpenShift Dedicated and other Red Hat supported services.

Access requirements

  • To appropriately manage the OpenShift Dedicated service, Red Hat must have the AdministratorAccess policy applied to the administrator role at all times.

    This policy only provides Red Hat with permissions and capabilities to change resources in the customer-provided GCP account.

  • Red Hat must have GCP console access to the customer-provided GCP account. This access is protected and managed by Red Hat.

  • The customer must not utilize the GCP account to elevate their permissions within the OpenShift Dedicated cluster.

  • Actions available in the OpenShift Cluster Manager Hybrid Cloud Console must not be directly performed in the customer-provided GCP account.

Support requirements

  • Red Hat recommends that the customer have at least Production Support from GCP.

  • Red Hat has authority from the customer to request GCP support on their behalf.

  • Red Hat has authority from the customer to request GCP resource limit increases on the customer-provided account.

  • Red Hat manages the restrictions, limitations, expectations, and defaults for all OpenShift Dedicated clusters in the same manner, unless otherwise specified in this requirements section.

Security requirements

  • The customer-provided IAM credentials must be unique to the customer-provided GCP account and must not be stored anywhere in the customer-provided GCP account.

  • Volume snapshots will remain within the customer-provided GCP account and customer-specified region.

  • Red Hat must have ingress access to the API server through white-listed Red Hat machines.

  • Red Hat must have egress allowed to forward system and audit logs to a Red Hat managed central logging stack.

Required customer procedure

The Customer Cloud Subscription (CCS) model allows Red Hat to deploy and manage OpenShift Dedicated into a customer’s Google Cloud Platform (GCP) project. Red Hat requires several prerequisites to provide these services.

To use OpenShift Dedicated in your GCP project, the following GCP organizational policy constraints cannot be in place:

  • constraints/iam.allowedPolicyMemberDomains

  • constraints/compute.restrictLoadBalancerCreationForTypes

  • constraints/compute.requireShieldedVm

  • constraints/compute.vmExternalIpAccess (This policy constraint is unsupported only during installation. You can re-enable the policy constraint after installation.)
Procedure

  1. Create a Google Cloud project to host the OpenShift Dedicated cluster.

    The project name must be 10 characters or less.

  2. Enable the following required APIs in the project that hosts your OpenShift Dedicated cluster:

    Table 1. Required API services
    API service Console service name

    Cloud Deployment Manager V2 API

    deploymentmanager.googleapis.com

    Compute Engine API

    compute.googleapis.com

    Google Cloud APIs

    cloudapis.googleapis.com

    Cloud Resource Manager API

    cloudresourcemanager.googleapis.com

    Google DNS API

    dns.googleapis.com

    Network Security API

    networksecurity.googleapis.com

    IAM Service Account Credentials API

    iamcredentials.googleapis.com

    Identity and Access Management (IAM) API

    iam.googleapis.com

    Service Management API

    servicemanagement.googleapis.com

    Service Usage API

    serviceusage.googleapis.com

    Google Cloud Storage JSON API

    storage-api.googleapis.com

    Cloud Storage

    storage-component.googleapis.com

  3. To ensure that Red Hat can perform necessary actions, you must create an osd-ccs-admin IAM service account user within the GCP project.

    The following roles must be granted to the service account:

    Table 2. Required roles
    Role Console role name

    Compute Admin

    roles/compute.admin

    DNS Admin

    roles/dns.admin

    Organizational Policy Viewer

    roles/orgpolicy.policyViewer

    Owner

    roles/owner

    Project IAM Admin

    roles/resourcemanager.projectIamAdmin

    Service Management Administrator

    roles/servicemanagement.admin

    Service Usage Admin

    roles/serviceusage.serviceUsageAdmin

    Storage Admin

    roles/storage.admin

  4. Create the service account key for the osd-ccs-admin IAM service account. Export the key to a file named osServiceAccount.json; this JSON file will be uploaded in Red Hat OpenShift Cluster Manager when you create your cluster.

Red Hat managed Google Cloud resources

Red Hat is responsible for creating and managing the following IAM Google Cloud Platform (GCP) resources.

IAM service account and roles

The osd-managed-admin IAM service account is created immediately after taking control of the customer-provided GCP account. This is the user that will perform the OpenShift Dedicated cluster installation.

The following roles are attached to the service account:

Table 3. IAM roles for osd-managed-admin
Role Console role name Description

Compute Admin

roles/compute.admin

Provides full control of all Compute Engine resources.

DNS Administrator

roles/dns.admin

Provides read-write access to all Cloud DNS resources.

Security Admin

roles/iam.securityAdmin

Security admin role, with permissions to get and set any IAM policy.

Storage Admin

roles/storage.admin

Grants full control of objects and buckets.

When applied to an individual bucket, control applies only to the specified bucket and objects within the bucket.

Service Account Admin

roles/iam.serviceAccountAdmin

Create and manage service accounts.

Service Account Key Admin

roles/iam.serviceAccountKeyAdmin

Create and manage (and rotate) service account keys.

Service Account User

roles/iam.serviceAccountUser

Run operations as the service account.

IAM group and roles

The sd-sre-platform-gcp-access Google group is granted access to the GCP project to allow Red Hat Site Reliability Engineering (SRE) access to the console for emergency troubleshooting purposes.

The following roles are attached to the group:

Table 4. IAM roles for sd-sre-platform-gcp-access
Role Console role name Description

Compute Admin

roles/compute.admin

Provides full control of all Compute Engine resources.

Editor

roles/editor

Provides all viewer permissions, plus permissions for actions that modify state.

Organization Policy Viewer

roles/orgpolicy.policyViewer

Provides access to view Organization Policies on resources.

Project IAM Admin

roles/resourcemanager.projectIamAdmin

Provides permissions to administer IAM policies on projects.

Quota Administrator

roles/servicemanagement.quotaAdmin

Provides access to administer service quotas.

Role Administrator

roles/iam.roleAdmin

Provides access to all custom roles in the project.

Service Account Admin

roles/iam.serviceAccountAdmin

Create and manage service accounts.

Service Usage Admin

roles/serviceusage.serviceUsageAdmin

Ability to enable, disable, and inspect service states, inspect operations, and consume quota and billing for a consumer project.

Tech Support Editor

roles/cloudsupport.techSupportEditor

Provides full read-write access to technical support cases.

Provisioned GCP Infrastructure

This is an overview of the provisioned Google Cloud Platform (GCP) components on a deployed OpenShift Dedicated cluster. For a more detailed listing of all provisioned GCP components, see the OpenShift Container Platform documentation.

Compute instances

GCP compute instances are required to deploy the control plane and data plane functions of OpenShift Dedicated in GCP. Instance types might vary for control plane and infrastructure nodes depending on worker node count.

  • Single availability zone

    • 2 infra nodes (custom machine type: 4 vCPU and 32 GB RAM)

    • 3 control plane nodes (custom machine type: 8 vCPU and 32 GB RAM)

    • 2 worker nodes (custom machine type: 4 vCPU and 16 GB RAM)

  • Multiple availability zones

    • 3 infra nodes (custom machine type: 4 vCPU and 32 GB RAM)

    • 3 control plane nodes (custom machine type: 8 vCPU and 32 GB RAM)

    • 3 worker nodes (custom machine type: 4 vCPU and 16 GB RAM)

Storage

  • Infrastructure volumes:

    • 128 GB SSD persistent disk (deleted on instance deletion)

    • 110 GB Standard persistent disk (kept on instance deletion)

  • Worker volumes:

    • 128 GB SSD persistent disk (deleted on instance deletion)

  • Control plane volumes:

    • 128 GB SSD persistent disk (deleted on instance deletion)

VPC

  • Subnets: One master subnet for the control plane workloads and one worker subnet for all others.

  • Router tables: One global route table per VPC.

  • Internet gateways: One internet gateway per cluster.

  • NAT gateways: One master NAT gateway and one worker NAT gateway per cluster.

Services

The following services must be enabled on a GCP CCS cluster:

  • Deploymentmanager

  • Compute

  • Cloudapis

  • Cloudresourcemanager

  • DNS

  • Iamcredentials

  • IAM

  • Servicemanagement

  • Serviceusage

  • Storage-api

  • Storage-component

Permissions

The following roles must be added to the support service account:

  • Compute.admin

  • Dns.admin

  • orgpolicy.policyViewer

  • Owner

  • resourcemanager.projectIamAdmin

  • Servicemanagement.admin

  • serviceusage.serviceUsageAdmin

  • storage.admin

GCP account limits

The OpenShift Dedicated cluster uses a number of Google Cloud Platform (GCP) components, but the default quotas do not affect your ability to install an OpenShift Dedicated cluster.

A standard OpenShift Dedicated cluster uses the following resources. Note that some resources are required only during the bootstrap process and are removed after the cluster deploys.

Table 5. GCP resources used in a default cluster
Service Component Location Total resources required Resources removed after bootstrap

Service account

IAM

Global

5

0

Firewall Rules

Compute

Global

11

1

Forwarding Rules

Compute

Global

2

0

In-use global IP addresses

Compute

Global

4

1

Health checks

Compute

Global

3

0

Images

Compute

Global

1

0

Networks

Compute

Global

2

0

Static IP addresses

Compute

Region

4

1

Routers

Compute

Global

1

0

Routes

Compute

Global

2

0

Subnetworks

Compute

Global

2

0

Target Pools

Compute

Global

3

0

CPUs

Compute

Region

28

4

Persistent Disk SSD (GB)

Compute

Region

896

128

If any of the quotas are insufficient during installation, the installation program displays an error that states both which quota was exceeded and the region.

Be sure to consider your actual cluster size, planned cluster growth, and any usage from other clusters that are associated with your account. The CPU, Static IP addresses, and Persistent Disk SSD (Storage) quotas are the ones that are most likely to be insufficient.

If you plan to deploy your cluster in one of the following regions, you will exceed the maximum storage quota and are likely to exceed the CPU quota limit:

  • asia-east2

  • asia-northeast2

  • asia-south1

  • australia-southeast1

  • europe-north1

  • europe-west2

  • europe-west3

  • europe-west6

  • northamerica-northeast1

  • southamerica-east1

  • us-west2

You can increase resource quotas from the GCP console, but you might need to file a support ticket. Be sure to plan your cluster size early so that you can allow time to resolve the support ticket before you install your OpenShift Dedicated cluster.