OpenShift docs are moving and will soon only be available at docs.redhat.com, the home of all Red Hat product documentation. Explore the new docs experience today.
  1. Documentation
    2. history bug_report picture_as_pdf
×

Image configuration resources for ROSA with HCP

edit

Use the following procedure to configure image registries.

edit

Image controller configuration parameters for ROSA with HCP

The image.config.openshift.io/cluster resource holds cluster-wide information about how to handle images. The resource exists, but it is read only and can only be changed through supported tools like ROSA CLI (rosa). The canonical and only valid name is cluster. It can be configured in Red Hat OpenShift Service on AWS hosted control planes through ROSA CLI (rosa) commands.

Parameters such as DisableScheduledImport, MaxImagesBulkImportedPerRepository, MaxScheduledImportsPerMinute, ScheduledImageImportMinimumIntervalSeconds, InternalRegistryHostname are not configurable.
Parameters for ROSA CLI Description

registry-config-allowed-registries

Registries for which image pull and push actions are allowed. To specify all subdomains, add the asterisk (*) wildcard character as a prefix to the domain name. For example, *.example.com. You can specify an individual repository within a registry. For example, reg1.io/myrepo/myapp:latest. All other registries are blocked. The format should be a comma-separated list of allowed registries. For example, allowed.io, allowed.io2.

registry-config-insecure-registries

Registries which do not have a valid TLS certificate or only support HTTP connections. To specify all subdomains, add the asterisk (*) wildcard character as a prefix to the domain name. For example, *.example.com. You can specify an individual repository within a registry. For example, reg1.io/myrepo/myapp:latest. The format should be a comma-separated list of insecure registries. For example, insecure.io, insecure.io2.

registry-config-blocked-registries

Registries for which image pull and push actions are denied. To specify all subdomains, add the asterisk (*) wildcard character as a prefix to the domain name. For example, *.example.com. You can specify an individual repository within a registry. For example, reg1.io/myrepo/myapp:latest. All other registries are allowed. The format should be a comma-separated list of blocked registries. For example, blocked.io, blocked.io2.

registry-config-allowed-registries-for-import

Contains configuration that determines how the container runtime should treat individual registries when accessing images for builds and pods. For example, whether or not to allow insecure access. It does not contain configuration for the internal cluster registry. Limits the container image registries from which normal users can import images. The format should be a comma-separated list of domainName:insecure. domainName specifies a domain name for the registry. insecure indicates whether the registry is secure or insecure.

registry-config-additional-trusted-ca

A JSON file containing the registry hostname as the key, and the PEM-encoded certificate as the value, for each additional registry CA to trust.

When the allowedRegistries parameter is defined, all registries are blocked unless explicitly listed. To prevent pod failure, a list of Red Hat registries is automatically whitelisted, as they are required by payload images within your environment. The current list consists of image-registry.openshift-image-registry.svc:5000,quay.io,registry.redhat.io and it is also visible when running the rosa describe cluster command.
edit

Configuring image registry settings for ROSA with HCP

You can configure image registry settings at cluster creation. The cluster’s nodes will use the required configuration after creation.

Procedure

  • Create ROSA with HCP clusters with image registry by running the following command:

    $ rosa create cluster —cluster-name=<cluster_name> --sts --mode=auto \
   --hosted-cp --operator-roles-prefix <operator_role_prefix> \
   --oidc-config-id <id_of_oidc_configuration> \
   --subnet-ids=<public_subnet_id>,<private_subnet_id> \
   --registry-config-insecure-registries <insecure_registries> \
   --registry-config-allowed-registries <allowed_registries> \
   --registry-config-allowed-registries-for-import <registry_name:insecure> \
   --registry-config-additional-trusted-ca <additional_trusted_ca_file>

    When using the allowedRegistries, blockedRegistries, or insecureRegistries parameter, you can specify an individual repository within a registry. For example: reg1.io/myrepo/myapp:latest.

    Avoid insecure external registries to reduce possible security risks. Parameters allowedRegistries, blockedRegistries are mutually exclusive.
Verification

  1. Run the rosa describe command to verify that your image registry is enabled by running the following command:

    $ rosa describe cluster --cluster=<cluster_name>
    Example output
    Name:                       rosa-hcp-test
Domain Prefix:              rosa-hcp-test
Display Name:               rosa-hcp-test
ID:                         <cluster_hcp_id>
External ID:                <cluster_hcp_id>
Control Plane:              ROSA Service Hosted
OpenShift Version:          4.Y.Z
Channel Group:              stable
DNS:                        <dns>
AWS Account:                <aws_id>
AWS Billing Account:        <aws_id>
API URL:                    <ocm_api>
Console URL:
Region:                     us-east-1
Availability:
 - Control Plane:           MultiAZ
 - Data Plane:              SingleAZ
Nodes:
 - Compute (desired):       2
 - Compute (current):       2
Network:
 - Type:                    OVNKubernetes
 - Service CIDR:            <service_cidr>
 - Machine CIDR:            <machine_cidr>
 - Pod CIDR:                <pod_cidr>
 - Host Prefix:             /23
 - Subnets:                 <subnet_ids>
EC2 Metadata Http Tokens:   optional
Role (STS) ARN:             arn:aws:iam::<aws_id>:role/<account_roles_prefix>-HCP-ROSA-Installer-Role
Support Role ARN:           arn:aws:iam::<aws_id>:role/<account_roles_prefix>-HCP-ROSA-Support-Role
Instance IAM Roles:
 - Worker:                  arn:aws:iam::<aws_id>:role/<account_roles_prefix>-HCP-ROSA-Worker-Role
Operator IAM Roles:
 - arn:aws:iam::<aws_id>:role/<operator_roles_prefix>-kube-system-capa-controller-manager
 - arn:aws:iam::<aws_id>:role/<operator_roles_prefix>-kube-system-control-plane-operator
 - arn:aws:iam::<aws_id>:role/<operator_roles_prefix>-kube-system-kms-provider
 - arn:aws:iam::<aws_id>:role/<operator_roles_prefix>-openshift-image-registry-installer-cloud-cred
 - arn:aws:iam::<aws_id>:role/<operator_roles_prefix>-openshift-ingress-operator-cloud-credentials
 - arn:aws:iam::<aws_id>:role/<operator_roles_prefix>-openshift-cluster-csi-drivers-ebs-cloud-credent
 - arn:aws:iam::<aws_id>:role/<operator_roles_prefix>-openshift-cloud-network-config-controller-cloud
Managed Policies:           Yes
State:                      ready
Private:                    No
Delete Protection:          Disabled
Created:                    Oct 01 2030 09:48:52 UTC
User Workload Monitoring:   Enabled
OIDC Endpoint URL:          https://<endpoint> (Managed)
Audit Log Forwarding:       Disabled
External Authentication:    Disabled
Etcd Encryption:            Disabled
Registry Configuration:
 - Allowed Registries: <allowed_registry>  (1) (2)
 - Insecure Registries: <insecure_registry> (3)
 - Allowed Registries for Import: (4)
    - Domain Name: <domain_name> (5)
    - Insecure: true (6)
 - Platform Allowlist: <platform_allowlist_id> (7)
    - Registries:      <list_of_registries> (8)
 - Additional Trusted CA: (9)
    - <registry_name> : REDACTED
    1 Allowed Registries: A comma-separated list of registries for which image pull and push actions are allowed.
    2 Blocked Registries: A comma-separated list of registries for which image pull and push actions are blocked. Parameters allowedRegistries, blockedRegistries are mutually exclusive.
    3 Insecure Registries: A comma-separated list of registries which do not have a valid TLS certificate or only support HTTP connections.
    4 Allowed Registries for Import: Limits the container image registries from which normal users can import images. The format should be a comma-separated list of domainName:insecure.
    5 domainName: Specifies a domain name for the registry.
    6 insecure: Indicates whether the registry is secure or insecure.
    7 Platform Allowlist: A reference to the id of the list of registries that needs to be whitelisted for the platform to work.
    8 Registries: The list of registries that needs to be whitelisted for the platform to work.
    9 Additional Trusted CA: A JSON file containing the registry hostname as the key, and the PEM-encoded certificate as the value, for each additional registry CA to trust.

  2. List your nodes to check the applied changes by running the following command:

    $ oc get nodes
    Example output
    NAME                                         STATUS                     ROLES                  AGE   VERSION
ip-10-0-137-182.us-east-2.compute.internal   Ready,SchedulingDisabled   worker                 65m   v1.30.3
ip-10-0-188-96.us-east-2.compute.internal    Ready                      worker                 65m   v1.30.3
ip-10-0-200-59.us-east-2.compute.internal    Ready                      worker                 63m   v1.30.3
edit

Editing image registry settings for ROSA with HCP

You can change the image registry config with the rosa edit command.

When the allowedRegistries parameter is defined, all registries are blocked unless explicitly listed. To prevent pod failure, a list of Red Hat registries is automatically whitelisted, as they are required by payload images within your environment. The current list consists of image-registry.openshift-image-registry.svc:5000,quay.io,registry.redhat.io and it is also visible when running the rosa describe cluster command.

You can change any registry-related parameter, which will trigger a rollout across all machine pools; all machine pool nodes will be recreated, following pod draining from each node.
Procedure

  • Update or edit the image registry for the cluster by running the following command:

    $ rosa edit cluster --registry-config-insecure-registries <insecure_registries> \
   --registry-config-allowed-registries <allowed_registries> \
   --registry-config-allowed-registries-for-import <registry_name:insecure> \
   --registry-config-additional-trusted-ca <additional_trusted_ca_file>
    Example output
    ? Changing any registry related parameter will trigger a rollout across all machinepools
(all machinepool nodes will be recreated, following pod draining from each node).
Do you want to proceed? Yes
I: Updated cluster '<cluster_name>'
Verification

  • Run the rosa describe command again, to see if the changes you made to your image registry updated by running the following command:

    $ rosa describe cluster --cluster=<cluster_name>
    Example output
    Name:                       rosa-hcp-test
Domain Prefix:              rosa-hcp-test
Display Name:               rosa-hcp-test
ID:                         <cluster_hcp_id>
External ID:                <cluster_hcp_id>
Control Plane:              ROSA Service Hosted
OpenShift Version:          4.Y.Z
Channel Group:              stable
DNS:                        <dns>
AWS Account:                <aws_id>
AWS Billing Account:        <aws_id>
API URL:                    <ocm_api>
Console URL:
Region:                     us-east-1
Availability:
 - Control Plane:           MultiAZ
 - Data Plane:              SingleAZ

Nodes:
 - Compute (desired):       2
 - Compute (current):       2
Network:
 - Type:                    OVNKubernetes
 - Service CIDR:            <service_cidr>
 - Machine CIDR:            <machine_cidr>
 - Pod CIDR:                <pod_cidr>
 - Host Prefix:             /23
 - Subnets:                 <subnet_ids>
EC2 Metadata Http Tokens:   optional
Role (STS) ARN:             arn:aws:iam::<aws_id>:role/<account_roles_prefix>-HCP-ROSA-Installer-Role
Support Role ARN:           arn:aws:iam::<aws_id>:role/<account_roles_prefix>-HCP-ROSA-Support-Role
Instance IAM Roles:
 - Worker:                  arn:aws:iam::<aws_id>:role/<account_roles_prefix>-HCP-ROSA-Worker-Role
Operator IAM Roles:
 - arn:aws:iam::<aws_id>:role/<operator_roles_prefix>-kube-system-capa-controller-manager
 - arn:aws:iam::<aws_id>:role/<operator_roles_prefix>-kube-system-control-plane-operator
 - arn:aws:iam::<aws_id>:role/<operator_roles_prefix>-kube-system-kms-provider
 - arn:aws:iam::<aws_id>:role/<operator_roles_prefix>-openshift-image-registry-installer-cloud-cred
 - arn:aws:iam::<aws_id>:role/<operator_roles_prefix>-openshift-ingress-operator-cloud-credentials
 - arn:aws:iam::<aws_id>:role/<operator_roles_prefix>-openshift-cluster-csi-drivers-ebs-cloud-credent
 - arn:aws:iam::<aws_id>:role/<operator_roles_prefix>-openshift-cloud-network-config-controller-cloud
Managed Policies:           Yes
State:                      ready
Private:                    No
Delete Protection:          Disabled
Created:                    Oct 01 2030 09:48:52 UTC
User Workload Monitoring:   Enabled
OIDC Endpoint URL:          https://<endpoint> (Managed)
Audit Log Forwarding:       Disabled
External Authentication:    Disabled
Etcd Encryption:            Disabled
Registry Configuration:
 - Allowed Registries: <allowed_registry>  (1) (2)
 - Insecure Registries: <insecure_registry> (3)
 - Allowed Registries for Import: (4)
    - Domain Name: <domain_name> (5)
    - Insecure: true (6)
 - Platform Allowlist: <platform_allowlist_id> (7)
    - Registries:      <list_of_registries> (8)
 - Additional Trusted CA: (9)
    - <registry_name> : REDACTED
    1 Allowed Registries: A comma-separated list of registries for which image pull and push actions are allowed.
    2 Blocked Registries: A comma-separated list of registries for which image pull and push actions are blocked. Parameters allowedRegistries, blockedRegistries are mutually exclusive.
    3 Insecure Registries: A comma-separated list of registries which do not have a valid TLS certificate or only support HTTP connections.
    4 Allowed Registries for Import: Limits the container image registries from which normal users can import images. The format should be a comma-separated list of domainName:insecure.
    5 domainName: Specifies a domain name for the registry.
    6 insecure: Indicates whether the registry is secure or insecure.
    7 Platform Allowlist: A reference to the id of the list of registries that needs to be whitelisted for the platform to work.
    8 Registries: The list of registries that needs to be whitelisted for the platform to work.
    9 Additional Trusted CA: A JSON file containing the registry hostname as the key, and the PEM-encoded certificate as the value, for each additional registry CA to trust.
edit

Updating platform allowlist for ROSA with HCP

A list of Red Hat registries is automatically allowed and it is visible when running rosa describe cluster. This list can be periodically updated to ensure platform can be operated correctly. Impacted clusters will receive a notification with the new allowlist ID. In such cases, the user must use this parameter to update from the previous expected ID to the newly expected ID. Update or edit the image registry for the cluster by running the following command:

$ rosa edit cluster --registry-config-platform-allowlist <newID>