OpenShift docs are moving and will soon only be available at docs.redhat.com, the home of all Red Hat product documentation. Explore the new docs experience today.
  1. Documentation
    2. history
×

Manually creating IAM for Azure

Manually create IAM

The Cloud Credential Operator can be put into manual mode prior to installation in environments where the cloud identity and access management (IAM) APIs are not reachable, or the administrator prefers not to store an administrator-level credential secret in the cluster kube-system namespace.

Procedure

  1. Run the OpenShift Container Platform installer to generate manifests:

    $ openshift-install create manifests --dir=mycluster

  2. Insert a config map into the manifests directory so that the Cloud Credential Operator is placed in manual mode:

    $ cat <<EOF > mycluster/manifests/cco-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: cloud-credential-operator-config
  namespace: openshift-cloud-credential-operator
  annotations:
    release.openshift.io/create-only: "true"
data:
  disabled: "true"
EOF

  3. Remove the admin credential secret created using your local cloud credentials. This removal prevents your admin credential from being stored in the cluster:

    $ rm mycluster/openshift/99_cloud-creds-secret.yaml

  4. Obtain the OpenShift Container Platform release image your openshift-install binary is built to use:

    $ bin/openshift-install version
    Example output
    release image quay.io/openshift-release-dev/ocp-release:4.z.z-x86_64

  5. Locate all CredentialsRequest objects in this release image that target the cloud you are deploying on:

    $ oc adm release extract quay.io/openshift-release-dev/ocp-release:4.z.z-x86_64 --to ./release-image

  6. Locate the CredentialsRequests in the extracted file:

    $ grep -l "apiVersion: cloudcredential.openshift.io" * | xargs cat

    In a future OpenShift Container Platform release, there will be a new oc adm release command to scan for the CredentialsRequests and display them.

    This displays the details for each request. Remember to ignore any CredentialsRequests where the spec.providerSpec.kind does not match the cloud provider you are installing to.

    Sample CredentialsRequest object
    apiVersion: cloudcredential.openshift.io/v1
kind: CredentialsRequest
metadata:
  labels:
    controller-tools.k8s.io: "1.0"
  name: openshift-image-registry-azure
  namespace: openshift-cloud-credential-operator
spec:
  secretRef:
    name: installer-cloud-credentials
    namespace: openshift-image-registry
  providerSpec:
    apiVersion: cloudcredential.openshift.io/v1
    kind: AzureProviderSpec
    roleBindings:
    - role: Contributor

  7. Create YAML files for secrets in the openshift-install manifests directory that you generated previously. The secrets must be stored using the namespace and secret name defined in each request.spec.secretRef. The format for the secret data varies for each cloud provider.

  8. Proceed with cluster creation:

    $ openshift-install create cluster --dir=mycluster

    Before performing an upgrade, you might need to adjust your credentials if permissions have changed in the next release. In the future, the Cloud Credential Operator might prevent you from upgrading until you have indicated that you have addressed updated permissions.

Admin credentials root secret format

Each cloud provider uses a credentials root secret in the kube-system namespace by convention, which is then used to satisfy all credentials requests and create their respective secrets. This is done either by minting new credentials, Mint Mode, or by copying the credentials root secret, Passthrough Mode.

The format for the secret varies by cloud, and is also used for each CredentialsRequest secret.

Microsoft Azure secret format
apiVersion: v1
kind: Secret
metadata:
  namespace: kube-system
  name: azure-credentials
stringData:
  azure_subscription_id: <SubscriptionID>
  azure_client_id: <ClientID>
  azure_client_secret: <ClientSecret>
  azure_tenant_id: <TenantID>
  azure_resource_prefix: <ResourcePrefix>
  azure_resourcegroup: <ResourceGroup>
  azure_region: <Region>

On Microsoft Azure, the credentials secret format includes two properties that must contain the cluster’s infrastructure ID, generated randomly for each cluster installation. This value can be found after running create manifests:

$ cat .openshift_install_state.json | jq '."*installconfig.ClusterID".InfraID' -r
Example output
mycluster-2mpcn

This value would be used in the secret data as follows:

azure_resource_prefix: mycluster-2mpcn
azure_resourcegroup: mycluster-2mpcn-rg

Upgrades

In a future release, improvements to the Cloud Credential Operator will prevent situations where a user might enter an upgrade that will fail because their manually maintained credentials have not been updated to match the CredentialsRequest objects in the upcoming release image.

Mint Mode

Mint Mode is supported for AWS, GCP, and Azure.

The default and recommended best practice for running OpenShift Container Platform is to run the installer with an administrator-level cloud credential. The admin credential is stored in the kube-system namespace, and then used by the Cloud Credential Operator to process the CredentialsRequest objects in the cluster and create new users for each with specific permissions.

The benefits of Mint Mode include:

  • Each cluster component only has the permissions it requires.

  • Automatic, on-going reconciliation for cloud credentials including upgrades, which might require additional credentials or permissions.

One drawback is that Mint Mode requires admin credential storage in a cluster kube-system secret.