Chapter 10. Tutorial: Use AWS Secrets Manager CSI on Red Hat OpenShift Service on AWS with STS

Procedure

1. Log in to your Red Hat OpenShift Service on AWS cluster by running the following command:

   $ oc login --token=<your-token> --server=<your-server-url>

   You can find your login token by accessing your cluster in pull secret from Red Hat OpenShift Cluster Manager.

2. Validate that your cluster has STS by running the following command:

   $ oc get authentication.config.openshift.io cluster -o json \
     | jq .spec.serviceAccountIssuer

   Example output

   "https://xxxxx.cloudfront.net/xxxxx"

   If your output is different, do not proceed. See Red Hat documentation on creating an STS cluster before continuing this process.

3. Set the SecurityContextConstraints permission to allow the Container Storage Interface (CSI) driver to run by running the following command:

   $ oc new-project csi-secrets-store
   $ oc adm policy add-scc-to-user privileged \
       system:serviceaccount:csi-secrets-store:secrets-store-csi-driver
   $ oc adm policy add-scc-to-user privileged \
       system:serviceaccount:csi-secrets-store:csi-secrets-store-provider-aws

4. Create environment variables to use later in this process by running the following command:

   $ export REGION=$(oc get infrastructure cluster -o=jsonpath="{.status.platformStatus.aws.region}")
   $ export OIDC_ENDPOINT=$(oc get authentication.config.openshift.io cluster \
      -o jsonpath='{.spec.serviceAccountIssuer}' | sed  's|^https://||')
   $ export AWS_ACCOUNT_ID=`aws sts get-caller-identity --query Account --output text`
   $ export AWS_PAGER=""

Procedure

1. Use Helm to register the Secrets Store CSI Driver by running the following command:

   $ helm repo add secrets-store-csi-driver \
       https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts

2. Update your Helm repositories by running the following command:

   $ helm repo update

3. Install the Secrets Store CSI Driver by running the following command:

   $ helm upgrade --install -n csi-secrets-store \
       csi-secrets-store-driver secrets-store-csi-driver/secrets-store-csi-driver

4. Deploy the AWS provider by running the following command:

   $ oc -n csi-secrets-store apply -f \
       https://raw.githubusercontent.com/rh-mobb/documentation/main/content/misc/secrets-store-csi/aws-provider-installer.yaml

5. Check that both Daemonsets are running by running the following command:

   $ oc -n csi-secrets-store get ds \
       csi-secrets-store-provider-aws \
       csi-secrets-store-driver-secrets-store-csi-driver

6. Label the Secrets Store CSI Driver to allow use with the restricted pod security profile by running the following command:

   $ oc label csidriver.storage.k8s.io/secrets-store.csi.k8s.io security.openshift.io/csi-ephemeral-volume-profile=restricted

Procedure

1. Create a secret in AWS Secrets Manager by running the following command:

   $ SECRET_ARN=$(aws --region "$REGION" secretsmanager create-secret \
       --name MySecret --secret-string \
       '{"username":"shadowman", "password":"hunter2"}' \
       --query ARN --output text); echo $SECRET_ARN

2. Create an IAM Access Policy document by running the following command:

   $ cat << EOF > policy.json
   {
      "Version": "2012-10-17",
      "Statement": [{
         "Effect": "Allow",
         "Action": [
           "secretsmanager:GetSecretValue",
           "secretsmanager:DescribeSecret"
         ],
         "Resource": ["$SECRET_ARN"]
         }]
   }
   EOF

3. Create an IAM Access Policy by running the following command:

   $ POLICY_ARN=$(aws --region "$REGION" --query Policy.Arn \
   --output text iam create-policy \
   --policy-name openshift-access-to-mysecret-policy \
   --policy-document file://policy.json); echo $POLICY_ARN

4. Create an IAM Role trust policy document by running the following command:

   Note

   The trust policy is locked down to the default service account of a namespace you create later in this process.

   $ cat <<EOF > trust-policy.json
   {
      "Version": "2012-10-17",
      "Statement": [
      {
      "Effect": "Allow",
      "Condition": {
        "StringEquals" : {
          "${OIDC_ENDPOINT}:sub": ["system:serviceaccount:my-application:default"]
         }
       },
       "Principal": {
          "Federated": "arn:aws:iam::$AWS_ACCOUNT_ID:oidc-provider/${OIDC_ENDPOINT}"
       },
       "Action": "sts:AssumeRoleWithWebIdentity"
       }
       ]
   }
   EOF

5. Create an IAM role by running the following command:

   $ ROLE_ARN=$(aws iam create-role --role-name openshift-access-to-mysecret \
   --assume-role-policy-document file://trust-policy.json \
   --query Role.Arn --output text); echo $ROLE_ARN

6. Attach the role to the policy by running the following command:

   $ aws iam attach-role-policy --role-name openshift-access-to-mysecret \
       --policy-arn $POLICY_ARN

Procedure

1. Create an OpenShift project by running the following command:

   $ oc new-project my-application

2. Annotate the default service account to use the Security Token Service (STS) role by running the following command:

   $ oc annotate -n my-application serviceaccount default \
       eks.amazonaws.com/role-arn=$ROLE_ARN

3. Create a secret provider class to access our secret by running the following command:

   $ cat << EOF | oc apply -f -
   apiVersion: secrets-store.csi.x-k8s.io/v1
   kind: SecretProviderClass
   metadata:
     name: my-application-aws-secrets
   spec:
     provider: aws
     parameters:
       objects: |
         - objectName: "MySecret"
           objectType: "secretsmanager"
   EOF

4. Create a deployment by using our secret in the following command:

   $ cat << EOF | oc apply -f -
   apiVersion: v1
   kind: Pod
   metadata:
     name: my-application
     labels:
       app: my-application
   spec:
     volumes:
     - name: secrets-store-inline
       csi:
         driver: secrets-store.csi.k8s.io
         readOnly: true
         volumeAttributes:
           secretProviderClass: "my-application-aws-secrets"
     containers:
     - name: my-application-deployment
       image: k8s.gcr.io/e2e-test-images/busybox:1.29
       command:
         - "/bin/sleep"
         - "10000"
       volumeMounts:
       - name: secrets-store-inline
         mountPath: "/mnt/secrets-store"
         readOnly: true
   EOF

Procedure

1. Delete the application by running the following command:

   $ oc delete project my-application

2. Delete the secrets store Container Storage Interface (CSI) driver by running the following command:

   $ helm delete -n csi-secrets-store csi-secrets-store-driver

3. Delete the security context constraints by running the following command:

   $ oc adm policy remove-scc-from-user privileged \
       system:serviceaccount:csi-secrets-store:secrets-store-csi-driver; oc adm policy remove-scc-from-user privileged \
       system:serviceaccount:csi-secrets-store:csi-secrets-store-provider-aws

4. Delete the AWS provider by running the following command:

   $ oc -n csi-secrets-store delete -f \
   https://raw.githubusercontent.com/rh-mobb/documentation/main/content/misc/secrets-store-csi/aws-provider-installer.yaml

5. Delete AWS roles and policies by running the following command:

   $ aws iam detach-role-policy --role-name openshift-access-to-mysecret \
       --policy-arn $POLICY_ARN; aws iam delete-role --role-name openshift-access-to-mysecret; aws iam delete-policy --policy-arn $POLICY_ARN

6. Delete the Secrets Manager secret by running the following command:

   $ aws secretsmanager --region $REGION delete-secret --secret-id $SECRET_ARN