Deploying the External DNS Operator on ROSA | Tutorials

Prerequisites
Setting up your environment
Setting up your custom domain
Preparing your AWS account
Installing the External DNS Operator
Deploying a sample application

Starting with Red Hat OpenShift Service on AWS 4.14, the Custom Domain Operator is deprecated. To manage Ingress in Red Hat OpenShift Service on AWS 4.14, use the Ingress Operator. The functionality is unchanged for Red Hat OpenShift Service on AWS 4.13 and earlier versions.

Configuring the Custom Domain Operator requires a wildcard CNAME DNS record in your Amazon Route 53 hosted zone. If you do not want to use a wildcard record, you can use the External DNS Operator to create individual entries for routes.

Use this tutorial to deploy and configure the External DNS Operator with a custom domain in Red Hat OpenShift Service on AWS (ROSA).

The External DNS Operator does not support STS using IAM Roles for Service Accounts (IRSA) and uses long-lived Identity Access Management (IAM) credentials instead. This tutorial will be updated when the Operator supports STS.

edit

Prerequisites

A ROSA cluster
A user account with dedicated-admin privileges
The OpenShift CLI (oc)
The Amazon Web Services (AWS) CLI (aws)
A unique domain, such as *.apps.<company_name>.io
An Amazon Route 53 public hosted zone for the above domain

Setting up your environment

Configure the following environment variables, replacing CLUSTER_NAME with the name of your cluster:

$ export DOMAIN=apps.<company_name>.io (1)
$ export AWS_PAGER=""
$ export CLUSTER_NAME=$(oc get infrastructure cluster -o=jsonpath="{.status.infrastructureName}"  | sed 's/-[a-z0-9]\{5\}$//')
$ export REGION=$(oc get infrastructure cluster -o=jsonpath="{.status.platformStatus.aws.region}")
$ export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
$ export SCRATCH="/tmp/${CLUSTER_NAME}/external-dns"
$ mkdir -p ${SCRATCH}

1	The custom domain.

Ensure all fields output correctly before moving to the next section:

$ echo "Cluster: ${CLUSTER_NAME}, Region: ${REGION}, AWS Account ID: ${AWS_ACCOUNT_ID}"

Setting up your custom domain

ROSA manages secondary Ingress Controllers using the Custom Domain Operator. Use the following procedure to deploy a secondary Ingress Controller using a custom domain.

Prerequisites

A unique domain, such as *.apps.<company_name>.io
A custom SAN or wildcard certificate, such as CN=*.apps.<company_name>.io

Procedure

Create a new project:
```
$ oc new-project external-dns-operator
```
Create a new TLS secret from a private key and a public certificate, where fullchain.pem is your full wildcard certificate chain (including any intermediaries) and privkey.pem is your wildcard certificate’s private key:
```
$ oc -n external-dns-operator create secret tls external-dns-tls --cert=fullchain.pem --key=privkey.pem
```

Create a new CustomDomain custom resource (CR):

Example external-dns-custom-domain.yaml

apiVersion: managed.openshift.io/v1alpha1
kind: CustomDomain
metadata:
  name: external-dns
spec:
  domain: apps.<company_name>.io (1)
  scope: External
  loadBalancerType: NLB
  certificate:
    name: external-dns-tls
    namespace: external-dns-operator

1	The custom domain.

Apply the CR:

$ oc apply -f external-dns-custom-domain.yaml

Verify that your custom domain Ingress Controller has been deployed and has a Ready status:

$ oc get customdomains

Example output

NAME               ENDPOINT                                                    DOMAIN                       STATUS
external-dns       xxrywp.<company_name>.cluster-01.opln.s1.openshiftapps.com  *.apps.<company_name>.io     Ready

Preparing your AWS account

Retrieve the Amazon Route 53 public hosted zone ID:

$ export ZONE_ID=$(aws route53 list-hosted-zones-by-name --output json \
  --dns-name "${DOMAIN}." --query 'HostedZones[0]'.Id --out text | sed 's/\/hostedzone\///')

Create an AWS IAM Policy document that allows the External DNS Operator to update only the custom domain public hosted zone:

$ cat << EOF > "${SCRATCH}/external-dns-policy.json"
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "route53:ChangeResourceRecordSets"
      ],
      "Resource": [
        "arn:aws:route53:::hostedzone/${ZONE_ID}"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "route53:ListHostedZones",
        "route53:ListResourceRecordSets"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}
EOF

Create an AWS IAM policy:

$ export POLICY_ARN=$(aws iam create-policy --policy-name "${CLUSTER_NAME}-AllowExternalDNSUpdates" \
  --policy-document file://${SCRATCH}/external-dns-policy.json \
  --query 'Policy.Arn' --output text)

Create an AWS IAM user:

$ aws iam create-user --user-name "${CLUSTER_NAME}-external-dns-operator"

Attach the policy:

$ aws iam attach-user-policy --user-name "${CLUSTER_NAME}-external-dns-operator" --policy-arn $POLICY_ARN

This will be changed to STS using IRSA in the future.

Create AWS keys for the IAM user:

$ SECRET_ACCESS_KEY=$(aws iam create-access-key --user-name "${CLUSTER_NAME}-external-dns-operator")

Create static credentials:

$ cat << EOF > "${SCRATCH}/credentials"
[default]
aws_access_key_id = $(echo $SECRET_ACCESS_KEY | jq -r '.AccessKey.AccessKeyId')
aws_secret_access_key = $(echo $SECRET_ACCESS_KEY | jq -r '.AccessKey.SecretAccessKey')
EOF

Installing the External DNS Operator

Install the External DNS Operator from OperatorHub:

$ cat << EOF | oc apply -f -
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: external-dns-group
  namespace: external-dns-operator
spec:
  targetNamespaces:
  - external-dns-operator
---
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: external-dns-operator
  namespace: external-dns-operator
spec:
  channel: stable-v1.1
  installPlanApproval: Automatic
  name: external-dns-operator
  source: redhat-operators
  sourceNamespace: openshift-marketplace
EOF

Wait until the External DNS Operator is running:

$ oc rollout status deploy external-dns-operator --timeout=300s

Create a secret from the AWS IAM user credentials:

$ oc -n external-dns-operator create secret generic external-dns \
  --from-file "${SCRATCH}/credentials"

Deploy the ExternalDNS controller:

$ cat << EOF | oc apply -f -
apiVersion: externaldns.olm.openshift.io/v1beta1
kind: ExternalDNS
metadata:
  name: ${DOMAIN}
spec:
  domains:
    - filterType: Include
      matchType: Exact
      name: ${DOMAIN}
  provider:
    aws:
      credentials:
        name: external-dns
    type: AWS
  source:
    openshiftRouteOptions:
      routerName: external-dns
    type: OpenShiftRoute
  zones:
    - ${ZONE_ID}
EOF

Wait until the controller is running:

$ oc rollout status deploy external-dns-${DOMAIN} --timeout=300s

Deploying a sample application

Now that the ExternalDNS controller is running, you can deploy a sample application to confirm that the custom domain is configured and trusted when you expose a new route.

Create a new project for your sample application:
```
$ oc new-project hello-world
```

Deploy a hello world application:

$ oc new-app -n hello-world --image=docker.io/openshift/hello-openshift

Create a route for the application specifying your custom domain name:

$ oc -n hello-world create route edge --service=hello-openshift hello-openshift-tls \
--hostname hello-openshift.${DOMAIN}

Check if the DNS record was created automatically by ExternalDNS:

It can take a few minutes for the record to appear in Amazon Route 53.

$ aws route53 list-resource-record-sets --hosted-zone-id ${ZONE_ID} \
   --query "ResourceRecordSets[?Type == 'CNAME']" | grep hello-openshift

Optional: You can also view the TXT records that indicate they were created by ExternalDNS:

$ aws route53 list-resource-record-sets --hosted-zone-id ${ZONE_ID} \
   --query "ResourceRecordSets[?Type == 'TXT']" | grep ${DOMAIN}

Navigate to your custom console domain in the browser where you see the OpenShift login:
```
$ echo console.${DOMAIN}
```

Tutorial: Deploying the External DNS Operator on ROSA

Prerequisites

Setting up your environment

Setting up your custom domain

Preparing your AWS account

Installing the External DNS Operator

Deploying a sample application