1. Documentation
    2. history bug_report picture_as_pdf
×

Tutorial: Deploying ROSA with a Custom DNS Resolver

edit

A custom DHCP option set enables you to customize your VPC with your own DNS server, domain name, and more. Red Hat OpenShift Service on AWS (ROSA) clusters support using custom DHCP option sets. By default, ROSA clusters require setting the "domain name servers" option to AmazonProvidedDNS to ensure successful cluster creation and operation. Customers who want to use custom DNS servers for DNS resolution must do additional configuration to ensure successful ROSA cluster creation and operation.

In this tutorial, we will configure our DNS server to forward DNS lookups for specific DNS zones (further detailed below) to an Amazon Route 53 Inbound Resolver.

This tutorial uses the open-source BIND DNS server (named) to demonstrate the configuration necessary to forward DNS lookups to an Amazon Route 53 Inbound Resolver located in the VPC you plan to deploy a ROSA cluster into. Refer to the documentation of your preferred DNS server for how to configure zone forwarding.
edit

Prerequisites

  • ROSA CLI (rosa)

  • AWS CLI (aws)

  • A manually created AWS VPC

  • A DHCP option set configured to point to a custom DNS server and set as the default for your VPC

Setting up your environment

  1. Configure the following environment variables:

    $ export VPC_ID=<vpc_ID> (1)
$ export REGION=<region> (2)
$ export VPC_CIDR=<vpc_CIDR> (3)
    1 Replace <vpc_ID> with the ID of the VPC you want to install your cluster into.
    2 Replace <region> with the AWS region you want to install your cluster into.
    3 Replace <vpc_CIDR> with the CIDR range of your VPC.

  2. Ensure all fields output correctly before moving to the next section:

    $ echo "VPC ID: ${VPC_ID}, VPC CIDR Range: ${VPC_CIDR}, Region: ${REGION}"

Create an Amazon Route 53 Inbound Resolver

Use the following procedure to deploy an Amazon Route 53 Inbound Resolver in the VPC we plan to deploy the cluster into.

In this example, we deploy the Amazon Route 53 Inbound Resolver into the same VPC the cluster will use. If you want to deploy it into a separate VPC, you must manually associate the private hosted zone(s) detailed below once cluster creation is started. You cannot associate the zone before the cluster creation process begins. Failure to associate the private hosted zone during the cluster creation process will result in cluster creation failures.

  1. Create a security group and allow access to ports 53/tcp and 53/udp from the VPC:

    $ SG_ID=$(aws ec2 create-security-group --group-name rosa-inbound-resolver --description "Security group for ROSA inbound resolver" --vpc-id ${VPC_ID} --region ${REGION} --output text)
$ aws ec2 authorize-security-group-ingress --group-id ${SG_ID} --protocol tcp --port 53 --cidr ${VPC_CIDR} --region ${REGION}
$ aws ec2 authorize-security-group-ingress --group-id ${SG_ID} --protocol udp --port 53 --cidr ${VPC_CIDR} --region ${REGION}

  2. Create an Amazon Route 53 Inbound Resolver in your VPC:

    $ RESOLVER_ID=$(aws route53resolver create-resolver-endpoint \
  --name rosa-inbound-resolver \
  --creator-request-id rosa-$(date '+%Y-%m-%d') \
  --security-group-ids ${SG_ID} \
  --direction INBOUND \
  --ip-addresses $(aws ec2 describe-subnets --filter Name=vpc-id,Values=${VPC_ID} --region ${REGION} | jq -jr '.Subnets | map("SubnetId=\(.SubnetId) ") | .[]') \
  --region ${REGION} \
  --output text \
  --query 'ResolverEndpoint.Id')

    The above command attaches Amazon Route 53 Inbound Resolver endpoints to all subnets in the provided VPC using dynamically allocated IP addresses. If you prefer to manually specify the subnets and/or IP addresses, run the following command instead:

    $ RESOLVER_ID=$(aws route53resolver create-resolver-endpoint \
  --name rosa-inbound-resolver \
  --creator-request-id rosa-$(date '+%Y-%m-%d') \
  --security-group-ids ${SG_ID} \
  --direction INBOUND \
  --ip-addresses SubnetId=<subnet_ID>,Ip=<endpoint_IP> SubnetId=<subnet_ID>,Ip=<endpoint_IP> \(1)
  --region ${REGION} \
  --output text \
  --query 'ResolverEndpoint.Id')
    1 Replace <subnet_ID> with the subnet IDs and <endpoint_IP> with the static IP addresses you want inbound resolver endpoints added to.

  3. Get the IP addresses of your inbound resolver endpoints to configure in your DNS server configuration:

    $ aws route53resolver list-resolver-endpoint-ip-addresses \
  --resolver-endpoint-id ${RESOLVER_ID} \
  --region=${REGION} \
  --query 'IpAddresses[*].Ip'
    Example output
    [
    "10.0.45.253",
    "10.0.23.131",
    "10.0.148.159"
]

Configure your DNS server

Use the following procedure to configure your DNS server to forward the necessary private hosted zones to your Amazon Route 53 Inbound Resolver.

ROSA with HCP

ROSA with HCP clusters require you to configure DNS forwarding for two private hosted zones:

  • <cluster-name>.hypershift.local

  • rosa.<domain-prefix>.<unique-ID>.p3.openshiftapps.com

These Amazon Route 53 private hosted zones are created during cluster creation. The cluster-name and domain-prefix are customer-specified values, but the unique-ID is randomly generated during cluster creation and cannot be preselected. As such, you must wait for the cluster creation process to begin before configuring forwarding for the p3.openshiftapps.com private hosted zone.

  1. Before the cluster is created, configure your DNS server to forward all DNS requests for <cluster-name>.hypershift.local to your Amazon Route 53 Inbound Resolver endpoints. For BIND DNS servers, edit your /etc/named.conf file in your favorite text editor and add a new zone using the below example:

    Example
    zone "<cluster-name>.hypershift.local" { (1)
  type forward;
  forward only;
  forwarders { (2)
    10.0.45.253;
    10.0.23.131;
    10.0.148.159;
  };
};
    1 Replace <cluster-name> with your ROSA HCP cluster name.
    2 Replace with the IP addresses of your inbound resolver endpoints collected above, ensuring that following each IP address there is a ;.

  2. Create your cluster.

  3. Once your cluster has begun the creation process, locate the newly created private hosted zone:

    $ aws route53 list-hosted-zones-by-vpc \
  --vpc-id ${VPC_ID} \
  --vpc-region ${REGION} \
  --query 'HostedZoneSummaries[*].Name' \
  --output table
    Example output
    --------------------------------------------------
|             ListHostedZonesByVPC               |
+------------------------------------------------+
|  rosa.domain-prefix.lkmb.p3.openshiftapps.com. |
|  cluster-name.hypershift.local.                |
+------------------------------------------------+

    It may take a few minutes for the cluster creation process to create the private hosted zones in Route 53. If you do not see an p3.openshiftapps.com domain, wait a few minutes and run the command again.

  4. Once you know the unique ID of the cluster domain, configure your DNS server to forward all DNS requests for rosa.<domain-prefix>.<unique-ID>.p3.openshiftapps.com to your Amazon Route 53 Inbound Resolver endpoints. For BIND DNS servers, edit your /etc/named.conf file in your favorite text editor and add a new zone using the below example:

    Example
    zone "rosa.<domain-prefix>.<unique-ID>.p3.openshiftapps.com" { (1)
  type forward;
  forward only;
  forwarders { (2)
    10.0.45.253;
    10.0.23.131;
    10.0.148.159;
  };
};
    1 Replace <domain-prefix> with your cluster domain prefix and <unique-ID> with your unique ID collected above.
    2 Replace with the IP addresses of your inbound resolver endpoints collected above, ensuring that following each IP address there is a ;.

ROSA Classic

ROSA Classic clusters require you to configure DNS forwarding for one private hosted zones:

  • <domain-prefix>.<unique-ID>.p1.openshiftapps.com

This Amazon Route 53 private hosted zones is created during cluster creation. The domain-prefix is a customer-specified value, but the unique-ID is randomly generated during cluster creation and cannot be preselected. As such, you must wait for the cluster creation process to begin before configuring forwarding for the p1.openshiftapps.com private hosted zone.

  1. Create your cluster.

  2. Once your cluster has begun the creation process, locate the newly created private hosted zone:

    $ aws route53 list-hosted-zones-by-vpc \
  --vpc-id ${VPC_ID} \
  --vpc-region ${REGION} \
  --query 'HostedZoneSummaries[*].Name' \
  --output table
    Example output
    ----------------------------------------------
|           ListHostedZonesByVPC             |
+--------------------------------------------+
|  domain-prefix.agls.p3.openshiftapps.com.  |
+--------------------------------------------+

    It may take a few minutes for the cluster creation process to create the private hosted zones in Route 53. If you do not see an p1.openshiftapps.com domain, wait a few minutes and run the command again.

  3. Once you know the unique ID of the cluster domain, configure your DNS server to forward all DNS requests for <domain-prefix>.<unique-ID>.p1.openshiftapps.com to your Amazon Route 53 Inbound Resolver endpoints. For BIND DNS servers, edit your /etc/named.conf file in your favorite text editor and add a new zone using the below example:

    Example
    zone "<domain-prefix>.<unique-ID>.p1.openshiftapps.com" { (1)
  type forward;
  forward only;
  forwarders { (2)
    10.0.45.253;
    10.0.23.131;
    10.0.148.159;
  };
};
    1 Replace <domain-prefix> with your cluster domain prefix and <unique-ID> with your unique ID collected above.
    2 Replace with the IP addresses of your inbound resolver endpoints collected above, ensuring that following each IP address there is a ;.