Chapter 2. Configuring private connections
2.1. Configuring private connections Copy linkLink copied to clipboard!
You can configure private connections for your Red Hat OpenShift Service on AWS cluster by using AWS Virtual Private Cloud (VPC) peering, AWS virtual private network (VPN), or AWS Direct Connect. These methods establish secure, dedicated network communication between your infrastructure and your cluster.
2.2. Configuring AWS VPC peering Copy linkLink copied to clipboard!
Configure an AWS Virtual Private Cloud (VPC) containing your Red Hat OpenShift Service on AWS cluster to peer with another AWS VPC network.
2.2.1. VPC peering terms Copy linkLink copied to clipboard!
When setting up a Virtual Private Cloud (VPC) peering connection between two VPCs on two separate AWS accounts, the following terms are used:
| Red Hat OpenShift Service on AWS AWS Account | The AWS account that contains the Red Hat OpenShift Service on AWS cluster. |
| Red Hat OpenShift Service on AWS Cluster VPC | The VPC that contains the Red Hat OpenShift Service on AWS cluster. |
| Customer AWS Account | Your non-Red Hat OpenShift Service on AWS AWS Account that you want to peer with. |
| Customer VPC | The VPC in your AWS Account that you want to peer with. |
| Customer VPC Region | The region where the customer’s VPC resides. |
As of July 2018, AWS supports inter-region VPC peering between all commercial regions excluding China.
2.2.2. Initiate the VPC peer request Copy linkLink copied to clipboard!
You can send a Virtual Private Cloud (VPC) peering connection request from the Red Hat OpenShift Service on AWS AWS Account to the Customer AWS Account.
Prerequisites
Gather the following information about the Customer VPC required to initiate the peering request:
- Customer AWS account number
- Customer VPC ID
- Customer VPC Region
- Customer VPC CIDR
- Check the CIDR block used by the Red Hat OpenShift Service on AWS Cluster VPC. If it overlaps or matches the CIDR block for the Customer VPC, then peering between these two VPCs is not possible; see the Amazon VPC Unsupported VPC Peering Configurations documentation for details. If the CIDR blocks do not overlap, you can continue with the procedure.
Procedure
- Log in to the Web Console for the Red Hat OpenShift Service on AWS AWS Account and navigate to the VPC Dashboard in the region where the cluster is being hosted.
- Go to the Peering Connections page and click the Create Peering Connection button.
Verify the details of the account you are logged in to and the details of the account and VPC you are connecting to:
- Peering connection name tag: Set a descriptive name for the VPC Peering Connection.
- VPC (Requester): Select the Red Hat OpenShift Service on AWS Cluster VPC ID from the list.
- Account: Select Another account and provide the Customer AWS Account number (without dashes).
- Region: If the Customer VPC Region differs from the current region, select Another Region and select the customer VPC Region from the list.
- VPC (Accepter): Set the Customer VPC ID.
- Click Create Peering Connection.
- Confirm that the request enters a Pending state. If it enters a Failed state, confirm the details and repeat the process.
2.2.3. Accept the VPC peer request Copy linkLink copied to clipboard!
After you create the Virtual Private Cloud (VPC) peering connection, you must accept the request in the Customer AWS Account.
Prerequisites
- Initiate the VPC peer request.
Procedure
- Log in to the AWS Web Console.
- Navigate to VPC Service.
- Go to Peering Connections.
- Click Pending peering connection.
- Confirm the AWS Account and VPC ID that the request originated from. This should be from the Red Hat OpenShift Service on AWS AWS Account and Red Hat OpenShift Service on AWS Cluster VPC.
- Click Accept Request.
2.2.4. Configure the routing tables Copy linkLink copied to clipboard!
After you accept the Virtual Private Cloud (VPC) peering request, both VPCs must configure their routes to communicate across the peering connection.
Prerequisites
- Initiate and accept the VPC peer request.
Procedure
- Log in to the AWS Web Console for the Red Hat OpenShift Service on AWS AWS Account.
- Navigate to the VPC Service, then Route tables.
Select the Route Table for the Red Hat OpenShift Service on AWS Cluster VPC.
NoteOn some clusters, there may be more than one route table for a particular VPC. Select the private one that has several explicitly associated subnets.
- Select the Routes tab, then Edit.
- Enter the Customer VPC CIDR block in the Destination text box.
- Enter the Peering Connection ID in the Target text box.
- Click Save.
You must complete the same process with the other VPC’s CIDR block:
-
Log into the Customer AWS Web Console
VPC Service Route Tables. - Select the Route Table for your VPC.
- Select the Routes tab, then Edit.
- Enter the Red Hat OpenShift Service on AWS Cluster VPC CIDR block in the Destination text box.
- Enter the Peering Connection ID in the Target text box.
- Click Save changes.
-
Log into the Customer AWS Web Console
2.2.5. Verify and troubleshoot VPC peering Copy linkLink copied to clipboard!
After you set up a Virtual Private Cloud (VPC) peering connection, it is best to confirm it has been configured and is working correctly.
Prerequisites
- Initiate and accept the VPC peer request.
- Configure the routing tables.
Procedure
In the AWS console, look at the route table for the cluster VPC that is peered. Ensure that the steps for configuring the routing tables were followed and that there is a route table entry pointing the VPC Classless Inter-Domain Routing (CIDR) range destination to the peering connection target.
If the routes look correct on both the Red Hat OpenShift Service on AWS Cluster VPC route table and Customer VPC route table, then the connection should be tested using the
netcatmethod below. If the test calls are successful, then VPC peering is working correctly.To test network connectivity to an endpoint device,
nc(ornetcat) is a helpful troubleshooting tool. It is included in the default image and provides quick and clear output if a connection can be established:Create a temporary pod using the
busyboximage, which cleans up after itself:$ oc run netcat-test \ --image=busybox -i -t \ --restart=Never --rm \ -- /bin/shCheck the connection using
nc.Example successful connection results:
/ nc -zvv 192.168.1.1 8080 10.181.3.180 (10.181.3.180:8080) open sent 0, rcvd 0Example failed connection results:
/ nc -zvv 192.168.1.2 8080 nc: 10.181.3.180 (10.181.3.180:8081): Connection refused sent 0, rcvd 0
Exit the container, which automatically deletes the Pod:
/ exit
2.3. Configuring AWS VPN Copy linkLink copied to clipboard!
Configure your Amazon Web Services (AWS) Red Hat OpenShift Service on AWS cluster to connect to an on-site hardware virtual private network (VPN) device.
AWS VPN does not currently provide a managed option to apply Network Address Translation (NAT) to VPN traffic. Routing all traffic, for example 0.0.0.0/0, through a private connection is not supported. This requires deleting the internet gateway, which disables Site Reliability Engineering (SRE) management traffic.
2.3.1. Create a customer gateway and virtual private gateway Copy linkLink copied to clipboard!
Create a customer gateway and, if needed, a virtual private gateway to prepare for a VPN connection to your Red Hat OpenShift Service on AWS cluster.
Prerequisites
- Hardware VPN gateway device model and software version, for example Cisco ASA running version 8.3. See the Amazon VPC Network Administrator Guide to confirm whether your gateway device is supported by AWS.
- Public, static IP address for the VPN gateway device.
- Border Gateway Protocol (BGP) or static routing: if using BGP, the Autonomous System Number (ASN) is required. If using static routing, you must configure at least one static route.
- Optional: IP and Port/Protocol of a reachable service to test the VPN connection.
Procedure
- Log in to the Red Hat OpenShift Service on AWS AWS Account Dashboard, and navigate to the VPC Dashboard.
- Under Virtual private cloud click Your VPCs and identify the name and VPC ID for the VPC containing the Red Hat OpenShift Service on AWS cluster.
- Under Virtual private network (VPN) click Customer gateways.
- Click Create customer gateway and give it a meaningful name.
- Enter the ASN of your customer gateway device in the BGP ASN field.
- In the IP address field, enter the IP address of an external interface for your customer gateway devices.
- Click Create customer gateway.
If you do not already have a Virtual Private Gateway attached to the intended VPC:
- From the VPC Dashboard, click Virtual Private Gateways.
- Click Create virtual private gateway, give it a meaningful name.
- Click Create virtual private gateway, leaving the Amazon default ASN.
- Select the newly created gateway.
- Select Actions from the list and click Attach to VPC.
- Select the newly created gateway under Available VPC’s and click Attach to VPC to attach it to the cluster VPC you identified earlier.
2.3.2. Establish the VPN connection and enable route propagation Copy linkLink copied to clipboard!
After you create the customer gateway and virtual private gateway, establish the Site-to-Site VPN connection and enable route propagation.
Prerequisites
- You created a customer gateway in your Red Hat OpenShift Service on AWS AWS account.
- You have a virtual private gateway attached to the cluster Virtual Private Cloud (VPC).
Procedure
- From the VPC dashboard, under Virtual private network (VPN) click Site-to-Site VPN connections.
Click Create VPN connection.
- Give it a meaningful name tag.
- Select the Virtual private gateway created previously.
- For Customer gateway, select Existing.
- Select the Customer gateway ID by name.
- If the VPN uses the Border Gateway Protocol (BGP), select Dynamic, otherwise select Static and enter the Static IP Classless Inter-Domain Routings (CIDRs). If there are multiple CIDRs, add each CIDR as Another Rule.
- Click Create VPN connection.
- Under State wait for the VPN status to change from Pending to Available, approximately 5 to 10 minutes.
Select the VPN you just created and click Download configuration.
- From the list, select the vendor, platform, and version of the customer gateway device, then click Download.
The Generic vendor configuration is also available for retrieving information in a plain text format.
NoteAfter the VPN connection has been established, set up Route Propagation or the VPN might not function as expected. Note the VPC subnet information, which you must add to your configuration as the remote network.
- From the VPC Dashboard, under Virtual private cloud, click on Route tables.
Select the private Route table associated with the VPC that contains your Red Hat OpenShift Service on AWS cluster.
NoteOn some clusters, there may be more than one route table for a particular VPC. Select the private one that has several explicitly associated subnets.
- Click on the Route Propagation tab.
In the table that shows, you see the Virtual Private Gateway you created. Check the value in the Propagate column.
- If Propagation is set to No, click Edit route propagation, and check the Enable checkbox in Propagation.
Click Save.
After you configure your VPN tunnel and AWS detects it as Up, your static or BGP routes are automatically added to the route table.
2.3.3. Verify the VPN connection Copy linkLink copied to clipboard!
After you have set up your side of the Virtual Private Network (VPN) tunnel, you can verify that the tunnel is up in the AWS console and that connectivity across the tunnel is working.
Prerequisites
- Created a VPN connection.
Procedure
Verify the tunnel is up in AWS.
- From the VPC Dashboard, under Virtual private network (VPN), click on Site-to-Site VPN connections.
- Select the VPN connection you created previously and click the Tunnel details tab.
- You should see that at least one of the VPN tunnels is in an Up status.
Verify the connection.
To test network connectivity to an endpoint device,
nc(ornetcat) is a helpful troubleshooting tool. It is included in the default image and provides quick and clear output if a connection can be established:Create a temporary pod using the
busyboximage, which cleans up after itself:$ oc run netcat-test \ --image=busybox -i -t \ --restart=Never --rm \ -- /bin/shCheck the connection using
nc.Example successful connection results:
/ nc -zvv 192.168.1.1 8080 10.181.3.180 (10.181.3.180:8080) open sent 0, rcvd 0Example failed connection results:
/ nc -zvv 192.168.1.2 8080 nc: 10.181.3.180 (10.181.3.180:8081): Connection refused sent 0, rcvd 0
Exit the container, which automatically deletes the Pod:
/ exit
2.3.4. Troubleshoot the VPN connection Copy linkLink copied to clipboard!
Review the following troubleshooting scenarios for common VPN connection issues.
2.3.4.1. Tunnel does not connect Copy linkLink copied to clipboard!
If the tunnel connection is still Down, there are several things you can verify:
- The AWS tunnel will not initiate a VPN connection. The connection attempt must be initiated from the Customer Gateway.
- Ensure that your source traffic is coming from the same IP as the configured customer gateway. AWS will silently drop all traffic to the gateway whose source IP address does not match.
- Ensure that your configuration matches values supported by AWS. This includes IKE versions, DH groups, IKE lifetime, and more.
- Recheck the route table for the VPC. Ensure that propagation is enabled and that there are entries in the route table that have the virtual private gateway you created earlier as a target.
- Confirm that you do not have any firewall rules that could be causing an interruption.
- Check if you are using a policy-based VPN as this can cause complications depending on how it is configured.
2.3.4.2. Tunnel does not stay connected Copy linkLink copied to clipboard!
If the tunnel connection doesn’t remain Up, check that all AWS tunnel connections must be initiated from your gateway. AWS tunnels do not initiate tunneling.
Red Hat recommends setting up an SLA Monitor (Cisco ASA) or some device on your side of the tunnel that constantly sends "interesting" traffic, for example ping, nc, or telnet, at any IP address configured within the VPC CIDR range. It does not matter whether the connection is successful, just that the traffic is being directed at the tunnel.
2.3.4.3. Secondary tunnel in Down state Copy linkLink copied to clipboard!
When a VPN tunnel is created, AWS creates an additional failover tunnel. Depending upon the gateway device, sometimes the secondary tunnel will be seen as in the Down state.
The AWS Notification is as follows:
You have new non-redundant VPN connections
One or more of your vpn connections are not using both tunnels. This mode of operation is not highly available and we strongly recommend you configure your second tunnel. View your non-redundant VPN connections.
2.4. Configuring AWS Direct Connect Copy linkLink copied to clipboard!
Accept an AWS Direct Connect virtual interface for use with your Red Hat OpenShift Service on AWS cluster.
2.4.1. AWS Direct Connect methods Copy linkLink copied to clipboard!
To access a remote VPC in the same or another account, a Direct Connect connection requires a hosted Virtual Interface (VIF) connected to a Direct Connect Gateway (DXGateway), which is in turn associated to a Virtual Gateway (VGW) or a Transit Gateway.
If you do not have an existing DXGateway, the typical process involves creating the hosted VIF, with the DXGateway and VGW being created in the Red Hat OpenShift Service on AWS AWS Account.
If you have an existing DXGateway connected to one or more existing VGWs, the process involves the Red Hat OpenShift Service on AWS AWS Account sending an Association Proposal to the DXGateway owner. The DXGateway owner must ensure that the proposed CIDR will not conflict with any other VGWs they have associated.
When connecting to an existing DXGateway, you are responsible for the costs. For more information, see Additional resources.
There are two configuration options available:
| Method 1 | Create the hosted VIF and then the DXGateway and VGW. |
| Method 2 | Request a connection by using an existing Direct Connect Gateway that you own. |
2.4.2. Determine the Direct Connect Virtual Interface type Copy linkLink copied to clipboard!
Determine whether your Direct Connect Virtual Interface (VIF) is private or public. This determines whether you create a Virtual Private Gateway or a Direct Connect Gateway.
Prerequisites
- Gather Red Hat OpenShift Service on AWS AWS Account ID.
Procedure
- Log in to the Red Hat OpenShift Service on AWS AWS Account Dashboard and select the correct region.
- Select Direct Connect from the Services menu.
- There will be one or more Virtual Interfaces waiting to be accepted, select one of them to view the Summary.
- View the Virtual Interface type: private or public.
- Record the Amazon side ASN value.
Next steps
If the Virtual Interface type is private, create a Virtual Private Gateway. If the Virtual Interface type is public, create a Direct Connect Gateway.
2.4.3. Create a Virtual Private Gateway for a private Direct Connect Copy linkLink copied to clipboard!
If your Direct Connect Virtual Interface type is private, create and attach a Virtual Private Gateway to your Red Hat OpenShift Service on AWS cluster VPC.
Prerequisites
- You determined the Virtual Interface type is private.
- You recorded the Amazon side ASN value.
Procedure
- Log in to the Red Hat OpenShift Service on AWS AWS Account Dashboard and select the correct region.
- From the AWS region, select VPC from the Services menu.
- From Virtual private network (VPN), select Virtual private gateways.
- Click Create virtual private gateway.
- Give the Virtual Private Gateway a suitable name.
- Select Custom ASN in the Enter custom ASN field.
- Enter the Amazon side ASN value gathered previously.
- Click Create virtual private gateway.
- Click the newly created Virtual Private Gateway and choose Attach to VPC from the Actions tab.
- Select the Red Hat OpenShift Service on AWS Cluster VPC from the list, and click Attach VPC.
Verification
After the Direct Connect Virtual Interfaces have been accepted, wait a short period and view the status of the Interfaces.
- Log in to the Red Hat OpenShift Service on AWS AWS Account Dashboard and select the correct region.
- From the Red Hat OpenShift Service on AWS AWS Account region, select Direct Connect from the Services menu.
- Select one of the Direct Connect Virtual Interfaces from the list.
- Check the Interface State has become Available.
- Check the Interface BGP Status has become Up.
Repeat this verification for any remaining Direct Connect Interfaces.
After the Direct Connect Virtual Interfaces are available, you can log in to the Red Hat OpenShift Service on AWS AWS Account Dashboard and download the Direct Connect configuration file for configuration on your side.
2.4.4. Create a Direct Connect Gateway for a public Direct Connect Copy linkLink copied to clipboard!
If your Direct Connect Virtual Interface type is public, create a Direct Connect Gateway.
Prerequisites
- You determined the Virtual Interface type is public.
- You recorded the Amazon side ASN value.
Procedure
- Log in to the Red Hat OpenShift Service on AWS AWS Account Dashboard and select the correct region.
- From the Red Hat OpenShift Service on AWS AWS Account region, select Direct Connect from the Services menu.
- Select Direct Connect gateways and Create Direct Connect gateway.
- Give the Direct Connect gateway a suitable name.
- In the Amazon side ASN, enter the Amazon side ASN value gathered previously.
- Click Create the Direct Connect gateway.
Verification
After the Direct Connect Virtual Interfaces have been accepted, wait a short period and view the status of the Interfaces.
- Log in to the Red Hat OpenShift Service on AWS AWS Account Dashboard and select the correct region.
- From the Red Hat OpenShift Service on AWS AWS Account region, select Direct Connect from the Services menu.
- Select one of the Direct Connect Virtual Interfaces from the list.
- Check the Interface State has become Available.
- Check the Interface BGP Status has become Up.
Repeat this verification for any remaining Direct Connect Interfaces.
After the Direct Connect Virtual Interfaces are available, you can log in to the Red Hat OpenShift Service on AWS AWS Account Dashboard and download the Direct Connect configuration file for configuration on your side.
2.4.5. Create a virtual private gateway for a Direct Connect connection Copy linkLink copied to clipboard!
Create a virtual private gateway in your Red Hat OpenShift Service on AWS AWS account to connect to an existing Direct Connect Gateway.
Prerequisites
- Confirm the Classless Inter-Domain Routing (CIDR) range of the Red Hat OpenShift Service on AWS Virtual Private Cloud (VPC) doesn’t conflict with any other VGWs you have associated.
Gather the following information:
- The Direct Connect Gateway ID.
- The AWS Account ID associated with the virtual interface.
- The BGP ASN assigned for the DXGateway. Optional: the Amazon default ASN may also be used.
Procedure
- Log in to the Red Hat OpenShift Service on AWS AWS Account Dashboard and select the correct region.
- From the Red Hat OpenShift Service on AWS AWS Account region, select VPC from the Services menu.
- From Virtual private network (VPN), select Virtual private gateways.
- Select Create virtual private gateway.
- Give the virtual private gateway a suitable name in the Details field.
- Click Custom ASN and enter the Amazon side ASN value gathered previously or use the Amazon Provided ASN.
- Click Create virtual private gateway.
2.4.6. Associate the Direct Connect gateway with the virtual private gateway Copy linkLink copied to clipboard!
After you create a virtual private gateway, associate it with your existing Direct Connect gateway.
Prerequisites
- You created a virtual private gateway in your Red Hat OpenShift Service on AWS AWS account.
Procedure
- From the Red Hat OpenShift Service on AWS AWS Account region, select Direct Connect from the Services menu.
- Click virtual private gateways and select the virtual private gateway.
- Click View details.
- Click the Direct Connect gateway associations tab.
- Click Associate Direct Connect gateway
- Under Association account type, for Account owner, click Another account.
- Under Association settings, for Direct Connect gateway ID, enter the ID of the Direct Connect gateway.
- For Direct Connect gateway owner, enter the ID of the AWS account that owns the Direct Connect gateway.
- Optional: Add prefixes to Allowed prefixes, separating them using commas or put them on separate lines.
- Click Associate Direct Connect gateway.
Next steps
After the Association Proposal has been sent, you must accept it. Follow the final steps you must perform are available in the Associate a Direct Connect virtual private gateway across accounts (AWS documentation).