Protect metadata service from DoS attacks with rate limiting

Limit metadata service query rates to protect against DoS attacks.

Before you begin

  • You have the oc command line tool installed on your workstation.
  • You are logged on to a workstation that has access to the RHOSO control plane as a user with cluster-admin privileges.
  • Your RHOSP environment uses IPv4 networking.

    Currently, the Networking service does not support metadata rate limiting on IPv6 networks.

  • You have a scheduled maintenance window.

    This procedure requires you to restart the OVN metadata service.

About this task

To add metadata rate limiting for a node set, complete these tasks:

  1. Create a ConfigMap custom resource (CR) to configure the nodes.
  2. Create a custom service for the feature that runs the playbook for the service.
  3. Include the ConfigMap CR in the custom service.

A detailed procedure follows.

Procedure

  1. Create a ConfigMap CR that defines a new configuration for metadata rate limiting, and save it to a YAML file on your workstation, for example, neutron-metadata-rate-limit.yaml.
    Note

    Do not use the name of the default configuration file, because it would override the infrastructure configuration, such as the transport_url.

    Set values for the following rate limiting parameters:

    rate_limit_enabled
    enables you to limit the rate of metadata requests. The default value is false. Set the value to true to enable metadata rate limiting.
    ip_versions
    the IP version, 4, used for metadata IP addresses on which you want to control query rates. RHOSP does not yet support metadata rate limiting for IPv6 networks.
    base_window_duration
    the time span, in seconds, during which query requests are limited. The default value is 10 seconds.
    base_query_rate_limit
    the maximum number of requests allowed during the base_window_duration. The default value is 10 requests.
    burst_window_duration
    the time span, in seconds, that a request rate higher than the base_window_duration is allowed. The default value is 10 seconds.
    burst_query_rate_limit
    the maximum number of requests allowed during the burst_window_duration. The default value is 10 requests.
    Example

    In this example, the Networking service is configured for a base time and rate that allows instances to query the IPv4 metadata service IP address 6 times over a 60 second period. The Networking service is also configured for a burst time and rate that allows a higher rate of 2 queries during shorter periods of 10 seconds each:

    apiVersion: v1
kind: ConfigMap
metadata:
  name: neutron-metadata-rate-limit
data:
   20-neutron-metadata-rate.conf: |
     [metadata_rate_limiting]
     rate_limit_enabled = True
     ip_versions = 4
     base_window_duration = 60
     base_query_rate_limit = 6
     burst_window_duration = 10
     burst_query_rate_limit = 2
     ...
  2. Create the ConfigMap object by using the ConfigMap CR file.
    Example
    $ oc create -f neutron-metadata-rate-limit.yaml -n openstack
  3. Create an OpenStackDataPlaneService CR that defines the metadata rate limit custom service, and save it to a YAML file on your workstation, for example neutron-metadata-rate-limit-service.yaml: 
    apiVersion: dataplane.openstack.org/v1beta1
kind: OpenStackDataPlaneService
metadata:
  name: neutron-metadata-rate-limit
  4. Add the ConfigMap CRs to the custom service, and specify the Secret CR for the cell that the node set that runs this service connects to: 
    apiVersion: dataplane.openstack.org/v1beta1
kind: OpenStackDataPlaneService
metadata:
  name: neutron-metadata-rate-limit
spec:
  dataSources:
    - configMapRef:
        name: neutron-metadata-rate-limit
    - secretRef:
        name: neutron-ovn-metadata-agent-neutron-config
    - secretRef:
        name: nova-metadata-neutron-config
    - configMapRef:
        name: neutron-metadata-rate-limit
  tlsCerts:
    default:
      contents:
      - dnsnames
      - ips
      networks:
      - ctlplane
      issuer: osp-rootca-issuer-ovn
      keyUsages:
        - digital signature
        - key encipherment
        - client auth
  caCerts: combined-ca-bundle
  containerImageFields:
  - EdpmNeutronMetadataAgentImage
  5. Specify the Ansible commands to create the custom service, by referencing an Ansible Playbook or by including the Ansible play in the playbookContents field: 
    apiVersion: dataplane.openstack.org/v1beta1
kind: OpenStackDataPlaneService
metadata:
  name: neutron-metadata-rate-limit
spec:
  playbook: osp.edpm.neutron_metadata
  dataSources:
    - configMapRef:
        name: neutron-metadata-rate-limit
    - secretRef:
        name: neutron-ovn-metadata-agent-neutron-config
    - secretRef:
        name: nova-metadata-neutron-config
  tlsCerts:
    default:
      contents:
      - dnsnames
      - ips
      networks:
      - ctlplane
      issuer: osp-rootca-issuer-ovn
      keyUsages:
        - digital signature
        - key encipherment
        - client auth
  caCerts: combined-ca-bundle
  containerImageFields:
  - EdpmNeutronMetadataAgentImage
  6. Create the metadata-rate-limit service: 
    $ oc apply -f neutron-metadata-rate-limit -n openstack

Results

  • Confirm that the custom service is created: 
    $ oc get openstackdataplaneservice neutron-metadata-rate-limit -o yaml -n openstack