OpenShift docs are moving and will soon only be available at docs.redhat.com, the home of all Red Hat product documentation. Explore the new docs experience today.
  1. Documentation
    2. history
×

Using admission controller enforcement

Red Hat Advanced Cluster Security for Kubernetes works with Kubernetes admission controllers and OpenShift Container Platform admission plug-ins to allow you to enforce security policies before Kubernetes or OpenShift Container Platform creates workloads, for example, deployments, daemon sets or jobs. The Red Hat Advanced Cluster Security for Kubernetes admission controller prevents users from creating workloads that violate policies you configure in Red Hat Advanced Cluster Security for Kubernetes. Beginning from the Red Hat Advanced Cluster Security for Kubernetes version 3.0.41, you can also configure the admission controller to prevent updates to workloads that violate policies.

Red Hat Advanced Cluster Security for Kubernetes uses the ValidatingAdmissionWebhook controller to verify that the resource being provisioned complies with the specified security policies. To handle this, Red Hat Advanced Cluster Security for Kubernetes creates a ValidatingWebhookConfiguration which contains multiple webhook rules. When the Kubernetes or OpenShift Container Platform API server receives a request that matches one of the webhook rules, the API server sends an AdmissionReview request to Red Hat Advanced Cluster Security for Kubernetes. Red Hat Advanced Cluster Security for Kubernetes then accepts or rejects the request based on the configured security policies.

To use admission controller enforcement on OpenShift Container Platform, you need the Red Hat Advanced Cluster Security for Kubernetes version 3.0.49 or newer.

Enabling admission controller enforcement

If you intend to use admission controller enforcement, consider the following:

  • API latency: Using admission controller enforcement increases Kubernetes or OpenShift Container Platform API latency because it involves additional API validation requests. Many standard Kubernetes libraries, such as fabric8, have short Kubernetes or OpenShift Container Platform API timeouts by default. Also, consider API timeouts in any custom automation you might be using.

  • Image scanning: You can choose whether the admission controller scans images while reviewing requests by setting the Contact Image Scanners option in the cluster configuration panel.

    • If you enable this setting, Red Hat Advanced Cluster Security for Kubernetes contacts image scanners if scan results are not already available, which adds considerable latency.

    • If you disable this setting, the enforcement decision only considers image scan criteria if cached scan results are available.

      • For Red Hat Advanced Cluster Security for Kubernetes version 3.0.41 and newer, the cached scan results are only available for images referenced by their image digest (@sha256:...).

  • You can use admission controller enforcement for:

    • Options in the pod securityContext.

    • Deployment configurations.

    • Image components and vulnerabilities.

  • You cannot use admission controller enforcement for:

    • Any runtime behavior, such as processes.

    • Any policies based on port exposure.

  • The admission controller might fail if there are connectivity issues between the Kubernetes or OpenShift Container Platform API server and Red Hat Advanced Cluster Security for Kubernetes Sensor. To resolve this issue, delete the ValidatingWebhookConfiguration object as described in the disabling admission controller enforcement section.

  • If you have deploy-time enforcement enabled for a policy and you enable the admission controller, Red Hat Advanced Cluster Security for Kubernetes attempts to block deployments that violate the policy. If a noncompliant deployment slips past the admission controller, for example, in case of a timeout, Red Hat Advanced Cluster Security for Kubernetes still applies other deploy-time enforcement mechanisms, such as scaling to zero replicas.

Enabling admission controller enforcement in Red Hat Advanced Cluster Security for Kubernetes version 3.0.55 and newer

You can enable admission controller enforcement from the Clusters view when you install Sensor or edit an existing cluster configuration.

Beginning from Red Hat Advanced Cluster Security for Kubernetes version 3.0.55.0, the admission controller webhook is deployed into the cluster by default.
Procedure

  1. On the RHACS portal, navigate to Platform Configuration → Clusters.

  2. Select an existing cluster from the list or select + New Cluster.

  3. In the cluster configuration panel, enter the details for your cluster.

  4. Red Hat recommends that you only turn on the Configure Admission Controller Webhook to listen on creates toggle if you are planning to use the admission controller to enforce on object create events.

  5. Red Hat recommends that you only turn on the Configure Admission Controller Webhook to listen on updates toggle if you are planning to use the admission controller to enforce on update events.

  6. Red Hat recommends that you only turn on the Enable Admission Controller Webhook to listen on exec and port-forward events toggle if you are planning to use the admission controller to enforce on pod execution and pod port forwards events.

  7. Configure the following options:

    • Enforce on Object Creates: This toggle controls the behavior of the admission control service. You must have the Configure Admission Controller Webhook to listen on creates toggle turned on for this to work.

    • Enforce on Object Updates: This toggle controls the behavior of the admission control service. You must have the Configure Admission Controller Webhook to listen on updates toggle turned on for this to work.

  8. Select Next.

  9. In the Download files section, select Download YAML Files and Keys.

    When enabling admission controller for an existing cluster, if you make any changes in the:

    • Static Configuration section, you must download the YAML files and redeploy the Sensor.

    • Dynamic Configuration section, you can skip downloading the files and deployment, as Red Hat Advanced Cluster Security for Kubernetes automatically syncs the Sensor and applies the changes.

  10. Select Finish.

Verification

  • After you provision a new cluster with the generated YAML, run the following command to verify if admission controller enforcement is configured correctly:

    $ oc get ValidatingWebhookConfiguration (1)
    1 If you use Kubernetes, enter kubectl instead of oc.
    Example output
    NAME       CREATED AT
stackrox   2019-09-24T06:07:34Z

Enabling admission controller enforcement in Red Hat Advanced Cluster Security for Kubernetes version 3.0.54 and older

You can enable admission controller enforcement from the Clusters view when you install Sensor or edit an existing cluster configuration.

Procedure

  1. On the RHACS portal, navigate to Platform Configuration → Clusters.

  2. Select an existing cluster from the list or select + New Cluster.

  3. In the cluster configuration panel, fill in the details for your cluster.

  4. Turn on the Create Admission Controller Webhook toggle in the Static Configuration section. This setting controls whether Kubernetes or OpenShift Container Platform is configured to contact Red Hat Advanced Cluster Security for Kubernetes with AdmissionReview requests.

  5. Red Hat recommends that you only turn on the Configure Admission Controller Webhook to listen on updates toggle if you are planning to use the admission controller to enforce on updates.

    The Configure Admission Controller Webhook to listen on updates option is only available in Red Hat Advanced Cluster Security for Kubernetes version 3.0.41 and newer.

    When you keep it turned off, Red Hat Advanced Cluster Security for Kubernetes creates the ValidatingWebhookConfiguration in a way that causes the Kubernetes or OpenShift Container Platform API server not to send object update events. Since the volume of object updates is usually higher than the object creates, leaving this turned off limits the load on the admission control service.

  6. Turn on the Enable Admission Controller toggle in the Dynamic Configuration section. This setting controls whether Red Hat Advanced Cluster Security for Kubernetes evaluates policies; if it is disabled, all AdmissionReview requests are automatically accepted.

  7. Configure the following options:

    • Enforce on Updates: This toggle controls the behavior of the admission control service. You must have the Configure Admission Controller Webhook to listen on updates toggle turned on for this to work.

      The Enforce on Updates option is only available in Red Hat Advanced Cluster Security for Kubernetes version 3.0.41 and newer.

    • Timeout: The maximum time in seconds, Red Hat Advanced Cluster Security for Kubernetes should wait while evaluating admission review requests. Use it to set request timeouts when you enable image scanning. If the image scan runs longer than the specified time, the Red Hat Advanced Cluster Security for Kubernetes accepts the request. Other enforcement options, such as scaling the deployment to zero replicas, are still applied later if the image violates applicable policies.

    • Contact Image Scanners: Turn on this toggle to enable image scanning. See information about image scanning before you enable this toggle.

    • Disable Use of Bypass Annotation: Turn on this toggle to disable bypassing the admission controller.

  8. Select Next.

  9. In the Download files section, select Download YAML Files and Keys.

    When enabling admission controller for an existing cluster, if you make any changes in the:

    • Static Configuration section, you must download the YAML files and redeploy the Sensor.

    • Dynamic Configuration section, you can skip downloading the files and deployment, because Red Hat Advanced Cluster Security for Kubernetes automatically syncs the Sensor and applies the changes.

  10. Select Finish.

Verification

  • After you provision a new cluster with the generated YAML, run the following command to verify if admission controller enforcement is set up:

    $ oc get ValidatingWebhookConfiguration (1)
    1 If you use Kubernetes, enter kubectl instead of oc.
    Example output
    NAME       CREATED AT
stackrox   2019-09-24T06:07:34Z

Bypassing admission controller enforcement

To bypass the admission controller, add the admission.stackrox.io/break-glass annotation to your configuration YAML. Bypassing the admission controller triggers a policy violation which includes deployment details. Red Hat recommends providing an issue-tracker link or some other reference as the value of this annotation so that others can understand why you bypassed the admission controller.

Disabling admission controller enforcement

Disabling admission controller enforcement in Red Hat Advanced Cluster Security for Kubernetes version 3.0.55 and newer

You can disable admission controller enforcement from the Clusters view on the RHACS portal.

Procedure

  1. On the RHACS portal, navigate to Platform Configuration → Clusters.

  2. Select an existing cluster from the list.

  3. Turn off the Enforce on Object Creates and Enforce on Object Updates toggles in the Dynamic Configuration section.

  4. Select Next.

  5. Select Finish.

Disabling admission controller enforcement in Red Hat Advanced Cluster Security for Kubernetes version 3.0.54 and older

You can disable admission controller enforcement from the Clusters view on the RHACS portal.

Procedure

  1. On the RHACS portal, navigate to Platform Configuration → Clusters.

  2. Select an existing cluster from the list.

  3. Turn off the Enforce on Object Creates and Enforce on Object Updates toggles in the Dynamic Configuration section.

  4. Select Next.

  5. Select Finish.

To disable the admission controller enforcement on pod executions and pod port forwards, you can either disable the system policies associated with the admission controller or you can disable the webhook.

Disabling associated policies

You can turn off the enforcement on relevant policies, which in turn instructs the admission controller to skip enforcements.

Procedure

  1. On the RHACS portal, navigate to Platform Configuration → System Policies.

  2. Disable enforcement on the default policies:

    • In the policies view, scroll down and select the power icon next to the Kubernetes Actions: Exec into Pod policy to disable that policy.

    • In the policies view, scroll down and select the power icon next to the Kubernetes Actions: Port Forward to Pod policy to disable that policy.

  3. Disable enforcement on any other custom policies that you have created by using criteria from the default Kubernetes Actions: Port Forward to Pod and Kubernetes Actions: Exec into Pod policies.

Disabling the webhook

You can disable admission controller enforcement from the Clusters view on the RHACS portal.

If you disable the admission controller by turning off the webhook, you must redeploy the Sensor bundle.
Procedure

  1. On the RHACS portal, navigate to Platform Configuration → Clusters.

  2. Select an existing cluster from the list.

  3. Turn off the Enable Admission Controller Webhook to listen on exec and port-forward events toggle in the Static Configuration section.

  4. Select Next to continue with Sensor setup.

  5. Click Download YAML File and Keys.

  6. From a system that has access to the monitored cluster, unzip and run the sensor script:

    $ unzip -d sensor sensor-<cluster_name>.zip
    $ ./sensor/sensor.sh

    If you get a warning that you do not have the required permissions to deploy the sensor, follow the on-screen instructions, or contact your cluster administrator for assistance.

    After the sensor is deployed, it contacts Central and provides cluster information.

  7. Return to the RHACS portal and check if the deployment is successful. If it is successful, a green checkmark appears under section #2. If you do not see a green checkmark, use the following command to check for problems:

    • On OpenShift Container Platform:

      $ oc get pod -n stackrox -w

    • On Kubernetes:

      $ kubectl get pod -n stackrox -w

  8. Select Finish.

When you disable the admission controller, Red Hat Advanced Cluster Security for Kubernetes does not delete the ValidatingWebhookConfiguration. However, instead of checking requests for violations, it accepts all AdmissionReview requests.

To remove the ValidatingWebhookConfiguration object, run the following command in the secured cluster:

  • On OpenShift Container Platform:

    $ oc delete ValidatingWebhookConfiguration/stackrox

  • On Kubernetes:

    $ kubectl delete ValidatingWebhookConfiguration/stackrox

ValidatingWebhookConfiguration YAML file changes

This section is only applicable if you are using Red Hat Advanced Cluster Security for Kubernetes version 3.0.41 or newer.

  • The webhook is not the part of Sensor anymore, and the reference service changed from Sensor to a dedicated admission control service. When you upgrade Red Hat Advanced Cluster Security for Kubernetes to version 3.0.41 or newer, the upgrade instructions handle this change.

  • With the new configuration, you can enforce security policies on object updates as well, in addition to enforcing on object creation.

  • Beginning from Red Hat Advanced Cluster Security for Kubernetes version 3.0.55, you can enforce security policies on pod execution and pod port forward events.

If Central or Sensor is unavailable

The admission controller requires an initial configuration from Sensor to work. Kubernetes or OpenShift Container Platform saves this configuration, and it remains accessible even if all admission control service replicas are rescheduled onto other nodes. If this initial configuration exists, the admission controller enforces all configured deploy-time policies.

If Sensor or Central becomes unavailable later:

  • you will not be able to run image scans, or query information about cached image scans. However, admission controller enforcement still functions based on the available information gathered before the timeout expires, even if the gathered information is incomplete.

  • you will not be able to disable the admission controller from the RHACS portal or modify enforcement for an existing policy as the changes will not get propagated to the admission control service.

If you need to disable admission control enforcement, you can delete the validating webhook configuration by running the following command:

  • On OpenShift Container Platform:

    $ oc delete ValidatingWebhookConfiguration/stackrox

  • On Kubernetes:

    $ kubectl delete ValidatingWebhookConfiguration/stackrox

Make the admission controller more reliable

Red Hat recommends that you schedule the admission control service on the control plane and not on worker nodes. The deployment YAML file includes a soft preference for running on the control plane, however it is not enforced.

By default, the admission control service runs 3 replicas. To increase reliability, you can increase the replicas by running the following command:

$ oc -n stackrox scale deploy/admission-control --replicas=<number_of_replicas> (1)
1 If you use Kubernetes, enter kubectl instead of oc.

Using with the roxctl CLI

You can use the following options when you generate a Sensor deployment YAML file:

  • --admission-controller-listen-on-updates: If you use this option, Red Hat Advanced Cluster Security for Kubernetes generates a Sensor bundle with a ValidatingWebhookConfiguration pre-configured to receive update events from the Kubernetes or OpenShift Container Platform API server.

  • --admission-controller-enforce-on-updates: If you use this option, Red Hat Advanced Cluster Security for Kubernetes configures Central such that the admission controller also enforces security policies object updates.

Both these options are optional, and are false by default.