OpenShift docs are moving and will soon only be available at docs.redhat.com, the home of all Red Hat product documentation. Explore the new docs experience today.
  1. Documentation
    2. history bug_report picture_as_pdf
×

Using admission controller enforcement

Red Hat Advanced Cluster Security for Kubernetes works with Kubernetes admission controllers and OpenShift Container Platform admission plugins to allow you to enforce security policies before Kubernetes or OpenShift Container Platform creates workloads, for example, deployments, daemon sets or jobs.

The RHACS admission controller prevents users from creating workloads that violate policies you configure in RHACS. Beginning from the RHACS version 3.0.41, you can also configure the admission controller to prevent updates to workloads that violate policies.

RHACS uses the ValidatingAdmissionWebhook controller to verify that the resource being provisioned complies with the specified security policies. To handle this, RHACS creates a ValidatingWebhookConfiguration which contains multiple webhook rules.

When the Kubernetes or OpenShift Container Platform API server receives a request that matches one of the webhook rules, the API server sends an AdmissionReview request to RHACS. RHACS then accepts or rejects the request based on the configured security policies.

To use admission controller enforcement on OpenShift Container Platform, you need the Red Hat Advanced Cluster Security for Kubernetes version 3.0.49 or newer.

Understanding admission controller enforcement

If you intend to use admission controller enforcement, consider the following:

  • API latency: Using admission controller enforcement increases Kubernetes or OpenShift Container Platform API latency because it involves additional API validation requests. Many standard Kubernetes libraries, such as fabric8, have short Kubernetes or OpenShift Container Platform API timeouts by default. Also, consider API timeouts in any custom automation you might be using.

  • Image scanning: You can choose whether the admission controller scans images while reviewing requests by setting the Contact Image Scanners option in the cluster configuration panel.

    • If you enable this setting, Red Hat Advanced Cluster Security for Kubernetes contacts the image scanners if the scan or image signature verification results are not already available, which adds considerable latency.

    • If you disable this setting, the enforcement decision only considers image scan criteria if cached scan and signature verification results are available.

  • You can use admission controller enforcement for:

    • Options in the pod securityContext.

    • Deployment configurations.

    • Image components and vulnerabilities.

  • You cannot use admission controller enforcement for:

    • Any runtime behavior, such as processes.

    • Any policies based on port exposure.

  • The admission controller might fail if there are connectivity issues between the Kubernetes or OpenShift Container Platform API server and RHACS Sensor. To resolve this issue, delete the ValidatingWebhookConfiguration object as described in the disabling admission controller enforcement section.

  • If you have deploy-time enforcement enabled for a policy and you enable the admission controller, RHACS attempts to block deployments that violate the policy. If a noncompliant deployment is not rejected by the admission controller, for example, in case of a timeout, RHACS still applies other deploy-time enforcement mechanisms, such as scaling to zero replicas.

Enabling admission controller enforcement

You can enable admission controller enforcement from the Clusters view when you install Sensor or edit an existing cluster configuration.

Procedure

  1. In the RHACS portal, go to Platform Configuration → Clusters.

  2. Select an existing cluster from the list or secure a new cluster by selecting Secure a cluster → Legacy installation method.

  3. If you are securing a new cluster, in the Static Configuration section of the cluster configuration panel, enter the details for your cluster.

  4. Red Hat recommends that you only turn on the Configure Admission Controller Webhook to listen on Object Creates toggle if you are planning to use the admission controller to enforce on object create events.

  5. Red Hat recommends that you only turn on the Configure Admission Controller Webhook to listen on Object Updates toggle if you are planning to use the admission controller to enforce on update events.

  6. Red Hat recommends that you only turn on the Enable Admission Controller Webhook to listen on exec and port-forward events toggle if you are planning to use the admission controller to enforce on pod execution and pod port forwards events.

  7. Configure the following options in the Dynamic Configuration section:

    • Enforce on Object Creates: This toggle controls the behavior of the admission control service. You must have the Configure Admission Controller Webhook to listen on Object Creates toggle turned on for this to work.

    • Enforce on Object Updates: This toggle controls the behavior of the admission control service. You must have the Configure Admission Controller Webhook to listen on Object Updates toggle turned on for this to work.

  8. Select Next.

  9. In the Download files section, select Download YAML files and keys.

    When enabling admission controller for an existing cluster, follow this guidance:

    • If you make any changes in the Static Configuration section, you must download the YAML files and redeploy the Sensor.

    • If you make any changes in the Dynamic Configuration section, you can skip downloading the files and deployment, as RHACS automatically synchronizes the Sensor and applies the changes.

  10. Select Finish.

Verification

  • After you provision a new cluster with the generated YAML, run the following command to verify if admission controller enforcement is configured correctly:

    $ oc get ValidatingWebhookConfiguration (1)
    1 If you use Kubernetes, enter kubectl instead of oc.
    Example output
    NAME       CREATED AT
stackrox   2019-09-24T06:07:34Z

Bypassing admission controller enforcement

To bypass the admission controller, add the admission.stackrox.io/break-glass annotation to your configuration YAML. Bypassing the admission controller triggers a policy violation which includes deployment details. Red Hat recommends providing an issue-tracker link or some other reference as the value of this annotation so that others can understand why you bypassed the admission controller.

Disabling admission controller enforcement

You can disable admission controller enforcement from the Clusters view on the Red Hat Advanced Cluster Security for Kubernetes (RHACS) portal.

Procedure

  1. In the RHACS portal, select Platform Configuration → Clusters.

  2. Select an existing cluster from the list.

  3. Turn off the Enforce on Object Creates and Enforce on Object Updates toggles in the Dynamic Configuration section.

  4. Select Next.

  5. Select Finish.

Disabling associated policies

You can turn off the enforcement on relevant policies, which in turn instructs the admission controller to skip enforcements.

Procedure

  1. In the RHACS portal, go to Platform Configuration → Policy Management.

  2. Disable enforcement on the default policies:

    • In the policies view, locate the Kubernetes Actions: Exec into Pod policy. Click the overflow menu, kebab, and then select Disable policy.

    • In the policies view, locate the Kubernetes Actions: Port Forward to Pod policy. Click the overflow menu, kebab, and then select Disable policy.

  3. Disable enforcement on any other custom policies that you have created by using criteria from the default Kubernetes Actions: Port Forward to Pod and Kubernetes Actions: Exec into Pod policies.

Disabling the webhook

You can disable admission controller enforcement from the Clusters view in the RHACS portal.

If you disable the admission controller by turning off the webhook, you must redeploy the Sensor bundle.
Procedure

  1. In the RHACS portal, go to Platform Configuration → Clusters.

  2. Select an existing cluster from the list.

  3. Turn off the Enable Admission Controller Webhook to listen on exec and port-forward events toggle in the Static Configuration section.

  4. Select Next to continue with Sensor setup.

  5. Click Download YAML file and keys.

  6. From a system that has access to the monitored cluster, extract and run the sensor script:

    $ unzip -d sensor sensor-<cluster_name>.zip
    $ ./sensor/sensor.sh

    If you get a warning that you do not have the required permissions to deploy the sensor, follow the on-screen instructions, or contact your cluster administrator for help.

    After the sensor is deployed, it contacts Central and provides cluster information.

  7. Return to the RHACS portal and check if the deployment is successful. If it is successful, a green checkmark appears under section #2. If you do not see a green checkmark, use the following command to check for problems:

    • On OpenShift Container Platform:

      $ oc get pod -n stackrox -w

    • On Kubernetes:

      $ kubectl get pod -n stackrox -w

  8. Select Finish.

When you disable the admission controller, RHACS does not delete the ValidatingWebhookConfiguration parameter. However, instead of checking requests for violations, it accepts all AdmissionReview requests.

To remove the ValidatingWebhookConfiguration object, run the following command in the secured cluster:

  • On OpenShift Container Platform:

    $ oc delete ValidatingWebhookConfiguration/stackrox

  • On Kubernetes:

    $ kubectl delete ValidatingWebhookConfiguration/stackrox

ValidatingWebhookConfiguration YAML file changes

With Red Hat Advanced Cluster Security for Kubernetes you can enforce security policies on:

  • Object creation

  • Object update

  • Pod execution

  • Pod port forward

If Central or Sensor is unavailable

The admission controller requires an initial configuration from Sensor to work. Kubernetes or OpenShift Container Platform saves this configuration, and it remains accessible even if all admission control service replicas are rescheduled onto other nodes. If this initial configuration exists, the admission controller enforces all configured deploy-time policies.

If Sensor or Central becomes unavailable later:

  • you will not be able to run image scans, or query information about cached image scans. However, admission controller enforcement still functions based on the available information gathered before the timeout expires, even if the gathered information is incomplete.

  • you will not be able to disable the admission controller from the RHACS portal or modify enforcement for an existing policy as the changes will not get propagated to the admission control service.

If you need to disable admission control enforcement, you can delete the validating webhook configuration by running the following command:

  • On OpenShift Container Platform:

    $ oc delete ValidatingWebhookConfiguration/stackrox

  • On Kubernetes:

    $ kubectl delete ValidatingWebhookConfiguration/stackrox

Make the admission controller more reliable

Red Hat recommends that you schedule the admission control service on the control plane and not on worker nodes. The deployment YAML file includes a soft preference for running on the control plane, however it is not enforced.

By default, the admission control service runs 3 replicas. To increase reliability, you can increase the replicas by running the following command:

$ oc -n stackrox scale deploy/admission-control --replicas=<number_of_replicas> (1)
1 If you use Kubernetes, enter kubectl instead of oc.

Using with the roxctl CLI

You can use the following options when you generate a Sensor deployment YAML file:

  • --admission-controller-listen-on-updates: If you use this option, Red Hat Advanced Cluster Security for Kubernetes generates a Sensor bundle with a ValidatingWebhookConfiguration pre-configured to receive update events from the Kubernetes or OpenShift Container Platform API server.

  • --admission-controller-enforce-on-updates: If you use this option, Red Hat Advanced Cluster Security for Kubernetes configures Central such that the admission controller also enforces security policies object updates.

Both these options are optional, and are false by default.