Manual updates required by CVE-2020-8663

The fix for CVE-2020-8663: envoy: Resource exhaustion when accepting too many connections added a configurable limit on downstream connections. The configuration option for this limit must be configured to mitigate this vulnerability.

These manual steps are required to mitigate this CVE whether you are using the 1.1 version or the 1.0 version of Red Hat OpenShift Service Mesh.

This new configuration option is called overload.global_downstream_max_connections, and it is configurable as a proxy runtime setting. Perform the following steps to configure limits at the Ingress Gateway.

Procedure
  1. Create a file named bootstrap-override.json with the following text to force the proxy to override the bootstrap template and load runtime configuration from disk:

    {
      "runtime": {
        "symlink_root": "/var/lib/istio/envoy/runtime"
      }
    }
  2. Create a secret from the bootstrap-override.json file, replacing <SMCPnamespace> with the namespace where you created the service mesh control plane (SMCP):

    $ oc create secret generic -n <SMCPnamespace> gateway-bootstrap --from-file=bootstrap-override.json

  3. Update the SMCP configuration to activate the override.

    Updated SMCP configuration example #1
    apiVersion: maistra.io/v1
    kind: ServiceMeshControlPlane
    spec:
      istio:
        gateways:
          istio-ingressgateway:
            env:
              ISTIO_BOOTSTRAP_OVERRIDE: /var/lib/istio/envoy/custom-bootstrap/bootstrap-override.json
            secretVolumes:
            - mountPath: /var/lib/istio/envoy/custom-bootstrap
              name: custom-bootstrap
              secretName: gateway-bootstrap
  4. To set the new configuration option, create a secret that has the desired value for the overload.global_downstream_max_connections setting. The following example uses a value of 10000:

    $ oc create secret generic -n <SMCPnamespace> gateway-settings --from-literal=overload.global_downstream_max_connections=10000

  5. Update the SMCP again to mount the secret in the location where Envoy is looking for runtime configuration:

Updated SMCP configuration example #2
apiVersion: maistra.io/v1
kind: ServiceMeshControlPlane
spec:
  template: default
#Change the version to "v1.0" if you are on the 1.0 stream.
  version: v1.1
  istio:
    gateways:
      istio-ingressgateway:
        env:
          ISTIO_BOOTSTRAP_OVERRIDE: /var/lib/istio/envoy/custom-bootstrap/bootstrap-override.json
        secretVolumes:
        - mountPath: /var/lib/istio/envoy/custom-bootstrap
          name: custom-bootstrap
          secretName: gateway-bootstrap
        # below is the new secret mount
        - mountPath: /var/lib/istio/envoy/runtime
          name: gateway-settings
          secretName: gateway-settings

Upgrading from Elasticsearch 5 to Elasticsearch 6

When updating from Elasticsearch 5 to Elasticsearch 6, you must delete your Jaeger instance, then recreate the Jaeger instance because of an issue with certificates. Re-creating the Jaeger instance triggers creating a new set of certificates. If you are using persistent storage the same volumes can be mounted for the new Jaeger instance as long as the Jaeger name and namespace for the new Jaeger instance are the same as the deleted Jaeger instance.

Procedure if Jaeger is installed as part of Red Hat Service Mesh
  1. Determine the name of your Jaeger custom resource file:

    $ oc get jaeger -n istio-system

    You should see something like the following:

    NAME     AGE
    jaeger   3d21h
  2. Copy the generated custom resource file into a temporary directory:

    $ oc get jaeger jaeger -oyaml -n istio-system > /tmp/jaeger-cr.yaml
  3. Delete the Jaeger instance:

    $ oc delete jaeger jaeger -n istio-system
  4. Recreate the Jaeger instance from your copy of the custom resource file:

    $ oc create -f /tmp/jaeger-cr.yaml -n istio-system
  5. Delete the copy of the generated custom resource file.

    $ rm /tmp/jaeger-cr.yaml
    Procedure if Jaeger not installed as part of Red Hat Service Mesh

    Before you begin, create a copy of your Jaeger custom resource file.

  6. Delete the Jaeger instance by deleting the custom resource file.

    $ oc delete -f <jaeger-cr-file>

    For example:

    $ oc delete -f jaeger-prod-elasticsearch.yaml
  7. Recreate your Jaeger instance from the backup copy of your custom resource file.

    $ oc create -f <jaeger-cr-file>
  8. Validate that your Pods have restarted:

    $ oc get pods -n jaeger-system -w

Manual updates from 1.0 to 1.1

If you are updating from Red Hat OpenShift Service Mesh 1.0 to 1.1, you must update the ServiceMeshControlPlane resource to update the control plane components to the new version.

  1. In the web console, click the Red Hat OpenShift Service Mesh Operator.

  2. Click the Project menu and choose the project where your ServiceMeshControlPlane is deployed from the list, for example istio-system.

  3. Click the name of your control plane, for example basic-install.

  4. Click YAML and add a version field to the spec: of your ServiceMeshControlPlane resource. For example, to update to Red Hat OpenShift Service Mesh 1.1.0, add version: v1.1.

spec:
  version: v1.1
  ...

The version field specifies the version of ServiceMesh to install and defaults to the latest available version.

Manual updates

If you choose to update manually, the Operator Lifecycle Manager (OLM) controls the installation, upgrade, and role-based access control (RBAC) of Operators in a cluster. OLM runs by default in OpenShift Container Platform. OLM uses CatalogSources, which use the Operator Registry API, to query for available Operators as well as upgrades for installed Operators.

  • For more information about how OpenShift Container Platform handled upgrades, refer to the Operator Lifecycle Manager documentation.