1. Documentation
    2. history bug_report picture_as_pdf
×

Upgrading Service Mesh

Understanding versioning

Red Hat uses semantic versioning for product releases. Semantic Versioning is a 3-component number in the format of X.Y.Z, where:

  • X stands for a Major version. Major releases usually denote some sort of breaking change: architectural changes, API changes, schema changes, and similar major updates.

  • Y stands for a Minor version. Minor releases contain new features and functionality while maintaining backwards compatibility.

  • Z stands for a Patch version (also known as a z-stream release). Patch releases are used to addresses Common Vulnerabilities and Exposures (CVEs) and release bug fixes. New features and functionality are generally not released as part of a Patch release.

How versioning affects Service Mesh upgrades

Depending on the version of the update you are making, the upgrade process is different.

  • Patch updates - Patch upgrades are managed by the Operator Lifecycle Manager (OLM); they happen automatically when you update your Operators.

  • Minor upgrades - Minor upgrades require both updating to the most recent Red Hat OpenShift Service Mesh Operator version and manually modifying the spec.version value in your ServiceMeshControlPlane resources.

  • Major upgrades - Major upgrades require both updating to the most recent Red Hat OpenShift Service Mesh Operator version and manually modifying the spec.version value in your ServiceMeshControlPlane resources. Because major upgrades can contain changes that are not backwards compatible, additional manual changes might be required.

Understanding Service Mesh versions

In order to understand what version of Red Hat OpenShift Service Mesh you have deployed on your system, you need to understand how each of the component versions is managed.

  • Operator version - The most current Operator version is 2.3. The Operator version number only indicates the version of the currently installed Operator. Because the Red Hat OpenShift Service Mesh Operator supports multiple versions of the Service Mesh control plane, the version of the Operator does not determine the version of your deployed ServiceMeshControlPlane resources.

    Upgrading to the latest Operator version automatically applies patch updates, but does not automatically upgrade your Service Mesh control plane to the latest minor version.

  • ServiceMeshControlPlane version - The ServiceMeshControlPlane version determines what version of Red Hat OpenShift Service Mesh you are using. The value of the spec.version field in the ServiceMeshControlPlane resource controls the architecture and configuration settings that are used to install and deploy Red Hat OpenShift Service Mesh. When you create the Service Mesh control plane you can set the version in one of two ways:

    • To configure in the Form View, select the version from the Control Plane Version menu.

    • To configure in the YAML View, set the value for spec.version in the YAML file.

Operator Lifecycle Manager (OLM) does not manage Service Mesh control plane upgrades, so the version number for your Operator and ServiceMeshControlPlane (SMCP) may not match, unless you have manually upgraded your SMCP.

Upgrade considerations

The maistra.io/ label or annotation should not be used on a user-created custom resource, because it indicates that the resource was generated by and should be managed by the Red Hat OpenShift Service Mesh Operator.

During the upgrade, the Operator makes changes, including deleting or replacing files, to resources that include the following labels or annotations that indicate that the resource is managed by the Operator.

Before upgrading check for user-created custom resources that include the following labels or annotations:

  • maistra.io/ AND the app.kubernetes.io/managed-by label set to maistra-istio-operator (Red Hat OpenShift Service Mesh)

  • kiali.io/ (Kiali)

  • jaegertracing.io/ (Red Hat OpenShift distributed tracing platform)

  • logging.openshift.io/ (Red Hat Elasticsearch)

Before upgrading, check your user-created custom resources for labels or annotations that indicate they are Operator managed. Remove the label or annotation from custom resources that you do not want to be managed by the Operator.

When upgrading to version 2.0, the Operator only deletes resources with these labels in the same namespace as the SMCP.

When upgrading to version 2.1, the Operator deletes resources with these labels in all namespaces.

Known issues that may affect upgrade

Known issues that may affect your upgrade include:

  • Red Hat OpenShift Service Mesh does not support the use of EnvoyFilter configuration except where explicitly documented. This is due to tight coupling with the underlying Envoy APIs, meaning that backward compatibility cannot be maintained. If you are using Envoy Filters, and the configuration generated by Istio has changed due to the lastest version of Envoy introduced by upgrading your ServiceMeshControlPlane, that has the potential to break any EnvoyFilter you may have implemented.

  • OSSM-1505 ServiceMeshExtension does not work with OpenShift Container Platform version 4.11. Because ServiceMeshExtension has been deprecated in Red Hat OpenShift Service Mesh 2.2, this known issue will not be fixed and you must migrate your extensions to WasmPluging

  • OSSM-1396 If a gateway resource contains the spec.externalIPs setting, rather than being recreated when the ServiceMeshControlPlane is updated, the gateway is removed and never recreated.

  • OSSM-1052 When configuring a Service ExternalIP for the ingressgateway in the Service Mesh control plane, the service is not created. The schema for the SMCP is missing the parameter for the service.

    Workaround: Disable the gateway creation in the SMCP spec and manage the gateway deployment entirely manually (including Service, Role and RoleBinding).

Upgrading the Operators

In order to keep your Service Mesh patched with the latest security fixes, bug fixes, and software updates, you must keep your Operators updated. You initiate patch updates by upgrading your Operators.

The version of the Operator does not determine the version of your service mesh. The version of your deployed Service Mesh control plane determines your version of Service Mesh.

Because the Red Hat OpenShift Service Mesh Operator supports multiple versions of the Service Mesh control plane, updating the Red Hat OpenShift Service Mesh Operator does not update the spec.version value of your deployed ServiceMeshControlPlane. Also note that the spec.version value is a two digit number, for example 2.2, and that patch updates, for example 2.2.1, are not reflected in the SMCP version value.

Operator Lifecycle Manager (OLM) controls the installation, upgrade, and role-based access control (RBAC) of Operators in a cluster. The OLM runs by default in OpenShift Container Platform. OLM queries for available Operators as well as upgrades for installed Operators.

Whether or not you have to take action to upgrade your Operators depends on the settings you selected when installing them. When you installed each of your Operators, you selected an Update Channel and an Approval Strategy. The combination of these two settings determine when and how your Operators are updated.

Table 1. Interaction of Update Channel and Approval Strategy
Versioned channel "Stable" or "Preview" Channel

Automatic

Automatically updates the Operator for minor and patch releases for that version only. Will not automatically update to the next major version (that is, from version 2.0 to 3.0). Manual change to Operator subscription required to update to the next major version.

Automatically updates Operator for all major, minor, and patch releases.

Manual

Manual updates required for minor and patch releases for the specified version. Manual change to Operator subscription required to update to the next major version.

Manual updates required for all major, minor, and patch releases.

When you update your Red Hat OpenShift Service Mesh Operator the Operator Lifecycle Manager (OLM) removes the old Operator pod and starts a new pod. Once the new Operator pod starts, the reconciliation process checks the ServiceMeshControlPlane (SMCP), and if there are updated container images available for any of the Service Mesh control plane components, it replaces those Service Mesh control plane pods with ones that use the new container images.

When you upgrade the Kiali and Red Hat OpenShift distributed tracing platform Operators, the OLM reconciliation process scans the cluster and upgrades the managed instances to the version of the new Operator. For example, if you update the Red Hat OpenShift distributed tracing platform Operator from version 1.30.2 to version 1.34.1, the Operator scans for running instances of distributed tracing platform and upgrades them to 1.34.1 as well.

To stay on a particular patch version of Red Hat OpenShift Service Mesh, you would need to disable automatic updates and remain on that specific version of the Operator.

For more information about upgrading Operators, refer to the Operator Lifecycle Manager documentation.

Upgrading the control plane

You must manually update the control plane for minor and major releases. The community Istio project recommends canary upgrades, Red Hat OpenShift Service Mesh only supports in-place upgrades. Red Hat OpenShift Service Mesh requires that you upgrade from each minor release to the next minor release in sequence. For example, you must upgrade from version 2.0 to version 2.1, and then upgrade to version 2.2. You cannot update from Red Hat OpenShift Service Mesh 2.0 to 2.2 directly.

When you upgrade the service mesh control plane, all Operator managed resources, for example gateways, are also upgraded.

Although you can deploy multiple versions of the control plane in the same cluster, Red Hat OpenShift Service Mesh does not support canary upgrades of the service mesh. That is, you can have different SCMP resources with different values for spec.version, but they cannot be managing the same mesh.

For more information about migrating your extensions, refer to Migrating from ServiceMeshExtension to WasmPlugin resources.

Upgrade changes from version 2.2 to version 2.3

Upgrading the Service Mesh control plane from version 2.2 to 2.3 introduces the following behavioral changes:

  • This release requires use of the WasmPlugin API. Support for the ServiceMeshExtension API, which was deprecated in 2.2, has now been removed. If you attempt to upgrade while using the ServiceMeshExtension API, then the upgrade fails.

Upgrade changes from version 2.1 to version 2.2

Upgrading the Service Mesh control plane from version 2.1 to 2.2 introduces the following behavioral changes:

  • The istio-node DaemonSet is renamed to istio-cni-node to match the name in upstream Istio.

  • Istio 1.10 updated Envoy to send traffic to the application container using eth0 rather than lo by default.

  • This release adds support for the WasmPlugin API and deprecates the ServiceMeshExtension API.

Upgrade changes from version 2.0 to version 2.1

Upgrading the Service Mesh control plane from version 2.0 to 2.1 introduces the following architectural and behavioral changes.

Architecture changes

Mixer has been completely removed in Red Hat OpenShift Service Mesh 2.1. Upgrading from a Red Hat OpenShift Service Mesh 2.0.x release to 2.1 will be blocked if Mixer is enabled.

If you see the following message when upgrading from v2.0 to v2.1, update the existing Mixer type to Istiod type in the existing Control Plane spec before you update the .spec.version field:

An error occurred
admission webhook smcp.validation.maistra.io denied the request: [support for policy.type "Mixer" and policy.Mixer options have been removed in v2.1, please use another alternative, support for telemetry.type "Mixer" and telemetry.Mixer options have been removed in v2.1, please use another alternative]”

For example:

apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
spec:
  policy:
    type: Istiod
  telemetry:
    type: Istiod
  version: v2.3
Behavioral changes

  • AuthorizationPolicy updates:

    • With the PROXY protocol, if you’re using ipBlocks and notIpBlocks to specify remote IP addresses, update the configuration to use remoteIpBlocks and notRemoteIpBlocks instead.

    • Added support for nested JSON Web Token (JWT) claims.

  • EnvoyFilter breaking changes>

    • Must use typed_config

    • xDS v2 is no longer supported

    • Deprecated filter names

  • Older versions of proxies may report 503 status codes when receiving 1xx or 204 status codes from newer proxies.

Upgrading the Service Mesh control plane

To upgrade Red Hat OpenShift Service Mesh, you must update the version field of the Red Hat OpenShift Service Mesh ServiceMeshControlPlane v2 resource. Then, once it is configured and applied, restart the application pods to update each sidecar proxy and its configuration.

Prerequisites

  • You are running OpenShift Container Platform 4.9 or later.

  • You have the latest Red Hat OpenShift Service Mesh Operator.

Procedure

  1. Switch to the project that contains your ServiceMeshControlPlane resource. In this example, istio-system is the name of the Service Mesh control plane project.

    $ oc project istio-system

  2. Check your v2 ServiceMeshControlPlane resource configuration to verify it is valid.

    1. Run the following command to view your ServiceMeshControlPlane resource as a v2 resource.

      $ oc get smcp -o yaml

      Back up your Service Mesh control plane configuration.

  3. Update the .spec.version field and apply the configuration.

    For example:

    apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
metadata:
  name: basic
spec:
  version: v2.3

    Alternatively, instead of using the command line, you can use the web console to edit the Service Mesh control plane. In the OpenShift Container Platform web console, click Project and select the project name you just entered.

    1. Click Operators → Installed Operators.

    2. Find your ServiceMeshControlPlane instance.

    3. Select YAML view and update text of the YAML file, as shown in the previous example.

    4. Click Save.

Migrating Red Hat OpenShift Service Mesh from version 1.1 to version 2.0

Upgrading from version 1.1 to 2.0 requires manual steps that migrate your workloads and application to a new instance of Red Hat OpenShift Service Mesh running the new version.

Prerequisites

  • You must upgrade to OpenShift Container Platform 4.7. before you upgrade to Red Hat OpenShift Service Mesh 2.0.

  • You must have Red Hat OpenShift Service Mesh version 2.0 operator. If you selected the automatic upgrade