Installing the Service Mesh involves installing the Operator, and then creating and managing a custom resource definition file to deploy the control plane.

Starting with Red Hat OpenShift Service Mesh 0.9.TechPreview, Mixer’s policy enforcement is disabled by default. You must enable it to run policy tasks. See Update Mixer policy enforcement for instructions on enabling Mixer policy enforcement.

Single-tenant control plane installations are known to cause issues with OpenShift Container Platform restarts and upgrades. Multi-tenant control plane installations are the default configuration starting with Red Hat OpenShift Service Mesh 0.12.TechPreview.

Prerequisites

Installing the Operators

The Service Mesh installation process introduces an Operator to manage the installation of the control plane within the istio-operator namespace. This Operator defines and monitors a custom resource related to the deployment, update, and deletion of the control plane.

Starting with Red Hat OpenShift Service Mesh 0.12.TechPreview, you must install the Jaeger Operator and the Kiali Operator before the Red Hat OpenShift Service Mesh Operator can install the control plane.

Installing the Jaeger Operator

You must install the Jaeger Operator for the Red Hat OpenShift Service Mesh Operator to install the control plane.

Prerequisites
  • An account with cluster administrator access.

Procedure
  1. Log into your OpenShift Container Platform installation as a cluster administrator.

  2. Run the following commands to install the Jaeger Operator:

    $ oc new-project observability # create the project for the jaeger operator
    $ oc create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/v1.13.1/deploy/crds/jaegertracing_v1_jaeger_crd.yaml
    $ oc create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/v1.13.1/deploy/service_account.yaml
    $ oc create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/v1.13.1/deploy/role.yaml
    $ oc create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/v1.13.1/deploy/role_binding.yaml
    $ oc create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/v1.13.1/deploy/operator.yaml

Installing the Kiali Operator

You must install the Kiali Operator for the Red Hat OpenShift Service Mesh Operator to install the control plane.

Prerequisites
  • Review the Kiali documentation.

  • An account with cluster administrator access.

Procedure
  1. Log into your OpenShift Container Platform installation as a cluster administrator.

  2. Run the following command to install the Kiali Operator:

    $ bash <(curl -L https://git.io/getLatestKialiOperator) --operator-image-version v1.0.0 --operator-watch-namespace '**' --accessible-namespaces '**' --operator-install-kiali false

Installing the Red Hat OpenShift Service Mesh Operator

Prerequisites
  • Review the Operator templates on GitHub.

  • An account with cluster administrator access.

  • The Jaeger Operator must be installed.

  • The Kiali Operator must be installed.

Procedure
  1. Log into your OpenShift Container Platform installation as a cluster administrator.

  2. If the istio-operator and istio-system namespaces do not exist, run these commands to create the namespaces:

    $ oc new-project istio-operator
    $ oc new-project istio-system
  3. Run this command to install the Service Mesh Operator. You can run them from any host with access to the cluster.

    $ oc apply -n istio-operator -f https://raw.githubusercontent.com/Maistra/istio-operator/maistra-0.12/deploy/servicemesh-operator.yaml

Verifying the Operator installation

Before proceeding with the install process, verify that the Operator is installed correctly.

Prerequisites
  • An account with cluster administrator access.

Procedure
  1. Log into your OpenShift Container Platform installation as a cluster administrator.

  2. Run this command to verify that the Operator is installed correctly.

    $ oc get pods -n istio-operator
  3. When the Operator reaches a running state, it is installed correctly.

    NAME                              READY     STATUS    RESTARTS   AGE
    istio-operator-5cd6bcf645-fvb57   1/1       Running   0          1h

Red Hat OpenShift Service Mesh custom resource

The istio-system namespace is used an example throughout the Service Mesh documentation, but you can use other namespaces as necessary.

To deploy the Service Mesh control plane, you must create a custom resource. A custom resource allows you to introduce your own API into a Kubernetes project or cluster. You create a custom resource yaml file that defines the project parameters and creates the object. This example custom resource yaml file contains all of the supported parameters and deploys Red Hat OpenShift Service Mesh 0.12.TechPreview images based on Red Hat Enterprise Linux (RHEL).

The 3scale Istio Adapter is deployed and configured in the custom resource file. It also requires a working 3scale account (SaaS or On-Premises).

Full example istio-installation.yaml
  apiVersion: maistra.io/v1
  kind: ServiceMeshControlPlane
  metadata:
    name: basic-install

    threeScale:
      enabled: false

    istio:
      global:
        proxy:
          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              cpu: 500m
              memory: 128Mi
        multitenant: true

      gateways:
        istio-egressgateway:
          autoscaleEnabled: false
        istio-ingressgateway:
          autoscaleEnabled: false
          ior_enabled: false

      mixer:
        policy:
          autoscaleEnabled: false

        telemetry:
          autoscaleEnabled: false
          resources:
            requests:
              cpu: 100m
              memory: 1G
            limits:
              cpu: 500m
              memory: 4G

      pilot:
        autoscaleEnabled: false
        traceSampling: 100.0

      kiali:
       dashboard:
          user: admin
          passphrase: admin
      tracing:
        enabled: true

Custom resource parameters

The following examples illustrate use of the supported custom resource parameters for Red Hat OpenShift Service Mesh and the tables provide additional information about supported parameters.

The resources you configure for Red Hat OpenShift Service Mesh with these custom resource parameters, including CPUs, memory, and the number of pods, are based on the configuration of your OpenShift cluster. Configure these parameters based on the available resources in your current cluster configuration.

Istio global example

Here is an example that illustrates the Istio global parameters for the Red Hat OpenShift Service Mesh custom resource and a description of the available parameters with appropriate values.

In order for the 3scale Istio Adapter to work, disablePolicyChecks must be false.

  istio:
    global:
      hub: `maistra/` or `registry.redhat.io/openshift-istio-tech-preview/`
      tag: 0.12.0
      proxy:
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 500m
            memory: 128Mi
      mtls:
        enabled: false
      disablePolicyChecks: true
      policyCheckFailOpen: false
      imagePullSecrets:
        - MyPullSecret

See the OpenShift documentation on Scalability and performance for additional details on CPU and memory resources for the containers in your pod.

Table 1. Global parameters
Parameter Description Values Default value

disablePolicyChecks

This boolean indicates whether to enable policy checks

true/false

true

policyCheckFailOpen

This boolean indicates whether traffic is allowed to pass through to the Envoy sidecar when the Mixer policy service cannot be reached

true/false

false

tag

The tag that the Operator uses to pull the Istio images

A valid container image tag

0.12.0

hub

The hub that the Operator uses to pull Istio images

A valid image repo

maistra/ or registry.redhat.io/openshift-istio-tech-preview/

mtls

This controls whether to enable Mutual Transport Layer Security (mTLS) between services by default

true/false

false

imagePullSecret

If access to the registry providing the Istio images is secure, list an imagePullSecret here

redhat-registry-pullsecret OR quay-pullsecret

None

Table 2. Proxy parameters
Type Parameter Description Values Default value

Resources

cpu

The percentage of CPU resources requested for Envoy proxy

CPU resources in millicores based on your environment’s configuration

100m

memory

The amount of memory requested for Envoy proxy

Available memory in bytes based on your environment’s configuration

128Mi

Limits

cpu

The maximum percentage of CPU resources requested for Envoy proxy

CPU resources in millicores based on your environment’s configuration

2000m

memory

The maximum amount of memory Envoy proxy is permitted to use

Available memory in bytes based on your environment’s configuration

128Mi

Container Network Interface (CNI) example

This example illustrates the CNI parameter and its values for Red Hat OpenShift Service Mesh.

If Container Network Interface (CNI) is enabled, manual sidecar injection will work, but pods will not be able to communicate with the control plane unless they are a part of the ServiceMeshMemberRoll resource.

CNI example
  apiVersion: maistra.io/v1
  kind: ServiceMeshControlPlane
  metadata:
   name: basic-install
  spec:

    istio:
      istio_cni:
        enabled: true
Table 3. CNI parameter
Type Parameter Description Values Default value

istio_cni

enabled

This parameter enables the Container Network Interface (CNI).

true/false

false

Istio gateway example

Here is an example that illustrates the Istio gateway parameters for the Red Hat OpenShift Service Mesh custom resource and a description of the available parameters with appropriate values.

Automatic route creation does not currently work with multi-tenancy. Set ior_enabled to false for multi-tenant installations.

  gateways:
       istio-egressgateway:
         autoscaleEnabled: false
         autoscaleMin: 1
         autoscaleMax: 5
       istio-ingressgateway:
         autoscaleEnabled: false
         autoscaleMin: 1
         autoscaleMax: 5
         ior_enabled: false
Table 4. Istio Gateway parameters
Type Parameter Description Values Default value

istio-egressgateway

autoscaleEnabled

This parameter enables autoscaling.

true/false

true

autoscaleMin

The minimum number of pods to deploy for the egress gateway based on the autoscaleEnabled setting

A valid number of allocatable pods based on your environment’s configuration

1

autoscaleMax

The maximum number of pods to deploy for the egress gateway based on the autoscaleEnabled setting

A valid number of allocatable pods based on your environment’s configuration

5

istio-ingressgateway

autoscaleEnabled

This parameter enables autoscaling.

true/false

true

autoscaleMin

The minimum number of pods to deploy for the ingress gateway based on the autoscaleEnabled setting

A valid number of allocatable pods based on your environment’s configuration

1

autoscaleMax

The maximum number of pods to deploy for the ingress gateway based on the autoscaleEnabled setting

A valid number of allocatable pods based on your environment’s configuration

5

ior_enabled

This parameter controls whether Istio routes are automatically configured in OpenShift

true/false

true

Istio Mixer example

Here is an example that illustrates the Mixer parameters for the Red Hat OpenShift Service Mesh custom resource and a description of the available parameters with appropriate values.

  mixer:
    enabled: true
       policy:
         autoscaleEnabled: false

       telemetry:
         autoscaleEnabled: false
         resources:
           requests:
             cpu: 100m
             memory: 1G
           limits:
             cpu: 500m
             memory: 4G
Table 5. Istio Mixer policy parameters
Parameter Description Values Default value

enabled

This enables Mixer

true/false

true

autoscaleEnabled

This controls whether to enable autoscaling. Disable this for small environments.

true/false

true

autoscaleMin

The minimum number of pods to deploy based on the autoscaleEnabled setting

A valid number of allocatable pods based on your environment’s configuration

1

autoscaleMax

The maximum number of pods to deploy based on the autoscaleEnabled setting

A valid number of allocatable pods based on your environment’s configuration

5

Table 6. Istio Mixer telemetry parameters
Type Parameter Description Values Default

Resources

cpu

The percentage of CPU resources requested for Mixer telemetry

CPU resources in millicores based on your environment’s configuration

1000m

memory

The amount of memory requested for Mixer telemetry

Available memory in bytes based on your environment’s configuration

1G

Limits

cpu

The maximum percentage of CPU resources Mixer telemetry is permitted to use

CPU resources in millicores based on your environment’s configuration

4800m

memory

The maximum amount of memory Mixer telemetry is permitted to use

Available memory in bytes based on your environment’s configuration

4G

Istio Pilot example

Here is an example that illustrates the Istio Pilot parameters for the Red Hat OpenShift Service Mesh custom resource and a description of the available parameters with appropriate values.

  pilot:
    resources:
      requests:
        cpu: 100m
     autoscaleEnabled: false
     traceSampling: 100.0
Table 7. Istio Pilot parameters
Parameter Description Values Default value

cpu

The percentage of CPU resources requested for Pilot

CPU resources in millicores based on your environment’s configuration

500m

memory

The amount of memory requested for Pilot

Available memory in bytes based on your environment’s configuration

2048Mi

traceSampling

This value controls how often random sampling occurs. Note: increase for development or testing.

A valid percentage

1.0

Tracing and Jaeger example

This example illustrates tracing parameters for the Red Hat OpenShift Service Mesh custom resource and a description of the available parameters with appropriate values.

  tracing:
      enabled: false
      jaeger:
        tag: 1.13.1
        template: all-in-one
        agentStrategy: DaemonSet
Table 8. Tracing and Jaeger parameters
Parameter Description Value Default value

enabled

This enables tracing in the environment

true/false

true

hub

The hub that the Operator uses to pull Jaeger images

A valid image repo

jaegertracing/ or registry.redhat.io/openshift-istio-tech-preview/

tag

The tag that the Operator uses to pull the Jaeger images

A valid container image tag.

1.13.1

template

The deployment template to use for Jaeger

The name of a template type

all-in-one/production-elasticsearch

agentStrategy

Deploy the Jaeger Agent to each compute node

DaemonSet if required

None

Kiali example

Here is an example that illustrates the Kiali parameters for the Red Hat OpenShift Service Mesh custom resource and a description of the available parameters with appropriate values.

Kiali supports OAuth authentication and hard-coded user credentials. By default, Kiali uses OpenShift OAuth, but you can add a user and passphrase.

  kiali:
     enabled: true
     hub: kiali/
     tag: v01.0.0
     dashboard:
       user: admin
       passphrase: admin
Table 9. Kiali parameters
Parameter Description Values Default value

enabled

This enables or disables Kiali in Service Mesh. Kiali is installed by default. If you do not want to install Kiali, change the enabled value to false.

true/false

true

hub

The hub that the operator uses to pull Kiali images

A valid image repo

kiali/ or registry.redhat.io/openshift-istio-tech-preview/

tag

The tag that the operator uses to pull the Istio images

A valid container image tag

1.0.0

user

The username to access the Kiali console. Note: This is not related to any OpenShift account.

A valid Kiali dashboard username

None

passphrase

The password used to access the Kiali console. Note: This is not related to any OpenShift account.

A valid Kiali dashboard passphrase

None

3scale example

Here is an example that illustrates the 3scale Istio Adapter parameters for the Red Hat OpenShift Service Mesh custom resource and a description of the available parameters with appropriate values.

  threeScale:
      enabled: false
      PARAM_THREESCALE_LISTEN_ADDR: 3333
      PARAM_THREESCALE_LOG_LEVEL: info
      PARAM_THREESCALE_LOG_JSON: true
      PARAM_THREESCALE_LOG_GRPC: false
      PARAM_THREESCALE_REPORT_METRICS: true
      PARAM_THREESCALE_METRICS_PORT: 8080
      PARAM_THREESCALE_CACHE_TTL_SECONDS: 300
      PARAM_THREESCALE_CACHE_REFRESH_SECONDS: 180
      PARAM_THREESCALE_CACHE_ENTRIES_MAX: 1000
      PARAM_THREESCALE_CACHE_REFRESH_RETRIES: 1
      PARAM_THREESCALE_ALLOW_INSECURE_CONN: false
      PARAM_THREESCALE_CLIENT_TIMEOUT_SECONDS: 10
      PARAM_THREESCALE_GRPC_CONN_MAX_SECONDS: 60
Table 10. 3scale parameters
Parameter Description Values Default value

enabled

Whether to use the 3scale adapter

true/false

false

PARAM_THREESCALE_LISTEN_ADDR

Sets the listen address for the gRPC server

Valid port number

3333

PARAM_THREESCALE_LOG_LEVEL

Sets the minimum log output level.

debug, info, warn, error, or none

info

PARAM_THREESCALE_LOG_JSON

Controls whether the log is formatted as JSON

true/false

true

PARAM_THREESCALE_REPORT_METRICS

Controls whether 3scale system and backend metrics are collected and reported to Prometheus

true/false

true

PARAM_THREESCALE_METRICS_PORT

Sets the port that the 3scale /metrics endpoint can be scrapped from

Valid port number

8080

PARAM_THREESCALE_CACHE_TTL_SECONDS

Time period, in seconds, to wait before purging expired items from the cache

Time period in seconds

300

PARAM_THREESCALE_CACHE_REFRESH_SECONDS

Time period before expiry when cache elements are attempted to be refreshed

Time period in seconds

180

PARAM_THREESCALE_CACHE_ENTRIES_MAX

Max number of items that can be stored in the cache at any time. Set to 0 to disable caching

Valid number

1000

PARAM_THREESCALE_CACHE_REFRESH_RETRIES

The number of times unreachable hosts are retried during a cache update loop

Valid number

1

PARAM_THREESCALE_ALLOW_INSECURE_CONN

Allow to skip certificate verification when calling 3scale APIs. Enabling this is not recommended.

true/false

false

PARAM_THREESCALE_CLIENT_TIMEOUT_SECONDS

Sets the number of seconds to wait before terminating requests to 3scale System and Backend

Time period in seconds

10

Updating Mixer policy enforcement

In previous versions of Red Hat OpenShift Service Mesh, Mixer’s policy enforcement was enabled by default. Mixer policy enforcement is now disabled by default. You must enable it before running policy tasks.

Procedure
  1. Run this command to check the current Mixer policy enforcement status:

    $ oc get cm -n istio-system istio -o jsonpath='{.data.mesh}' | grep disablePolicyChecks
  2. If disablePolicyChecks: true, edit the Service Mesh ConfigMap:

    $ oc edit cm -n istio-system istio
  3. Locate disablePolicyChecks: true within the ConfigMap and change the value to false.

  4. Save the configuration and exit the editor.

  5. Re-check the Mixer policy enforcement status to ensure it is set to false.

Deploying the control plane

With the introduction of OpenShift Container Platform 4.1, the network capabilities of the host are now based on nftables rather than iptables. This change impacts the initialization of the Red Hat OpenShift Service Mesh application components. Service Mesh needs to know what host operating system OpenShift is running on to correctly initialize Service Mesh networking components.

You do not need to follow this procedure if you are using OpenShift Container Platform 4.1.

If the OpenShift installation is deployed on a Red Hat Enterprise Linux (RHEL) 7 host, then the custom resource must explicitly request the RHEL 7 proxy-init container image by including the following:

Enabling the proxy-init container for RHEL 7 hosts
  apiVersion: maistra.io/v1
   kind: ServiceMeshControlPlane
   spec:
     istio:
       global:
        proxy_init:
          image: proxy-init

Use the custom resource definition file you created to deploy the Service Mesh control plane.

Procedure
  1. Create a custom resource definition file named istio-installation.yaml.

  2. Run this command to deploy the control plane:

    $ oc create -n istio-system -f istio-installation.yaml
  3. Run this command to watch the progress of the pods during the installation process:

    $ oc get pods -n istio-system -w

Verifying the control plane installation

Verify that the control plane installation completed successfully.

Procedure

The name of the resource is istio-installation.

  1. Run this command to determine if the Operator finished deploying the control plane:

    $ oc get servicemeshcontrolplane/basic-install -n istio-system --template='{{range .status.conditions}}{{printf "%s=%s, reason=%s, message=%s\n\n" .type .status .reason .message}}{{end}}'

    When the control plane installation is finished, the output is similar to the following:

    Installed=True, reason=InstallSuccessful, message=%!s()
    
    Reconciled=True, reason=InstallSuccessful, message=%!s()
  2. After the control plane is deployed, run this command to check the status of the pods:

    $ oc get pods -n istio-system
  3. Verify that the pods are in a state similar to this:

    The results returned when you run this verification step vary depending on your configuration including the number of nodes in the cluster, and whether you are using 3scale, Jaeger, Kiali, or Prometheus.

    NAME                                          READY     STATUS      RESTARTS   AGE
    3scale-istio-adapter-67b96f97b5-cwvgt         1/1       Running     0          99s
    grafana-75f4cbbc6-xw99s                       1/1       Running     0          54m
    istio-citadel-8489b8bb96-ttqfd                1/1       Running     0          54m
    stio-egressgateway-5ccd4d5ddd-wtp2h           1/1       Running     0          52m
    istio-galley-58ff8db57c-jrpkz                 1/1       Running     0          54m
    istio-ingressgateway-698674848f-bk57s         1/1       Running     0          52m
    istio-node-2d764                              1/1       Running     0          54m
    istio-node-4h926                              1/1       Running     0          54m
    istio-node-6qxcj                              1/1       Running     0          54m
    istio-node-8fxqz                              1/1       Running     0          54m
    istio-node-gzg5v                              1/1       Running     0          54m
    istio-node-vxx5p                              1/1       Running     0          54m
    istio-pilot-764966cf69-9nlhp                  2/2       Running     0          19m
    istio-policy-7c856f7d5f-4fjk4                 2/2       Running     2          53m
    istio-sidecar-injector-757b8ccdbf-znggc       1/1       Running     0          49m
    istio-telemetry-65d8b47c98-jrp9h              2/2       Running     2          53m
    jaeger-f775b79f8-cmbb2	                      2/2	      Running	    0	         54m
    kiali-7646d796cd-kfx29	                      1/1	      Running	    0	         20m
    prometheus-56cb9c859b-dlqmn	                  2/2	      Running	    0	         54m
Next steps
  • Prepare to deploy applications on Red Hat OpenShift Service Mesh.