OpenShift docs are moving and will soon only be available at docs.redhat.com, the home of all Red Hat product documentation. Explore the new docs experience today.
  1. Documentation
    2. history
×

Creating the ServiceMeshControlPlane

About ServiceMeshControlPlane

The control plane includes Istiod, Ingress and Egress Gateways, and other components, such as Kiali and Jaeger. The control plane must be deployed in a separate namespace than the Service Mesh Operators and the data plane applications and services. You can deploy a basic installation of the ServiceMeshControlPlane(SMCP) from the OpenShift Container Platform web console or the command line using the oc client tool.

This basic installation is configured based on the default OpenShift Container Platform settings and is not designed for production use. Use this default installation to verify your installation, and then configure your ServiceMeshControlPlane settings for your environment.

Red Hat OpenShift Service on AWS (ROSA) places additional restrictions on where you can create resources, and as a result, the default deployment does not work. See Installing Service Mesh on Red Hat OpenShift Service on AWS for additional requirements before deploying your SMCP in a ROSA environment.

The Service Mesh documentation uses istio-system as the example project, but you can deploy the service mesh to any project.

Deploying the Service Mesh control plane from the web console

You can deploy a basic ServiceMeshControlPlane by using the web console. In this example, istio-system is the name of the Service Mesh control plane project.

Prerequisites

  • The Red Hat OpenShift Service Mesh Operator must be installed.

  • An account with the cluster-admin role.

Procedure

  1. Log in to the OpenShift Container Platform web console as a user with the cluster-admin role. If you use Red Hat OpenShift Dedicated, you must have an account with the dedicated-admin role.

  2. Create a project named istio-system.

    1. Navigate to Home → Projects.

    2. Click Create Project.

    3. In the Name field, enter istio-system. The ServiceMeshControlPlane resource must be installed in a project that is separate from your microservices and Operators.

      These steps use istio-system as an example, but you can deploy your Service Mesh control plane in any project as long as it is separate from the project that contains your services.

    4. Click Create.

  3. Navigate to Operators → Installed Operators.

  4. Click the Red Hat OpenShift Service Mesh Operator, then click Istio Service Mesh Control Plane.

  5. On the Istio Service Mesh Control Plane tab, click Create ServiceMeshControlPlane.

  6. On the Create ServiceMeshControlPlane page, accept the default Service Mesh control plane version to take advantage of the features available in the most current version of the product. The version of the control plane determines the features available regardless of the version of the Operator.

    1. Click Create. The Operator creates pods, services, and Service Mesh control plane components based on your configuration parameters. You can configure ServiceMeshControlPlane settings later.

  7. To verify the control plane installed correctly, click the Istio Service Mesh Control Plane tab.

    1. Click the name of the new control plane.

    2. Click the Resources tab to see the Red Hat OpenShift Service Mesh control plane resources the Operator created and configured.

Deploying the Service Mesh control plane using the CLI

You can deploy a basic ServiceMeshControlPlane from the command line.

Prerequisites

  • The Red Hat OpenShift Service Mesh Operator must be installed.

  • Access to the OpenShift CLI (oc).

Procedure

  1. Log in to the OpenShift Container Platform CLI as a user with the cluster-admin role. If you use Red Hat OpenShift Dedicated, you must have an account with the dedicated-admin role.

    $ oc login --username=<NAMEOFUSER> https://<HOSTNAME>:6443

  2. Create a project named istio-system.

    $ oc new-project istio-system

  3. Create a ServiceMeshControlPlane file named istio-installation.yaml using the following example. The version of the Service Mesh control plane determines the features available regardless of the version of the Operator.

    Example version 2.4 istio-installation.yaml
    apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
metadata:
  name: basic
  namespace: istio-system
spec:
  version: v2.4
  tracing:
    type: Jaeger
    sampling: 10000
  addons:
    jaeger:
      name: jaeger
      install:
        storage:
          type: Memory
    kiali:
      enabled: true
      name: kiali
    grafana:
      enabled: true

  4. Run the following command to deploy the Service Mesh control plane, where <istio_installation.yaml> includes the full path to your file.

    $ oc create -n istio-system -f <istio_installation.yaml>

  5. To watch the progress of the pod deployment, run the following command:

    $ oc get pods -n istio-system -w

    You should see output similar to the following:

    NAME                                   READY   STATUS    RESTARTS   AGE
grafana-b4d59bd7-mrgbr                 2/2     Running   0          65m
istio-egressgateway-678dc97b4c-wrjkp   1/1     Running   0          108s
istio-ingressgateway-b45c9d54d-4qg6n   1/1     Running   0          108s
istiod-basic-55d78bbbcd-j5556          1/1     Running   0          108s
jaeger-67c75bd6dc-jv6k6                2/2     Running   0          65m
kiali-6476c7656c-x5msp                 1/1     Running   0          43m
prometheus-58954b8d6b-m5std            2/2     Running   0          66m

Validating your SMCP installation with the CLI

You can validate the creation of the ServiceMeshControlPlane from the command line.

Procedure

  1. Log in to the OpenShift Container Platform CLI as a user with the cluster-admin role. If you use Red Hat OpenShift Dedicated, you must have an account with the dedicated-admin role.

    $ oc login https://<HOSTNAME>:6443

  2. Run the following command to verify the Service Mesh control plane installation, where istio-system is the namespace where you installed the Service Mesh control plane.

    $ oc get smcp -n istio-system

    The installation has finished successfully when the STATUS column is ComponentsReady.

    NAME    READY   STATUS            PROFILES      VERSION   AGE
basic   10/10   ComponentsReady   ["default"]   2.1.1     66m

About control plane components and infrastructure nodes

Infrastructure nodes provide a way to isolate infrastructure workloads for two primary purposes:

  • To prevent incurring billing costs against subscription counts

  • To separate maintenance and management of infrastructure workloads

You can configure some or all of the Service Mesh control plane components to run on infrastructure nodes.

Configuring all control plane components to run on infrastructure nodes using the web console

Perform this task if all of the components deployed by the Service Mesh control plane will run on infrastructure nodes. These deployed components include Istiod, Ingress Gateway, and Egress Gateway, and optional applications such as Prometheus, Grafana, and Distributed Tracing.

If the control plane will run on a worker node, skip this task.

Prerequisites

  • You have installed the Red Hat OpenShift Service Mesh Operator.

  • You are logged in as a user with the cluster-admin role. If you use Red Hat OpenShift Dedicated, you are logged in as a user with the dedicated-admin role.

Procedure

  1. Log in to the OpenShift Container Platform web console.

  2. Navigate to Operators → Installed Operators.

  3. Click the Red Hat OpenShift Service Mesh Operator, and then click Istio Service Mesh Control Plane.

  4. Click the name of the control plane resource. For example, basic.

  5. Click YAML.

  6. Add the nodeSelector and tolerations fields to the spec.runtime.defaults.pod specification in the ServiceMeshControlPlane resource, as shown in the following example:

    spec:
  runtime:
    defaults:
      pod:
        nodeSelector: (1)
          node-role.kubernetes.io/infra: ""
        tolerations: (2)
        - effect: NoSchedule
          key: node-role.kubernetes.io/infra
          value: reserved
        - effect: NoExecute
          key: node-role.kubernetes.io/infra
          value: reserved
    1 Ensures that the ServiceMeshControlPlane pod is only scheduled on an infrastructure node.
    2 Ensures that the pod is accepted by the infrastructure node for execution.

  7. Click Save.

  8. Click Reload.

Configuring individual control plane components to run on infrastructure nodes using the web console

Perform this task if individual components deployed by the Service Mesh control plane will run on infrastructure nodes. These deployed components include Istiod, the Ingress Gateway, and the Egress Gateway.

If the control plane will run on a worker node, skip this task.

Prerequisites

  • You have installed the Red Hat OpenShift Service Mesh Operator.

  • You are logged in as a user with the cluster-admin role. If you use Red Hat OpenShift Dedicated, you are logged in as a user with the dedicated-admin role.

Procedure

  1. Log in to the OpenShift Container Platform web console.

  2. Navigate to Operators → Installed Operators.

  3. Click the Red Hat OpenShift Service Mesh Operator, and then click Istio Service Mesh Control Plane.

  4. Click the name of the control plane resource. For example, basic.

  5. Click YAML.

  6. Add the nodeSelector and tolerations fields to the spec.runtime.components.pilot.pod specification in the ServiceMeshControlPlane resource, as shown in the following example:

    spec:
  runtime:
    components:
      pilot:
        pod:
          nodeSelector: (1)
            node-role.kubernetes.io/infra: ""
          tolerations: (2)
          - effect: NoSchedule
            key: node-role.kubernetes.io/infra
            value: reserved
          - effect: NoExecute
            key: node-role.kubernetes.io/infra
            value: reserved
    1 Ensures that the Istiod pod is only scheduled on an infrastructure node.
    2 Ensures that the pod is accepted by the infrastructure node for execution.

  7. Add the nodeSelector and the tolerations fields to the spec.gateways.ingress.runtime.pod and spec.gateways.egress.runtime.pod specifications in the ServiceMeshControlPlane resource, as shown in the following example:

    spec:
  gateways:
    ingress:
      runtime:
        pod:
          nodeSelector: (1)
            node-role.kubernetes.io/infra: ""
          tolerations: (2)
          - effect: NoSchedule
            key: node-role.kubernetes.io/infra
            value: reserved
          - effect: NoExecute
            key: node-role.kubernetes.io/infra
            value: reserved
    egress:
      runtime:
        pod:
          nodeSelector: (1)
            node-role.kubernetes.io/infra: ""
          tolerations: (2)
          - effect: NoSchedule
            key: node-role.kubernetes.io/infra
            value: reserved
          - effect: NoExecute
            key: node-role.kubernetes.io/infra
            value: reserved
    1 Ensures that the gateway pod is only scheduled on an infrastructure node
    2 Ensures that the pod is accepted by the infrastructure node for execution.

  8. Click Save.

  9. Click Reload.

Configuring all control plane components to run on infrastructure nodes using the CLI

Perform this task if all of the components deployed by the Service Mesh control plane will run on infrastructure nodes. These deployed components include Istiod, Ingress Gateway, and Egress Gateway, and optional applications such as Prometheus, Grafana, and Distributed Tracing.

If the control plane will run on a worker node, skip this task.

Prerequisites

  • You have installed the Red Hat OpenShift Service Mesh Operator.

  • You are logged in as a user with the cluster-admin role. If you use Red Hat OpenShift Dedicated, you are logged in as a user with the dedicated-admin role.

Procedure

  1. Open the ServiceMeshControlPlane resource as a YAML file:

    $ oc -n istio-system edit smcp <name> (1)
    1 <name> represents the name of the ServiceMeshControlPlane resource.

  2. To run all of the Service Mesh components deployed by the ServiceMeshControlPlane on infrastructure nodes, add the nodeSelector and tolerations fields to the spec.runtime.defaults.pod spec in the ServiceMeshControlPlane resource:

    spec:
  runtime:
    defaults:
      pod:
        nodeSelector: (1)
          node-role.kubernetes.io/infra: ""
        tolerations: (2)
        - effect: NoSchedule
          key: node-role.kubernetes.io/infra
          value: reserved
        - effect: NoExecute
          key: node-role.kubernetes.io/infra
          value: reserved
    1 Ensures that the SMCP pods are only scheduled on an infrastructure node.
    2 Ensures that the pods are accepted by the infrastructure node.

Configuring individual control plane components to run on infrastructure nodes using the CLI

Perform this task if individual components deployed by the Service Mesh control plane will run on infrastructure nodes. These deployed components include Istiod, the Ingress Gateway, and the Egress Gateway.

If the control plane will run on a worker node, skip this task.

Prerequisites

  • You have installed the Red Hat OpenShift Service Mesh Operator.

  • You are logged in as a user with the cluster-admin role. If you use Red Hat OpenShift Dedicated, you are logged in as a user with the dedicated-admin role.

Procedure

  1. Open the ServiceMeshControlPlane resource as a YAML file.

    $ oc -n istio-system edit smcp <name> (1)
    1 <name> represents the name of the ServiceMeshControlPlane resource.

  2. To run the Istiod component on an infrastructure node, add the nodeSelector and the tolerations fields to the spec.runtime.components.pilot.pod spec in the ServiceMeshControlPlane resource.

    spec:
  runtime:
    components:
      pilot:
        pod:
          nodeSelector: (1)
            node-role.kubernetes.io/infra: ""
          tolerations: (2)
          - effect: NoSchedule
            key: node-role.kubernetes.io/infra
            value: reserved
          - effect: NoExecute
            key: node-role.kubernetes.io/infra
            value: reserved
    1 Ensures that the Istiod pod is only scheduled on an infrastructure node.
    2 Ensures that the pod is accepted by the infrastructure node.

  3. To run Ingress and Egress Gateways on infrastructure nodes, add the nodeSelector and the tolerations fields to the spec.gateways.ingress.runtime.pod spec and the spec.gateways.egress.runtime.pod spec in the ServiceMeshControlPlane resource.

    spec:
  gateways:
    ingress:
      runtime:
        pod:
          nodeSelector: (1)
            node-role.kubernetes.io/infra: ""
          tolerations: (2)
          - effect: NoSchedule
            key: node-role.kubernetes.io/infra
            value: reserved
          - effect: NoExecute
            key: node-role.kubernetes.io/infra
            value: reserved
    egress:
      runtime:
        pod:
          nodeSelector: (1)
            node-role.kubernetes.io/infra: ""
          tolerations: (2)
          - effect: NoSchedule
            key: node-role.kubernetes.io/infra
            value: reserved
          - effect: NoExecute
            key: node-role.kubernetes.io/infra
            value: reserved
    1 Ensures that the gateway pod is only scheduled on an infrastructure node
    2 Ensures that the pod is accepted by the infrastructure node.

Verifying the Service Mesh control plane is running on infrastructure nodes

Procedure

  • Confirm that the nodes associated with Istiod, Ingress Gateway, and Egress Gateway pods are infrastructure nodes:

    $ oc -n istio-system get pods -owide

About control plane and cluster-wide deployments

A cluster-wide deployment contains a Service Mesh Control Plane that monitors resources for an entire cluster. Monitoring resources for an entire cluster closely resembles Istio functionality in that the control plane uses a single query across all namespaces to monitor Istio and Kubernetes resources. As a result, cluster-wide deployments decrease the number of requests sent to the API server.

You can configure the Service Mesh Control Plane for cluster-wide deployments using either the OpenShift Container Platform web console or the CLI.

Configuring the control plane for cluster-wide deployment with the web console

You can configure the ServiceMeshControlPlane resource for cluster-wide deployment using the OpenShift Container Platform web console. In this example, istio-system is the name of the Service Mesh control plane project.

Prerequisites

  • The Red Hat OpenShift Service Mesh Operator is installed.

  • You are logged in using an account with the cluster-admin role, or if you use Red Hat OpenShift Dedicated with the dedicated-admin role.

Procedure

  1. Create a project named istio-system.

    1. Navigate to Home → Projects.

    2. Click Create Project.

    3. In the Name field, enter istio-system. The ServiceMeshControlPlane resource must be installed in a project that is separate from your microservices and Operators.

      These steps use istio-system as an example. You can deploy the Service Mesh control plane to any project as long as it is separate from the project that contains your services.

    4. Click Create.

  2. Navigate to Operators → Installed Operators.

  3. Click the Red Hat OpenShift Service Mesh Operator, then click Istio Service Mesh Control Plane.

  4. On the Istio Service Mesh Control Plane tab, click Create ServiceMeshControlPlane.

  5. Click YAML view. The version of the Service Mesh control plane determines the features available regardless of the version of the Operator.

  6. Modify the spec.mode field of the YAML file to specify ClusterWide.

    Example version 2.4 istio-installation.yaml
    apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
metadata:
  name: basic
  namespace: istio-system
spec:
  version: v2.4
  mode: ClusterWide

  7. Click Create. The Operator creates pods, services, and Service Mesh control plane components based on your configuration parameters. The operator also creates the ServiceMeshMemberRoll if it does not exist as part of the default configuration.

  8. To verify that the control plane installed correctly, click the Istio Service Mesh Control Plane tab.

    1. Click the name of the new ServiceMeshControlPlane object.

    2. Click the Resources tab to see the Red Hat OpenShift Service Mesh control plane resources that the Operator created and configured.

This module is included in the following assemblies: * service_mesh/v2x/ossm-create-smcp.adoc :_content-type: PROCEDURE

Configuring the control plane for cluster-wide deployment with the CLI

You can configure the ServiceMeshControlPlane resource for cluster-wide deployment using the CLI. In this example, istio-system is the name of the Service Mesh control plane namespace.

Prerequisites

  • The Red Hat OpenShift Service Mesh Operator is installed.

  • You have access to the OpenShift CLI (oc).

Procedure

  1. Log in to the OpenShift Container Platform CLI as a user with the cluster-admin role. If you use Red Hat OpenShift Dedicated, you must have an account with the dedicated-admin role.

    $ oc login --username=<NAMEOFUSER> https://<HOSTNAME>:6443

  2. Create a project named istio-system.

    $ oc new-project istio-system

  3. Create a ServiceMeshControlPlane file named istio-installation.yaml using the following example.

    Example version 2.4 istio-installation.yaml
    apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
metadata:
  name: basic
  namespace: istio-system
spec:
  version: v2.4
  mode: ClusterWide

  4. Run the following command to deploy the Service Mesh control plane, where <istio_installation.yaml> includes the full path to your file.

    $ oc create -n istio-system -f <istio_installation.yaml>

  5. To monitor the progress of the pod deployment, run the following command:

    $ oc get pods -n istio-system -w

    You should see output similar to the following example:

    Example output
    NAME                                   READY   STATUS    RESTARTS   AGE
grafana-b4d59bd7-mrgbr                 2/2     Running   0          65m
istio-egressgateway-678dc97b4c-wrjkp   1/1     Running   0          108s
istio-ingressgateway-b45c9d54d-4qg6n   1/1     Running   0          108s
istiod-basic-55d78bbbcd-j5556          1/1     Running   0          108s
jaeger-67c75bd6dc-jv6k6                2/2     Running   0          65m
kiali-6476c7656c-x5msp                 1/1     Running   0          43m
prometheus-58954b8d6b-m5std            2/2     Running   0          66m

This module is included in the following assemblies: * service_mesh/v2x/ossm-create-smcp.adoc

Customizing the member roll for a cluster-wide mesh

In cluster-wide mode, when you create the ServiceMeshControlPlane resource, the ServiceMeshMemberRoll resource is also created. You can modify the ServiceMeshMemberRoll resource after it gets created. After you modify the resource, the Service Mesh operator no longer changes it. If you modify the ServiceMeshMemberRoll resource by using the OpenShift Container Platform web console, accept the prompt to overwrite the modifications.

Alternatively, you can create a ServiceMeshMemberRoll resource before deploying the ServiceMeshControlPlane resource. When you create the ServiceMeshControlPlane resource, the Service Mesh Operator will not modify the ServiceMeshMemberRoll.

The ServiceMeshMemberRoll resource name must be named default and must be created in the same project namespace as the ServiceMeshControlPlane resource.

There are two ways to add a namespace to the mesh. You can either add the namespace by specifying its name in the spec.members list, or configure a set of namespace label selectors to include or exclude namespaces based on their labels.

Regardless of how members are specified in the ServiceMeshMemberRoll resource, you can also add members to the mesh by creating the ServiceMeshMember resource in each namespace.

Validating your SMCP installation with Kiali

You can use the Kiali console to validate your Service Mesh installation. The Kiali console offers several ways to validate your Service Mesh components are deployed and configured properly.

Procedure

  1. Log in to the OpenShift Container Platform web console as a user with cluster-admin rights. If you use Red Hat OpenShift Dedicated, you must have an account with the dedicated-admin role.

  2. Navigate to Networking → Routes.

  3. On the Routes page, select the Service Mesh control plane project, for example istio-system, from the Namespace menu.

    The Location column displays the linked address for each route.

  4. If necessary, use the filter to find the route for the Kiali console. Click the route Location to launch the console.

  5. Click Log In With OpenShift.

    When you first log in to the Kiali Console, you see the Overview page which displays all the namespaces in your service mesh that you have permission to view. When there are multiple namespaces shown on the Overview page, Kiali shows namespaces with health or validation problems first.

    Kiali Overview page showing istio-system
    Figure 1. Kiali Overview page

    The tile for each namespace displays the number of labels, the Istio Config health, the number of and Applications health, and Traffic for the namespace. If you are validating the console installation and namespaces have not yet been added to the mesh, there might not be any data to display other than istio-system.

  6. Kiali has four dashboards specifically for the namespace where the Service Mesh control plane is installed. To view these dashboards, click the Options menu kebab on the tile for the control plane namespace, for example, istio-system, and select one of the following options:

    • Istio Mesh Dashboard

    • Istio Control Plane Dashboard

    • Istio Performance Dashboard

    • Istio Wasm Exetension Dashboard

      Istio Control Plane Dashboard showing data for bookinfo sample project
      Figure 2. Grafana Istio Control Plane Dashboard

      Kiali also installs two additional Grafana dashboards, available from the Grafana Home page:

    • Istio Workload Dashboard

    • Istio Service Dashboard

  7. To view the Service Mesh control plane nodes, click the Graph page, select the Namespace where you installed the ServiceMeshControlPlane from the menu, for example istio-system.

    1. If necessary, click Display idle nodes.

    2. To learn more about the Graph page, click the Graph tour link.

    3. To view the mesh topology, select one or more additional namespaces from the Service Mesh Member Roll from the Namespace menu.

  8. To view the list of applications in the istio-system namespace, click the Applications page. Kiali displays the health of the applications.

    1. Hover your mouse over the information icon to view any additional information noted in the Details column.

  9. To view the list of workloads in the istio-system namespace, click the Workloads page. Kiali displays the health of the workloads.

    1. Hover your mouse over the information icon to view any additional information noted in the Details column.

  10. To view the list of services in the istio-system namespace, click the Services page. Kiali displays the health of the services and of the configurations.

    1. Hover your mouse over the information icon to view any additional information noted in the Details column.

  11. To view a list of the Istio Configuration objects in the istio-system namespace, click the Istio Config page. Kiali displays the health of the configuration.

    1. If there are configuration errors, click the row and Kiali opens the configuration file with the error highlighted.

Installing on Red Hat OpenShift Service on AWS (ROSA)

Starting with version 2.2, Red Hat OpenShift Service Mesh supports installation on Red Hat OpenShift Service on AWS (ROSA). This section documents the additional requirements when installing Service Mesh on this platform.

Installation location

You must create a new namespace, for example istio-system, when installing Red Hat OpenShift Service Mesh and creating the ServiceMeshControlPlane.

Required Service Mesh control plane configuration

The default configuration in the ServiceMeshControlPlane file does not work on a ROSA cluster. You must modify the default SMCP and set spec.security.identity.type=ThirdParty when installing on Red Hat OpenShift Service on AWS.

Example ServiceMeshControlPlane resource for ROSA
apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
metadata:
  name: basic
  namespace: istio-system
spec:
  version: v2.4
  security:
    identity:
      type: ThirdParty  #required setting for ROSA
  tracing:
    type: Jaeger
    sampling: 10000
  policy:
    type: Istiod
  addons:
    grafana:
      enabled: true
    jaeger:
      install:
        storage:
          type: Memory
    kiali:
      enabled: true
    prometheus:
      enabled: true
  telemetry:
    type: Istiod

Restrictions on Kiali configuration

Red Hat OpenShift Service on AWS places additional restrictions on where you can create resources and does not let you create the Kiali resource in a Red Hat managed namespace.

This means that the following common settings for spec.deployment.accessible_namespaces are not allowed in a ROSA cluster:

  • ['**'] (all namespaces)

  • default

  • codeready-*

  • openshift-*

  • redhat-*

The validation error message provides a complete list of all the restricted namespaces.

Example Kiali resource for ROSA
apiVersion: kiali.io/v1alpha1
kind: Kiali
metadata:
  name: kiali
  namespace: istio-system
spec:
  auth:
    strategy: openshift
  deployment:
    accessible_namespaces:   #restricted setting for ROSA
      - istio-system
    image_pull_policy: ''
    ingress_enabled: true
    namespace: istio-system

Additional resources

Red Hat OpenShift Service Mesh supports multiple independent control planes within the cluster. You can create reusable configurations with ServiceMeshControlPlane profiles. For more information, see Creating control plane profiles.

Next steps