$ oc adm must-gather --image=registry.redhat.io/openshift-service-mesh/istio-must-gather-rhel7
Red Hat OpenShift Service Mesh is a platform that provides behavioral insight and operational control over the service mesh, providing a uniform way to connect, secure, and monitor microservice applications.
The term service mesh describes the network of microservices that make up applications in a distributed microservice architecture and the interactions between those microservices. As a service mesh grows in size and complexity, it can become harder to understand and manage.
Based on the open source Istio project, Red Hat OpenShift Service Mesh adds a transparent layer on existing distributed applications without requiring any changes to the service code. You add Red Hat OpenShift Service Mesh support to services by deploying a special sidecar proxy throughout your environment that intercepts all network communication between microservices. You configure and manage the service mesh using the control plane features.
Red Hat OpenShift Service Mesh provides an easy way to create a network of deployed services that provides discovery, load balancing, service-to-service authentication, failure recovery, metrics, and monitoring. A service mesh also provides more complex operational functionality, including A/B testing, canary releases, rate limiting, access control, and end-to-end authentication.
Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see our CTO Chris Wright’s message.
If you experience difficulty with a procedure described in this documentation, or with OpenShift Container Platform in general, visit the Red Hat Customer Portal. From the Customer Portal, you can:
Search or browse through the Red Hat Knowledgebase of articles and solutions relating to Red Hat products.
Submit a support case to Red Hat Support.
Access other product documentation.
To identify issues with your cluster, you can use Insights in Red Hat OpenShift Cluster Manager. Insights provides details about issues and, if available, information on how to solve a problem.
If you have a suggestion for improving this documentation or have found an error, please submit a Bugzilla report against the OpenShift Container Platform product for the Documentation component. Please provide specific details, such as the section name and OpenShift Container Platform version.
When opening a support case, it is helpful to provide debugging information about your cluster to Red Hat Support.
The must-gather tool enables you to collect diagnostic information about your
OpenShift Container Platform cluster, including virtual machines and other data related to
Red Hat OpenShift Service Mesh.
For prompt support, supply diagnostic information for both OpenShift Container Platform and Red Hat OpenShift Service Mesh.
The oc adm must-gather CLI command collects the information from your cluster that is most likely needed for debugging issues, such as:
Resource definitions
Audit logs
Service logs
You can specify one or more images when you run the command by including the --image argument. When you specify an image, the tool collects data related to that feature or product.
When you run oc adm must-gather, a new pod is created on the cluster. The data is collected on that pod and saved in a new directory that starts with must-gather.local. This directory is created in the current working directory.
Access to the cluster as a user with the cluster-admin role.
The OpenShift Container Platform CLI (oc) installed.
You can use the oc adm must-gather CLI command to collect information about your cluster, including features and objects associated with Red Hat OpenShift Service Mesh.
To collect Red Hat OpenShift Service Mesh data with must-gather, you must specify the Red Hat OpenShift Service Mesh image:
$ oc adm must-gather --image=registry.redhat.io/openshift-service-mesh/istio-must-gather-rhel7The following are the only supported configurations for the Red Hat OpenShift Service Mesh:
Red Hat OpenShift Container Platform version 4.x.
|
OpenShift Online and OpenShift Dedicated are not supported for Red Hat OpenShift Service Mesh. |
The deployment must be contained to a single OpenShift Container Platform cluster that is not federated.
This release of Red Hat OpenShift Service Mesh is only available on OpenShift Container Platform x86_64, IBM Z, and IBM Power Systems.
IBM Z is only supported on OpenShift Container Platform 4.6 and later.
IBM Power Systems is only supported on OpenShift Container Platform 4.6 and later.
This release only supports configurations where all Service Mesh components are contained in the OpenShift cluster in which it operates. It does not support management of microservices that reside outside of the cluster, or in a multi-cluster scenario.
This release only supports configurations that do not integrate external services such as virtual machines.
For additional information about Red Hat OpenShift Service Mesh lifecycle and supported configurations, refer to the Support Policy.
Red Hat OpenShift Service Mesh supports the following network configurations.
Openshift-SDN
OVN-Kubernetes is supported as a technology preview in OpenShift Container Platform version 4.7.
Red Hat OpenShift Service Mesh provides a number of key capabilities uniformly across a network of services:
Traffic Management - Control the flow of traffic and API calls between services, make calls more reliable, and make the network more robust in the face of adverse conditions.
Service Identity and Security - Provide services in the mesh with a verifiable identity and provide the ability to protect service traffic as it flows over networks of varying degrees of trustworthiness.
Policy Enforcement - Apply organizational policy to the interaction between services, ensure access policies are enforced and resources are fairly distributed among consumers. Policy changes are made by configuring the mesh, not by changing application code.
Telemetry - Gain understanding of the dependencies between services and the nature and flow of traffic between them, providing the ability to quickly identify issues.
| Component | Version |
|---|---|
Istio |
1.6.14 |
Jaeger |
1.20.3 |
Kiali |
1.24.6 |
3scale Istio Adapter |
2.0.0 |
This release of Red Hat OpenShift Service Mesh adds support for IBM Z and IBM Power Systems. It also addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
This release of Red Hat OpenShift Service Mesh addresses Common Vulnerabilities and Exposures (CVEs) and bug fixes.
This release of Red Hat OpenShift Service Mesh adds support for Istio 1.6.5, Jaeger 1.20.0, Kiali 1.24.2, and the 3scale Istio Adapter 2.0 and OpenShift Container Platform 4.6.
In addition, this release has the following new features:
Introduces a re-architected control plane. The Mixer component has been deprecated and will be removed in a future release. The other control plane components, Pilot, Galley, Citadel, have been combined into a single binary known as Istiod. The "d" stands for daemon.
Simplifies installation, upgrades, and management of the control plane.
Reduces the control plane’s resource usage and startup time.
Improves performance by reducing inter-control plane communication over networking.
Adds support for Envoy’s Secret Discovery Service (SDS). SDS is a more secure and efficient mechanism for delivering secrets to Envoy side car proxies.
Removes the need to use Kubernetes Secrets, which have well known security risks.
Improves performance during certificate rotation, as proxies no longer require a restart to recognize new certificates.
Adds support for Istio’s Telemetry v2 architecture, which is built using WebAssembly extensions. This new architecture brings significant performance improvements.
Updates the ServiceMeshControlPlane resource to v2 with a streamlined configuration to make it easier to manage the Control Plane.
Introduces WebAssembly extensions as a Technology Preview feature.
|
Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see https://access.redhat.com/support/offerings/techpreview/. |
Red Hat OpenShift Service Mesh 2.0.0 introduces support for WebAssembly extensions to Envoy Proxy.
Up through release 1.5, Istio implemented extensions using the Mixer Telemetry and Policy components. In Istio 1.5 Mixer was deprecated and WebAssembly was introduced as the new mechanism for extensions in Istio. Envoy now allows extensions using WebAssembly (“WASM”) - a format for executing code written in multiple programming languages. Mixer has been deprecated as of Istio 1.5, and will be removed in 1.8. Going forward, extensions to Istio will be implemented with Envoy plugins written with WebAssembly.
The new Telemetry architecture is based on these WebAssembly extensions. For Service Mesh 2.0, we are introducing WebAssembly extensions as a Tech Preview feature. WebAssembly extensions is the new way of extending Istio functionality, replacing the Mixer component, which has been deprecated and will eventually be removed.
|
Note that built-in Istio WASM extensions are not included in the proxy binary and that WASM filters from the upstream Istio community are not supported in Red Hat OpenShift Service Mesh 2.0. |
Red Hat OpenShift Service Mesh 2.0.1 introduces support for the OVN-Kubernetes network type on OpenShift Container Platform 4.7.
For more information about WebAssembly extensions, see Extensions.
Some features available in previous releases have been deprecated or removed.
Deprecated functionality is still included in OpenShift Container Platform and continues to be supported; however, it will be removed in a future release of this product and is not recommended for new deployments.
The Mixer component was deprecated in release 2.0 and will be removed in release 2.1. While using Mixer for implementing extensions was still supported in release 2.0, extensions should have been migrated to the new WebAssembly mechanism.
The following resource types are no longer supported in Red Hat OpenShift Service Mesh 2.0:
Policy (authentication.istio.io/v1alpha1) is no longer supported. Depending on the specific configuration in your Policy resource, you may have to configure multiple resources to achieve the same effect.
Use RequestAuthentication (security.istio.io/v1beta1)
Use PeerAuthentication (security.istio.io/v1beta1)
ServiceMeshPolicy (maistra.io/v1) is no longer supported.
Use RequestAuthentication or PeerAuthentication, as mentioned above, but place in the control plane namespace.
RbacConfig (rbac.istio.io/v1alpha1) is no longer supported.
Replaced by AuthorizationPolicy (security.istio.io/v1beta1), which encompasses behavior of RbacConfig, ServiceRole, and ServiceRoleBinding.
ServiceMeshRbacConfig (maistra.io/v1) is no longer supported.
Use AuthorizationPolicy as above, but place in control plane namespace.
ServiceRole (rbac.istio.io/v1alpha1) is no longer supported.
ServiceRoleBinding (rbac.istio.io/v1alpha1) is no longer supported.
In Kiali, the login and LDAP strategies are deprecated. A future version will introduce authentication using OpenID providers.
These limitations exist in Red Hat OpenShift Service Mesh:
Red Hat OpenShift Service Mesh does not support IPv6, as it is not supported by the upstream Istio project, nor fully supported by OpenShift.
Graph layout - The layout for the Kiali graph can render differently, depending on your application architecture and the data to display (number of graph nodes and their interactions). Because it is difficult if not impossible to create a single layout that renders nicely for every situation, Kiali offers a choice of several different layouts. To choose a different layout, you can choose a different Layout Schema from the Graph Settings menu.
The first time you access related services such as Jaeger and Grafana, from the Kiali console, you must accept the certificate and re-authenticate using your OpenShift Container Platform login credentials. This happens due to an issue with how the framework displays embedded pages in the console.
The Bookinfo sample application cannot be installed on IBM Z and IBM Power Systems.
WebAssembly is unsupported on IBM Z and IBM Power Systems.
These are the known issues in Red Hat OpenShift Service Mesh:
Istio-14743 Due to limitations in the version of Istio that this release of Red Hat OpenShift Service Mesh is based on, there are several applications that are currently incompatible with Service Mesh. See the linked community issue for details.
OSSM-285 When trying to access the Kiali console, receive the following error message "Error trying to get OAuth Metadata". The workaround is to restart the Kiali pod.
MAISTRA-1947 Technology Preview Updates to ServiceMeshExtensions are not applied. The workaround is to remove and recreate the ServiceMeshExtensions.
MAISTRA-1621 Migration to 2.0 Prometheus scraping (spec.addons.prometheus.scrape set to true) does not work when mTLS is enabled. Additionally, Kiali displays extraneous graph data when mTLS is disabled.
This problem can be addressed by excluding port 15020 from proxy configuration, for example,
spec:
proxy:
networking:
trafficControl:
inbound:
excludedPorts:
- 15020MAISTRA-806 Evicted Istio Operator Pod causes mesh and CNI not to deploy.
If the istio-operator pod is evicted while deploying the control pane, delete the evicted istio-operator pod.
MAISTRA-681 When the control plane has many namespaces, it can lead to performance issues.
MAISTRA-465 The Maistra Operator fails to create a service for operator metrics.
MAISTRA-453 If you create a new project and deploy pods immediately, sidecar injection does not occur. The operator fails to add the maistra.io/member-of before the pods are created, therefore the pods must be deleted and recreated for sidecar injection to occur.
MAISTRA-158 Applying multiple gateways referencing the same hostname will cause all gateways to stop functioning.
|
New issues for Kiali should be created in the OpenShift Service Mesh project with the |
These are the known issues in Kiali:
KIALI-2206 When you are accessing the Kiali console for the first time, and there is no cached browser data for Kiali, the “View in Grafana” link on the Metrics tab of the Kiali Service Details page redirects to the wrong location. The only way you would encounter this issue is if you are accessing Kiali for the first time.
KIALI-507 Kiali does not support Internet Explorer 11. This is because the underlying frameworks do not support Internet Explorer. To access the Kiali console, use one of the two most recent versions of the Chrome, Edge, Firefox or Safari browser.
These limitations exist in Jaeger:
Apache Spark is not supported.
Jaeger streaming via AMQ/Kafka is unsupported on IBM Z and IBM Power Systems.
These are the known issues in Jaeger:
BZ-1918920 The Elasticsearch pods does not get restarted automatically after an update. As a workaround, restart the pods manually.
TRACING-809 Jaeger Ingester is incompatible with Kafka 2.3. When there are two or more instances of the Jaeger Ingester and enough traffic it will continuously generate rebalancing messages in the logs. This is due to a regression in Kafka 2.3 that was fixed in Kafka 2.3.1. For more information, see Jaegertracing-1819.
The following issues been resolved in the current release:
OSSM-296 When adding health configuration to the Kiali custom resource (CR) is it not being replicated to the Kiali configmap.
OSSM-291 In the Kiali console, on the Applications, Services, and Workloads pages, the "Remove Label from Filters" function is not working.
OSSM-289 In the Kiali console, on the Service Details pages for the 'istio-ingressgateway' and 'jaeger-query' services there are no Traces being displayed. The traces exist in Jaeger.
OSSM-287 In the Kiali console there are no traces being displayed on the Graph Service.
MAISTRA-2010 AuthorizationPolicy does not support request.regex.headers field. The validatingwebhook rejects any AuthorizationPolicy with the field, and even if you disable that, Pilot tries to validate it using the same code, and it does not work.
MAISTRA-1979 Migration to 2.0 The conversion webhook drops the following important fields when converting SMCP.status from v2 to v1:
conditions
components
observedGeneration
annotations
Upgrading the operator to 2.0 might break client tools that read the SMCP status using the maistra.io/v1 version of the resource.
This also causes the READY and STATUS columns to be empty when you run oc get servicemeshcontrolplanes.v1.maistra.io.
MAISTRA-1089 Migration to 2.0 Gateways created in a non-control plane namespace are automatically deleted. Users will need to manually delete these resources after removing the gateway definition from the SMCP spec.
MAISTRA-1983 Migration to 2.0 Upgrading to 2.0.0 with an existing invalid ServiceMeshControlPlane cannot easily be repaired. The invalid items in the ServiceMeshControlPlane resource caused an unrecoverable error. The fix makes the errors recoverable. You can delete the invalid resource and replace it with a new one or edit the resource to fix the errors. For more information about editing your resource, see [Configuring the Red Hat OpenShift Service Mesh installation].
Maistra-1502 As a result of CVEs fixes in version 1.0.10, the Istio dashboards are not available from the Home Dashboard menu in Grafana. The Istio dashboards still exist. To access them, click the Dashboard menu in the navigation panel and select the Manage tab.
MAISTRA-858 The following Envoy log messages describing deprecated options and configurations associated with Istio 1.1.x are expected:
[2019-06-03 07:03:28.943][19][warning][misc] [external/envoy/source/common/protobuf/utility.cc:129] Using deprecated option 'envoy.api.v2.listener.Filter.config'. This configuration will be removed from Envoy soon.
[2019-08-12 22:12:59.001][13][warning][misc] [external/envoy/source/common/protobuf/utility.cc:174] Using deprecated option 'envoy.api.v2.Listener.use_original_dst' from file lds.proto. This configuration will be removed from Envoy soon.
MAISTRA-193 Unexpected console info messages are visible when health checking is enabled for citadel.
Bug 1821432 Toggle controls in OpenShift Container Platform Control Resource details page do not update the CR correctly. UI Toggle controls in the Service Mesh Control Plane (SMCP) Overview page in the OpenShift Container Platform web console sometimes update the wrong field in the resource. To update a SMCP, edit the YAML content directly or update the resource from the command line instead of clicking the toggle controls.
TRACING-1725 Follow-up to TRACING-1631. Additional fix to ensure that Elasticsearch certificates are properly reconciled when there are multiple Jaeger production instances, using same name but within different namespaces. See also BZ-1918920.
TRACING-1631 Multiple Jaeger production instances, using same name but within different namespaces, causing Elasticsearch certificate issue. When multiple service meshes were installed, all of the Jaeger Elasticsearch instances had the same Elasticsearch secret instead of individual secrets, which prevented the OpenShift Elasticsearch Operator from communicating with all of the Elasticsearch clusters.
TRACING-1300 Failed connection between Agent and Collector when using Istio sidecar. An update of the Jaeger Operator enabled TLS communication by default between a Jaeger sidecar agent and the Jaeger Collector.
TRACING-1208 Authentication "500 Internal Error" when accessing Jaeger UI. When trying to authenticate to the UI using OAuth, I get a 500 error because oauth-proxy sidecar doesn’t trust the custom CA bundle defined at installation time with the additionalTrustBundle.
TRACING-1166 It is not currently possible to use the Jaeger streaming strategy within a disconnected environment. When a Kafka cluster is being provisioned, it results in a error: Failed to pull image registry.redhat.io/amq7/amq-streams-kafka-24-rhel7@sha256:f9ceca004f1b7dccb3b82d9a8027961f9fe4104e0ed69752c0bdd8078b4a1076.