×

You can use Red Hat OpenShift distributed tracing with OpenShift Serverless to monitor and troubleshoot serverless applications.

Using Red Hat OpenShift distributed tracing to enable distributed tracing

Red Hat OpenShift distributed tracing is made up of several components that work together to collect, store, and display tracing data.

Prerequisites
  • You have access to an OpenShift Container Platform account with cluster administrator access.

  • You have not yet installed the OpenShift Serverless Operator, Knative Serving, and Knative Eventing. These must be installed after the Red Hat OpenShift distributed tracing installation.

  • You have installed Red Hat OpenShift distributed tracing by following the OpenShift Container Platform "Installing distributed tracing" documentation.

  • You have installed the OpenShift CLI (oc).

  • You have created a project or have access to a project with the appropriate roles and permissions to create applications and other workloads in OpenShift Container Platform.

Procedure
  1. Create an OpenTelemetryCollector custom resource (CR):

    Example OpenTelemetryCollector CR
    apiVersion: opentelemetry.io/v1alpha1
    kind: OpenTelemetryCollector
    metadata:
      name: cluster-collector
      namespace: <namespace>
    spec:
      mode: deployment
      config: |
        receivers:
          zipkin:
        processors:
        exporters:
          jaeger:
            endpoint: jaeger-all-in-one-inmemory-collector-headless.tracing-system.svc:14250
            tls:
              ca_file: "/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt"
          logging:
        service:
          pipelines:
            traces:
              receivers: [zipkin]
              processors: []
              exporters: [jaeger, logging]
  2. Verify that you have two pods running in the namespace where Red Hat OpenShift distributed tracing is installed:

    $ oc get pods -n <namespace>
    Example output
    NAME                                          READY   STATUS    RESTARTS   AGE
    cluster-collector-collector-85c766b5c-b5g99   1/1     Running   0          5m56s
    jaeger-all-in-one-inmemory-ccbc9df4b-ndkl5    2/2     Running   0          15m
  3. Verify that the following headless services have been created:

    $ oc get svc -n <namespace> | grep headless
    Example output
    cluster-collector-collector-headless            ClusterIP   None             <none>        9411/TCP                                 7m28s
    jaeger-all-in-one-inmemory-collector-headless   ClusterIP   None             <none>        9411/TCP,14250/TCP,14267/TCP,14268/TCP   16m

    These services are used to configure Jaeger, Knative Serving, and Knative Eventing. The name of the Jaeger service may vary.

  4. Install the OpenShift Serverless Operator by following the "Installing the OpenShift Serverless Operator" documentation.

  5. Install Knative Serving by creating the following KnativeServing CR:

    Example KnativeServing CR
    apiVersion: operator.knative.dev/v1beta1
    kind: KnativeServing
    metadata:
        name: knative-serving
        namespace: knative-serving
    spec:
      config:
        tracing:
          backend: "zipkin"
          zipkin-endpoint: "http://cluster-collector-collector-headless.tracing-system.svc:9411/api/v2/spans"
          debug: "false"
          sample-rate: "0.1" (1)
    1 The sample-rate defines sampling probability. Using sample-rate: "0.1" means that 1 in 10 traces are sampled.
  6. Install Knative Eventing by creating the following KnativeEventing CR:

    Example KnativeEventing CR
    apiVersion: operator.knative.dev/v1beta1
    kind: KnativeEventing
    metadata:
        name: knative-eventing
        namespace: knative-eventing
    spec:
      config:
        tracing:
          backend: "zipkin"
          zipkin-endpoint: "http://cluster-collector-collector-headless.tracing-system.svc:9411/api/v2/spans"
          debug: "false"
          sample-rate: "0.1" (1)
    1 The sample-rate defines sampling probability. Using sample-rate: "0.1" means that 1 in 10 traces are sampled.
  7. Create a Knative service:

    Example service
    apiVersion: serving.knative.dev/v1
    kind: Service
    metadata:
      name: helloworld-go
    spec:
      template:
        metadata:
          labels:
            app: helloworld-go
          annotations:
            autoscaling.knative.dev/minScale: "1"
            autoscaling.knative.dev/target: "1"
        spec:
          containers:
          - image: quay.io/openshift-knative/helloworld:v1.2
            imagePullPolicy: Always
            resources:
              requests:
                cpu: "200m"
            env:
            - name: TARGET
              value: "Go Sample v1"
  8. Make some requests to the service:

    Example HTTPS request
    $ curl https://helloworld-go.example.com
  9. Get the URL for the Jaeger web console:

    Example command
    $ oc get route jaeger-all-in-one-inmemory  -o jsonpath='{.spec.host}' -n <namespace>

    You can now examine traces by using the Jaeger console.