Installation - Distributed tracing platform (Tempo) | Distributed tracing

Installing the distributed tracing platform (Tempo) from the web console
Installing the distributed tracing platform (Tempo) by using the CLI
Additional resources

The Tempo Operator is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

Installing the distributed tracing platform (Tempo) involves the following steps:

Setting up supported object storage.
Installing the Tempo Operator.
Creating a secret for the object storage credentials.
Creating a namespace for a TempoStack instance.
Creating a TempoStack custom resource to deploy at least one TempoStack instance.

Installing the distributed tracing platform (Tempo) from the web console

You can install the distributed tracing platform (Tempo) from the Administrator view of the web console.

Prerequisites

You are logged in to the OpenShift Container Platform web console as a cluster administrator with the cluster-admin role.
For Red Hat OpenShift Dedicated, you must be logged in using an account with the dedicated-admin role.
You are using a supported provider of object storage: Red Hat OpenShift Data Foundation, MinIO, Amazon S3, Azure Blob Storage, Google Cloud Storage.

Procedure

Install the Tempo Operator:

Go to Operators → OperatorHub and search for Tempo Operator.

Select the Tempo Operator that is OpenShift Operator for Tempo → Install → Install → View Operator.

This installs the Operator with the default presets:

Update channel → stable
Installation mode → All namespaces on the cluster
Installed Namespace → openshift-tempo-operator
Update approval → Automatic

In the Details tab of the page of the installed Operator, under ClusterServiceVersion details, verify that the installation Status is Succeeded.

Create a project of your choice for the TempoStack instance that you will create in a subsequent step: go to Home → Projects → Create Project.

In the project that you created for the TempoStack instance, create a secret for your object storage bucket: go to Workloads → Secrets → Create → From YAML.

Table 1. Required secret parameters
Storage provider	Secret parameters
Red Hat OpenShift Data Foundation	`name: tempostack-dev-odf # example` `bucket: <bucket_name> # requires an ObjectBucketClaim` `endpoint: https://s3.openshift-storage.svc` `access_key_id: <data_foundation_access_key_id>` `access_key_secret: <data_foundation_access_key_secret>`
MinIO	See MinIO Operator. `name: tempostack-dev-minio # example` `bucket: <minio_bucket_name> # MinIO documentation` `endpoint: <minio_bucket_endpoint>` `access_key_id: <minio_access_key_id>` `access_key_secret: <minio_access_key_secret>`
Amazon S3	`name: tempostack-dev-s3 # example` `bucket: <s3_bucket_name> # Amazon S3 documentation` `endpoint: <s3_bucket_endpoint>` `access_key_id: <s3_access_key_id>` `access_key_secret: <s3_access_key_secret>`
Microsoft Azure Blob Storage	`name: tempostack-dev-azure # example` `container: <azure_blob_storage_container_name> # Microsoft Azure documentation` `account_name: <azure_blob_storage_account_name>` `account_key: <azure_blob_storage_account_key>`
Google Cloud Storage on Google Cloud Platform (GCP)	`name: tempostack-dev-gcs # example` `bucketname: <google_cloud_storage_bucket_name> # requires a bucket created in a GCP project` `key.json: <path/to/key.json> # requires a service account in the bucket’s GCP project for GCP authentication`

Example secret for Amazon S3 and MinIO storage

apiVersion: v1
kind: Secret
metadata:
  name: minio-test
stringData:
  endpoint: http://minio.minio.svc:9000
  bucket: tempo
  access_key_id: tempo
  access_key_secret: <secret>
type: Opaque

Create a TempoStack instance.

You can create multiple TempoStack instances in separate projects on the same cluster.

Go to Operators → Installed Operators.
Select TempoStack → Create TempoStack → YAML view.

In the YAML view, customize the TempoStack custom resource (CR):

apiVersion: tempo.grafana.com/v1alpha1
kind: TempoStack
metadata:
  name: sample
  namespace: <project_of_tempostack_instance>
spec:
  storageSize: 1Gi
  storage:
    secret:
      name: <secret-name> (1)
      type: <secret-provider> (2)
  template:
    queryFrontend:
      jaegerQuery:
        enabled: true
        ingress:
          route:
            termination: edge
          type: route

1	The value of the `name` in the `metadata` of the secret.
2	The accepted values are `azure` for Azure Blob Storage; `gcs` for Google Cloud Storage; and `s3` for Amazon S3, MinIO, or Red Hat OpenShift Data Foundation.

Example of a TempoStack CR for AWS S3 and MinIO storage

apiVersion: tempo.grafana.com/v1alpha1
kind: TempoStack
metadata:
  name: simplest
  namespace: <project_of_tempostack_instance>
spec:
  storageSize: 1Gi
  storage:
    secret:
      name: minio-test
      type: s3
  resources:
    total:
      limits:
        memory: 2Gi
        cpu: 2000m
  template:
    queryFrontend:
      jaegerQuery:
        enabled: true
        ingress:
          route:
            termination: edge
          type: route

The stack deployed in this example is configured to receive Jaeger Thrift over HTTP and OpenTelemetry Protocol (OTLP), which permits visualizing the data with the Jaeger UI.

Select Create.

Verification

Use the Project: dropdown list to select the project of the TempoStack instance.
Go to Operators → Installed Operators to verify that the Status of the TempoStack instance is Condition: Ready.
Go to Workloads → Pods to verify that all the component pods of the TempoStack instance are running.
Access the Tempo console:
1. Go to Networking → Routes and Ctrl+F to search for tempo.
2. In the Location column, open the URL to access the Tempo console.
3. Select Log In With OpenShift to use your cluster administrator credentials for the web console.
  
  The Tempo console initially shows no trace data following the Tempo console installation.

Installing the distributed tracing platform (Tempo) by using the CLI

You can install the distributed tracing platform (Tempo) from the command line.

Prerequisites

An active OpenShift CLI (oc) session by a cluster administrator with the cluster-admin role.
Ensure that your OpenShift CLI (oc) version is up to date and matches your OpenShift Container Platform version.

Run oc login:

$ oc login --username=<your_username>
You are using a supported provider of object storage: Red Hat OpenShift Data Foundation, MinIO, Amazon S3, Azure Blob Storage, Google Cloud Storage.

Procedure

Install the Tempo Operator:

Create a project for the Tempo Operator by running the following command:

$ oc apply -f - << EOF
apiVersion: project.openshift.io/v1
kind: Project
metadata:
  labels:
    kubernetes.io/metadata.name: openshift-tempo-operator
    openshift.io/cluster-monitoring: "true"
  name: openshift-tempo-operator
EOF

Create an operator group by running the following command:

$ oc apply -f - << EOF
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: openshift-tempo-operator
  namespace: openshift-tempo-operator
spec:
  upgradeStrategy: Default
EOF

Create a subscription by running the following command:

$ oc apply -f - << EOF
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: tempo-product
  namespace: openshift-tempo-operator
spec:
  channel: stable
  installPlanApproval: Automatic
  name: tempo-product
  source: redhat-operators
  sourceNamespace: openshift-marketplace
EOF

Check the operator status by running the following command:
```
$ oc get csv -n openshift-tempo-operator
```

Create a project of your choice for the TempoStack instance that you will create in a subsequent step:
- To create a project from standard input without metadata:
  $ oc new-project <project_of_tempostack_instance>
- To create a project from standard input with metadata:
  $ oc apply -f - << EOF apiVersion: project.openshift.io/v1 kind: Project metadata: name: <project_of_tempostack_instance> EOF

In the project that you created for the TempoStack instance, create a secret for your object storage bucket by running one of the following commands:

To create a secret from a YAML file:
```
$ oc apply -f <secret_file>.yaml
```

To create a secret from standard input:

$ oc apply -f - << EOF
<object_storage_secret>
EOF

Table 2. Required secret parameters
Storage provider	Secret parameters
Red Hat OpenShift Data Foundation	`name: tempostack-dev-odf # example` `bucket: <bucket_name> # requires an ObjectBucketClaim` `endpoint: https://s3.openshift-storage.svc` `access_key_id: <data_foundation_access_key_id>` `access_key_secret: <data_foundation_access_key_secret>`
MinIO	See MinIO Operator. `name: tempostack-dev-minio # example` `bucket: <minio_bucket_name> # MinIO documentation` `endpoint: <minio_bucket_endpoint>` `access_key_id: <minio_access_key_id>` `access_key_secret: <minio_access_key_secret>`
Amazon S3	`name: tempostack-dev-s3 # example` `bucket: <s3_bucket_name> # Amazon S3 documentation` `endpoint: <s3_bucket_endpoint>` `access_key_id: <s3_access_key_id>` `access_key_secret: <s3_access_key_secret>`
Microsoft Azure Blob Storage	`name: tempostack-dev-azure # example` `container: <azure_blob_storage_container_name> # Microsoft Azure documentation` `account_name: <azure_blob_storage_account_name>` `account_key: <azure_blob_storage_account_key>`
Google Cloud Storage on Google Cloud Platform (GCP)	`name: tempostack-dev-gcs # example` `bucketname: <google_cloud_storage_bucket_name> # requires a bucket created in a GCP project` `key.json: <path/to/key.json> # requires a service account in the bucket’s GCP project for GCP authentication`

Example secret for Amazon S3 and MinIO storage

apiVersion: v1
kind: Secret
metadata:
  name: minio-test
stringData:
  endpoint: http://minio.minio.svc:9000
  bucket: tempo
  access_key_id: tempo
  access_key_secret: <secret>
type: Opaque

Create a TempoStack instance in the project that you created for the TempoStack instance.

You can create multiple TempoStack instances in separate projects on the same cluster.

Customize the TempoStack custom resource (CR):

apiVersion: tempo.grafana.com/v1alpha1
kind: TempoStack
metadata:
  name: sample
  namespace: <project_of_tempostack_instance>
spec:
  storageSize: 1Gi
  storage:
      secret:
          name: <secret-name> (1)
          type: <secret-provider> (2)
  template:
    queryFrontend:
      jaegerQuery:
        enabled: true
        ingress:
          route:
            termination: edge
          type: route

1	The value of the `name` in the `metadata` of the secret.
2	The accepted values are `azure` for Azure Blob Storage; `gcs` for Google Cloud Storage; and `s3` for Amazon S3, MinIO, or Red Hat OpenShift Data Foundation.

TempoStack CR for AWS S3 and MinIO storage

apiVersion: tempo.grafana.com/v1alpha1
kind: TempoStack
metadata:
  name: simplest
  namespace: project_of_tempostack_instance
spec:
  storageSize: 1Gi
  storage:
    secret:
      name: minio-test
      type: s3
  resources:
    total:
      limits:
        memory: 2Gi
        cpu: 2000m
  template:
    queryFrontend:
      jaegerQuery:
        enabled: true
        ingress:
          route:
            termination: edge
          type: route

The stack deployed in this example is configured to receive Jaeger Thrift over HTTP and OpenTelemetry Protocol (OTLP), which permits visualizing the data with the Jaeger UI.

Apply the customized CR by running the following command.

$ oc apply -f - << EOF
<TempoStack_custom_resource>
EOF

Verification

Verify that the status of all TempoStack components is Running and the conditions are type: Ready by running the following command:
```
$ oc get tempostacks.tempo.grafana.com simplest -o yaml
```
Verify that all the TempoStack component pods are running by running the following command:
```
$ oc get pods
```
Access the Tempo console:
1. Query the route details by running the following command:
  $ export TEMPO_URL=$(oc get route -n <control_plane_namespace> tempo -o jsonpath='{.spec.host}')
2. Open https://<route_from_previous_step> in a web browser.
3. Log in using your cluster administrator credentials for the web console.
  
  The Tempo console initially shows no trace data following the Tempo console installation.

Installing the distributed tracing platform (Tempo)

Installing the distributed tracing platform (Tempo) from the web console

Installing the distributed tracing platform (Tempo) by using the CLI

Additional resources