Chapter 7. Forwarding telemetry data

Procedure

1. Create a service account for the OpenTelemetry Collector.

   Example ServiceAccount

   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: otel-collector-deployment

2. Create a cluster role for the service account.

   Example ClusterRole

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRole
   metadata:
     name: otel-collector
   rules:
     1
     2
   - apiGroups: ["", "config.openshift.io"]
     resources: ["pods", "namespaces", "infrastructures", "infrastructures/status"]
     verbs: ["get", "watch", "list"]

   1

   The k8sattributesprocessor requires permissions for pods and namespaces resources.

   2

   The resourcedetectionprocessor requires permissions for infrastructures and status.

3. Bind the cluster role to the service account.

   Example ClusterRoleBinding

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
     name: otel-collector
   subjects:
   - kind: ServiceAccount
     name: otel-collector-deployment
     namespace: otel-collector-example
   roleRef:
     kind: ClusterRole
     name: otel-collector
     apiGroup: rbac.authorization.k8s.io

4. Create the YAML file to define the OpenTelemetryCollector custom resource (CR).

   Example OpenTelemetryCollector

   apiVersion: opentelemetry.io/v1beta1
   kind: OpenTelemetryCollector
   metadata:
     name: otel
   spec:
     mode: deployment
     serviceAccount: otel-collector-deployment
     config:
       receivers:
         jaeger:
           protocols:
             grpc: {}
             thrift_binary: {}
             thrift_compact: {}
             thrift_http: {}
         opencensus: {}
         otlp:
           protocols:
             grpc: {}
             http: {}
         zipkin: {}
       processors:
         batch: {}
         k8sattributes: {}
         memory_limiter:
           check_interval: 1s
           limit_percentage: 50
           spike_limit_percentage: 30
         resourcedetection:
           detectors: [openshift]
       exporters:
         otlp:
           endpoint: "tempo-simplest-distributor:4317" 1
           tls:
             insecure: true
       service:
         pipelines:
           traces:
             receivers: [jaeger, opencensus, otlp, zipkin] 2
             processors: [memory_limiter, k8sattributes, resourcedetection, batch]
             exporters: [otlp]

   1

   The Collector exporter is configured to export OTLP and points to the Tempo distributor endpoint, "tempo-simplest-distributor:4317" in this example, which is already created.

   2

   The Collector is configured with a receiver for Jaeger traces, OpenCensus traces over the OpenCensus protocol, Zipkin traces over the Zipkin protocol, and OTLP traces over the gRPC protocol.

Procedure

1. Create a service account for the OpenTelemetry Collector.

   Example ServiceAccount object

   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: otel-collector-deployment
     namespace: openshift-logging

2. Create a cluster role that grants the Collector’s service account the permissions to push logs to the LokiStack application tenant.

   Example ClusterRole object

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRole
   metadata:
     name: otel-collector-logs-writer
   rules:
    - apiGroups: ["loki.grafana.com"]
      resourceNames: ["logs"]
      resources: ["application"]
      verbs: ["create"]
    - apiGroups: [""]
      resources: ["pods", "namespaces", "nodes"]
      verbs: ["get", "watch", "list"]
    - apiGroups: ["apps"]
      resources: ["replicasets"]
      verbs: ["get", "list", "watch"]
    - apiGroups: ["extensions"]
      resources: ["replicasets"]
      verbs: ["get", "list", "watch"]

3. Bind the cluster role to the service account.

   Example ClusterRoleBinding object

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
     name: otel-collector-logs-writer
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: ClusterRole
     name: otel-collector-logs-writer
   subjects:
     - kind: ServiceAccount
       name: otel-collector-deployment
       namespace: openshift-logging

4. Create an OpenTelemetryCollector custom resource (CR) object.

   Example OpenTelemetryCollector CR object

   apiVersion: opentelemetry.io/v1beta1
   kind: OpenTelemetryCollector
   metadata:
     name: otel
     namespace: openshift-logging
   spec:
     serviceAccount: otel-collector-deployment
     config:
       extensions:
         bearertokenauth:
           filename: "/var/run/secrets/kubernetes.io/serviceaccount/token"
       receivers:
         otlp:
           protocols:
             grpc: {}
             http: {}
       processors:
         k8sattributes: {}
         resource:
           attributes: 1
             - key:  kubernetes.namespace_name
               from_attribute: k8s.namespace.name
               action: upsert
             - key:  kubernetes.pod_name
               from_attribute: k8s.pod.name
               action: upsert
             - key: kubernetes.container_name
               from_attribute: k8s.container.name
               action: upsert
             - key: log_type
               value: application
               action: upsert
         transform:
           log_statements:
             - context: log
               statements:
                 - set(attributes["level"], ConvertCase(severity_text, "lower"))
       exporters:
         otlphttp:
           endpoint: https://logging-loki-gateway-http.openshift-logging.svc.cluster.local:8080/api/logs/v1/application/otlp
           encoding: json
           tls:
             ca_file: "/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt"
           auth:
             authenticator: bearertokenauth
         debug:
           verbosity: detailed
       service:
         extensions: [bearertokenauth] 2
         pipelines:
           logs:
             receivers: [otlp]
             processors: [k8sattributes, transform, resource]
             exporters: [otlphttp] 3
           logs/test:
             receivers: [otlp]
             processors: []
             exporters: [debug]

   1

   Provides the following resource attributes to be used by the web console: kubernetes.namespace_name, kubernetes.pod_name, kubernetes.container_name, and log_type.

   2

   Enables the BearerTokenAuth Extension that is required by the OTLP HTTP Exporter.

   3

   Enables the OTLP HTTP Exporter to export logs from the Collector.