Table of Contents

Prometheus is an open source system and service monitoring toolkit. Prometheus collects metrics from configured targets at specified intervals, evaluates rule expressions, displays the results, and can trigger alerts if some condition is observed to be true. Grafana or other API consumers can be used to visualize the collected data.

Querying Prometheus metrics

This tutorial uses Service Mesh and the bookinfo tutorial to demonstrate how you can query for metrics using Prometheus.

Prerequisites:
  • OpenShift Container Platform 3.11 or higher installed.

  • Red Hat OpenShift Service Mesh 0.12.TechPreview installed.

  • Bookinfo demonstration application installed.

After you have verified the Bookinfo application has deployed, you will need to generate calls to the application before you query for metrics.

Procedure
  1. Verify that the prometheus Service is running in your cluster. In Kubernetes environments, execute the following command:

    $ oc get svc prometheus -n istio-system

    You will see something like the following:

    NAME         CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
    prometheus   10.59.241.54   <none>        9090/TCP   2m
  2. Generate network traffic by accessing the Bookinfo application:

    $ curl -o /dev/null http://$GATEWAY_URL/productpage
  3. A route to access the Prometheus user interface already exists. Query for details of the route:

    $ export PROMETHEUS_URL=$(oc get route -n istio-system prometheus -o jsonpath='{.spec.host}')
  4. Launch a browser and navigate to http://<PROMETHEUS_URL>;. You will see the Prometheus home screen, similar to the following figure:

    ossm prometheus home screen
  5. In the Expression field, enter istio_request_duration_seconds_count, and click the Execute button. You will see a screen similar to the following figure:

    ossm prometheus metrics
  6. You can narrow down queries by using selectors. For example istio_request_duration_seconds_count{destination_workload="reviews-v2"} shows only counters with the matching destination_workload label. For more information about using queries, see the Prometheus documentation.

  7. To list all available Prometheus metrics, run the following command:

    $ oc get prometheus -n istio-system -o jsonpath='{.items[*].spec.metrics[*].name}' requests_total request_duration_seconds request_bytes response_bytes tcp_sent_bytes_total tcp_received_bytes_total

    Note that returned metric names must be prepended with istio_ when used in queries, for example, requests_total is istio_requests_total.