Chapter 1. LokiStack dashboards
Use the LokiStack dashboards to monitor write throughput, read performance, chunk storage, and retention in your logging deployment.
1.1. Prerequisites
-
You have created a
LokiStackcustom resource.
1.2. LokiStack dashboards
When you deploy a LokiStack custom resource, the Loki Operator automatically creates four Grafana dashboards that provide observability into Loki performance and resource usage. These dashboards help you monitor ingestion rates, query performance, storage operations, and retention policies.
1.2.1. Access the LokiStack dashboards
The dashboards are available in the OpenShift Container Platform web console. Navigate to Observe
1.2.2. Writes dashboard
The LokiStack Writes dashboard shows log ingestion and write-path performance metrics:
- Write QPS (queries per second): The rate of log ingestion requests to the distributor and ingester components.
- Write latency: The time to process and acknowledge log writes.
- Ingester metrics: Memory usage, active streams, and flush operations.
- Distributor metrics: Request rate, batch size, and forwarding latency.
Use this dashboard to identify write-path bottlenecks, such as distributor overload or ingester memory pressure.
1.2.3. Reads dashboard
The LokiStack Reads dashboard shows log query and read-path performance metrics:
- Read QPS (queries per second): The rate of query requests through the gateway, query front-end, and querier components.
- Query latency: The time to run queries at each component (p50, p99, p999 percentiles).
- Cache hit rates: Effectiveness of query result and chunk caching.
- Querier metrics: Active queries, queue depth, and resource usage.
This dashboard provides QPS metrics for sizing decisions. The LokiStack sizing table shows average QPS at 200ms query latency, which corresponds to the query front-end latency metrics in this dashboard.
Use this dashboard to identify read-path bottlenecks, such as slow queries, insufficient querier replicas, or poor cache performance.
1.2.4. Chunks dashboard
The LokiStack Chunks dashboard shows chunk storage and compaction metrics:
- Chunk operations: Upload rate, download rate, and operation latency to object storage
- Chunk cache: Hit rates and eviction rates for the chunk cache
- Compactor metrics: Compaction job duration, objects processed, and delete operations
- Storage metrics: Object storage request rates and error rates
Use this dashboard to monitor object storage performance and identify storage-related bottlenecks or misconfigurations.
1.2.5. Retention dashboard
The LokiStack Retention dashboard shows log retention and deletion metrics:
- Retention period: Configured and actual retention for each tenant
- Delete operations: Rate of log deletion and compaction cleanup
- Storage reclamation: Space freed by retention policy enforcement
- Compactor status: Active compaction jobs and completion times
Use this dashboard to verify that retention policies are working as expected and to monitor storage growth trends.
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}