Chapter 6. Azure Monitor Logs Ingestion API

The Azure Monitor Logs Ingestion API provides a modern, secure method for forwarding logs to Azure Monitor. This API uses Data Collection Rules (DCR) to define log schemas and routing, replacing the Data Collector API.

The azureLogsIngestion output type implements this API and offers the following advantages over the deprecated azureMonitor output:

The deprecated azureMonitor output type uses the Data Collector API. Microsoft will disable this API on September 14, 2026. Existing deployments using azureMonitor should migrate to azureLogsIngestion before this date.

Both output types can coexist during the migration period. You can configure both outputs in the same ClusterLogForwarder custom resource (CR) to validate the new configuration before removing the deprecated output.

Before you can forward logs to Azure Monitor by using the Logs Ingestion API, you must create a Data Collection Rule (DCR) in your Azure environment. The DCR defines the schema for your logs and controls which Log Analytics workspace receives the data.

The azureLogsIngestion output supports two authentication methods for {entra-id}: service principal credentials with a client secret, or {entra-id} Workload Identity. Your choice depends on your security requirements and Azure environment configuration.

When you forward logs to Azure Monitor by using the Logs Ingestion API, the log collector transforms OpenShift log records into the schema defined by your Data Collection Rule (DCR). Understanding this transformation helps you configure DCRs and query logs in Azure Monitor.

The Azure Monitor Logs Ingestion API requires logs in a JSON array format that matches the schema defined in your Data Collection Rule. Unlike other output types that accept arbitrary JSON, Azure Monitor requires you to pre-define the schema and field mappings in the DCR.

The log collector sends HTTP POST requests to the Data Collection Endpoint (DCE) with the following structure:

The DCR controls the transformation from OpenShift log record fields to Azure Monitor table columns. You define this mapping by using Kusto Query Language (KQL) in the DCR’s transformation section.

{
  "kubernetes": {
    "namespace_name": "my-app",
    "pod_name": "web-1",
    "container_name": "nginx"
  },
  "message": "Error connecting to database",
  "level": "error",
  "@timestamp": "2024-01-15T10:30:00Z"
}

The log collector transforms this into an Azure Monitor Logs Ingestion API request body:

[
  {
    "TimeGenerated": "2024-01-15T10:30:00Z",
    "Namespace": "my-app",
    "PodName": "web-1",
    "ContainerName": "nginx",
    "LogMessage": "Error connecting to database",
    "LogLevel": "error",
    "RawLog": "{\"kubernetes\":{\"namespace_name\":\"my-app\",\"pod_name\":\"web-1\",\"container_name\":\"nginx\"},\"message\":\"Error connecting to database\",\"level\":\"error\",\"@timestamp\":\"2024-01-15T10:30:00Z\"}"
  }
]

You define the transformation from OpenShift fields to Azure columns by using KQL in the DCR:

source
| extend TimeGenerated = todatetime(['@timestamp'])
| extend Namespace = tostring(kubernetes.namespace_name)
| extend PodName = tostring(kubernetes.pod_name)
| extend ContainerName = tostring(kubernetes.container_name)
| extend LogMessage = tostring(message)
| extend LogLevel = tostring(level)
| project TimeGenerated, Namespace, PodName, ContainerName, LogMessage, LogLevel

The DCR transformation extracts nested fields from the OpenShift log record structure and maps them to flat column names in the Azure Monitor table. The TimeGenerated field is typically required and must be mapped from the log record’s @timestamp field.