Cloud native applications such as virtual RAN (vRAN) require access to notifications about hardware timing events that are critical to the functioning of the overall network. Precision Time Protocol (PTP) clock synchronization errors can negatively affect the performance and reliability of your low-latency application, for example, a vRAN application running in a distributed unit (DU).
Loss of PTP synchronization is a critical error for a RAN network. If synchronization is lost on a node, the radio might be shut down and the network Over the Air (OTA) traffic might be shifted to another node in the wireless network. Fast event notifications mitigate against workload errors by allowing cluster nodes to communicate PTP clock sync status to the vRAN application running in the DU.
Event notifications are available to vRAN applications running on the same DU node. A publish/subscribe REST API passes events notifications to the messaging bus. Publish/subscribe messaging, or pub-sub messaging, is an asynchronous service-to-service communication architecture where any message published to a topic is immediately received by all of the subscribers to the topic.
The PTP Operator generates fast event notifications for every PTP-capable network interface. You can access the events by using a cloud-event-proxy
sidecar container over an HTTP or Advanced Message Queuing Protocol (AMQP) message bus.
PTP fast event notifications are available for network interfaces configured to use PTP ordinary clocks, PTP grandmaster clocks, or PTP boundary clocks. |
HTTP transport is the default transport for PTP and bare-metal events. Use HTTP transport instead of AMQP for PTP and bare-metal events where possible. AMQ Interconnect is EOL from 30 June 2024. Extended life cycle support (ELS) for AMQ Interconnect ends 29 November 2029. For more information see, Red Hat AMQ Interconnect support status. |
Use the Precision Time Protocol (PTP) fast event notifications framework to subscribe cluster applications to PTP events that the bare-metal cluster node generates.
The fast events notifications framework uses a REST API for communication. The REST API is based on the O-RAN O-Cloud Notification API Specification for Event Consumers 3.0 that is available from O-RAN ALLIANCE Specifications. |
The framework consists of a publisher, subscriber, and an AMQ or HTTP messaging protocol to handle communications between the publisher and subscriber applications.
Applications run the cloud-event-proxy
container in a sidecar pattern to subscribe to PTP events.
The cloud-event-proxy
sidecar container can access the same resources as the primary application container without using any of the resources of the primary application and with no significant latency.
HTTP transport is the default transport for PTP and bare-metal events. Use HTTP transport instead of AMQP for PTP and bare-metal events where possible. AMQ Interconnect is EOL from 30 June 2024. Extended life cycle support (ELS) for AMQ Interconnect ends 29 November 2029. For more information see, Red Hat AMQ Interconnect support status. |
linuxptp-daemon
in the PTP Operator-managed pod runs as a Kubernetes DaemonSet
and manages the various linuxptp
processes (ptp4l
, phc2sys
, and optionally for grandmaster clocks, ts2phc
).
The linuxptp-daemon
passes the event to the UNIX domain socket.
The PTP plugin reads the event from the UNIX domain socket and passes it to the cloud-event-proxy
sidecar in the PTP Operator-managed pod.
cloud-event-proxy
delivers the event from the Kubernetes infrastructure to Cloud-Native Network Functions (CNFs) with low latency.
The cloud-event-proxy
sidecar in the PTP Operator-managed pod processes the event and publishes the cloud-native event by using a REST API.
The message transporter transports the event to the cloud-event-proxy
sidecar in the application pod over HTTP or AMQP 1.0 QPID.
The cloud-event-proxy
sidecar in the Application pod processes the event and makes it available by using the REST API.
The consumer application sends an API request to the cloud-event-proxy
sidecar in the application pod to create a PTP events subscription.
The cloud-event-proxy
sidecar creates an AMQ or HTTP messaging listener protocol for the resource specified in the subscription.
The cloud-event-proxy
sidecar in the application pod receives the event from the PTP Operator-managed pod, unwraps the cloud events object to retrieve the data, and posts the event to the consumer application.
The consumer application listens to the address specified in the resource qualifier and receives and processes the PTP event.
To start using PTP fast event notifications for a network interface in your cluster, you must enable the fast event publisher in the PTP Operator PtpOperatorConfig
custom resource (CR) and configure ptpClockThreshold
values in a PtpConfig
CR that you create.
You have installed the OpenShift Container Platform CLI (oc
).
You have logged in as a user with cluster-admin
privileges.
You have installed the PTP Operator.
Modify the default PTP Operator config to enable PTP fast events.
Save the following YAML in the ptp-operatorconfig.yaml
file:
apiVersion: ptp.openshift.io/v1
kind: PtpOperatorConfig
metadata:
name: default
namespace: openshift-ptp
spec:
daemonNodeSelector:
node-role.kubernetes.io/worker: ""
ptpEventConfig:
enableEventPublisher: true (1)
1 | Set enableEventPublisher to true to enable PTP fast event notifications. |
In OpenShift Container Platform 4.13 or later, you do not need to set the |
Update the PtpOperatorConfig
CR:
$ oc apply -f ptp-operatorconfig.yaml
Create a PtpConfig
custom resource (CR) for the PTP enabled interface, and set the required values for ptpClockThreshold
and ptp4lOpts
.
The following YAML illustrates the required values that you must set in the PtpConfig
CR:
spec:
profile:
- name: "profile1"
interface: "enp5s0f0"
ptp4lOpts: "-2 -s --summary_interval -4" (1)
phc2sysOpts: "-a -r -m -n 24 -N 8 -R 16" (2)
ptp4lConf: "" (3)
ptpClockThreshold: (4)
holdOverTimeout: 5
maxOffsetThreshold: 100
minOffsetThreshold: -100
1 | Append --summary_interval -4 to use PTP fast events. |
2 | Required phc2sysOpts values. -m prints messages to stdout . The linuxptp-daemon DaemonSet parses the logs and generates Prometheus metrics. |
3 | Specify a string that contains the configuration to replace the default /etc/ptp4l.conf file. To use the default configuration, leave the field empty. |
4 | Optional. If the ptpClockThreshold stanza is not present, default values are used for the ptpClockThreshold fields. The stanza shows default ptpClockThreshold values. The ptpClockThreshold values configure how long after the PTP master clock is disconnected before PTP events are triggered. holdOverTimeout is the time value in seconds before the PTP clock event state changes to FREERUN when the PTP master clock is disconnected. The maxOffsetThreshold and minOffsetThreshold settings configure offset values in nanoseconds that compare against the values for CLOCK_REALTIME (phc2sys ) or master offset (ptp4l ). When the ptp4l or phc2sys offset value is outside this range, the PTP clock state is set to FREERUN . When the offset value is within this range, the PTP clock state is set to LOCKED . |
For a complete example CR that configures linuxptp
services as an ordinary clock with PTP fast events, see Configuring linuxptp services as ordinary clock.
If you have previously deployed PTP or bare-metal events consumer applications, you need to update the applications to use HTTP message transport.
You have installed the OpenShift CLI (oc
).
You have logged in as a user with cluster-admin
privileges.
You have updated the PTP Operator or Bare Metal Event Relay to version 4.13+ which uses HTTP transport by default.
Update your events consumer application to use HTTP transport.
Set the http-event-publishers
variable for the cloud event sidecar deployment.
For example, in a cluster with PTP events configured, the following YAML snippet illustrates a cloud event sidecar deployment:
containers:
- name: cloud-event-sidecar
image: cloud-event-sidecar
args:
- "--metrics-addr=127.0.0.1:9091"
- "--store-path=/store"
- "--transport-host=consumer-events-subscription-service.cloud-events.svc.cluster.local:9043"
- "--http-event-publishers=ptp-event-publisher-service-NODE_NAME.openshift-ptp.svc.cluster.local:9043" (1)
- "--api-port=8089"
1 | The PTP Operator automatically resolves NODE_NAME to the host that is generating the PTP events.
For example, compute-1.example.com . |
In a cluster with bare-metal events configured, set the http-event-publishers
field to hw-event-publisher-service.openshift-bare-metal-events.svc.cluster.local:9043
in the cloud event sidecar deployment CR.
Deploy the consumer-events-subscription-service
service alongside the events consumer application.
For example:
apiVersion: v1
kind: Service
metadata:
annotations:
prometheus.io/scrape: "true"
service.alpha.openshift.io/serving-cert-secret-name: sidecar-consumer-secret
name: consumer-events-subscription-service
namespace: cloud-events
labels:
app: consumer-service
spec:
ports:
- name: sub-port
port: 9043
selector:
app: consumer
clusterIP: None
sessionAffinity: None
type: ClusterIP
To pass PTP fast event notifications between publisher and subscriber on a node, you can install and configure an AMQ messaging bus to run locally on the node. To use AMQ messaging, you must install the AMQ Interconnect Operator.
HTTP transport is the default transport for PTP and bare-metal events. Use HTTP transport instead of AMQP for PTP and bare-metal events where possible. AMQ Interconnect is EOL from 30 June 2024. Extended life cycle support (ELS) for AMQ Interconnect ends 29 November 2029. For more information see, Red Hat AMQ Interconnect support status. |
Install the OpenShift Container Platform CLI (oc
).
Log in as a user with cluster-admin
privileges.
Install the AMQ Interconnect Operator to its own amq-interconnect
namespace. See Adding the Red Hat Integration - AMQ Interconnect Operator.
Check that the AMQ Interconnect Operator is available and the required pods are running:
$ oc get pods -n amq-interconnect
NAME READY STATUS RESTARTS AGE
amq-interconnect-645db76c76-k8ghs 1/1 Running 0 23h
interconnect-operator-5cb5fc7cc-4v7qm 1/1 Running 0 23h
Check that the required linuxptp-daemon
PTP event producer pods are running in the openshift-ptp
namespace.
$ oc get pods -n openshift-ptp
NAME READY STATUS RESTARTS AGE
linuxptp-daemon-2t78p 3/3 Running 0 12h
linuxptp-daemon-k8n88 3/3 Running 0 12h
Subscribe applications to PTP events by using the resource address /cluster/node/<node_name>/ptp
, where <node_name>
is the cluster node running the DU application.
Deploy your cloud-event-consumer
DU application container and cloud-event-proxy
sidecar container in a separate DU application pod. The cloud-event-consumer
DU application subscribes to the cloud-event-proxy
container in the application pod.
Use the following API endpoints to subscribe the cloud-event-consumer
DU application to PTP events posted by the cloud-event-proxy
container at http://localhost:8089/api/ocloudNotifications/v1/
in the DU application pod:
/api/ocloudNotifications/v1/subscriptions
POST
: Creates a new subscription
GET
: Retrieves a list of subscriptions
DELETE
: Deletes all subscriptions
/api/ocloudNotifications/v1/subscriptions/{subscription_id}
GET
: Returns details for the specified subscription ID
DELETE
: Deletes the subscription associated with the specified subscription ID
/api/ocloudNotifications/v1/health
GET
: Returns the health status of ocloudNotifications
API
api/ocloudNotifications/v1/publishers
GET
: Returns an array of os-clock-sync-state
, ptp-clock-class-change
, lock-state
, and gnss-sync-status
messages for the cluster node
/api/ocloudnotifications/v1/{resource_address}/CurrentState
GET
: Returns the current state of one the following event types: os-clock-sync-state
, ptp-clock-class-change
, lock-state
, or gnss-state-change
events
|
Use the PTP event notifications REST API to subscribe a cluster application to the PTP events that are generated on the parent node.
GET api/ocloudNotifications/v1/subscriptions
Returns a list of subscriptions. If subscriptions exist, a 200 OK
status code is returned along with the list of subscriptions.
[
{
"id": "75b1ad8f-c807-4c23-acf5-56f4b7ee3826",
"endpointUri": "http://localhost:9089/event",
"uriLocation": "http://localhost:8089/api/ocloudNotifications/v1/subscriptions/75b1ad8f-c807-4c23-acf5-56f4b7ee3826",
"resource": "/cluster/node/compute-1.example.com/ptp"
}
]
POST api/ocloudNotifications/v1/subscriptions
Creates a new subscription. If a subscription is successfully created, or if it already exists, a 201 Created
status code is returned.
Parameter | Type |
---|---|
subscription |
data |
{
"uriLocation": "http://localhost:8089/api/ocloudNotifications/v1/subscriptions",
"resource": "/cluster/node/compute-1.example.com/ptp"
}
DELETE api/ocloudNotifications/v1/subscriptions
Deletes all subscriptions.
{
"status": "deleted all subscriptions"
}
GET api/ocloudNotifications/v1/subscriptions/{subscription_id}
Returns details for the subscription with ID subscription_id
.
Parameter | Type |
---|---|
|
string |
{
"id":"48210fb3-45be-4ce0-aa9b-41a0e58730ab",
"endpointUri": "http://localhost:9089/event",
"uriLocation":"http://localhost:8089/api/ocloudNotifications/v1/subscriptions/48210fb3-45be-4ce0-aa9b-41a0e58730ab",
"resource":"/cluster/node/compute-1.example.com/ptp"
}
DELETE api/ocloudNotifications/v1/subscriptions/{subscription_id}
Deletes the subscription with ID subscription_id
.
Parameter | Type |
---|---|
|
string |
{
"status": "OK"
}
GET api/ocloudNotifications/v1/health/
Returns the health status for the ocloudNotifications
REST API.
OK
GET api/ocloudNotifications/v1/publishers
Returns an array of os-clock-sync-state
, ptp-clock-class-change
, lock-state
, and gnss-sync-status
details for the cluster node.
The system generates notifications when the relevant equipment state changes.
os-clock-sync-state
notifications describe the host operating system clock synchronization state. Can be in LOCKED
or FREERUN
state.
ptp-clock-class-change
notifications describe the current state of the PTP clock class.
lock-state
notifications describe the current status of the PTP equipment lock state. Can be in LOCKED
, HOLDOVER
or FREERUN
state.
gnss-sync-status
notifications describe the GPS synchronization state with regard to the external GNSS clock signal. Can be in LOCKED
or FREERUN
state.
You can use equipment synchronization status subscriptions together to deliver a detailed view of the overall synchronization health of the system.
[
{
"id": "0fa415ae-a3cf-4299-876a-589438bacf75",
"endpointUri": "http://localhost:9085/api/ocloudNotifications/v1/dummy",
"uriLocation": "http://localhost:9085/api/ocloudNotifications/v1/publishers/0fa415ae-a3cf-4299-876a-589438bacf75",
"resource": "/cluster/node/compute-1.example.com/sync/sync-status/os-clock-sync-state"
},
{
"id": "28cd82df-8436-4f50-bbd9-7a9742828a71",
"endpointUri": "http://localhost:9085/api/ocloudNotifications/v1/dummy",
"uriLocation": "http://localhost:9085/api/ocloudNotifications/v1/publishers/28cd82df-8436-4f50-bbd9-7a9742828a71",
"resource": "/cluster/node/compute-1.example.com/sync/ptp-status/ptp-clock-class-change"
},
{
"id": "44aa480d-7347-48b0-a5b0-e0af01fa9677",
"endpointUri": "http://localhost:9085/api/ocloudNotifications/v1/dummy",
"uriLocation": "http://localhost:9085/api/ocloudNotifications/v1/publishers/44aa480d-7347-48b0-a5b0-e0af01fa9677",
"resource": "/cluster/node/compute-1.example.com/sync/ptp-status/lock-state"
},
{
"id": "778da345d-4567-67b0-a43f0-rty885a456",
"endpointUri": "http://localhost:9085/api/ocloudNotifications/v1/dummy",
"uriLocation": "http://localhost:9085/api/ocloudNotifications/v1/publishers/778da345d-4567-67b0-a43f0-rty885a456",
"resource": "/cluster/node/compute-1.example.com/sync/gnss-status/gnss-sync-status"
}
]
You can find os-clock-sync-state
, ptp-clock-class-change
, lock-state
, and gnss-sync-status
events in the logs for the cloud-event-proxy
container. For example:
$ oc logs -f linuxptp-daemon-cvgr6 -n openshift-ptp -c cloud-event-proxy
{
"id":"c8a784d1-5f4a-4c16-9a81-a3b4313affe5",
"type":"event.sync.sync-status.os-clock-sync-state-change",
"source":"/cluster/compute-1.example.com/ptp/CLOCK_REALTIME",
"dataContentType":"application/json",
"time":"2022-05-06T15:31:23.906277159Z",
"data":{
"version":"v1",
"values":[
{
"resource":"/sync/sync-status/os-clock-sync-state",
"dataType":"notification",
"valueType":"enumeration",
"value":"LOCKED"
},
{
"resource":"/sync/sync-status/os-clock-sync-state",
"dataType":"metric",
"valueType":"decimal64.3",
"value":"-53"
}
]
}
}
{
"id":"69eddb52-1650-4e56-b325-86d44688d02b",
"type":"event.sync.ptp-status.ptp-clock-class-change",
"source":"/cluster/compute-1.example.com/ptp/ens2fx/master",
"dataContentType":"application/json",
"time":"2022-05-06T15:31:23.147100033Z",
"data":{
"version":"v1",
"values":[
{
"resource":"/sync/ptp-status/ptp-clock-class-change",
"dataType":"metric",
"valueType":"decimal64.3",
"value":"135"
}
]
}
}
{
"id":"305ec18b-1472-47b3-aadd-8f37933249a9",
"type":"event.sync.ptp-status.ptp-state-change",
"source":"/cluster/compute-1.example.com/ptp/ens2fx/master",
"dataContentType":"application/json",
"time":"2022-05-06T15:31:23.467684081Z",
"data":{
"version":"v1",
"values":[
{
"resource":"/sync/ptp-status/lock-state",
"dataType":"notification",
"valueType":"enumeration",
"value":"LOCKED"
},
{
"resource":"/sync/ptp-status/lock-state",
"dataType":"metric",
"valueType":"decimal64.3",
"value":"62"
}
]
}
}
{
"id": "435e1f2a-6854-4555-8520-767325c087d7",
"type": "event.sync.gnss-status.gnss-state-change",
"source": "/cluster/node/compute-1.example.com/sync/gnss-status/gnss-sync-status",
"dataContentType": "application/json",
"time": "2023-09-27T19:35:33.42347206Z",
"data": {
"version": "v1",
"values": [
{
"resource": "/cluster/node/compute-1.example.com/ens2fx/master",
"dataType": "notification",
"valueType": "enumeration",
"value": "LOCKED"
},
{
"resource": "/cluster/node/compute-1.example.com/ens2fx/master",
"dataType": "metric",
"valueType": "decimal64.3",
"value": "5"
}
]
}
}
GET api/ocloudNotifications/v1/cluster/node/<node_name>/sync/ptp-status/lock-state/CurrentState
GET api/ocloudNotifications/v1/cluster/node/<node_name>/sync/sync-status/os-clock-sync-state/CurrentState
GET api/ocloudNotifications/v1/cluster/node/<node_name>/sync/ptp-status/ptp-clock-class-change/CurrentState
Configure the CurrentState
API endpoint to return the current state of the os-clock-sync-state
, ptp-clock-class-change
, lock-state
events for the cluster node.
os-clock-sync-state
notifications describe the host operating system clock synchronization state. Can be in LOCKED
or FREERUN
state.
ptp-clock-class-change
notifications describe the current state of the PTP clock class.
lock-state
notifications describe the current status of the PTP equipment lock state. Can be in LOCKED
, HOLDOVER
or FREERUN
state.
Parameter | Type |
---|---|
|
string |
{
"id": "c1ac3aa5-1195-4786-84f8-da0ea4462921",
"type": "event.sync.ptp-status.ptp-state-change",
"source": "/cluster/node/compute-1.example.com/sync/ptp-status/lock-state",
"dataContentType": "application/json",
"time": "2023-01-10T02:41:57.094981478Z",
"data": {
"version": "v1",
"values": [
{
"resource": "/cluster/node/compute-1.example.com/ens5fx/master",
"dataType": "notification",
"valueType": "enumeration",
"value": "LOCKED"
},
{
"resource": "/cluster/node/compute-1.example.com/ens5fx/master",
"dataType": "metric",
"valueType": "decimal64.3",
"value": "29"
}
]
}
}
{
"specversion": "0.3",
"id": "4f51fe99-feaa-4e66-9112-66c5c9b9afcb",
"source": "/cluster/node/compute-1.example.com/sync/sync-status/os-clock-sync-state",
"type": "event.sync.sync-status.os-clock-sync-state-change",
"subject": "/cluster/node/compute-1.example.com/sync/sync-status/os-clock-sync-state",
"datacontenttype": "application/json",
"time": "2022-11-29T17:44:22.202Z",
"data": {
"version": "v1",
"values": [
{
"resource": "/cluster/node/compute-1.example.com/CLOCK_REALTIME",
"dataType": "notification",
"valueType": "enumeration",
"value": "LOCKED"
},
{
"resource": "/cluster/node/compute-1.example.com/CLOCK_REALTIME",
"dataType": "metric",
"valueType": "decimal64.3",
"value": "27"
}
]
}
}
{
"id": "064c9e67-5ad4-4afb-98ff-189c6aa9c205",
"type": "event.sync.ptp-status.ptp-clock-class-change",
"source": "/cluster/node/compute-1.example.com/sync/ptp-status/ptp-clock-class-change",
"dataContentType": "application/json",
"time": "2023-01-10T02:41:56.785673989Z",
"data": {
"version": "v1",
"values": [
{
"resource": "/cluster/node/compute-1.example.com/ens5fx/master",
"dataType": "metric",
"valueType": "decimal64.3",
"value": "165"
}
]
}
}
You can monitor PTP fast events metrics from cluster nodes where the linuxptp-daemon
is running.
You can also monitor PTP fast event metrics in the OpenShift Container Platform web console by using the preconfigured and self-updating Prometheus monitoring stack.
Install the OpenShift Container Platform CLI oc
.
Log in as a user with cluster-admin
privileges.
Install and configure the PTP Operator on a node with PTP-capable hardware.
Start a debug pod for the node by running the following command:
$ oc debug node/<node_name>
Check for PTP metrics exposed by the linuxptp-daemon
container. For example, run the following command:
sh-4.4# curl http://localhost:9091/metrics
# HELP cne_api_events_published Metric to get number of events published by the rest api # TYPE cne_api_events_published gauge cne_api_events_published{address="/cluster/node/compute-1.example.com/sync/gnss-status/gnss-sync-status",status="success"} 1 cne_api_events_published{address="/cluster/node/compute-1.example.com/sync/ptp-status/lock-state",status="success"} 94 cne_api_events_published{address="/cluster/node/compute-1.example.com/sync/ptp-status/ptp-clock-class-change",status="success"} 18 cne_api_events_published{address="/cluster/node/compute-1.example.com/sync/sync-status/os-clock-sync-state",status="success"} 27
To view the PTP event in the OpenShift Container Platform web console, copy the name of the PTP metric you want to query, for example, openshift_ptp_offset_ns
.
In the OpenShift Container Platform web console, click Observe → Metrics.
Paste the PTP metric name into the Expression field, and click Run queries.
The following table describes the PTP fast events metrics that are available from cluster nodes where the linuxptp-daemon
service is running.
Metric | Description | Example |
---|---|---|
|
Returns the PTP clock class for the interface.
Possible values for PTP clock class are 6 ( |
|
|
Returns the current PTP clock state for the interface.
Possible values for PTP clock state are |
|
|
Returns the delay in nanoseconds between the primary clock sending the timing packet and the secondary clock receiving the timing packet. |
|
|
Returns the current status of the highly available system clock when there are multiple time sources on different NICs.
Possible values are 0 ( |
|
|
Returns the frequency adjustment in nanoseconds between 2 PTP clocks.
For example, between the upstream clock and the NIC, between the system clock and the NIC, or between the PTP hardware clock ( |
|
|
Returns the configured PTP clock role for the interface.
Possible values are 0 ( |
|
|
Returns the maximum offset in nanoseconds between 2 clocks or interfaces.
For example, between the upstream GNSS clock and the NIC ( |
|
|
Returns the offset in nanoseconds between the DPLL clock or the GNSS clock source and the NIC hardware clock. |
|
|
Returns a count of the number of times the |
|
|
Returns a status code that shows whether the PTP processes are running or not. |
|
|
Returns values for
|
|
The following table describes the PTP fast event metrics that are available only when PTP grandmaster clock (T-GM) is enabled.
Metric | Description | Example |
---|---|---|
|
Returns the current status of the digital phase-locked loop (DPLL) frequency for the NIC.
Possible values are -1 ( |
|
|
Returns the current status of the NMEA connection.
NMEA is the protocol that is used for 1PPS NIC connections.
Possible values are 0 ( |
|
|
Returns the status of the DPLL phase for the NIC.
Possible values are -1 ( |
|
|
Returns the current status of the NIC 1PPS connection.
You use the 1PPS connection to synchronize timing between connected NICs.
Possible values are 0 ( |
|
|
Returns the current status of the global navigation satellite system (GNSS) connection.
GNSS provides satellite-based positioning, navigation, and timing services globally.
Possible values are 0 ( |
|