×

Knative Kafka provides integration options for you to use supported versions of the Apache Kafka message streaming platform with OpenShift Serverless. Kafka provides options for event source, channel, broker, and event sink capabilities.

In addition to the Knative Eventing components that are provided as part of a core OpenShift Serverless installation, cluster or dedicated administrators can install the KnativeKafka custom resource (CR).

The KnativeKafka CR provides users with additional options, such as:

  • Kafka source

  • Kafka channel

  • Kafka broker (Technology Preview)

  • Kafka sink (Technology Preview)

Installing Knative Kafka

Knative Kafka provides integration options for you to use supported versions of the Apache Kafka message streaming platform with OpenShift Serverless. Knative Kafka functionality is available in an OpenShift Serverless installation if you have installed the KnativeKafka custom resource.

Prerequisites
  • You have installed the OpenShift Serverless Operator and Knative Eventing on your cluster.

  • You have access to a Red Hat AMQ Streams cluster.

  • Install the OpenShift CLI (oc) if you want to use the verification steps.

  • You have cluster or dedicated administrator permissions on OpenShift Dedicated.

  • You are logged in to the OpenShift Dedicated web console.

Procedure
  1. In the Administrator perspective, navigate to OperatorsInstalled Operators.

  2. Check that the Project dropdown at the top of the page is set to Project: knative-eventing.

  3. In the list of Provided APIs for the OpenShift Serverless Operator, find the Knative Kafka box and click Create Instance.

  4. Configure the KnativeKafka object in the Create Knative Kafka page.

    To use the Kafka channel, source, broker, or sink on your cluster, you must toggle the enabled switch for the options you want to use to true. These switches are set to false by default. Additionally, to use the Kafka channel, broker, or sink you must specify the bootstrap servers.

    Example KnativeKafka custom resource
    apiVersion: operator.serverless.openshift.io/v1alpha1
    kind: KnativeKafka
    metadata:
        name: knative-kafka
        namespace: knative-eventing
    spec:
        channel:
            enabled: true (1)
            bootstrapServers: <bootstrap_servers> (2)
        source:
            enabled: true (3)
        broker:
            enabled: true (4)
            defaultConfig:
                bootstrapServers: <bootstrap_servers> (5)
                numPartitions: <num_partitions> (6)
                replicationFactor: <replication_factor> (7)
        sink:
            enabled: true (8)
    1 Enables developers to use the KafkaChannel channel type in the cluster.
    2 A comma-separated list of bootstrap servers from your AMQ Streams cluster.
    3 Enables developers to use the KafkaSource event source type in the cluster.
    4 Enables developers to use the Knative Kafka broker implementation in the cluster.
    5 A comma-separated list of bootstrap servers from your Red Hat AMQ Streams cluster.
    6 Defines the number of partitions of the Kafka topics, backed by the Broker objects. The default is 10.
    7 Defines the replication factor of the Kafka topics, backed by the Broker objects. The default is 3.
    8 Enables developers to use a Kafka sink in the cluster.

    The replicationFactor value must be less than or equal to the number of nodes of your Red Hat AMQ Streams cluster.

    1. Using the form is recommended for simpler configurations that do not require full control of KnativeKafka object creation.

    2. Editing the YAML is recommended for more complex configurations that require full control of KnativeKafka object creation. You can access the YAML by clicking the Edit YAML link in the top right of the Create Knative Kafka page.

  5. Click Create after you have completed any of the optional configurations for Kafka. You are automatically directed to the Knative Kafka tab where knative-kafka is in the list of resources.

Verification
  1. Click on the knative-kafka resource in the Knative Kafka tab. You are automatically directed to the Knative Kafka Overview page.

  2. View the list of Conditions for the resource and confirm that they have a status of True.

    Kafka Knative Overview page showing Conditions

    If the conditions have a status of Unknown or False, wait a few moments to refresh the page.

  3. Check that the Knative Kafka resources have been created:

    $ oc get pods -n knative-eventing
    Example output
    NAME                                            READY   STATUS    RESTARTS   AGE
    kafka-ch-controller-85f879d577-xcbjh            1/1     Running   0          44s
    kafka-ch-dispatcher-55d76d7db8-ggqjl            1/1     Running   0          44s
    kafka-controller-manager-bc994c465-pt7qd        1/1     Running   0          40s
    kafka-webhook-54646f474f-wr7bb                  1/1     Running   0          42s

Configuring TLS authentication for Kafka brokers

Transport Layer Security (TLS) is used by Apache Kafka clients and servers to encrypt traffic between Knative and Kafka, as well as for authentication. You can set up TLS for Kafka brokers by modifying the KnativeKafka custom resource (CR).

Prerequisites
  • You have cluster or dedicated administrator permissions on OpenShift Dedicated.

  • The OpenShift Serverless Operator, Knative Eventing, and the KnativeKafka CR are installed on your OpenShift Dedicated cluster.

  • You have created a project or have access to a project with the appropriate roles and permissions to create applications and other workloads in OpenShift Dedicated.

  • You have a Kafka cluster CA certificate stored as a .pem file.

  • You have a Kafka cluster client certificate and a key stored as .pem files.

  • Install the OpenShift (oc) CLI.

Procedure
  1. Create the certificate files as a secret in the knative-eventing namespace:

    $ oc create secret -n knative-eventing generic <secret_name> \
      --from-literal=protocol=SSL \
      --from-file=ca.crt=caroot.pem \
      --from-file=user.crt=certificate.pem \
      --from-file=user.key=key.pem

    Use the key names ca.crt, user.crt, and user.key. Do not change them.

  2. Edit the KnativeKafka CR and add a reference to your secret in the broker spec:

    apiVersion: operator.serverless.openshift.io/v1alpha1
    kind: KnativeKafka
    metadata:
      namespace: knative-eventing
      name: knative-kafka
    spec:
      broker:
        enabled: true
        defaultConfig:
          authSecretName: <secret_name>
    ...

Configuring SASL authentication for Kafka brokers

Simple Authentication and Security Layer (SASL) is used by Apache Kafka for authentication. If you use SASL authentication on your cluster, users must provide credentials to Knative for communicating with the Kafka cluster, otherwise events cannot be produced or consumed. You can set up SASL for Kafka brokers by modifying the KnativeKafka custom resource (CR).

Prerequisites
  • You have cluster or dedicated administrator permissions on OpenShift Dedicated.

  • The OpenShift Serverless Operator, Knative Eventing, and the KnativeKafka CR are installed on your OpenShift Dedicated cluster.

  • You have created a project or have access to a project with the appropriate roles and permissions to create applications and other workloads in OpenShift Dedicated.

  • You have a username and password for a Kafka cluster.

  • You have chosen the SASL mechanism to use, for example PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512.

  • If TLS is enabled, you also need the ca.crt certificate file for the Kafka cluster.

  • Install the OpenShift CLI (oc).

It is recommended to enable TLS in addition to SASL.

Procedure
  1. Create the certificate files as a secret in the knative-eventing namespace:

    $ oc create secret -n knative-eventing generic <secret_name> \
      --from-literal=protocol=SASL_SSL \
      --from-literal=sasl.mechanism=<sasl_mechanism> \
      --from-file=ca.crt=caroot.pem \
      --from-literal=password="SecretPassword" \
      --from-literal=user="my-sasl-user"

    Use the key names ca.crt, password, and sasl.mechanism. Do not change them.

  2. Edit the KnativeKafka CR and add a reference to your secret in the broker spec:

    apiVersion: operator.serverless.openshift.io/v1alpha1
    kind: KnativeKafka
    metadata:
      namespace: knative-eventing
      name: knative-kafka
    spec:
      broker:
        enabled: true
        defaultConfig:
          authSecretName: <secret_name>
    ...

Additional resources