Release notes | Serverless | OpenShift Container Platform 4.8

Release notes contain information about new and deprecated features, breaking changes, and known issues. The following release notes apply for the most recent OpenShift Serverless releases on OpenShift Container Platform.

For an overview of OpenShift Serverless functionality, see About OpenShift Serverless.

OpenShift Serverless is based on the open source Knative project.

For details about the latest Knative component releases, see the Knative blog.

About API versions

API versions are an important measure of the development status of certain features and custom resources in OpenShift Serverless. Creating resources on your cluster that do not use the correct API version can cause issues in your deployment.

The OpenShift Serverless Operator automatically upgrades older resources that use deprecated versions of APIs to use the latest version. For example, if you have created resources on your cluster that use older versions of the ApiServerSource API, such as v1beta1, the OpenShift Serverless Operator automatically updates these resources to use the v1 version of the API when this is available and the v1beta1 version is deprecated.

After they have been deprecated, older versions of APIs might be removed in any upcoming release. Using deprecated versions of APIs does not cause resources to fail. However, if you try to use a version of an API that has been removed, it will cause resources to fail. Ensure that your manifests are updated to use the latest version to avoid issues.

Generally Available and Technology Preview features

Features which are Generally Available (GA) are fully supported and are suitable for production use. Technology Preview (TP) features are experimental features and are not intended for production use. See the Technology Preview scope of support on the Red Hat Customer Portal for more information about TP features.

The following table provides information about which OpenShift Serverless features are GA and which are TP:

Table 1. Generally Available and Technology Preview features tracker
Feature	1.20	1.21	1.22	1.23
`kn func`	TP	TP	TP	TP
`kn func invoke`	-	TP	TP	TP
Service Mesh mTLS	GA	GA	GA	GA
`emptyDir` volumes	GA	GA	GA	GA
HTTPS redirection	GA	GA	GA	GA
Kafka broker	TP	TP	TP	TP
Kafka sink	-	TP	TP	TP
Init containers support for Knative services	-	-	TP	TP
PVC support for Knative services	-	-	TP	TP

Deprecated and removed features

Some features that were Generally Available (GA) or a Technology Preview (TP) in previous releases have been deprecated or removed. Deprecated functionality is still included in OpenShift Serverless and continues to be supported; however, it will be removed in a future release of this product and is not recommended for new deployments.

For the most recent list of major functionality deprecated and removed within OpenShift Serverless, refer to the following table:

Table 2. Deprecated and removed features tracker
Feature	1.20	1.21	1.22	1.23
`KafkaBinding` API	Deprecated	Deprecated	Removed	Removed
`kn func emit` (`kn func invoke` in 1.21+)	Deprecated	Removed	Removed	Removed

Release notes for Red Hat OpenShift Serverless 1.23.0

OpenShift Serverless 1.23.0 is now available. New features, changes, and known issues that pertain to OpenShift Serverless on OpenShift Container Platform are included in this topic.

New features

OpenShift Serverless now uses Knative Serving 1.2.
OpenShift Serverless now uses Knative Eventing 1.2.
OpenShift Serverless now uses Kourier 1.2.
OpenShift Serverless now uses Knative kn CLI 1.2.
OpenShift Serverless now uses Knative Kafka 1.2.
The kn func CLI plug-in now uses func 0.24.
It is now possible to use the kafka.eventing.knative.dev/external.topic annotation with the Kafka broker. This annotation makes it possible to use an existing externally managed topic instead of the broker creating its own internal topic.
The kafka-ch-controller and kafka-webhook Kafka components no longer exist. These components have been replaced by the kafka-webhook-eventing component.
The OpenShift Serverless Functions Technology Preview now uses Source-to-Image (S2I) by default to build container images.

Known issues

The Federal Information Processing Standards (FIPS) mode is disabled for Kafka broker, Kafka source, and Kafka sink.
If you delete a namespace that includes a Kafka broker, the namespace finalizer may fail to be removed if the broker’s auth.secret.ref.name secret is deleted before the broker.

Running OpenShift Serverless with a large number of Knative services can cause Knative activator pods to run close to their default memory limits of 600MB. These pods might be restarted if memory consumption reaches this limit. Requests and limits for the activator deployment can be configured by modifying the KnativeServing custom resource:

apiVersion: operator.knative.dev/v1alpha1
kind: KnativeServing
metadata:
  name: knative-serving
  namespace: knative-serving
spec:
  deployments:
  - name: activator
    resources:
    - container: activator
      requests:
        cpu: 300m
        memory: 60Mi
      limits:
        cpu: 1000m
        memory: 1000Mi

If you are using Cloud Native Buildpacks as the local build strategy for a function, kn func is unable to automatically start podman or use an SSH tunnel to a remote daemon. The workaround for these issues is to have a Docker or podman daemon already running on the local development computer before deploying a function.
On-cluster function builds currently fail for Quarkus and Golang runtimes. They work correctly for Node, Typescript, Python, and Springboot runtimes.

Additional resources

Source-to-Image

Release notes for Red Hat OpenShift Serverless 1.22.0

OpenShift Serverless 1.22.0 is now available. New features, changes, and known issues that pertain to OpenShift Serverless on OpenShift Container Platform are included in this topic.

New features

OpenShift Serverless now uses Knative Serving 1.1.
OpenShift Serverless now uses Knative Eventing 1.1.
OpenShift Serverless now uses Kourier 1.1.
OpenShift Serverless now uses Knative kn CLI 1.1.
OpenShift Serverless now uses Knative Kafka 1.1.
The kn func CLI plug-in now uses func 0.23.
Init containers support for Knative services is now available as a Technology Preview.
Persistent volume claim (PVC) support for Knative services is now available as a Technology Preview.
The knative-serving, knative-serving-ingress, knative-eventing and knative-kafka system namespaces now have the knative.openshift.io/part-of: "openshift-serverless" label by default.
The Knative Eventing - Kafka Broker/Trigger dashboard has been added, which allows visualizing Kafka broker and trigger metrics in the web console.
The Knative Eventing - KafkaSink dashboard has been added, which allows visualizing KafkaSink metrics in the web console.
The Knative Eventing - Broker/Trigger dashboard is now called Knative Eventing - Channel-based Broker/Trigger.
The knative.openshift.io/part-of: "openshift-serverless" label has substituted the knative.openshift.io/system-namespace label.
Naming style in Knative Serving YAML configuration files changed from camel case (ExampleName) to hyphen style (example-name). Beginning with this release, use the hyphen style notation when creating or editing Knative Serving YAML configuration files.

Known issues

The Federal Information Processing Standards (FIPS) mode is disabled for Kafka broker, Kafka source, and Kafka sink.

Release notes for Red Hat OpenShift Serverless 1.21.0

OpenShift Serverless 1.21.0 is now available. New features, changes, and known issues that pertain to OpenShift Serverless on OpenShift Container Platform are included in this topic.

New features

OpenShift Serverless now uses Knative Serving 1.0
OpenShift Serverless now uses Knative Eventing 1.0.
OpenShift Serverless now uses Kourier 1.0.
OpenShift Serverless now uses Knative kn CLI 1.0.
OpenShift Serverless now uses Knative Kafka 1.0.
The kn func CLI plug-in now uses func 0.21.
The Kafka sink is now available as a Technology Preview.
The Knative open source project has begun to deprecate camel-cased configuration keys in favor of using kebab-cased keys consistently. As a result, the defaultExternalScheme key, previously mentioned in the OpenShift Serverless 1.18.0 release notes, is now deprecated and replaced by the default-external-scheme key. Usage instructions for the key remain the same.

Fixed issues

In OpenShift Serverless 1.20.0, there was an event delivery issue affecting the use of kn event send to send events to a service. This issue is now fixed.
In OpenShift Serverless 1.20.0 (func 0.20), TypeScript functions created with the http template failed to deploy on the cluster. This issue is now fixed.
In OpenShift Serverless 1.20.0 (func 0.20), deploying a function using the gcr.io registry failed with an error. This issue is now fixed.
In OpenShift Serverless 1.20.0 (func 0.20), creating a Springboot function project directory with the kn func create command and then running the kn func build command failed with an error message. This issue is now fixed.
In OpenShift Serverless 1.19.0 (func 0.19), some runtimes were unable to build a function by using podman. This issue is now fixed.

Known issues

Currently, the domain mapping controller cannot process the URI of a broker, which contains a path that is currently not supported.

This means that, if you want to use a DomainMapping custom resource (CR) to map a custom domain to a broker, you must configure the DomainMapping CR with the broker’s ingress service, and append the exact path of the broker to the custom domain:
Example DomainMapping CR
```
apiVersion: serving.knative.dev/v1alpha1
kind: DomainMapping
metadata:
  name: <domain-name>
  namespace: knative-eventing
spec:
  ref:
    name: broker-ingress
    kind: Service
    apiVersion: v1
```
The URI for the broker is then <domain-name>/<broker-namespace>/<broker-name>.

Release notes for Red Hat OpenShift Serverless 1.20.0

OpenShift Serverless 1.20.0 is now available. New features, changes, and known issues that pertain to OpenShift Serverless on OpenShift Container Platform are included in this topic.

New features

OpenShift Serverless now uses Knative Serving 0.26.
OpenShift Serverless now uses Knative Eventing 0.26.
OpenShift Serverless now uses Kourier 0.26.
OpenShift Serverless now uses Knative kn CLI 0.26.
OpenShift Serverless now uses Knative Kafka 0.26.
The kn func CLI plug-in now uses func 0.20.
The Kafka broker is now available as a Technology Preview.

The Kafka broker, which is currently in Technology Preview, is not supported on FIPS.
The kn event plug-in is now available as a Technology Preview.
The --min-scale and --max-scale flags for the kn service create command have been deprecated. Use the --scale-min and --scale-max flags instead.

Known issues

OpenShift Serverless deploys Knative services with a default address that uses HTTPS. When sending an event to a resource inside the cluster, the sender does not have the cluster certificate authority (CA) configured. This causes event delivery to fail, unless the cluster uses globally accepted certificates.

For example, an event delivery to a publicly accessible address works:
```
$ kn event send --to-url https://ce-api.foo.example.com/
```
On the other hand, this delivery fails if the service uses a public address with an HTTPS certificate issued by a custom CA:
```
$ kn event send --to Service:serving.knative.dev/v1:event-display
```
Sending an event to other addressable objects, such as brokers or channels, is not affected by this issue and works as expected.
The Kafka broker currently does not work on a cluster with Federal Information Processing Standards (FIPS) mode enabled.
If you create a Springboot function project directory with the kn func create command, subsequent running of the kn func build command fails with this error message:
```
[analyzer] no stack metadata found at path ''
[analyzer] ERROR: failed to : set API for buildpack 'paketo-buildpacks/ca-certificates@3.0.2': buildpack API version '0.7' is incompatible with the lifecycle
```
As a workaround, you can change the builder property to gcr.io/paketo-buildpacks/builder:base in the function configuration file func.yaml.
Deploying a function using the gcr.io registry fails with this error message:
```
Error: failed to get credentials: failed to verify credentials: status code: 404
```
As a workaround, use a different registry than gcr.io, such as quay.io or docker.io.
TypeScript functions created with the http template fail to deploy on the cluster.

As a workaround, in the func.yaml file, replace the following section:
```
buildEnvs: []
```
with this:
```
buildEnvs:
- name: BP_NODE_RUN_SCRIPTS
  value: build
```
In func version 0.20, some runtimes might be unable to build a function by using podman. You might see an error message similar to the following:
```
ERROR: failed to image: error during connect: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.40/info": EOF
```
- The following workaround exists for this issue:
  1. Update the podman service by adding --time=0 to the service ExecStart definition:
    
    Example service configuration
    
    ExecStart=/usr/bin/podman $LOGGING system service --time=0
  2. Restart the podman service by running the following commands:
    
    $ systemctl --user daemon-reload
    
    $ systemctl restart --user podman.socket
- Alternatively, you can expose the podman API by using TCP:
  $ podman system service --time=0 tcp:127.0.0.1:5534 & export DOCKER_HOST=tcp://127.0.0.1:5534

Release notes for Red Hat OpenShift Serverless 1.19.0

OpenShift Serverless 1.19.0 is now available. New features, changes, and known issues that pertain to OpenShift Serverless on OpenShift Container Platform are included in this topic.

New features

OpenShift Serverless now uses Knative Serving 0.25.
OpenShift Serverless now uses Knative Eventing 0.25.
OpenShift Serverless now uses Kourier 0.25.
OpenShift Serverless now uses Knative kn CLI 0.25.
OpenShift Serverless now uses Knative Kafka 0.25.
The kn func CLI plug-in now uses func 0.19.
The KafkaBinding API is deprecated in OpenShift Serverless 1.19.0 and will be removed in a future release.
HTTPS redirection is now supported and can be configured either globally for a cluster or per each Knative service.

Fixed issues

In previous releases, the Kafka channel dispatcher waited only for the local commit to succeed before responding, which might have caused lost events in the case of an Apache Kafka node failure. The Kafka channel dispatcher now waits for all in-sync replicas to commit before responding.

Known issues

In func version 0.19, some runtimes might be unable to build a function by using podman. You might see an error message similar to the following:
```
ERROR: failed to image: error during connect: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.40/info": EOF
```
- The following workaround exists for this issue:
  1. Update the podman service by adding --time=0 to the service ExecStart definition:
    
    Example service configuration
    
    ExecStart=/usr/bin/podman $LOGGING system service --time=0
  2. Restart the podman service by running the following commands:
    
    $ systemctl --user daemon-reload
    
    $ systemctl restart --user podman.socket
- Alternatively, you can expose the podman API by using TCP:
  $ podman system service --time=0 tcp:127.0.0.1:5534 & export DOCKER_HOST=tcp://127.0.0.1:5534

Release notes for Red Hat OpenShift Serverless 1.18.0

OpenShift Serverless 1.18.0 is now available. New features, changes, and known issues that pertain to OpenShift Serverless on OpenShift Container Platform are included in this topic.

New features

OpenShift Serverless now uses Knative Serving 0.24.0.
OpenShift Serverless now uses Knative Eventing 0.24.0.
OpenShift Serverless now uses Kourier 0.24.0.
OpenShift Serverless now uses Knative kn CLI 0.24.0.
OpenShift Serverless now uses Knative Kafka 0.24.7.
The kn func CLI plug-in now uses func 0.18.0.
In the upcoming OpenShift Serverless 1.19.0 release, the URL scheme of external routes will default to HTTPS for enhanced security.

If you do not want this change to apply for your workloads, you can override the default setting before upgrading to 1.19.0, by adding the following YAML to your KnativeServing custom resource (CR):
```
...
spec:
  config:
    network:
      defaultExternalScheme: "http"
...
```
If you want the change to apply in 1.18.0 already, add the following YAML:
```
...
spec:
  config:
    network:
      defaultExternalScheme: "https"
...
```
In the upcoming OpenShift Serverless 1.19.0 release, the default service type by which the Kourier Gateway is exposed will be ClusterIP and not LoadBalancer.

If you do not want this change to apply to your workloads, you can override the default setting before upgrading to 1.19.0, by adding the following YAML to your KnativeServing custom resource (CR):
```
...
spec:
  ingress:
    kourier:
      service-type: LoadBalancer
...
```
You can now use emptyDir volumes with OpenShift Serverless. See the OpenShift Serverless documentation about Knative Serving for details.
Rust templates are now available when you create a function using kn func.

Fixed issues

The prior 1.4 version of Camel-K was not compatible with OpenShift Serverless 1.17.0. The issue in Camel-K has been fixed, and Camel-K version 1.4.1 can be used with OpenShift Serverless 1.17.0.
Previously, if you created a new subscription for a Kafka channel, or a new Kafka source, a delay was possible in the Kafka data plane becoming ready to dispatch messages after the newly created subscription or sink reported a ready status.

As a result, messages that were sent during the time when the data plane was not reporting a ready status, might not have been delivered to the subscriber or sink.

In OpenShift Serverless 1.18.0, the issue is fixed and the initial messages are no longer lost. For more information about the issue, see Knowledgebase Article #6343981.

Known issues

Older versions of the Knative kn CLI might use older versions of the Knative Serving and Knative Eventing APIs. For example, version 0.23.2 of the kn CLI uses the v1alpha1 API version.

On the other hand, newer releases of OpenShift Serverless might no longer support older API versions. For example, OpenShift Serverless 1.18.0 no longer supports version v1alpha1 of the kafkasources.sources.knative.dev API.

Consequently, using an older version of the Knative kn CLI with a newer OpenShift Serverless might produce an error because the kn cannot find the outdated API. For example, version 0.23.2 of the kn CLI does not work with OpenShift Serverless 1.18.0.

To avoid issues, use the latest kn CLI version available for your OpenShift Serverless release. For OpenShift Serverless 1.18.0, use Knative kn CLI 0.24.0.

Release Notes for Red Hat OpenShift Serverless 1.17.0

OpenShift Serverless 1.17.0 is now available. New features, changes, and known issues that pertain to OpenShift Serverless on OpenShift Container Platform are included in this topic.

New features

OpenShift Serverless now uses Knative Serving 0.23.0.
OpenShift Serverless now uses Knative Eventing 0.23.0.
OpenShift Serverless now uses Kourier 0.23.0.
OpenShift Serverless now uses Knative kn CLI 0.23.0.
OpenShift Serverless now uses Knative Kafka 0.23.0.
The kn func CLI plug-in now uses func 0.17.0.
In the upcoming OpenShift Serverless 1.19.0 release, the URL scheme of external routes will default to HTTPS for enhanced security.

If you do not want this change to apply for your workloads, you can override the default setting before upgrading to 1.19.0, by adding the following YAML to your KnativeServing custom resource (CR):
```
...
spec:
  config:
    network:
      defaultExternalScheme: "http"
...
```
mTLS functionality is now Generally Available (GA).
TypeScript templates are now available when you create a function using kn func.
Changes to API versions in Knative Eventing 0.23.0:
- The v1alpha1 version of the KafkaChannel API, which was deprecated in OpenShift Serverless version 1.14.0, has been removed. If the ChannelTemplateSpec parameters of your config maps contain references to this older version, you must update this part of the spec to use the correct API version.

Known issues

If you try to use an older version of the Knative kn CLI with a newer OpenShift Serverless release, the API is not found and an error occurs.

For example, if you use the 1.16.0 release of the kn CLI, which uses version 0.22.0, with the 1.17.0 OpenShift Serverless release, which uses the 0.23.0 versions of the Knative Serving and Knative Eventing APIs, the CLI does not work because it continues to look for the outdated 0.22.0 API versions.

Ensure that you are using the latest kn CLI version for your OpenShift Serverless release to avoid issues.
Kafka channel metrics are not monitored or shown in the corresponding web console dashboard in this release. This is due to a breaking change in the Kafka dispatcher reconciling process.
If you create a new subscription for a Kafka channel, or a new Kafka source, there might be a delay in the Kafka data plane becoming ready to dispatch messages after the newly created subscription or sink reports a ready status.

As a result, messages that are sent during the time when the data plane is not reporting a ready status might not be delivered to the subscriber or sink.

For more information about this issue and possible workarounds, see Knowledge Article #6343981.
The Camel-K 1.4 release is not compatible with OpenShift Serverless version 1.17.0. This is because Camel-K 1.4 uses APIs that were removed in Knative version 0.23.0. There is currently no workaround available for this issue. If you need to use Camel-K 1.4 with OpenShift Serverless, do not upgrade to OpenShift Serverless version 1.17.0.

The issue has been fixed, and Camel-K version 1.4.1 is compatible with OpenShift Serverless 1.17.0.

Release Notes for Red Hat OpenShift Serverless 1.16.0

OpenShift Serverless 1.16.0 is now available. New features, changes, and known issues that pertain to OpenShift Serverless on OpenShift Container Platform are included in this topic.

New features

OpenShift Serverless now uses Knative Serving 0.22.0.
OpenShift Serverless now uses Knative Eventing 0.22.0.
OpenShift Serverless now uses Kourier 0.22.0.
OpenShift Serverless now uses Knative kn CLI 0.22.0.
OpenShift Serverless now uses Knative Kafka 0.22.0.
The kn func CLI plug-in now uses func 0.16.0.
The kn func emit command has been added to the functions kn plug-in. You can use this command to send events to test locally deployed functions.

Known issues

You must upgrade OpenShift Container Platform to version 4.6.30, 4.7.11, or higher before upgrading to OpenShift Serverless 1.16.0.
The AMQ Streams Operator might prevent the installation or upgrade of the OpenShift Serverless Operator. If this happens, the following error is thrown by Operator Lifecycle Manager (OLM):
```
WARNING: found multiple channel heads: [amqstreams.v1.7.2 amqstreams.v1.6.2], please check the `replaces`/`skipRange` fields of the operator bundles.
```
You can fix this issue by uninstalling the AMQ Streams Operator before installing or upgrading the OpenShift Serverless Operator. You can then reinstall the AMQ Streams Operator.
If Service Mesh is enabled with mTLS, metrics for Knative Serving are disabled by default because Service Mesh prevents Prometheus from scraping metrics. For instructions on enabling Knative Serving metrics for use with Service Mesh and mTLS, see the "Integrating Service Mesh with OpenShift Serverless" section of the Serverless documentation.

If you deploy Service Mesh CRs with the Istio ingress enabled, you might see the following warning in the istio-ingressgateway pod:

2021-05-02T12:56:17.700398Z warning envoy config [external/envoy/source/common/config/grpc_subscription_impl.cc:101] gRPC config for type.googleapis.com/envoy.api.v2.Listener rejected: Error adding/updating listener(s) 0.0.0.0_8081: duplicate listener 0.0.0.0_8081 found

Your Knative services might also not be accessible.

You can use the following workaround to fix this issue by recreating the knative-local-gateway service:

Delete the existing knative-local-gateway service in the istio-system namespace:
```
$ oc delete services -n istio-system knative-local-gateway
```

Create and apply a knative-local-gateway service that contains the following YAML:

apiVersion: v1
kind: Service
metadata:
 name: knative-local-gateway
 namespace: istio-system
 labels:
   experimental.istio.io/disable-gateway-port-translation: "true"
spec:
 type: ClusterIP
 selector:
   istio: ingressgateway
 ports:
   - name: http2
     port: 80
     targetPort: 8081

If you have 1000 Knative services on a cluster, and then perform a reinstall or upgrade of Knative Serving, there is a delay when you create the first new service after the KnativeServing custom resource (CR) becomes Ready.

The 3scale-kourier-control service reconciles all previously existing Knative services before processing the creation of a new service, which causes the new service to spend approximately 800 seconds in an IngressNotConfigured or Unknown state before the state updates to Ready.
If you create a new subscription for a Kafka channel, or a new Kafka source, there might be a delay in the Kafka data plane becoming ready to dispatch messages after the newly created subscription or sink reports a ready status.

As a result, messages that are sent during the time when the data plane is not reporting a ready status might not be delivered to the subscriber or sink.

For more information about this issue and possible workarounds, see Knowledge Article #6343981.