-
myapp.<namespace>
-
myapp.<namespace>.svc
-
myapp.<namespace>.svc.cluster.local
You can enable OpenShift Serverless Serving transport encryption to allow transporting data over secured and encrypted HTTPS connections using TLS.
OpenShift Serverless Serving transport encryption is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope. |
Serving Transport Encryption is only available for Kourier as an ingress layer. For Red Hat OpenShift Service Mesh, use the service mesh mTLS capabilities to ensure encrypted traffic. |
There are three parts to OpenShift Serverless Serving transport encryption:
Transport encryption on the ingress layer external to the cluster. For example, a cluster-external domain, such as myapp-<namespace>.example.com
.
Transport encryption on the ingress layer internal to the cluster. For example, cluster-local domains, such as myapp.<namespace>.svc.cluster.local
.
Transport encryption between the ingress-gateway
, activator
, and queue-proxy
Knative internal components.
Control-plane traffic, including Kubernetes PreStopHooks, metadata, and metrics, contains no user data and is not encrypted. |
The different parts are independent of each other and can be enabled and disabled individually. They can use the same or different Certificate Authorities (CAs) to sign the necessary certificates.
For diagrams illustrating transport encryption, see OpenShift Serverless Serving Transport Encryption. |
The transport encryption for external domains is handled by the ingress layer of the cluster, which is either the OpenShift Container Platform ingress or Red Hat OpenShift Service Mesh.
Cluster-local encryption enables transport encryption for cluster-local domains. It has the following properties:
The Certificate Common Name (CN) or Subject Alternative Name (SAN) contains the cluster-local domains of a Knative Service, for example myapp.namespace.svc.cluster.local
, myapp.namespace.svc
, myapp.namespace
.
The cluster-local endpoint of the ingress-controller
component uses SNI to select the certificates.
To create the certificates, Knative relies on cert-manager
, which needs to be installed and configured for the feature to work. For more information, see cert-manager Operator for Red Hat OpenShift.
The caller must trust the CA that signed the cluster-local certificates. This is out of the scope of OpenShift Serverless. |
System-internal encryption enables transport encryption for the ingress-gateway
, activator
, and queue-proxy
Knative internal components. These components host TLS endpoints when this configuration is used.
The following prerequisites must be satisfied to use this feature:
For OpenShift Serverless to get the certificates, cert-manager
must be installed and configured.
Specific SANs are used to verify each connection. Each component must trust the CA that signed the certificates. To satisfy this requirement, OpenShift Serverless system components consume and trust a bundle of CAs. The CA bundle must be provided by the cluster administrator.
Issuers refer to cert-manager
issuers and cluster issuers. They represent certificate authorities (CAs) that can generate signed certificates by honoring certificate signing requests. For more information, see cert-manager documentation on issuers.
Depending on the encryption features that you use, OpenShift Serverless requires your certificate issuer to be able to sign certain certificates. To identify your certificate issuer, refer to the list of cert-manager integrations, which contains examples for the following:
A custom CA stored in a Kubernetes secret
HTTP-01 challenges
DNS-01 challenges
Self-signed issuers
Not all issuer types work for each Knative Serving encryption feature.
For cluster-local encryption, the issuer must be able to sign certificates for the following cluster-local domain types:
myapp.<namespace>
myapp.<namespace>.svc
myapp.<namespace>.svc.cluster.local
As the CA usually is not within the cluster, verification using the Automated Certificate Management Environment (ACME) protocol (DNS01/HTTP01) is not possible. You can use an issuer that allows creating these certificates, such as the cert-manager
CA issuer.
For system-internal encryption, the issuer must be able to sign certificates with the following Subject Alternative Names (SANs):
kn-routing
names of format kn-user-<namespace>
, where <namespace>
is a namespace where Knative Services are created
data-plane.knative.dev
Knative requires these SANs to verify connections between the internal components. Because this is not possible using the ACME protocol (DNS01/HTTP01), you must configure an issuer that allows creating these certificates, for example, cert-manager
CA issuer.
You have access to an OpenShift Container Platform account with cluster administrator access.
Install the {oc-first}.
Install the cert-manager Operator for Red Hat OpenShift.
Install the OpenShift Serverless Operator.
If you install the OpenShift Serverless Operator before installing the cert-manager Operator for Red Hat OpenShift, you must restart the controller and activator deployments in the |
The following procedure uses a SelfSigned
issuer as the root certificate. For information about the implications and limitations of this method, see the SelfSigned cert-manager documentation.
If you manage your own company-specific Private Key Infrastructure (PKI), use the CA issuer. For more information, see cert-manager documentation on CA issuers.
Create a SelfSigned
ClusterIssuer
custom resource (CR):
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: knative-serving-selfsigned-issuer
spec:
selfSigned: {}
Apply the ClusterIssuer
CR by running the following command:
$ oc apply -f <filename>
Create a root certificate that refers to the ClusterIssuer
CR:
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: knative-serving-selfsigned-ca
namespace: cert-manager (1)
spec:
secretName: knative-serving-ca (2)
isCA: true
commonName: selfsigned-ca
privateKey:
algorithm: ECDSA
size: 256
issuerRef:
name: knative-serving-selfsigned-issuer
kind: ClusterIssuer
group: cert-manager.io
1 | The cert-manager Operator for Red Hat OpenShift namespace, cert-manager by default. |
2 | Secret name later used for the ClusterIssuer CR for Knative Serving. |
Apply the Certificate
CR by running the following command:
$ oc apply -f <filename>
To enable the use of certificates by Serving, you must create a cluster issuer.
Create the knative-serving-ca-issuer
ClusterIssuer
for Serving:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: knative-serving-ca-issuer
spec:
ca:
secretName: knative-serving-ca (1)
1 | Secret name in the cert-manager Operator for Red Hat OpenShift namespace (cert-manager by default) containing the certificate that can be used by OpenShift Serverless Serving components for new certificates. |
Apply the ClusterIssuer
resource by running the following command:
$ oc apply -f <filename>
Configuring transport encryption consists of two parts:
Specifying the ClusterIssuer
issuer to use:
clusterLocalIssuerRef
: issuer for cluster-local-domain certificates used for ingress.
systemInternalIssuerRef
: issuer for certificates for system-internal-tls certificates used by Knative internal components.
Specifying transport encryption features to use:
cluster-local-domain-tls
: Enables the transport encryption feature for cluster-local domains
system-internal-tls
: Enables the transport encryption feature for OpenShift Serverless Serving internal components.
Enable transport encryption in the KnativeServing
resource:
apiVersion: operator.knative.dev/v1beta1
kind: KnativeServing
metadata:
name: knative-serving
namespace: knative-serving
spec:
...
config:
certmanager:
clusterLocalIssuerRef: |
kind: ClusterIssuer
name: knative-serving-ca-issuer (1)
systemInternalIssuerRef: |
kind: ClusterIssuer
name: knative-serving-ca-issuer (2)
network:
cluster-local-domain-tls: Enabled (3)
system-internal-tls: Enabled (4)
1 | Define the cluster issuer for each feature. The same or individual cluster issuers can be used. |
2 | Define the cluster issuer. |
3 | Enable the cluster-local-domain-tls feature. This and other features can be enabled or disabled individually. |
4 | Enable the system-internal-tls feature. |
Apply the KnativeServing
resource by running the following command:
$ oc apply -f <filename>
Optionally, change the defaultCertificate
value in the Ingress Controller:
apiVersion: operator.openshift.io/v1
kind: IngressController
...
spec:
defaultCertificate:
name: ca-ingress-cert
If you changed the defaultCertificate
value, you must specify the custom certificate name in the openshift-ingress-default-certificate
field in the KnativeServing
custom resource.
For example, if the custom certificate name is ca-ingress-cert
, add the following configuration:
...
spec:
config:
network:
system-internal-tls: Enabled
openshift-ingress-default-certificate: "ca-ingress-cert"
...
If you enabled cluster-local-domain-tls
or system-internal-tls
, restart the Controller component by running the following command.
When either the |
$ oc rollout restart deploy/controller -n knative-serving
If you enabled system-internal-tls
, restart the Activator component by running the following command.
When the |
$ oc rollout restart deploy/activator -n knative-serving
When you enable any of the transport encryption features, you must make sure that all clients calling trust the Certificate Authority (CA) issuing the certificates used for the transport encryption.
There are multiple places where trust must be ensured:
Cluster external client, such as Browser or other application. This is out of the scope of OpenShift Serverless.
OpenShift Serverless system components, such as Activator, Queue-Proxy, and Ingress-Controller.
Cluster internal client, such as a Knative Service or other workload.
To ensure that OpenShift Serverless Serving components and Knative Services trust the CA that issues certificates, you can create a ConfigMap in the following namespaces with the label networking.knative.dev/trust-bundle: true
:
knative-serving
for the system components of OpenShift Serverless Serving.
knative-serving-ingress
for the ingress layer of OpenShift Serverless Serving.
istio-system
or your own Service Mesh namespacewhen the Service Mesh integration is enabled.
Knative reads all data keys in ConfigMaps with this label, regardless of the name. One key can contain one or multiple CAs or intermediate certificates. If they are valid, they are added to the trust store of the Knative components.
This is an example ConfigMap:
apiVersion: v1
data:
cacerts.pem: | (1)
-----BEGIN CERTIFICATE-----
MIIDDTCCAfWgAwIBAgIQMQuip05h7NLQq2TB+j9ZmTANBgkqhkiG9w0BAQsFADAW
MRQwEgYDVQQDEwtrbmF0aXZlLmRldjAeFw0yMzExMjIwOTAwNDhaFw0yNDAyMjAw
OTAwNDhaMBYxFDASBgNVBAMTC2tuYXRpdmUuZGV2MIIBIjANBgkqhkiG9w0BAQEF
AAOCAQ8AMIIBCgKCAQEA3clC3CV7sy0TpUKNuTku6QmP9z8JUCbLCPCLACCUc1zG
FEokqOva6TakgvAntXLkB3TEsbdCJlNm6qFbbko6DBfX6rEggqZs40x3/T+KH66u
4PvMT3fzEtaMJDK/KQOBIvVHrKmPkvccUYK/qWY7rgBjVjjLVSJrCn4dKaEZ2JNr
Fd0KNnaaW/dP9/FvviLqVJvHnTMHH5qyRRr1kUGTrc8njRKwpHcnUdauiDoWRKxo
Zlyy+MhQfdbbyapX984WsDjCvrDXzkdGgbRNAf+erl6yUm6pHpQhyFFo/zndx6Uq
QXA7jYvM2M3qCnXmaFowidoLDsDyhwoxD7WT8zur/QIDAQABo1cwVTAOBgNVHQ8B
Af8EBAMCAgQwEwYDVR0lBAwwCgYIKwYBBQUHAwEwDwYDVR0TAQH/BAUwAwEB/zAd
BgNVHQ4EFgQU7p4VuECNOcnrP9ulOjc4J37Q2VUwDQYJKoZIhvcNAQELBQADggEB
AAv26Vnk+ptQrppouF7yHV8fZbfnehpm07HIZkmnXO2vAP+MZJDNrHjy8JAVzXjt
+OlzqAL0cRQLsUptB0btoJuw23eq8RXgJo05OLOPQ2iGNbAATQh2kLwBWd/CMg+V
KJ4EIEpF4dmwOohsNR6xa/JoArIYH0D7gh2CwjrdGZr/tq1eMSL+uZcuX5OiE44A
2oXF9/jsqerOcH7QUMejSnB8N7X0LmUvH4jAesQgr7jo1JTOBs7GF6wb+U76NzFa
8ms2iAWhoplQ+EHR52wffWb0k6trXspq4O6v/J+nq9Ky3vC36so+G1ZFkMhCdTVJ
ZmrBsSMWeT2l07qeei2UFRU=
-----END CERTIFICATE-----
kind: ConfigMap
metadata:
labels:
networking.knative.dev/trust-bundle: "true"
name: knative-bundle (2)
namespace: knative-serving
1 | Serving components trust all keys containing valid PEM-encoded CA bundles. |
2 | You can use an arbitrary name. |
When a CA bundle ConfigMap is created or updated, the Serving components automatically pick them up and add the CAs or intermediate certificates to their CA trust store. The trust store is refreshed for every new HTTP connection. |
As OpenShift Serverless Serving does not control all workloads and managing trust is highly dependent on your runtime and language, custom workloads are out of the scope of OpenShift Serverless. The following are other options for custom workloads:
Adding a CA bundle to a Container image on build-time. Note that this complicates CA rotation, as you must rebuild and redeploy every application when the CA rotates.
Mounting a CA bundle to the filesystem, such as from a Secret or ConfigMap, and making sure your application uses it to verify TLS connections.
Reading a CA bundle from an environment variable and making sure that your application uses it to verify TLS connections.
Accessing a CA bundle from a secret or ConfigMap using Kubernetes API and making sure your application uses it to verify TLS connections.
Ensuring seamless CA rotation is essential to avoid service downtime, or to deal with an emergency.
Create a new CA certificate.
Add the public key of the new CA certificate to the CA trust bundles as described in the "Trust configuration for OpenShift Serverless Operator Serving components and Knative Services" section. Retain the public key of the existing CA.
Ensure that all clients have consumed the latest set of CA trust bundles. OpenShift Serverless Serving components will automatically reload the changed CA trust bundles.
If you have custom workload consuming trust bundles, reload or restart them accordingly.
Update the knative-serving-ca-issuer
cluster issuer to reference the secret containing the new CA certificate.
Either wait for cert-manager
to renew all your certificates or enforce it to renew all the certificates. For more information, see the cert-manager
documentation.
Once the CA rotation is fully completed, you can remove the public key of the old CA from the trust bundle configmap. Allow enough time for all components to apply the changes.
Create a Knative Service:
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: test-webapp
namespace: test-namespace
spec:
template:
spec:
containers:
- image: docker.io/openshift/hello-openshift
env:
- name: RESPONSE
value: "Hello Serverless!"
Apply the Knative Service YAML by running the following command:
$ oc apply -f <filename>
Examine status of the Knative Service:
$ oc get ksvc -n test-namespace -o yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
name: test-webapp
namespace: test-namespace
# spec:
# ...
status:
address:
# cluster-local-domain:
url: https://helloworld.test.svc.cluster.local (1)
1 | If you have enabled cluster-local-domain-tls , you will see the HTTPS URL. |
To verify that system-internal-tls
is enabled, check the output of Queue-Proxy logs by running the following command:
$ oc logs your-pod -n test-namespace -c queue-proxy | grep -E 'certDir|Certificate|tls'
If you see lines similar to the following, system-internal-tls
is enabled:
{"severity":"INFO","timestamp":"2024-01-03T07:07:32.892810888Z","logger":"queueproxy","caller":"certificate/watcher.go:62","message":"Starting to watch the following directories for changes{certDir 15 0 /var/lib/knative/certs <nil>} {keyDir 15 0 /var/lib/knative/certs <nil>}","commit":"86420f2-dirty","knative.dev/key":"first/helloworld-00001","knative.dev/pod":"helloworld-00001-deployment-75fbb7d488-qgmxx"}
{"severity":"INFO","timestamp":"2024-01-03T07:07:32.89397512Z","logger":"queueproxy","caller":"certificate/watcher.go:131","message":"Certificate and/or key have changed on disk and were reloaded.","commit":"86420f2-dirty","knative.dev/key":"first/helloworld-00001","knative.dev/pod":"helloworld-00001-deployment-75fbb7d488-qgmxx"}
{"severity":"INFO","timestamp":"2024-01-03T07:07:32.894232939Z","logger":"queueproxy","caller":"sharedmain/main.go:282","message":"Starting tls server admin:8022","commit":"86420f2-dirty","knative.dev/key":"first/helloworld-00001","knative.dev/pod":"helloworld-00001-deployment-75fbb7d488-qgmxx"}
{"severity":"INFO","timestamp":"2024-01-03T07:07:32.894268548Z","logger":"queueproxy","caller":"sharedmain/main.go:282","message":"Starting tls server main:8112","commit":"86420f2-dirty","knative.dev/key":"first/helloworld-00001","knative.dev/pod":"helloworld-00001-deployment-75fbb7d488-qgmxx"}