$ mkdir -p ./out
You can use PolicyGenTemplate
CRs to deploy custom functionality in your managed clusters.
If you require cluster configuration changes outside of the base GitOps ZTP pipeline configuration, there are three options:
When the GitOps ZTP pipeline deployment is complete, the deployed cluster is ready for application workloads. At this point, you can install additional Operators and apply configurations specific to your requirements. Ensure that additional configurations do not negatively affect the performance of the platform or allocated CPU budget.
The base source custom resources (CRs) that you deploy with the GitOps ZTP pipeline can be augmented with custom content as required.
Extra manifests are applied during installation and make the installation process more efficient.
Providing additional source CRs or modifying existing source CRs can significantly impact the performance or CPU profile of OpenShift Container Platform. |
See Customizing extra installation manifests in the ZTP GitOps pipeline for information about adding extra manifests.
PolicyGenTemplate
custom resources (CRs) allow you to overlay additional configuration details on top of the base source CRs provided with the GitOps plugin in the ztp-site-generate
container. You can think of PolicyGenTemplate
CRs as a logical merge or patch to the base CR. Use PolicyGenTemplate
CRs to update a single field of the base CR, or overlay the entire contents of the base CR. You can update values and insert fields that are not in the base CR.
The following example procedure describes how to update fields in the generated PerformanceProfile
CR for the reference configuration based on the PolicyGenTemplate
CR in the group-du-sno-ranGen.yaml
file. Use the procedure as a basis for modifying other parts of the PolicyGenTemplate
based on your requirements.
Create a Git repository where you manage your custom site configuration data. The repository must be accessible from the hub cluster and be defined as a source repository for Argo CD.
Review the baseline source CR for existing content. You can review the source CRs listed in the reference PolicyGenTemplate
CRs by extracting them from the zero touch provisioning (ZTP) container.
Create an /out
folder:
$ mkdir -p ./out
Extract the source CRs:
$ podman run --log-driver=none --rm registry.redhat.io/openshift4/ztp-site-generate-rhel8:v4.11.1 extract /home/ztp --tar | tar x -C ./out
Review the baseline PerformanceProfile
CR in ./out/source-crs/PerformanceProfile.yaml
:
apiVersion: performance.openshift.io/v2
kind: PerformanceProfile
metadata:
name: $name
annotations:
ran.openshift.io/ztp-deploy-wave: "10"
spec:
additionalKernelArgs:
- "idle=poll"
- "rcupdate.rcu_normal_after_boot=0"
cpu:
isolated: $isolated
reserved: $reserved
hugepages:
defaultHugepagesSize: $defaultHugepagesSize
pages:
- size: $size
count: $count
node: $node
machineConfigPoolSelector:
pools.operator.machineconfiguration.openshift.io/$mcp: ""
net:
userLevelNetworking: true
nodeSelector:
node-role.kubernetes.io/$mcp: ''
numa:
topologyPolicy: "restricted"
realTimeKernel:
enabled: true
Any fields in the source CR which contain |
Update the PolicyGenTemplate
entry for PerformanceProfile
in the group-du-sno-ranGen.yaml
reference file. The following example PolicyGenTemplate
CR stanza supplies appropriate CPU specifications, sets the hugepages
configuration, and adds a new field that sets globallyDisableIrqLoadBalancing
to false.
- fileName: PerformanceProfile.yaml
policyName: "config-policy"
metadata:
name: openshift-node-performance-profile
spec:
cpu:
# These must be tailored for the specific hardware platform
isolated: "2-19,22-39"
reserved: "0-1,20-21"
hugepages:
defaultHugepagesSize: 1G
pages:
- size: 1G
count: 10
globallyDisableIrqLoadBalancing: false
Commit the PolicyGenTemplate
change in Git, and then push to the Git repository being monitored by the GitOps ZTP argo CD application.
The ZTP application generates an RHACM policy that contains the generated PerformanceProfile
CR. The contents of that CR are derived by merging the metadata
and spec
contents from the PerformanceProfile
entry in the PolicyGenTemplate
onto the source CR. The resulting CR has the following content:
---
apiVersion: performance.openshift.io/v2
kind: PerformanceProfile
metadata:
name: openshift-node-performance-profile
spec:
additionalKernelArgs:
- idle=poll
- rcupdate.rcu_normal_after_boot=0
cpu:
isolated: 2-19,22-39
reserved: 0-1,20-21
globallyDisableIrqLoadBalancing: false
hugepages:
defaultHugepagesSize: 1G
pages:
- count: 10
size: 1G
machineConfigPoolSelector:
pools.operator.machineconfiguration.openshift.io/master: ""
net:
userLevelNetworking: true
nodeSelector:
node-role.kubernetes.io/master: ""
numa:
topologyPolicy: restricted
realTimeKernel:
enabled: true
In the An exception to this is the
The |
The source CRs in the GitOps ZTP site generator container provide a set of critical features and node tuning settings for RAN Distributed Unit (DU) applications. These are applied to the clusters that you deploy with ZTP. To add or modify existing source CRs in the ztp-site-generate
container, rebuild the ztp-site-generate
container and make it available to the hub cluster, typically from the disconnected registry associated with the hub cluster. Any valid OpenShift Container Platform CR can be added.
Perform the following procedure to add new content to the ZTP pipeline.
Create a directory containing a Containerfile and the source CR YAML files that you want to include in the updated ztp-site-generate
container, for example:
ztp-update/
├── example-cr1.yaml
├── example-cr2.yaml
└── ztp-update.in
Add the following content to the ztp-update.in
Containerfile:
FROM registry.redhat.io/openshift4/ztp-site-generate-rhel8:v4.11
ADD example-cr2.yaml /kustomize/plugin/ran.openshift.io/v1/policygentemplate/source-crs/
ADD example-cr1.yaml /kustomize/plugin/ran.openshift.io/v1/policygentemplate/source-crs/
Open a terminal at the ztp-update/
folder and rebuild the container:
$ podman build -t ztp-site-generate-rhel8-custom:v4.11-custom-1
Push the built container image to your disconnected registry, for example:
$ podman push localhost/ztp-site-generate-rhel8-custom:v4.11-custom-1 registry.example.com:5000/ztp-site-generate-rhel8-custom:v4.11-custom-1
Patch the Argo CD instance on the hub cluster to point to the newly built container image:
$ oc patch -n openshift-gitops argocd openshift-gitops --type=json -p '[{"op": "replace", "path":"/spec/repo/initContainers/0/image", "value": "registry.example.com:5000/ztp-site-generate-rhel8-custom:v4.11-custom-1"} ]'
When the Argo CD instance is patched, the openshift-gitops-repo-server
pod automatically restarts.
Verify that the new openshift-gitops-repo-server
pod has completed initialization and that the previous repo pod is terminated:
$ oc get pods -n openshift-gitops | grep openshift-gitops-repo-server
openshift-gitops-server-7df86f9774-db682 1/1 Running 1 28s
You must wait until the new openshift-gitops-repo-server
pod has completed initialization and the previous pod is terminated before the newly added container image content is available.
Alternatively, you can patch the ArgoCD instance as described in Configuring the hub cluster with ArgoCD by modifying argocd-openshift-gitops-patch.json
with an updated initContainer
image before applying the patch file.
Use Red Hat Advanced Cluster Management (RHACM) installed on a hub cluster to monitor and report on whether your managed clusters are compliant with applied policies. RHACM uses policy templates to apply predefined policy controllers and policies. Policy controllers are Kubernetes custom resource definition (CRD) instances.
You can override the default policy evaluation intervals with PolicyGenTemplate
custom resources (CRs). You configure duration settings that define how long a ConfigurationPolicy
CR can be in a state of policy compliance or non-compliance before RHACM re-evaluates the applied cluster policies.
The zero touch provisioning (ZTP) policy generator generates ConfigurationPolicy
CR policies with pre-defined policy evaluation intervals. The default value for the noncompliant
state is 10 seconds. The default value for the compliant
state is 10 minutes. To disable the evaluation interval, set the value to never
.
You have installed the OpenShift CLI (oc
).
You have logged in to the hub cluster as a user with cluster-admin
privileges.
You have created a Git repository where you manage your custom site configuration data.
To configure the evaluation interval for all policies in a PolicyGenTemplate
CR, add evaluationInterval
to the spec
field, and then set the appropriate compliant
and noncompliant
values. For example:
spec:
evaluationInterval:
compliant: 30m
noncompliant: 20s
To configure the evaluation interval for the spec.sourceFiles
object in a PolicyGenTemplate
CR, add evaluationInterval
to the sourceFiles
field, for example:
spec:
sourceFiles:
- fileName: SriovSubscription.yaml
policyName: "sriov-sub-policy"
evaluationInterval:
compliant: never
noncompliant: 10s
Commit the PolicyGenTemplate
CRs files in the Git repository and push your changes.
Check that the managed spoke cluster policies are monitored at the expected intervals.
Log in as a user with cluster-admin
privileges on the managed cluster.
Get the pods that are running in the open-cluster-management-agent-addon
namespa