OpenShift docs are moving and will soon only be available at docs.redhat.com, the home of all Red Hat product documentation. Explore the new docs experience today.
  1. Documentation
    2. history bug_report picture_as_pdf
×

Configuring an OpenShift cluster by deploying an application with cluster configurations

Prerequisites

  • You have logged in to the OpenShift Container Platform cluster as an administrator.

  • You have installed the Red Hat OpenShift GitOps Operator on your OpenShift Container Platform cluster.

Using an Argo CD instance to manage cluster-scoped resources

To manage cluster-scoped resources, update the existing Subscription object for the Red Hat OpenShift GitOps Operator and add the namespace of the Argo CD instance to the ARGOCD_CLUSTER_CONFIG_NAMESPACES environment variable in the spec section.

Procedure

  1. In the Administrator perspective of the web console, navigate to Operators → Installed Operators → Red Hat OpenShift GitOps → Subscription.

  2. Click the Actions list and then click Edit Subscription.

  3. On the openshift-gitops-operator Subscription details page, under the YAML tab, edit the Subscription YAML file by adding the namespace of the Argo CD instance to the ARGOCD_CLUSTER_CONFIG_NAMESPACES environment variable in the spec section:

    apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: openshift-gitops-operator
  namespace: openshift-gitops-operator
# ...
spec:
  config:
    env:
    - name: ARGOCD_CLUSTER_CONFIG_NAMESPACES
      value: openshift-gitops, <list of namespaces of cluster-scoped Argo CD instances>
# ...

  4. Click Save and Reload.

  5. To verify that the Argo CD instance is configured with a cluster role to manage cluster-scoped resources, perform the following steps:

    1. Navigate to User Management → Roles and from the Filter list select Cluster-wide Roles.

    2. Search for the argocd-application-controller by using the Search by name field.

      The Roles page displays the created cluster role.

      Alternatively, in the OpenShift CLI, run the following command:

      oc auth can-i create oauth -n openshift-gitops --as system:serviceaccount:openshift-gitops:openshift-gitops-argocd-application-controller

      The output yes verifies that the Argo instance is configured with a cluster role to manage cluster-scoped resources. Else, check your configurations and take necessary steps as required.

Default permissions of an Argo CD instance

By default Argo CD instance has the following permissions:

  • Argo CD instance has the admin privileges to manage resources only in the namespace where it is deployed. For instance, an Argo CD instance deployed in the foo namespace has the admin privileges to manage resources only for that namespace.

  • Argo CD has the following cluster-scoped permissions because Argo CD requires cluster-wide read privileges on resources to function appropriately:

    - verbs:
    - get
    - list
    - watch
   apiGroups:
    - '*'
   resources:
    - '*'
 - verbs:
    - get
    - list
   nonResourceURLs:
    - '*'

  • You can edit the cluster roles used by the argocd-server and argocd-application-controller components where Argo CD is running such that the write privileges are limited to only the namespaces and resources that you wish Argo CD to manage.

    $ oc edit clusterrole argocd-server
$ oc edit clusterrole argocd-application-controller

Running the Argo CD instance at the cluster-level

The default Argo CD instance and the accompanying controllers, installed by the Red Hat OpenShift GitOps Operator, can now run on the infrastructure nodes of the cluster by setting a simple configuration toggle.

Procedure

  1. Label the existing nodes:

    $ oc label node <node-name> node-role.kubernetes.io/infra=""

  2. Optional: If required, you can also apply taints and isolate the workloads on infrastructure nodes and prevent other workloads from scheduling on these nodes:

    $ oc adm taint nodes -l node-role.kubernetes.io/infra \
infra=reserved:NoSchedule infra=reserved:NoExecute

  3. Add the runOnInfra toggle in the GitOpsService custom resource:

    apiVersion: pipelines.openshift.io/v1alpha1
kind: GitopsService
metadata:
  name: cluster
spec:
  runOnInfra: true

  4. Optional: If taints have been added to the nodes, then add tolerations to the GitOpsService custom resource.

    Example
    apiVersion: pipelines.openshift.io/v1alpha1
kind: GitopsService
metadata:
  name: cluster
  spec:
    runOnInfra: true
    tolerations:
    - effect: NoSchedule
      key: infra
      value: reserved
    - effect: NoExecute
      key: infra
      value: reserved

  5. Verify that the workloads in the openshift-gitops namespace are now scheduled on the infrastructure nodes by viewing Pods → Pod details for any pod in the console UI.

Any nodeSelectors and tolerations manually added to the default Argo CD custom resource are overwritten by the toggle and tolerations in the GitOpsService custom resource.
Additional resources

Creating an application by using the Argo CD dashboard

Argo CD provides a dashboard which allows you to create applications.

This sample workflow walks you through the process of configuring Argo CD to recursively sync the content of the cluster directory to the cluster-configs application. The directory defines the OpenShift Container Platform web console cluster configurations that add a link to the Red Hat Developer Blog - Kubernetes under the red hat applications menu icon menu in the web console, and defines a namespace spring-petclinic on the cluster.

Prerequisites

  • You have logged in to the OpenShift Container Platform cluster as an administrator.

  • You have installed the Red Hat OpenShift GitOps Operator on your OpenShift Container Platform cluster.

  • You have logged in to Argo CD instance.

Procedure

  1. In the Argo CD dashboard, click NEW APP to add a new Argo CD application.

  2. For this workflow, create a cluster-configs application with the following configurations:

    Application Name

    cluster-configs

    Project

    default

    Sync Policy

    Manual

    Repository URL

    https://github.com/redhat-developer/openshift-gitops-getting-started

    Revision

    HEAD

    Path

    cluster

    Destination

    https://kubernetes.default.svc

    Namespace

    spring-petclinic

    Directory Recurse

    checked

  3. Click CREATE to create your application.

  4. Open the Administrator perspective of the web console and expand Administration → Namespaces.

  5. Search for and select the namespace, then enter argocd.argoproj.io/managed-by=openshift-gitops in the Label field so that the Argo CD instance in the openshift-gitops namespace can manage your namespace.

Creating an application by using the oc tool

You can create Argo CD applications in your terminal by using the oc tool.

Prerequisites

  • You have installed the Red Hat OpenShift GitOps Operator on your OpenShift Container Platform cluster.

  • You have logged in to an Argo CD instance.

Procedure

  1. Download the sample application:

    $ git clone git@github.com:redhat-developer/openshift-gitops-getting-started.git

  2. Create the application:

    $ oc create -f openshift-gitops-getting-started/argo/app.yaml

  3. Run the oc get command to review the created application:

    $ oc get application -n openshift-gitops

  4. Add a label to the namespace your application is deployed in so that the Argo CD instance in the openshift-gitops namespace can manage it:

    $ oc label namespace spring-petclinic argocd.argoproj.io/managed-by=openshift-gitops

Creating an application in the default mode by using the GitOps CLI

You can create applications in the default mode by using the GitOps argocd CLI.

This sample workflow walks you through the process of configuring Argo CD to recursively sync the content of the cluster directory to the cluster-configs application. The directory defines the OpenShift Container Platform cluster configurations and the spring-petclinic namespace on the cluster.

Prerequisites

  • You have installed the Red Hat OpenShift GitOps Operator on your OpenShift Container Platform cluster.

  • You have installed the OpenShift CLI (oc).

  • You have installed the Red Hat OpenShift GitOps argocd CLI.

  • You have logged in to Argo CD instance.

Procedure

  1. Get the admin account password for the Argo CD server:

    $ ADMIN_PASSWD=$(oc get secret openshift-gitops-cluster -n openshift-gitops -o jsonpath='{.data.admin\.password}' | base64 -d)

  2. Get the Argo CD server URL:

    $ SERVER_URL=$(oc get routes openshift-gitops-server -n openshift-gitops -o jsonpath='{.status.ingress[0].host}')

  3. Log in to the Argo CD server by using the admin account password and enclosing it in single quotes:

    Enclosing the password in single quotes ensures that special characters, such as $, are not misinterpreted by the shell. Always use single quotes to enclose the literal value of the password.

    		 
    $ argocd login --username admin --password ${ADMIN_PASSWD} ${SERVER_URL}
    Example
    $ argocd login --username admin --password '<password>' openshift-gitops.openshift-gitops.apps-crc.testing

  4. Verify that you are able to run argocd commands in the default mode by listing all applications:

    $ argocd app list

    If the configuration is correct, then existing applications will be listed with the following header:

    Sample output
    NAME CLUSTER NAMESPACE  PROJECT  STATUS  HEALTH   SYNCPOLICY  CONDITIONS  REPO PATH TARGET

  5. Create an application in the default mode:

    $ argocd app create app-cluster-configs \
    --repo https://github.com/redhat-developer/openshift-gitops-getting-started.git \
    --path cluster \
    --revision main \
    --dest-server  https://kubernetes.default.svc \
    --dest-namespace spring-petclinic \
    --directory-recurse \
    --sync-policy none \
    --sync-option Prune=true \
    --sync-option CreateNamespace=true

  6. Label the spring-petclinic destination namespace to be managed by the openshif-gitops Argo CD instance:

    $ oc label ns spring-petclinic "argocd.argoproj.io/managed-by=openshift-gitops"

  7. List the available applications to confirm that the application is created successfully:

    $ argocd app list

    Even though the cluster-configs Argo CD application has the Healthy status, it is not automatically synced due to its none sync policy, causing it to remain in the OutOfSync status.

Creating an application in core mode by using the GitOps CLI

You can create applications in core mode by using the GitOps argocd CLI.

This sample workflow walks you through the process of configuring Argo CD to recursively sync the content of the cluster directory to the cluster-configs application. The directory defines the OpenShift Container Platform cluster configurations and the spring-petclinic namespace on the cluster.

Prerequisites

  • You have installed the Red Hat OpenShift GitOps Operator on your OpenShift Container Platform cluster.

  • You have installed the OpenShift CLI (oc).

  • You have installed the Red Hat OpenShift GitOps argocd CLI.

Procedure

  1. Log in to the OpenShift Container Platform cluster by using the oc CLI tool:

    $ oc login -u <username> -p <password> <server_url>
    Example
    $ oc login -u kubeadmin -p '<password>' https://api.crc.testing:6443

  2. Check whether the context is set correctly in the kubeconfig file:

    $ oc config current-context

  3. Set the default namespace of the current context to openshift-gitops:

    $ oc config set-context --current --namespace openshift-gitops

  4. Set the following environment variable to override the Argo CD component names:

    $ export ARGOCD_REPO_SERVER_NAME=openshift-gitops-repo-server

  5. Verify that you are able to run argocd commands in core mode by listing all applications:

    $ argocd app list --core

    If the configuration is correct, then existing applications will be listed with the following header:

    Sample output
    NAME CLUSTER NAMESPACE  PROJECT  STATUS  HEALTH   SYNCPOLICY  CONDITIONS  REPO PATH TARGET

  6. Create an application in core mode:

    $ argocd app create app-cluster-configs --core \
    --repo https://github.com/redhat-developer/openshift-gitops-getting-started.git \
    --path cluster \
    --revision main \
    --dest-server  https://kubernetes.default.svc \
    --dest-namespace spring-petclinic \
    --directory-recurse \
    --sync-policy none \
    --sync-option Prune=true \
    --sync-option CreateNamespace=true

  7. Label the spring-petclinic destination namespace to be managed by the openshif-gitops Argo CD instance:

    $ oc label ns spring-petclinic "argocd.argoproj.io/managed-by=openshift-gitops"

  8. List the available applications to confirm that the application is created successfully:

    $ argocd app list --core

    Even though the cluster-configs Argo CD application has the Healthy status, it is not automatically synced due to its none sync policy, causing it to remain in the OutOfSync status.

Synchronizing your application with your Git repository

You can synchronize your application with your Git repository by modifying the synchronization policy for Argo CD. The policy modification automatically applies the changes in your cluster configurations from your Git repository to the cluster.

Procedure

  1. In the Argo CD dashboard, notice that the cluster-configs Argo CD application has the statuses Missing and OutOfSync. Because the application was configured with a manual sync policy, Argo CD does not sync it automatically.

  2. Click SYNC on the cluster-configs tile, review the changes, and then click SYNCHRONIZE. Argo CD will detect any changes in the Git repository automatically. If the configurations are changed, Argo CD will change the status of the cluster-configs to OutOfSync. You can modify the synchronization policy for Argo CD to automatically apply changes from your Git repository to the cluster.

  3. Notice that the cluster-configs Argo CD application now has the statuses Healthy and Synced. Click the cluster-configs tile to check the details of the synchronized resources and their status on the cluster.

  4. Navigate to the OpenShift Container Platform web console and click red hat applications menu icon to verify that a link to the Red Hat Developer Blog - Kubernetes is now present there.

  5. Navigate to the Project page and search for the spring-petclinic namespace to verify that it has been added to the cluster.

    Your cluster configurations have been successfully synchronized to the cluster.

Synchronizing an application in the default mode by using the GitOps CLI

You can synchronize applications in the default mode by using the GitOps argocd CLI.

This sample workflow walks you through the process of configuring Argo CD to recursively sync the content of the cluster directory to the cluster-configs application. The directory defines the OpenShift Container Platform cluster configurations and the spring-petclinic namespace on the cluster.

Prerequisites

  • You have installed the Red Hat OpenShift GitOps Operator on your OpenShift Container Platform cluster.

  • You have logged in to Argo CD instance.

  • You have installed the OpenShift CLI (oc).

  • You have installed the Red Hat OpenShift GitOps argocd CLI.

Procedure

  1. Get the admin account password for the Argo CD server:

    $ ADMIN_PASSWD=$(oc get secret openshift-gitops-cluster -n openshift-gitops -o jsonpath='{.data.admin\.password}' | base64 -d)

  2. Get the Argo CD server URL:

    $ SERVER_URL=$(oc get routes openshift-gitops-server -n openshift-gitops -o jsonpath='{.status.ingress[0].host}')

  3. Log in to the Argo CD server by using the admin account password and enclosing it in single quotes:

    Enclosing the password in single quotes ensures that special characters, such as $, are not misinterpreted by the shell. Always use single quotes to enclose the literal value of the password.

    		 
    $ argocd login --username admin --password ${ADMIN_PASSWD} ${SERVER_URL}
    Example
    $ argocd login --username admin --password '<password>' openshift-gitops.openshift-gitops.apps-crc.testing

  4. Because the application is configured with the none sync policy, you must manually trigger the sync operation:

    $ argocd app sync openshift-gitops/app-cluster-configs

  5. List the application to confirm that the application has the Healthy and Synced statuses:

    $ argocd app list

Synchronizing an application in core mode by using the GitOps CLI

You can synchronize applications in core mode by using the GitOps argocd CLI.

This sample workflow walks you through the process of configuring Argo CD to recursively sync the content of the cluster directory to the cluster-configs application. The directory defines the OpenShift Container Platform cluster configurations and the spring-petclinic namespace on the cluster.

Prerequisites

  • You have installed the Red Hat OpenShift GitOps Operator on your OpenShift Container Platform cluster.

  • You have installed the OpenShift CLI (oc).

  • You have installed the Red Hat OpenShift GitOps argocd CLI.

Procedure

  1. Log in to the OpenShift Container Platform cluster by using the oc CLI tool:

    $ oc login -u <username> -p <password> <server_url>
    Example
    $ oc login -u kubeadmin -p '<password>' https://api.crc.testing:6443

  2. Check whether the context is set correctly in the kubeconfig file:

    $ oc config current-context

  3. Set the default namespace of the current context to openshift-gitops:

    $ oc config set-context --current --namespace openshift-gitops

  4. Set the following environment variable to override the Argo CD component names:

    $ export ARGOCD_REPO_SERVER_NAME=openshift-gitops-repo-server

  5. Because the application is configured with the none sync policy, you must manually trigger the sync operation:

    $ argocd app sync --core openshift-gitops/app-cluster-configs

  6. List the application to confirm that the application has the Healthy and Synced statuses:

    $ argocd app list --core

In-built permissions for cluster configuration

By default, the Argo CD instance has permissions to manage specific cluster-scoped resources such as cluster Operators, optional OLM Operators and user management.

Argo CD does not have cluster-admin permissions.

Permissions for the Argo CD instance:

Resources Descriptions

Resource Groups

Configure the user or administrator

operators.coreos.com

Optional Operators managed by OLM

user.openshift.io , rbac.authorization.k8s.io

Groups, Users and their permissions

config.openshift.io

Control plane Operators managed by CVO used to configure cluster-wide build configuration, registry configuration and scheduler policies

storage.k8s.io

Storage

console.openshift.io

Console customization

Adding permissions for cluster configuration

You can grant permissions for an Argo CD instance to manage cluster configuration. Create a cluster role with additional permissions and then create a new cluster role binding to associate the cluster role with a service account.

Prerequisites

  • You have access to an OpenShift Container Platform cluster with cluster-admin privileges and are logged in to the web console.

  • You have installed the Red Hat OpenShift GitOps Operator on your OpenShift Container Platform cluster.

Procedure

  1. In the web console, select User Management → Roles → Create Role. Use the following ClusterRole YAML template to add rules to specify the additional permissions.

    apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: secrets-cluster-role
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["*"]

  2. Click Create to add the cluster role.

  3. To create the cluster role binding, select User Management → Role Bindings → Create Binding.

  4. Select All Projects from the Project list.

  5. Click Create binding.

  6. Select Binding type as Cluster-wide role binding (ClusterRoleBinding).

  7. Enter a unique value for the RoleBinding name.

  8. Select the newly created cluster role or an existing cluster role from the drop-down list.

  9. Select the Subject as ServiceAccount and the provide the Subject namespace and name.

    1. Subject namespace: openshift-gitops

    2. Subject name: openshift-gitops-argocd-application-controller

      The value of Subject name depends on the GitOps control plane components for which you create the cluster roles and cluster role bindings.

  10. Click Create. The YAML file for the ClusterRoleBinding object is as follows:

    kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: cluster-role-binding
subjects:
  - kind: ServiceAccount
    name: openshift-gitops-argocd-application-controller
    namespace: openshift-gitops
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: secrets-cluster-role
Additional resources

Installing OLM Operators using Red Hat OpenShift GitOps

Red Hat OpenShift GitOps with cluster configurations manages specific cluster-scoped resources and takes care of installing cluster Operators or any namespace-scoped OLM Operators.

Consider a case where as a cluster administrator, you have to install an OLM Operator such as Tekton. You use the OpenShift Container Platform web console to manually install a Tekton Operator or the OpenShift CLI to manually install a Tekton subscription and Tekton Operator group on your cluster.

Red Hat OpenShift GitOps places your Kubernetes resources in your Git repository. As a cluster administrator, use Red Hat OpenShift GitOps to manage and automate the installation of other OLM Operators without any manual procedures. For example, after you place the Tekton subscription in your Git repository by using Red Hat OpenShift GitOps, the Red Hat OpenShift GitOps automatically takes this Tekton subscription from your Git repository and installs the Tekton Operator on your cluster.

Installing cluster-scoped Operators

Operator Lifecycle Manager (OLM) uses a default global-operators Operator group in the openshift-operators namespace for cluster-scoped Operators. Hence you do not have to manage the OperatorGroup resource in your Gitops repository. However, for namespace-scoped Operators, you must manage the OperatorGroup resource in that namespace.

To install cluster-scoped Operators, create and place the Subscription resource of the required Operator in your Git repository.

Example: Grafana Operator subscription
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: grafana
spec:
  channel: v4
  installPlanApproval: Automatic
  name: grafana-operator
  source: redhat-operators
  sourceNamespace: openshift-marketplace

Installing namepace-scoped Operators

To install namespace-scoped Operators, create and place the Subscription and OperatorGroup resources of the required Operator in your Git repository.

Example: Ansible Automation Platform Resource Operator
# ...
apiVersion: v1
kind: Namespace
metadata:
  labels:
    openshift.io/cluster-monitoring: "true"
  name: ansible-automation-platform
# ...
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: ansible-automation-platform-operator
  namespace: ansible-automation-platform
spec:
  targetNamespaces:
    - ansible-automation-platform
# ...
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: ansible-automation-platform
  namespace: ansible-automation-platform
spec:
  channel: patch-me
  installPlanApproval: Automatic
  name: ansible-automation-platform-operator
  source: redhat-operators
  sourceNamespace: openshift-marketplace
# ...

When deploying multiple Operators using Red Hat OpenShift GitOps, you must create only a single Operator group in the corresponding namespace. If more than one Operator group exists in a single namespace, any CSV created in that namespace transition to a failure state with the TooManyOperatorGroups reason. After the number of Operator groups in their corresponding namespaces reaches one, all the previous failure state CSVs transition to pending state. You must manually approve the pending install plan to complete the Operator installation.

Additional resources