With OpenShift Pipelines, you can create a customized CI/CD solution to build, test, and deploy your application.

To create a full-fledged, self-serving CI/CD Pipeline for an application, you must perform the following tasks:

  • Create custom Tasks, or install existing reusable Tasks.

  • Create a Pipeline and PipelineResources to define the delivery Pipeline for your application.

  • Create a PipelineRun to instantiate and invoke the Pipeline.

  • Add Triggers to capture any events in the source repository.

This section uses the pipelines-tutorial example to demonstrate the preceding tasks. The example uses a simple application which consists of:

  • A front-end interface vote-ui, with the source code in ui-repo Git repository.

  • A back-end interface vote-api, with the source code in api-repo Git repository.

  • apply_manifest and update-deployment Tasks in pipelines-tutorial Git repository.

Prerequisites
  • You have access to an OpenShift Container Platform cluster.

  • You have installed OpenShift Pipelines using the OpenShift Pipelines Operator listed in the OpenShift OperatorHub. Once installed, it is applicable to the entire cluster.

  • You have installed OpenShift Pipelines CLI.

  • You have forked the front-end ui-repo and back-end api-repo Git repositories using your GitHub ID.

  • You have Administrator access to your repositories.

Creating a project and checking your Pipeline ServiceAccount

Procedure
  1. Log in to your OpenShift Container Platform cluster:

    $ oc login -u <login> -p <password> https://openshift.example.com:6443
  2. Create a project for the sample application. For this example workflow, create the pipelines-tutorial project:

    $ oc new-project pipelines-tutorial

    If you create a project with a different name, be sure to update the resource URLs used in the example with your project name.

  3. View the pipeline ServiceAccount:

    OpenShift Pipelines Operator adds and configures a ServiceAccount named pipeline that has sufficient permissions to build and push an image. This ServiceAccount is used by PipelineRun.

    $ oc get serviceaccount pipeline

About Tasks

Tasks are the building blocks of a Pipeline and consist of sequentially executed Steps. Steps are a series of commands that achieve a specific goal, such as building an image.

Every Task runs as a Pod and each Step runs in its own container within the same Pod. Because Steps run within the same Pod, they have access to the same volumes for caching files, configmaps, and secrets.

A Task uses inputs parameters, such as a Git resource, and outputs parameters, such as an image in a registry, to interact with other Tasks. They are reusable and can be used in multiple Pipelines.

Here is an example of a Maven Task with a single Step to build a Maven-based Java application.

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: maven-build
spec:
  resources:
    inputs:
    - name: workspace-git
      targetPath: /
      type: git
  steps:
  - name: build
    image: maven:3.6.0-jdk-8-slim
    command:
    - /usr/bin/mvn
    args:
    - install

This Task starts the Pod and runs a container inside that Pod using the maven:3.6.0-jdk-8-slim image to run the specified commands. It receives an input directory called workspace-git that contains the source code of the application.

The Task only declares the placeholder for the Git repository, it does not specify which Git repository to use. This allows Tasks to be reusable for multiple Pipelines and purposes.

Creating Pipeline Tasks

Procedure
  1. Install the apply-manifests and update-deployment Tasks from the pipelines-tutorial repository, which contains a list of reusable Tasks for Pipelines:

    $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/release-tech-preview-1/01_pipeline/01_apply_manifest_task.yaml
    $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/release-tech-preview-1/01_pipeline/02_update_deployment_task.yaml
  2. Use the tkn task list command to list the Tasks you created:

    $ tkn task list

    The output verifies that the apply-manifests and update-deployment Tasks were created:

    NAME                DESCRIPTION   AGE
    apply-manifests                   1 minute ago
    update-deployment                 48 seconds ago
  3. Use the tkn clustertasks list command to list the Operator-installed additional ClusterTasks, for example --buildah and s2i-python-3:

    $ tkn clustertasks list

    The output lists the Operator-installed ClusterTasks:

    NAME                       DESCRIPTION   AGE
    buildah                                  1 day ago
    buildah-v0-11-3                          1 day ago
    git-clone                                1 day ago
    jib-maven                                1 day ago
    kn                                       1 day ago
    maven                                    1 day ago
    openshift-client                         1 day ago
    openshift-client-v0-11-3                 1 day ago
    s2i                                      1 day ago
    s2i-dotnet-3                             1 day ago
    s2i-dotnet-3-v0-11-3                     1 day ago
    s2i-go                                   1 day ago
    s2i-go-v0-11-3                           1 day ago
    s2i-java-11                              1 day ago
    s2i-java-11-v0-11-3                      1 day ago
    s2i-java-8                               1 day ago
    s2i-java-8-v0-11-3                       1 day ago
    s2i-nodejs                               1 day ago
    s2i-nodejs-v0-11-3                       1 day ago
    s2i-perl                                 1 day ago
    s2i-perl-v0-11-3                         1 day ago
    s2i-php                                  1 day ago
    s2i-php-v0-11-3                          1 day ago
    s2i-python-3                             1 day ago
    s2i-python-3-v0-11-3                     1 day ago
    s2i-ruby                                 1 day ago
    s2i-ruby-v0-11-3                         1 day ago
    s2i-v0-11-3                              1 day ago
    tkn                                      1 day ago

Defining and creating PipelineResources

PipelineResources are artifacts that are used as inputs or outputs of a Task.

After you create Tasks, create PipelineResources that contain the specifics of the Git repository and the image registry to be used in the Pipeline during execution:

If you are not in the pipelines-tutorial namespace, and are using another namespace, ensure you update the front-end and back-end image resource to the correct URL with your namespace in the steps below. For example:

image-registry.openshift-image-registry.svc:5000/<namespace-name>/vote-api:latest
Procedure
  1. Create a PipelineResource that defines the Git repository for the front-end application:

    $ tkn resource create
    ? Enter a name for a pipeline resource : ui-repo
    ? Select a resource type to create : git
    ? Enter a value for url : http://github.com/openshift-pipelines/vote-ui.git
    ? Enter a value for revision : release-tech-preview-1

    The output verifies that the ui-repo PipelineResource was created.

    New git resource "ui-repo" has been created
  2. Create a PipelineResource that defines the OpenShift Container Platform internal image registry to where you want to push the front-end image:

    $ tkn resource create
    ? Enter a name for a pipeline resource : ui-image
    ? Select a resource type to create : image
    ? Enter a value for url : image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/ui:latest
    ? Enter a value for digest :

    The output verifies that the ui-image PipelineResource was created.

    New image resource "ui-image" has been created
  3. Create a PipelineResource that defines the Git repository for the back-end application:

    $ tkn resource create
    ? Enter a name for a pipeline resource : api-repo
    ? Select a resource type to create : git
    ? Enter a value for url : http://github.com/openshift-pipelines/vote-api.git
    ? Enter a value for revision : release-tech-preview-1

    The output verifies that the api-repo PipelineResource was created.

    New git resource "api-repo" has been created
  4. Create a PipelineResource that defines the OpenShift Container Platform internal image registry to where you want to push the back-end image:

    $ tkn resource create
    ? Enter a name for a pipeline resource : api-image
    ? Select a resource type to create : image
    ? Enter a value for url : image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/api:latest
    ? Enter a value for digest :

    The output verifies that the api-image PipelineResource was created.

    New image resource "api-image" has been created
  5. View the list of resources created:

    $ tkn resource list

    The output lists all the PipelineResource that were created.

    NAME        TYPE    DETAILS
    api-repo    git     url: http://github.com/openshift-pipelines/vote-api.git
    ui-repo     git     url: http://github.com/openshift-pipelines/vote-ui.git
    api-image   image   url: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/api:latest
    ui-image    image   url: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/ui:latest

Assembling a Pipeline

A Pipeline represents a CI/CD flow and is defined by the Tasks to be executed. It specifies how the Tasks interact with each other and their order of execution using the inputs , outputs, and runAfter parameters. It is designed to be generic and reusable in multiple applications and environments.

In this section, you will create a Pipeline that takes the source code of the application from GitHub and then builds and deploys it on OpenShift Container Platform.

The Pipeline performs the following tasks for the back-end application vote-api and front-end application vote-ui:

  • Clones the source code of the application from the Git repositories api-repo and ui-repo.

  • Builds the container images api-image and ui-image using the buildah ClusterTask.

  • Pushes the api-image and ui-image images to the internal image registry.

  • Deploys the new images on OpenShift Container Platform using the apply-manifests and update-deployment Tasks.

Procedure
  1. Copy the contents of the following sample Pipeline YAML file and save it:

    apiVersion: tekton.dev/v1beta1
    kind: Pipeline
    metadata:
      name: build-and-deploy
    spec:
      resources:
      - name: git-repo
        type: git
      - name: image
        type: image
      params:
      - name: deployment-name
        type: string
        description: name of the deployment to be patched
      tasks:
      - name: build-image
        taskRef:
          name: buildah
          kind: ClusterTask
        resources:
          inputs:
          - name: source
            resource: git-repo
          outputs:
          - name: image
            resource: image
        params:
        - name: TLSVERIFY
          value: "false"
      - name: apply-manifests
        taskRef:
          name: apply-manifests
        resources:
          inputs:
          - name: source
            resource: git-repo
        runAfter:
        - build-image
      - name: update-deployment
        taskRef:
          name: update-deployment
        resources:
          inputs:
          - name: image
            resource: image
        params:
        - name: deployment
          value: $(params.deployment-name)
        runAfter:
        - apply-manifests

    Notice that the Pipeline definition abstracts away the specifics of the Git source repository and image registries to be used during the Pipeline execution.

  2. Create the Pipeline:

    $ oc create -f <pipeline-yaml-file-name.yaml>

    Alternatively, you can also execute the YAML file directly from the Git repository:

    $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/release-tech-preview-1/01_pipeline/04_pipeline.yaml
  3. Use the tkn pipeline list command to verify that the Pipeline is added to the application:

    $ tkn pipeline list

    The output verifies that the build-and-deploy Pipeline was created:

    NAME               AGE            LAST RUN   STARTED   DURATION   STATUS
    build-and-deploy   1 minute ago   ---        ---       ---        ---

Running a Pipeline

A PipelineRun starts a Pipeline and ties it to the Git and image resources that should be used for the specific invocation. It automatically creates and starts the TaskRuns for each Task in the Pipeline.

Procedure
  1. Start the Pipeline for the back-end application:

    $ tkn pipeline start build-and-deploy -r git-repo=api-repo -r image=api-image -p deployment-name=vote-api

    Note the PipelineRun ID returned in the command output.

  2. Track the PipelineRun progress:

    $ tkn pipelinerun logs <pipelinerun ID> -f
  3. Start the Pipeline for the front-end application:

    $ tkn pipeline start build-and-deploy -r git-repo=ui-repo -r image=ui-image -p deployment-name=vote-ui

    Note the PipelineRun ID returned in the command output.

  4. Track the PipelineRun progress:

    $ tkn pipelinerun logs <pipelinerun ID> -f
  5. After a few minutes, use tkn pipelinerun list command to verify that the Pipeline ran successfully by listing all the PipelineRuns:

    $ tkn pipelinerun list

    The output lists the PipelineRuns:

     NAME                         STARTED      DURATION     STATUS
     build-and-deploy-run-xy7rw   1 hour ago   2 minutes    Succeeded
     build-and-deploy-run-z2rz8   1 hour ago   19 minutes   Succeeded
  6. Get the application route:

    $ oc get route vote-ui --template='http://{{.spec.host}}'

    Note the output of the previous command. You can access the application using this route.

  7. To rerun the last PipelineRun, using the PipelineResources and ServiceAccount of the previous Pipeline, run:

    $ tkn pipeline start build-and-deploy --last

About Triggers

Use Triggers in conjunction with Pipelines to create a full-fledged CI/CD system where the Kubernetes resources define the entire CI/CD execution. Pipeline Triggers capture the external events and process them to extract key pieces of information. Mapping this event data to a set of predefined parameters triggers a series of tasks that can then create and deploy Kubernetes resources.

For example, you define a CI/CD workflow using OpenShift Pipelines for your application. The PipelineRun must start for any new changes to take effect in the application repository. Triggers automate this process by capturing and processing any change events, and by triggering a PipelineRun that deploys the new image with the latest changes.

Triggers consist of the following main components that work together to form a reusable, decoupled, and self-sustaining CI/CD system:

  • EventListeners provide endpoints, or an event sink, that listen for incoming HTTP-based events with a JSON payload. The EventListener performs lightweight event processing on the payload using Event Interceptors, which identify the type of payload and optionally modify it. Currently, Pipeline Triggers support four types of Interceptors: Webhook Interceptors, GitHub Interceptors, GitLab Interceptors, and Common Expression Language (CEL) Interceptors.

  • TriggerBindings extract the fields from an event payload and store them as parameters.

  • TriggerTemplates specify how to use the parameterized data from the TriggerBindings. A TriggerTemplate defines a resource template that receives input from the TriggerBindings, while then performing a series of actions that result in creation of new PipelineResources and initiation of a new PipelineRun.

EventListeners tie the concepts of TriggerBindings and TriggerTemplates together. The EventListener listens for the incoming event, handles basic filtering using Interceptors, extracts data using TriggerBindings, and then processes this data to create Kubernetes resources using TriggerTemplates.

Adding Triggers to a Pipeline

After you have assembled and started the Pipeline for the application, add TriggerBindings, TriggerTemplates, and an EventListener to capture GitHub events.

Procedure
  1. Copy the content of the following sample TriggerBinding YAML file and save it:

    apiVersion: triggers.tekton.dev/v1alpha1
    kind: TriggerBinding
    metadata:
      name: vote-app
    spec:
      params:
      - name: git-repo-url
        value: $(body.repository.url)
      - name: git-repo-name
        value: $(body.repository.name)
      - name: git-revision
        value: $(body.head_commit.id)
  2. Create the TriggerBinding:

    $ oc create -f <triggerbinding-yaml-file-name.yaml>

    Alternatively, you can create the TriggerBinding directly from the pipelines-tutorial Git repository:

    $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/release-tech-preview-1/03_triggers/01_binding.yaml
  3. Copy the content of the following sample TriggerTemplate YAML file and save it:

    apiVersion: triggers.tekton.dev/v1alpha1
    kind: TriggerTemplate
    metadata:
      name: vote-app
    spec:
      params:
      - name: git-repo-url
        description: The git repository url
      - name: git-revision
        description: The git revision
        default: master
      - name: git-repo-name
        description: The name of the deployment to be created / patched
    
      resourcetemplates:
      - apiVersion: tekton.dev/v1alpha1
        kind: PipelineResource
        metadata:
          name: $(params.git-repo-name)-git-repo-$(uid)
        spec:
          type: git
          params:
          - name: revision
            value: $(params.git-revision)
          - name: url
            value: $(params.git-repo-url)
    
      - apiVersion: tekton.dev/v1alpha1
        kind: PipelineResource
        metadata:
          name: $(params.git-repo-name)-image-$(uid)
        spec:
          type: image
          params:
          - name: url
            value: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/$(params.git-repo-name):latest
    
      - apiVersion: tekton.dev/v1beta1
        kind: PipelineRun
        metadata:
          name: build-deploy-$(params.git-repo-name)-$(uid)
        spec:
          serviceAccountName: pipeline
          pipelineRef:
            name: build-and-deploy
          resources:
          - name: git-repo
            resourceRef:
              name: $(params.git-repo-name)-git-repo-$(uid)
          - name: image
            resourceRef:
              name: $(params.git-repo-name)-image-$(uid)
          params:
          - name: deployment-name
            value: $(params.git-repo-name)
  4. Create the TriggerTemplate:

    $ oc create -f <triggertemplate-yaml-file-name.yaml>

    Alternatively, you can create the TriggerTemplate directly from the pipelines-tutorial Git repository:

    $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/release-tech-preview-1/03_triggers/02_template.yaml
  5. Copy the contents of the following sample EventListener YAML file and save it:

    apiVersion: triggers.tekton.dev/v1alpha1
    kind: EventListener
    metadata:
      name: vote-app
    spec:
      serviceAccountName: pipeline
      triggers:
      - bindings:
        - name: vote-app
        template:
          name: vote-app
  6. Create the EventListener:

    $ oc create -f <eventlistener-yaml-file-name.yaml>

    Alternatively, you can create the EvenListener directly from the pipelines-tutorial Git repository:

    $ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/release-tech-preview-1/03_triggers/03_event_listener.yaml
  7. Expose the EventListener service as an OpenShift Container Platform route to make it publicly accessible:

    $ oc expose svc el-vote-app

Creating Webhooks

Webhooks are HTTP POST messages that are received by the EventListeners whenever a configured event occurs in your repository. The event payload is then mapped to TriggerBindings, and processed by TriggerTemplates. The TriggerTemplates eventually start one or more PipelineRuns, leading to the creation and deployment of Kubernetes resources.

In this section, you will configure a Webhook URL on your forked Git repositories vote-ui and vote-api. This URL points to the publicly accessible EventListener service route.

Adding Webhooks requires administrative privileges to the repository. If you do not have administrative access to your repository, contact your system administrator for adding Webhooks.

Procedure
  1. Get the Webhook URL:

    $ echo "URL: $(oc  get route el-vote-app --template='http://{{.spec.host}}')"

    Note the URL obtained in the output.

  2. Configure Webhooks manually on the front-end repository:

    1. Open the front-end Git repository vote-ui in your browser.

    2. Click SettingsWebhooksAdd Webhook

    3. On the Webhooks/Add Webhook page:

      1. Enter the Webhook URL from step 1 in Payload URL field

      2. Select application/json for the Content type

      3. Specify the secret in the Secret field

      4. Ensure that the Just the push event is selected

      5. Select Active

      6. Click Add Webhook

  3. Repeat step 2 for the back-end repository vote-api.

Triggering a PipelineRun

Whenever a push event occurs in the Git repository, the configured Webhook sends an event payload to the publicly exposed EventListener service route. The EventListener service of the application processes the payload, and passes it to the relevant TriggerBindings and TriggerTemplates pair. The TriggerBinding extracts the parameters and the TriggerTemplate uses these parameters to create resources. This may rebuild and redeploy the application.

In this section, you will push an empty commit to the front-end vote-api repository, which will trigger the PipelineRun.

Procedure
  1. From the terminal, clone your forked Git repository vote-api:

    $ git clone git@github.com:<your GitHub ID>/vote-api.git -b release-tech-preview-1
  2. Push an empty commit:

    $ git commit -m "empty-commit" --allow-empty && git push origin release-tech-preview-1
  3. Check if the PipelineRun was triggered:

    $ tkn pipelinerun list

    Notice that a new PipelineRun was initiated.

Additional resources