×

Red Hat OpenShift Pipelines is a cloud-native, continuous integration and continuous delivery (CI/CD) solution based on Kubernetes resources. It uses Tekton building blocks to automate deployments across multiple platforms by abstracting away the underlying implementation details. Tekton introduces a number of standard custom resource definitions (CRDs) for defining CI/CD pipelines that are portable across Kubernetes distributions.

Key features

  • Red Hat OpenShift Pipelines is a serverless CI/CD system that runs pipelines with all the required dependencies in isolated containers.

  • Red Hat OpenShift Pipelines are designed for decentralized teams that work on microservice-based architecture.

  • Red Hat OpenShift Pipelines use standard CI/CD pipeline definitions that are easy to extend and integrate with the existing Kubernetes tools, enabling you to scale on-demand.

  • You can use Red Hat OpenShift Pipelines to build images with Kubernetes tools such as Source-to-Image (S2I), Buildah, Buildpacks, and Kaniko that are portable across any Kubernetes platform.

  • You can use the OpenShift Container Platform Developer console to create Tekton resources, view logs of pipeline runs, and manage pipelines in your OpenShift Container Platform namespaces.

OpenShift Pipeline Concepts

This guide provides a detailed view of the various pipeline concepts.

Tasks

Tasks are the building blocks of a pipeline and consists of sequentially executed steps. It is essentially a function of inputs and outputs. A task can run individually or as a part of the pipeline. Tasks are reusable and can be used in multiple Pipelines.

Steps are a series of commands that are sequentially executed by the task and achieve a specific goal, such as building an image. Every task runs as a pod, and each step runs as a container within that pod. Because steps run within the same pod, they can access the same volumes for caching files, config maps, and secrets.

The following example shows the apply-manifests task.

apiVersion: tekton.dev/v1beta1 (1)
kind: Task (2)
metadata:
  name: apply-manifests (3)
spec: (4)
  workspaces:
  - name: source
  params:
    - name: manifest_dir
      description: The directory in source that contains yaml manifests
      type: string
      default: "k8s"
  steps:
    - name: apply
      image: image-registry.openshift-image-registry.svc:5000/openshift/cli:latest
      workingDir: /workspace/source
      command: ["/bin/bash", "-c"]
      args:
        - |-
          echo Applying manifests in $(params.manifest_dir) directory
          oc apply -f $(params.manifest_dir)
          echo -----------------------------------
1 The task API version, v1beta1.
2 The type of Kubernetes object, Task.
3 The unique name of this task.
4 The list of parameters and steps in the task and the workspace used by the task.

This task starts the pod and runs a container inside that pod using the specified image to run the specified commands.

Starting with Pipelines 1.6, the following defaults from the step YAML file are removed:

  • The HOME environment variable does not default to the /tekton/home directory

  • The workingDir field does not default to the /workspace directory

Instead, the container for the step defines the HOME environment variable and the workingDir field. However, you can override the default values by specifying the custom values in the YAML file for the step.

As a temporary measure, to maintain backward compatibility with the older Pipelines versions, you can set the following fields in the TektonConfig custom resource definition to false:

spec:
  pipeline:
    disable-working-directory-overwrite: false
    disable-home-env-overwrite: false

When expression

When expressions guard task execution by setting criteria for the execution of tasks within a pipeline. They contain a list of components that allows a task to run only when certain criteria are met. When expressions are also supported in the final set of tasks that are specified using the finally field in the pipeline YAML file.

The key components of a when expression are as follows:

  • input: Specifies static inputs or variables such as a parameter, task result, and execution status. You must enter a valid input. If you do not enter a valid input, its value defaults to an empty string.

  • operator: Specifies the relationship of an input to a set of values. Enter in or notin as your operator values.

  • values: Specifies an array of string values. Enter a non-empty array of static values or variables such as parameters, results, and a bound state of a workspace.

The declared when expressions are evaluated before the task is run. If the value of a when expression is True, the task is run. If the value of a when expression is False, the task is skipped.

You can use the when expressions in various use cases. For example, whether:

  • The result of a previous task is as expected.

  • A file in a Git repository has changed in the previous commits.

  • An image exists in the registry.

  • An optional workspace is available.

The following example shows the when expressions for a pipeline run. The pipeline run will execute the create-file task only if the following criteria are met: the path parameter is README.md, and the echo-file-exists task executed only if the exists result from the check-file task is yes.

apiVersion: tekton.dev/v1beta1
kind: PipelineRun (1)
metadata:
  generateName: guarded-pr-
spec:
  serviceAccountName: 'pipeline'
  pipelineSpec:
    params:
      - name: path
        type: string
        description: The path of the file to be created
    workspaces:
      - name: source
        description: |
          This workspace is shared among all the pipeline tasks to read/write common resources
    tasks:
      - name: create-file (2)
        when:
          - input: "$(params.path)"
            operator: in
            values: ["README.md"]
        workspaces:
          - name: source
            workspace: source
        taskSpec:
          workspaces:
            - name: source
              description: The workspace to create the readme file in
          steps:
            - name: write-new-stuff
              image: ubuntu
              script: 'touch $(workspaces.source.path)/README.md'
      - name: check-file
        params:
          - name: path
            value: "$(params.path)"
        workspaces:
          - name: source
            workspace: source
        runAfter:
          - create-file
        taskSpec:
          params:
            - name: path
          workspaces:
            - name: source
              description: The workspace to check for the file
          results:
            - name: exists
              description: indicates whether the file exists or is missing
          steps:
            - name: check-file
              image: alpine
              script: |
                if test -f $(workspaces.source.path)/$(params.path); then
                  printf yes | tee /tekton/results/exists
                else
                  printf no | tee /tekton/results/exists
                fi
      - name: echo-file-exists
        when: (3)
          - input: "$(tasks.check-file.results.exists)"
            operator: in
            values: ["yes"]
        taskSpec:
          steps:
            - name: echo
              image: ubuntu
              script: 'echo file exists'
...
      - name: task-should-be-skipped-1
        when: (4)
          - input: "$(params.path)"
            operator: notin
            values: ["README.md"]
        taskSpec:
          steps:
            - name: echo
              image: ubuntu
              script: exit 1
...
    finally:
      - name: finally-task-should-be-executed
        when: (5)
          - input: "$(tasks.echo-file-exists.status)"
            operator: in
            values: ["Succeeded"]
          - input: "$(tasks.status)"
            operator: in
            values: ["Succeeded"]
          - input: "$(tasks.check-file.results.exists)"
            operator: in
            values: ["yes"]
          - input: "$(params.path)"
            operator: in
            values: ["README.md"]
        taskSpec:
          steps:
            - name: echo
              image: ubuntu
              script: 'echo finally done'
  params:
    - name: path
      value: README.md
  workspaces:
    - name: source
      volumeClaimTemplate:
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 16Mi
1 Specifies the type of Kubernetes object. In this example, PipelineRun.
2 Task create-file used in the Pipeline.
3 when expression that specifies to execute the echo-file-exists task only if the exists result from the check-file task is yes.
4 when expression that specifies to skip the task-should-be-skipped-1 task only if the path parameter is README.md.
5 when expression that specifies to execute the finally-task-should-be-executed task only if the execution status of the echo-file-exists task and the task status is Succeeded, the exists result from the check-file task is yes, and the path parameter is README.md.

The Pipeline Run details page of the OpenShift Container Platform web console shows the status of the tasks and when expressions as follows:

  • All the criteria are met: Tasks and the when expression symbol, which is represented by a diamond shape are green.

  • Any one of the criteria are not met: Task is skipped. Skipped tasks and the when expression symbol are grey.

  • None of the criteria are met: Task is skipped. Skipped tasks and the when expression symbol are grey.

  • Task run fails: Failed tasks and the when expression symbol are red.

Finally tasks

The finally tasks are the final set of tasks specified using the finally field in the pipeline YAML file. A finally task always executes the tasks within the pipeline, irrespective of whether the pipeline runs are executed successfully. The finally tasks are executed in parallel after all the pipeline tasks are run, before the corresponding pipeline exits.

You can configure a finally task to consume the results of any task within the same pipeline. This approach does not change the order in which this final task is run. It is executed in parallel with other final tasks after all the non-final tasks are executed.

The following example shows a code snippet of the clone-cleanup-workspace pipeline. This code clones the repository into a shared workspace and cleans up the workspace. After executing the pipeline tasks, the cleanup task specified in the finally section of the pipeline YAML file cleans up the workspace.

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: clone-cleanup-workspace (1)
spec:
  workspaces:
    - name: git-source (2)
  tasks:
    - name: clone-app-repo (3)
      taskRef:
        name: git-clone-from-catalog
      params:
        - name: url
          value: https://github.com/tektoncd/community.git
        - name: subdirectory
          value: application
      workspaces:
        - name: output
          workspace: git-source
  finally:
    - name: cleanup (4)
      taskRef: (5)
        name: cleanup-workspace
      workspaces: (6)
        - name: source
          workspace: git-source
    - name: check-git-commit
      params: (7)
        - name: commit
          value: $(tasks.clone-app-repo.results.commit)
      taskSpec: (8)
        params:
          - name: commit
        steps:
          - name: check-commit-initialized
            image: alpine
            script: |
              if [[ ! $(params.commit) ]]; then
                exit 1
              fi
1 Unique name of the Pipeline.
2 The shared workspace where the git repository is cloned.
3 The task to clone the application repository to the shared workspace.
4 The task to clean-up the shared workspace.
5 A reference to the task that is to be executed in the TaskRun.
6 A shared storage volume that a Task in a Pipeline needs at runtime to receive input or provide output.
7 A list of parameters required for a task. If a parameter does not have an implicit default value, you must explicitly set its value.
8 Embedded task definition.

TaskRun

A TaskRun instantiates a Task for execution with specific inputs, outputs, and execution parameters on a cluster. It can be invoked on its own or as part of a PipelineRun for each Task in a pipeline.

A Task consists of one or more Steps that execute container images, and each container image performs a specific piece of build work. A TaskRun executes the Steps in a Task in the specified order, until all Steps execute successfully or a failure occurs. A TaskRun is automatically created by a PipelineRun for each Task in a Pipeline.

The following example shows a TaskRun that runs the apply-manifests Task with the relevant input parameters:

apiVersion: tekton.dev/v1beta1 (1)
kind: TaskRun (2)
metadata:
  name: apply-manifests-taskrun (3)
spec: (4)
  serviceAccountName: pipeline
  taskRef: (5)
    kind: Task
    name: apply-manifests
  workspaces: (6)
  - name: source
    persistentVolumeClaim:
      claimName: source-pvc
1 TaskRun API version v1beta1.
2 Specifies the type of Kubernetes object. In this example, TaskRun.
3 Unique name to identify this TaskRun.
4 Definition of the TaskRun. For this TaskRun, the Task and the required workspace are specified.
5 Name of the Task reference used for this TaskRun. This TaskRun executes the apply-manifests Task.
6 Workspace used by the TaskRun.

Pipelines

A Pipeline is a collection of Task resources arranged in a specific order of execution. They are executed to construct complex workflows that automate the build, deployment and delivery of applications. You can define a CI/CD workflow for your application using pipelines containing one or more tasks.

A Pipeline resource definition consists of a number of fields or attributes, which together enable the pipeline to accomplish a specific goal. Each Pipeline resource definition must contain at least one Task resource, which ingests specific inputs and produces specific outputs. The pipeline definition can also optionally include Conditions, Workspaces, Parameters, or Resources depending on the application requirements.

The following example shows the build-and-deploy pipeline, which builds an application image from a Git repository using the buildah ClusterTask resource:

apiVersion: tekton.dev/v1beta1 (1)
kind: Pipeline (2)
metadata:
  name: build-and-deploy (3)
spec: (4)
  workspaces: (5)
  - name: shared-workspace
  params: (6)
  - name: deployment-name
    type: string
    description: name of the deployment to be patched
  - name: git-url
    type: string
    description: url of the git repo for the code of deployment
  - name: git-revision
    type: string
    description: revision to be used from repo of the code for deployment
    default: "pipelines-1.8"
  - name: IMAGE
    type: string
    description: image to be built from the code
  tasks: (7)
  - name: fetch-repository
    taskRef:
      name: git-clone
      kind: ClusterTask
    workspaces:
    - name: output
      workspace: shared-workspace
    params:
    - name: url
      value: $(params.git-url)
    - name: subdirectory
      value: ""
    - name: deleteExisting
      value: "true"
    - name: revision
      value: $(params.git-revision)
  - name: build-image (8)
    taskRef:
      name: buildah
      kind: ClusterTask
    params:
    - name: TLSVERIFY
      value: "false"
    - name: IMAGE
      value: $(params.IMAGE)
    workspaces:
    - name: source
      workspace: shared-workspace
    runAfter:
    - fetch-repository
  - name: apply-manifests (9)
    taskRef:
      name: apply-manifests
    workspaces:
    - name: source
      workspace: shared-workspace
    runAfter: (10)