Soon, all Red Hat OpenShift documentation will only be available on docs.redhat.com, the official location of all Red Hat product documentation. Get familiar with the updated experience today!
  1. Documentation
    2. history
×

Understanding OpenShift Pipelines

Red Hat OpenShift Pipelines is a cloud-native, continuous integration and continuous delivery (CI/CD) solution based on Kubernetes resources. It uses Tekton building blocks to automate deployments across multiple platforms by abstracting away the underlying implementation details. Tekton introduces a number of standard custom resource definitions (CRDs) for defining CI/CD pipelines that are portable across Kubernetes distributions.

Key features

  • Red Hat OpenShift Pipelines is a serverless CI/CD system that runs pipelines with all the required dependencies in isolated containers.

  • Red Hat OpenShift Pipelines are designed for decentralized teams that work on microservice-based architecture.

  • Red Hat OpenShift Pipelines use standard CI/CD pipeline definitions that are easy to extend and integrate with the existing Kubernetes tools, enabling you to scale on-demand.

  • You can use Red Hat OpenShift Pipelines to build images with Kubernetes tools such as Source-to-Image (S2I), Buildah, Buildpacks, and Kaniko that are portable across any Kubernetes platform.

  • You can use the OpenShift Container Platform Developer console to create Tekton resources, view logs of pipeline runs, and manage pipelines in your OpenShift Container Platform namespaces.

OpenShift Pipeline Concepts

This guide provides a detailed view of the various pipeline concepts.

Tasks

Tasks are the building blocks of a Pipeline and consists of sequentially executed steps. It is essentially a function of inputs and outputs. A Task can run individually or as a part of the pipeline. Tasks are reusable and can be used in multiple Pipelines.

Steps are a series of commands that are sequentially executed by the Task and achieve a specific goal, such as building an image. Every Task runs as a pod, and each Step runs as a container within that pod. Because Steps run within the same pod, they can access the same volumes for caching files, config maps, and secrets.

The following example shows the apply-manifests Task.

apiVersion: tekton.dev/v1beta1 (1)
kind: Task (2)
metadata:
  name: apply-manifests (3)
spec: (4)
  workspaces:
  - name: source
  params:
    - name: manifest_dir
      description: The directory in source that contains yaml manifests
      type: string
      default: "k8s"
  steps:
    - name: apply
      image: image-registry.openshift-image-registry.svc:5000/openshift/cli:latest
      workingDir: /workspace/source
      command: ["/bin/bash", "-c"]
      args:
        - |-
          echo Applying manifests in $(params.manifest_dir) directory
          oc apply -f $(params.manifest_dir)
          echo -----------------------------------
1 The Task API version, v1beta1.
2 The type of Kubernetes object, Task.
3 The unique name of this Task.
4 The list of parameters and Steps in the Task and the workspace used by the Task.

This Task starts the pod and runs a container inside that pod using the specified image to run the specified commands.

TaskRun

A TaskRun instantiates a Task for execution with specific inputs, outputs, and execution parameters on a cluster. It can be invoked on its own or as part of a PipelineRun for each Task in a pipeline.

A Task consists of one or more Steps that execute container images, and each container image performs a specific piece of build work. A TaskRun executes the Steps in a Task in the specified order, until all Steps execute successfully or a failure occurs. A TaskRun is automatically created by a PipelineRun for each Task in a Pipeline.

The following example shows a TaskRun that runs the apply-manifests Task with the relevant input parameters:

apiVersion: tekton.dev/v1beta1 (1)
kind: TaskRun (2)
metadata:
  name: apply-manifests-taskrun (3)
spec: (4)
  serviceAccountName: pipeline
  taskRef: (5)
    kind: Task
    name: apply-manifests
  workspaces: (6)
  - name: source
    persistentVolumeClaim:
      claimName: source-pvc
1 TaskRun API version v1beta1.
2 Specifies the type of Kubernetes object. In this example, TaskRun.
3 Unique name to identify this TaskRun.
4 Definition of the TaskRun. For this TaskRun, the Task and the required workspace are specified.
5 Name of the Task reference used for this TaskRun. This TaskRun executes the apply-manifests Task.
6 Workspace used by the TaskRun.

Pipelines

A Pipeline is a collection of Task resources arranged in a specific order of execution. They are executed to construct complex workflows that automate the build, deployment and delivery of applications. You can define a CI/CD workflow for your application using pipelines containing one or more tasks.

A Pipeline resource definition consists of a number of fields or attributes, which together enable the pipeline to accomplish a specific goal. Each Pipeline resource definition must contain at least one Task resource, which ingests specific inputs and produces specific outputs. The pipeline definition can also optionally include Conditions, Workspaces, Parameters, or Resources depending on the application requirements.

The following example shows the build-and-deploy pipeline, which builds an application image from a Git repository using the buildah ClusterTask resource:

apiVersion: tekton.dev/v1beta1 (1)
kind: Pipeline (2)
metadata:
  name: build-and-deploy (3)
spec: (4)
  workspaces: (5)
  - name: shared-workspace
  params: (6)
  - name: deployment-name
    type: string
    description: name of the deployment to be patched
  - name: git-url
    type: string
    description: url of the git repo for the code of deployment
  - name: git-revision
    type: string
    description: revision to be used from repo of the code for deployment
    default: "pipelines-1.4"
  - name: IMAGE
    type: string
    description: image to be built from the code
  tasks: (7)
  - name: fetch-repository
    taskRef:
      name: git-clone
      kind: ClusterTask
    workspaces:
    - name: output
      workspace: shared-workspace
    params:
    - name: url
      value: $(params.git-url)
    - name: subdirectory
      value: ""
    - name: deleteExisting
      value: "true"
    - name: revision
      value: $(params.git-revision)
  - name: build-image (8)
    taskRef:
      name: buildah
      kind: ClusterTask
    params:
    - name: TLSVERIFY
      value: "false"
    - name: IMAGE
      value: $(params.IMAGE)
    workspaces:
    - name: source
      workspace: shared-workspace
    runAfter:
    - fetch-repository
  - name: apply-manifests (9)
    taskRef:
      name: apply-manifests
    workspaces:
    - name: source
      workspace: shared-workspace
    runAfter: (10)
    - build-image
  - name: update-deployment
    taskRef:
      name: update-deployment
    workspaces:
    - name: source
      workspace: shared-workspace
    params:
    - name: deployment
      value: $(params.deployment-name)
    - name: IMAGE
      value: $(params.IMAGE)
    runAfter:
    - apply-manifests
1 Pipeline API version v1beta1.
2 Specifies the type of Kubernetes object. In this example, Pipeline.
3 Unique name of this Pipeline.
4 Specifies the definition and structure of the Pipeline.
5 Workspaces used across all the Tasks in the Pipeline.
6 Parameters used across all the Tasks in the Pipeline.
7 Specifies the list of Tasks used in the Pipeline.
8 Task build-image, which uses the buildah ClusterTask to build application images from a given Git repository.
9 Task apply-manifests, which uses a user-defined Task with the same name.
10 Specifies the sequence in which Tasks are run in a Pipeline. In this example, the apply-manifests Task is run only after the build-image Task is completed.

PipelineRun

A PipelineRun is the running instance of a Pipeline. It instantiates a Pipeline for execution with specific inputs, outputs, and execution parameters on a cluster. A corresponding TaskRun is created for each Task automatically in the PipelineRun.

All the Tasks in the Pipeline are executed in the defined sequence until all Tasks are successful or a Task fails. The status field tracks and stores the progress of each TaskRun in the PipelineRun for monitoring and auditing purpose.

The following example shows a PipelineRun to run the build-and-deploy Pipeline with relevant resources and parameters:

apiVersion: tekton.dev/v1beta1 (1)
kind: PipelineRun (2)
metadata:
  name: build-deploy-api-pipelinerun (3)
spec:
  pipelineRef:
    name: build-and-deploy (4)
  params: (5)
  - name: deployment-name
    value: vote-api
  - name: git-url
    value: https://github.com/openshift-pipelines/vote-api.git
  - name: IMAGE
    value: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/vote-api
  workspaces: (6)
  - name: shared-workspace
    volumeClaimTemplate:
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 500Mi
1 PipelineRun API version v1beta1.
2 Specifies the type of Kubernetes object. In this example, PipelineRun.
3 Unique name to identify this PipelineRun.
4 Name of the Pipeline to be run. In this example, build-and-deploy.
5 Specifies the list of parameters required to run the Pipeline.
6 Workspace used by the PipelineRun.

Workspaces

It is recommended that you use Workspaces instead of PipelineResources in OpenShift Pipelines, as PipelineResources are difficult to debug, limited in scope, and make Tasks less reusable.

Workspaces declare shared storage volumes that a Task in a Pipeline needs at runtime to receive input or provide output. Instead of specifying the actual location of the volumes, Workspaces enable you to declare the filesystem or parts of the filesystem that would be required at runtime. A Task or Pipeline declares the Workspace and you must provide the specific location details of the volume. It is then mounted into that Workspace in a TaskRun or a PipelineRun. This separation of volume declaration from runtime storage volumes makes the Tasks reusable, flexible, and independent of the user environment.

With Workspaces, you can:

  • Store Task inputs and outputs

  • Share data among Tasks

  • Use it as a mount point for credentials held in Secrets

  • Use it as a mount point for configurations held in ConfigMaps

  • Use it as a mount point for common tools shared by an organization

  • Create a cache of build artifacts that speed up jobs

You can specify Workspaces in the TaskRun or PipelineRun using:

  • A read-only ConfigMaps or Secret

  • An existing PersistentVolumeClaim shared with other Tasks

  • A PersistentVolumeClaim from a provided VolumeClaimTemplate

  • An emptyDir that is discarded when the TaskRun completes

The following example shows a code snippet of the build-and-deploy Pipeline, which declares a shared-workspace Workspace for the build-image and apply-manifests Tasks as defined in the Pipeline.

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: build-and-deploy
spec:
  workspaces: (1)
  - name: shared-workspace
  params:
...
  tasks: (2)
  - name: build-image
    taskRef:
      name: buildah
      kind: ClusterTask
    params:
    - name: TLSVERIFY
      value: "false"
    - name: IMAGE
      value: $(params.IMAGE)
    workspaces: (3)
    - name: source (4)
      workspace: shared-workspace (5)
    runAfter:
    - fetch-repository
  - name: apply-manifests
    taskRef:
      name: apply-manifests
    workspaces: (6)
    - name: source
      workspace: shared-workspace
    runAfter:
      - build-image
...
1 List of Workspaces shared between the Tasks defined in the Pipeline. A Pipeline can define as many Workspaces as required. In this example, only one Workspace named shared-workspace is declared.
2 Definition of Tasks used in the Pipeline. This snippet defines two Tasks, build-image and apply-manifests, which share a common Workspace.
3 List of Workspaces used in the build-image Task. A Task definition can include as many Workspaces as it requires. However, it is recommended that a Task uses at most one writable Workspace.
4 Name that uniquely identifies the Workspace used in the Task. This Task uses one Workspace named source.
5 Name of the Pipeline Workspace used by the Task. Note that the Workspace source in turn uses the Pipeline Workspace named shared-workspace.
6 List of Workspaces used in the apply-manifests Task. Note that this Task shares the source Workspace with the build-image Task.

Workspaces help tasks share data, and allow you to specify one or more volumes that each task in the pipeline requires during execution. You can create a persistent volume claim or provide a volume claim template that creates a persistent volume claim for you.

The following code snippet of the build-deploy-api-pipelinerun PipelineRun uses a volume claim template to create a persistent volume claim for defining the storage volume for the shared-workspace Workspace used in the build-and-deploy Pipeline.

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: build-deploy-api-pipelinerun
spec:
  pipelineRef:
    name: build-and-deploy
  params:
...

  workspaces: (1)
  - name: shared-workspace (2)
    volumeClaimTemplate: (3)
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 500Mi
1 Specifies the list of Pipeline Workspaces for which volume binding will be provided in the PipelineRun.
2 The name of the Workspace in the Pipeline for which the volume is being provided.
3 Specifies a volume claim template that creates a persistent volume claim to define the storage volume for the workspace.

Triggers

Use Triggers in conjunction with pipelines to create a full-fledged CI/CD system where Kubernetes resources define the entire CI/CD execution. Triggers capture the external events, such as a Git pull request, and process them to extract key pieces of information. Mapping this event data to a set of predefined parameters triggers a series of tasks that can then create and deploy Kubernetes resources and instantiate the pipeline.

For example, you define a CI/CD workflow using Red Hat OpenShift Pipelines for your application. The pipeline must start for any new changes to take effect in the application repository. Triggers automate this process by capturing and processing any change event and by triggering a pipeline run that deploys the new image with the latest changes.

Triggers consist of the following main resources that work together to form a reusable, decoupled, and self-sustaining CI/CD system:

  • The TriggerBinding resource validates events, extracts the fields from an event payload, and stores them as parameters.

    The following example shows a code snippet of the TriggerBinding resource, which extracts the Git repository information from the received event payload:

    apiVersion: triggers.tekton.dev/v1alpha1 (1)
kind: TriggerBinding (2)
metadata:
  name: vote-app (3)
spec:
  params: (4)
  - name: git-repo-url
    value: $(body.repository.url)
  - name: git-repo-name
    value: $(body.repository.name)
  - name: git-revision
    value: $(body.head_commit.id)
    1 The API version of the TriggerBinding resource. In this example, v1alpha1.
    2 Specifies the type of Kubernetes object. In this example, TriggerBinding.
    3 Unique name to identify the TriggerBinding resource.
    4 List of parameters which will be extracted from the received event payload and passed to the TriggerTemplate resource. In this example, the Git repository URL, name, and revision are extracted from the body of the event payload.

  • The TriggerTemplate resource acts as a standard for the way resources must be created. It specifies the way parameterized data from the TriggerBinding resource should be used. A trigger template receives input from the trigger binding, and then performs a series of actions that results in creation of new pipeline resources, and initiation of a new pipeline run.

    The following example shows a code snippet of a TriggerTemplate resource, which creates a pipeline run using the Git repository information received from the TriggerBinding resource you just created:

    apiVersion: triggers.tekton.dev/v1alpha1 (1)
kind: TriggerTemplate (2)
metadata:
  name: vote-app (3)
spec:
  params: (4)
  - name: git-repo-url
    description: The git repository url
  - name: git-revision
    description: The git revision
    default: pipelines-1.4
  - name: git-repo-name
    description: The name of the deployment to be created / patched

  resourcetemplates: (5)
  - apiVersion: tekton.dev/v1beta1
    kind: PipelineRun
    metadata:
      name: build-deploy-$(tt.params.git-repo-name)-$(uid)
    spec:
      serviceAccountName: pipeline
      pipelineRef:
        name: build-and-deploy
      params:
      - name: deployment-name
        value: $(tt.params.git-repo-name)
      - name: git-url
        value: $(tt.params.git-repo-url)
      - name: git-revision
        value: $(tt.params.git-revision)
      - name: IMAGE
        value: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/$(tt.params.git-repo-name)
      workspaces:
      - name: shared-workspace
        volumeClaimTemplate:
         spec:
          accessModes:
           - ReadWriteOnce
          resources:
            requests:
              storage: 500Mi
    1 The API version of the TriggerTemplate resource. In this example, v1alpha1.
    2 Specifies the type of Kubernetes object. In this example, TriggerTemplate.
    3 Unique name to identify the TriggerTemplate resource.
    4 Parameters supplied by the TriggerBinding or EventListerner resources.
    5 List of templates that specify the way resources must be created using the parameters received through the TriggerBinding or EventListener resources.

  • The Trigger resource connects the TriggerBinding and TriggerTemplate resources, and this Trigger resource is referenced in the EventListener specification.

    The following example shows a code snippet of a Trigger resource, named vote-trigger that connects the TriggerBinding and TriggerTemplate resources.

    apiVersion: triggers.tekton.dev/v1alpha1 (1)
kind: Trigger (2)
metadata:
  name: vote-trigger (3)
spec:
  serviceAccountName: pipeline (4)
  bindings:
    - ref: vote-app (5)
  template: (6)
     ref: vote-app
    1 The API version of the Trigger resource. In this example, v1alpha1.
    2 Specifies the type of Kubernetes object. In this example, Trigger.
    3 Unique name to identify the Trigger resource.
    4 Service account name to be used.
    5 Name of the TriggerBinding resource to be connected to the TriggerTemplate resource.
    6 Name of the TriggerTemplate resource to be connected to the TriggerBinding resource.

  • The EventListener resource provides an endpoint, or an event sink, that listens for incoming HTTP-based events with a JSON payload. It extracts event parameters from each TriggerBinding resource, and then processes this data to create Kubernetes resources as specified by the corresponding TriggerTemplate resource. The EventListener resource also performs lightweight event processing or basic filtering on the payload using event interceptors, which identify the type of payload and optionally modify it. Currently, pipeline triggers support four types of interceptors: Webhook Interceptors, GitHub Interceptors, GitLab Interceptors, and Common Expression Language (CEL) Interceptors.

    The following example shows an EventListener resource, which references the Trigger resource named vote-trigger.

    apiVersion: triggers.tekton.dev/v1alpha1 (1)
kind: EventListener (2)
metadata:
  name: vote-app (3)
spec:
  serviceAccountName: pipeline (4)
  triggers:
    - triggerRef: vote-trigger (5)
    1 The API version of the EventListener resource. In this example, v1alpha1.
    2 Specifies the type of Kubernetes object. In this example, EventListener.
    3 Unique name to identify the EventListener resource.
    4 Service account name to be used.
    5 Name of the Trigger resource referenced by the EventListener resource.

Triggers in Red Hat OpenShift Pipelines support both HTTP (insecure) and HTTPS (secure HTTP) connections to the Eventlistener resource. With the secure HTTPS connection, you get end-to-end secure connection within and outside the cluster. After you create a namespace, you can enable this secure HTTPS connection for the Eventlistener resource by adding the operator.tekton.dev/enable-annotation=enabled label to the namespace, and then creating a Trigger resource and a secured route using re-encrypt TLS termination.

Additional resources