Builds and Image Streams - Core Concepts | Architecture

Builds
Image Streams

Builds

A build is the process of transforming input parameters into a resulting object. Most often, the process is used to transform input parameters or source code into a runnable image. A BuildConfig object is the definition of the entire build process.

OpenShift leverages Kubernetes by creating Docker containers from build images and pushing them to a Docker registry.

Build objects share common characteristics: inputs for a build, the need to complete a build process, logging the build process, publishing resources from successful builds, and publishing the final status of the build. Builds take advantage of resource restrictions, specifying limitations on resources such as CPU usage, memory usage, and build or pod execution time.

The OpenShift build system provides extensible support for build strategies that are based on selectable types specified in the build API. There are three build strategies available:

Docker build
Source-to-Image (S2I) build
Custom build

By default, Docker builds and S2I builds are supported.

The resulting object of a build depends on the builder used to create it. For Docker and S2I builds, the resulting objects are runnable images. For Custom builds, the resulting objects are whatever the builder image author has specified.

For a list of build commands, see the Developer’s Guide.

For more information on how OpenShift leverages Docker for builds, see the upstream documentation.

Docker Build

The Docker build strategy invokes the plain docker build command, and it therefore expects a repository with a Dockerfile and all required artifacts in it to produce a runnable image.

Source-to-Image (S2I) Build

Source-to-Image (S2I) is a tool for building reproducible Docker images. It produces ready-to-run images by injecting application source into a Docker image and assembling a new Docker image. The new image incorporates the base image (the builder) and built source and is ready to use with the docker run command. S2I supports incremental builds, which re-use previously downloaded dependencies, previously built artifacts, etc.

The advantages of S2I include the following:

Image flexibility	S2I scripts can be written to inject application code into almost any existing Docker image, taking advantage of the existing ecosystem. Note that, currently, S2I relies on `tar` to inject application source, so the image needs to be able to process tarred content.
Speed	With S2I, the assemble process can perform a large number of complex operations without creating a new layer at each step, resulting in a fast process. In addition, S2I scripts can be written to re-use artifacts stored in a previous version of the application image, rather than having to download or build them each time the build is run.
Patchability	S2I allows you to rebuild the application consistently if an underlying image needs a patch due to a security issue.
Operational efficiency	By restricting build operations instead of allowing arbitrary actions, as a *Dockerfile* would allow, the PaaS operator can avoid accidental or intentional abuses of the build system.
Operational security	Building an arbitrary *Dockerfile* exposes the host system to root privilege escalation. This can be exploited by a malicious user because the entire Docker build process is run as a user with Docker privileges. S2I restricts the operations performed as a root user and can run the scripts as a non-root user.
User efficiency	S2I prevents developers from performing arbitrary `yum install` type operations, which could slow down development iteration, during their application build.
Ecosystem	S2I encourages a shared ecosystem of images where you can leverage best practices for your applications.

Custom Build

The Custom build strategy allows developers to define a specific builder image responsible for the entire build process. Using your own builder image allows you to customize your build process.

The Custom builder image is a plain Docker image with embedded build process logic, such as building RPMs or building base Docker images. The openshift/origin-custom-docker-builder image is used by default.

Image Streams

An image stream can be used to automatically perform an action, such as updating a deployment, when a new image, such as a new version of the base image that is used in that deployment, is created.

An image stream comprises one or more Docker images identified by tags. It presents a single virtual view of related images, similar to a Docker image repository, and may contain images from any of the following:

Its own image repository in OpenShift’s integrated Docker Registry
Other image streams
Docker image repositories from external registries

OpenShift components such as builds and deployments can watch an image stream to receive notifications when new images are added and react by performing a build or a deployment.

Example 1. Image Stream Object Definition

{
  "kind": "ImageStream",
  "apiVersion": "v1",
  "metadata": {
    "name": "origin-ruby-sample",
    "namespace": "p1",
    "selfLink": "/osapi/v1/namesapces/p1/imageStreams/origin-ruby-sample",
    "uid": "480dfe73-f340-11e4-97b5-001c422dcd49",
    "resourceVersion": "293",
    "creationTimestamp": "2015-05-05T16:03:34Z",
    "labels": {
      "template": "application-template-stibuild"
    }
  },
  "spec": {},
  "status": {
    "dockerImageRepository": "172.30.30.129:5000/p1/origin-ruby-sample",
    "tags": [
      {
        "tag": "latest",
        "items": [
          {
            "created": "2015-05-05T16:05:47Z",
            "dockerImageReference": "172.30.30.129:5000/p1/origin-ruby-sample@sha256:4d3a646b58685449179a0c61ad4baa19a8df8ba668e0f0704b9ad16f5e16e642",
            "image": "sha256:4d3a646b58685449179a0c61ad4baa19a8df8ba668e0f0704b9ad16f5e16e642"
          }
        ]
      }
    ]
  }
}

Image Stream Mappings

When the integrated OpenShift Docker Registry receives a new image, it creates and sends an ImageStreamMapping to OpenShift. This informs OpenShift of the image’s namespace, name, tag, and Docker metadata. OpenShift uses this information to create a new image (if it does not already exist) and to tag the image into the image stream. OpenShift stores complete metadata about each image (e.g., command, entrypoint, environment variables, etc.). Note that images in OpenShift are immutable. Also, note that the maximum name length is 63 characters.

The example ImageStreamMapping below results in an image being tagged as test/origin-ruby-sample:latest.

Example 2. Image Stream Mapping Object Definition

{
  "kind": "ImageStreamMapping",
  "apiVersion": "v1",
  "metadata": {
    "name": "origin-ruby-sample",
    "namespace": "test"
  },
  "image": {
    "metadata": {
      "name": "a2f15cc10423c165ca221f4a7beb1f2949fb0f5acbbc8e3a0250eb7d5593ae64"
    },
    "dockerImageReference": "172.30.17.3:5001/test/origin-ruby-sample:a2f15cc10423c165ca221f4a7beb1f2949fb0f5acbbc8e3a0250eb7d5593ae64",
    "dockerImageMetadata": {
      "kind": "DockerImage",
      "apiVersion": "1.0",
      "Id": "a2f15cc10423c165ca221f4a7beb1f2949fb0f5acbbc8e3a0250eb7d5593ae64",
      "Parent": "3bb14bfe4832874535814184c13e01527239633627cdc38f18fa186e73a6b62c",
      "Created": "2015-01-23T21:47:04Z",
      "Container": "f81db8980c62d7650683326173a361c3b09f3bc41471918b6319f7df67943b54",
      "ContainerConfig": {
        "Hostname": "f81db8980c62",
        "User": "ruby",
        "AttachStdout": true,
        "ExposedPorts": {
          "9292/tcp": {}
        },
        "OpenStdin": true,
        "StdinOnce": true,
        "Env": [
          "OPENSHIFT_BUILD_NAME=4bf65438-a349-11e4-bead-001c42c44ee1",
          "OPENSHIFT_BUILD_NAMESPACE=test",
          "OPENSHIFT_BUILD_SOURCE=https://github.com/openshift/ruby-hello-world",
          "PATH=/opt/ruby/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
          "STI_SCRIPTS_URL=https://raw.githubusercontent.com/openshift/sti-ruby/master/2.0/.sti/bin",
          "APP_ROOT=.",
          "HOME=/opt/ruby"
        ],
        "Cmd": [
          "/bin/sh",
          "-c",
          "tar -C /tmp -xf - \u0026\u0026 /tmp/scripts/assemble"
        ],
        "Image": "openshift/ruby-20-centos7",
        "WorkingDir": "/opt/ruby/src"
      },
      "DockerVersion": "1.4.1-dev",
      "Config": {
        "User": "ruby",
        "ExposedPorts": {
          "9292/tcp": {}
        },
        "Env": [
          "OPENSHIFT_BUILD_NAME=4bf65438-a349-11e4-bead-001c42c44ee1",
          "OPENSHIFT_BUILD_NAMESPACE=test",
          "OPENSHIFT_BUILD_SOURCE=https://github.com/openshift/ruby-hello-world",
          "PATH=/opt/ruby/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
          "STI_SCRIPTS_URL=https://raw.githubusercontent.com/openshift/sti-ruby/master/2.0/.sti/bin",
          "APP_ROOT=.",
          "HOME=/opt/ruby"
        ],
        "Cmd": [
          "/tmp/scripts/run"
        ],
        "WorkingDir": "/opt/ruby/src"
      },
      "Architecture": "amd64",
      "Size": 11710004
    },
    "dockerImageMetadataVersion": "1.0"
  },
  "tag": "latest"
}

Referencing Images in Image Streams

An ImageStreamTag is used to reference or retrieve an image for a given image stream and tag. It uses the following convention for its name: <image stream name>:<tag>.

An ImageStreamImage is used to reference or retrieve an image for a given image stream and image name. It uses the following convention for its name: <image stream name>@<name>.

The sample image below is from the ruby image stream and was retrieved by asking for the ImageStreamImage with the name ruby@371829c:

Example 3. Definition of an Image Object retrieved via ImageStreamImage

{
    "kind": "ImageStreamImage",
    "apiVersion": "v1",
    "metadata": {
        "name": "ruby@371829c",
        "uid": "a48b40d7-18e2-11e5-9ba2-001c422dcd49",
        "resourceVersion": "1888",
        "creationTimestamp": "2015-06-22T13:29:00Z"
    },
    "image": {
        "metadata": {
            "name": "371829c6d5cf05924db2ab21ed79dd0937986a817c7940b00cec40616e9b12eb",
            "uid": "a48b40d7-18e2-11e5-9ba2-001c422dcd49",
            "resourceVersion": "1888",
            "creationTimestamp": "2015-06-22T13:29:00Z"
        },
        "dockerImageReference": "openshift/ruby-20-centos7:latest",
        "dockerImageMetadata": {
            "kind": "DockerImage",
            "apiVersion": "1.0",
            "Id": "371829c6d5cf05924db2ab21ed79dd0937986a817c7940b00cec40616e9b12eb",
            "Parent": "8c7059377eaf86bc913e915f064c073ff45552e8921ceeb1a3b7cbf9215ecb66",
            "Created": "2015-06-20T23:02:23Z",
            "ContainerConfig": {},
            "DockerVersion": "1.6.0",
            "Author": "Jakub Hadvig \u003cjhadvig@redhat.com\u003e",
            "Config": {
                "User": "1001",
                "ExposedPorts": {
                    "8080/tcp": {}
                },
                "Env": [
                    "PATH=/opt/openshift/src/bin:/opt/openshift/bin:/usr/local/sti:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                    "STI_SCRIPTS_URL=image:///usr/local/sti",
                    "HOME=/opt/openshift/src",
                    "BASH_ENV=/opt/openshift/etc/scl_enable",
                    "ENV=/opt/openshift/etc/scl_enable",
                    "PROMPT_COMMAND=. /opt/openshift/etc/scl_enable",
                    "RUBY_VERSION=2.0"
                ],
                "Cmd": [
                    "usage"
                ],
                "Image": "8c7059377eaf86bc913e915f064c073ff45552e8921ceeb1a3b7cbf9215ecb66",
                "WorkingDir": "/opt/openshift/src",
                "Labels": {
                    "io.openshift.s2i.scripts-url": "image:///usr/local/sti",
                    "k8s.io/description": "Platform for building and running Ruby 2.0 applications",
                    "k8s.io/display-name": "Ruby 2.0",
                    "openshift.io/expose-services": "8080:http",
                    "openshift.io/tags": "builder,ruby,ruby20"
                }
            },
            "Architecture": "amd64",
            "Size": 53950504
        },
        "dockerImageMetadataVersion": "1.0"
    }
}

Image Pull Policy

Each container in a pod has a Docker image. Once you have created an image and pushed it to a registry, you can then refer to it in the pod.

When OpenShift creates containers, it uses the container’s imagePullPolicy to determine if the image should be pulled prior to starting the container. There are three possible values for imagePullPolicy:

Always - always pull the image.
IfNotPresent - only pull the image if it does not already exist on the node.
Never - never pull the image.

If a container’s imagePullPolicy parameter is not specified, OpenShift sets it based on the image’s tag:

If the tag is latest, OpenShift defaults imagePullPolicy to Always.
Otherwise, OpenShift defaults imagePullPolicy to IfNotPresent.

Importing Tag and Image Metadata

An image stream can be configured to import tag and image metadata from an image repository in an external Docker image registry. See Image Registry for more details.

Tag Tracking

An image stream can also be configured so that a tag "tracks" another one. For example, you can configure the latest tag to always refer to the current image for the tag "2.0":

{
  "kind": "ImageStream",
  "apiVersion": "v1",
  "metadata": {
    "name": "ruby"
  },
  "spec": {
    "tags": [
      {
        "name": "latest",
        "from": {
          "kind": "ImageStreamTag",
          "name": "2.0"
        }
      }
    ]
  }
}

$ oc tag ruby:latest ruby:2.0

Tag Removal

You can stop tracking a tag by removing it. For example, you can stop tracking the latest tag you set above:

$ oc tag -d ruby:latest

The above command removes the tag from the image stream spec, but not from the image stream status. The image stream spec is user-defined, whereas the image stream status reflects the information the system has from the specification. To remove a tag completely from an image stream:

$ oc delete istag/ruby:latest

You can also do the same using the oc tag command:

oc tag ruby:latest ruby:2.0

Importing Images from Insecure Registries

An image stream can be configured to import tag and image metadata from an image repository that is signed with a self-signed certificate or from one using plain http instead of https. To do that, add openshift.io/image.insecureRepository annotation set to true. This setting will bypass certificate validation when connecting to registry.

kind: ImageStream
apiVersion: v1
metadata:
  name: ruby
  annotations:
    openshift.io/image.insecureRepository: "true"
  spec:
    dockerImageRepository: "my.repo.com:5000/myimage"

The above definition will only affect importing tag and image metadata. For this image to be used in the cluster (be able to do docker pull) each node needs to have docker configured with --insecure-registry flag. See Installing Docker in Host Preparation for information on the topic.