1. Documentation
    2. history
×

Deploying a Docker Registry

Overview

OpenShift can build Docker images from your source code, deploy them, and manage their lifecycle. To enable this, OpenShift provides an internal, integrated Docker registry that can be deployed in your OpenShift environment to locally manage images.

Deploying the Registry

To deploy the integrated Docker registry, use the oadm registry command as a user with cluster administrator privileges. For example:

$ oadm registry --config=/etc/openshift/master/admin.kubeconfig \(1)
    --credentials=/etc/openshift/master/openshift-registry.kubeconfig \(2)
    --images='registry.access.redhat.com/openshift3/ose-${component}:${version}' (3)
1 --config is the path to the CLI configuration file for the cluster administrator.
2 --credentials is the path to the CLI configuration file for the openshift-registry.
3 Required to pull the correct image for OpenShift Enterprise.

This creates a service and a deployment configuration, both called docker-registry. Once deployed successfully, a pod is created with a name similar to docker-registry-1-cpty9.

To see a full list of options that you can specify when creating the registry:

$ oadm registry --help

Storage for the Registry

The registry stores Docker images and metadata. If you simply deploy a pod with the registry, it uses an ephemeral volume that is destroyed if the pod exits. Any images anyone has built or pushed into the registry would disappear.

Production Use

For production use, attach a remote volume or define and use persistent storage using NFS.

For example, to use an existing persistent volume claim:

$ oc volume deploymentconfigs/docker-registry --add --name=v1 -t pvc \
     --claim-name=<pvc_name> --overwrite

Or, to attach an existing NFS volume to the registry:

$ oc volume deploymentconfigs/docker-registry \
     --add --overwrite --name=registry-storage --mount-path=/registry \
     --source='{"nfs": { "server": "<fqdn>", "path": "/path/to/export"}}'

For non-production use, you can use the --mount-host=<path> option to specify a directory for the registry to use for persistent storage. The registry volume is then created as a host-mount at the specified <path>.

The --mount-host option mounts a directory from the node on which the registry container lives. If you scale up the docker-registry deployment configuration, it is possible that your registry pods and containers will run on different nodes, which can result in two or more registry containers, each with its own local storage. This will lead to unpredictable behavior, as subsequent requests to pull the same image repeatedly may not always succeed, depending on which container the request ultimately goes to.

The --mount-host option requires that the registry container run in privileged mode. This is automatically enabled when you specify --mount-host. However, not all pods are allowed to run privileged containers by default. If you still want to use this option:

  1. Create a new service account in the default project for the registry to run as. The following example creates a service account named registry:

    $ echo \
    '{"kind":"ServiceAccount","apiVersion":"v1","metadata":{"name":"registry"}}' \
    | oc create -n default -f -

  2. To add the new registry service account to the list of users allowed to run privileged containers:

    1. Edit the privileged security context constraint (SCC):

      $ oc edit scc privileged

    2. Add a line under users with the user name system:serviceaccount:default:registry.

  3. Create the registry and specify that it use the new registry service account:

    $ oadm registry --service-account=registry \
    --config=/etc/openshift/master/admin.kubeconfig \
    --credentials=/etc/openshift/master/openshift-registry.kubeconfig \
    --images='registry.access.redhat.com/openshift3/ose-${component}:${version}' \
    --mount-host=<path>

Viewing Logs

To view the logs for the Docker registry, run the oc logs indicating the desired pod:

$ oc logs docker-registry-1-da73t
2015-05-01T19:48:36.300593110Z time="2015-05-01T19:48:36Z" level=info msg="version=v2.0.0+unknown"
2015-05-01T19:48:36.303294724Z time="2015-05-01T19:48:36Z" level=info msg="redis not configured" instance.id=9ed6c43d-23ee-453f-9a4b-031fea646002
2015-05-01T19:48:36.303422845Z time="2015-05-01T19:48:36Z" level=info msg="using inmemory layerinfo cache" instance.id=9ed6c43d-23ee-453f-9a4b-031fea646002
2015-05-01T19:48:36.303433991Z time="2015-05-01T19:48:36Z" level=info msg="Using OpenShift Auth handler"
2015-05-01T19:48:36.303439084Z time="2015-05-01T19:48:36Z" level=info msg="listening on :5000" instance.id=9ed6c43d-23ee-453f-9a4b-031fea646002

File Storage

Tag and image metadata is stored in OpenShift, but the registry stores layer and signature data in a volume that is mounted into the registry container at /registry. As oc exec does not work on privileged containers, to view a registry’s contents you must manually SSH into the node housing the registry pod’s container, then run docker exec on the container itself:

  1. List the current pods to find the pod name of your Docker registry:

    # oc get pods

    Then, use oc describe to find the host name for the node running the container:

    # oc describe pod <pod_name>

  2. Log into the desired node:

    # ssh node.example.com

  3. List the running containers on the node host and identify the container ID for the Docker registry:

    # docker ps | grep ose-docker-registry

  4. List the registry contents using the docker exec command:

    # docker exec -it 4c01db0b339c find /registry
/registry/docker
/registry/docker/registry
/registry/docker/registry/v2
/registry/docker/registry/v2/blobs (1)
/registry/docker/registry/v2/blobs/sha256
/registry/docker/registry/v2/blobs/sha256/ed
/registry/docker/registry/v2/blobs/sha256/ed/ede17b139a271d6b1331ca3d83c648c24f92cece5f89d95ac6c34ce751111810
/registry/docker/registry/v2/blobs/sha256/ed/ede17b139a271d6b1331ca3d83c648c24f92cece5f89d95ac6c34ce751111810/data (2)
/registry/docker/registry/v2/blobs/sha256/a3
/registry/docker/registry/v2/blobs/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
/registry/docker/registry/v2/blobs/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4/data
/registry/docker/registry/v2/blobs/sha256/f7
/registry/docker/registry/v2/blobs/sha256/f7/f72a00a23f01987b42cb26f259582bb33502bdb0fcf5011e03c60577c4284845
/registry/docker/registry/v2/blobs/sha256/f7/f72a00a23f01987b42cb26f259582bb33502bdb0fcf5011e03c60577c4284845/data
/registry/docker/registry/v2/repositories (3)
/registry/docker/registry/v2/repositories/p1
/registry/docker/registry/v2/repositories/p1/pause (4)
/registry/docker/registry/v2/repositories/p1/pause/_manifests
/registry/docker/registry/v2/repositories/p1/pause/_manifests/revisions
/registry/docker/registry/v2/repositories/p1/pause/_manifests/revisions/sha256
/registry/docker/registry/v2/repositories/p1/pause/_manifests/revisions/sha256/e9a2ac6418981897b399d3709f1b4a6d2723cd38a4909215ce2752a5c068b1cf
/registry/docker/registry/v2/repositories/p1/pause/_manifests/revisions/sha256/e9a2ac6418981897b399d3709f1b4a6d2723cd38a4909215ce2752a5c068b1cf/signatures (5)
/registry/docker/registry/v2/repositories/p1/pause/_manifests/revisions/sha256/e9a2ac6418981897b399d3709f1b4a6d2723cd38a4909215ce2752a5c068b1cf/signatures/sha256
/registry/docker/registry/v2/repositories/p1/pause/_manifests/revisions/sha256/e9a2ac6418981897b399d3709f1b4a6d2723cd38a4909215ce2752a5c068b1cf/signatures/sha256/ede17b139a271d6b1331ca3d83c648c24f92cece5f89d95ac6c34ce751111810
/registry/docker/registry/v2/repositories/p1/pause/_manifests/revisions/sha256/e9a2ac6418981897b399d3709f1b4a6d2723cd38a4909215ce2752a5c068b1cf/signatures/sha256/ede17b139a271d6b1331ca3d83c648c24f92cece5f89d95ac6c34ce751111810/link (6)
/registry/docker/registry/v2/repositories/p1/pause/_uploads (7)
/registry/docker/registry/v2/repositories/p1/pause/_layers (8)
/registry/docker/registry/v2/repositories/p1/pause/_layers/sha256
/registry/docker/registry/v2/repositories/p1/pause/_layers/sha256/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
/registry/docker/registry/v2/repositories/p1/pause/_layers/sha256/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4/link (9)
/registry/docker/registry/v2/repositories/p1/pause/_layers/sha256/f72a00a23f01987b42cb26f259582bb33502bdb0fcf5011e03c60577c4284845
/registry/docker/registry/v2/repositories/p1/pause/_layers/sha256/f72a00a23f01987b42cb26f259582bb33502bdb0fcf5011e03c60577c4284845/link
    1 This directory stores all layers and signatures as blobs.
    2 This file contains the blob’s contents.
    3 This directory stores all the image repositories.
    4 This directory is for a single image repository p1/pause.
    5 This directory contains signatures for a particular image manifest revision.
    6 This file contains a reference back to a blob (which contains the signature data).
    7 This directory contains any layers that are currently being uploaded and staged for the given repository.
    8 This directory contains links to all the layers this repository references.
    9 This file contains a reference to a specific layer that has been linked into this repository via an image.

Accessing the Registry

To access the registry directly, such as to perform docker push or docker pull operations, you must first log in to the registry using an access token.

  1. Ensure you are logged in to OpenShift as a regular user:

    $ oc login

    System users, such as system:admin, cannot obtain access tokens, and therefore cannot be used to access the registry directly.

  2. Get your access token:

    $ oc whoami -t

  3. Log in to the Docker registry:

    $ docker login -u <username> -e <any_email_address> \
    -p <token_value> <registry_service_host:port>

You can now perform docker pull and docker push operations against your registry. For example:

  1. Pull an arbitrary image:

    $ docker pull docker.io/busybox

  2. Tag the new image with the form <registry_ip:port>/<project>/<image>:

    $ docker tag docker.io/busybox 172.30.124.220:5000/openshift/busybox

  3. Push the newly-tagged image to your registry:

    $ docker push 172.30.124.220:5000/openshift/busybox
...
cf2616975b4a: Image successfully pushed
Digest: sha256:3662dd821983bc4326bee12caec61367e7fb6f6a3ee547cbaff98f77403cab55

Securing the Registry

Optionally, you can secure the registry so that it serves traffic via TLS:

  1. Deploy the registry.

  2. Fetch the service IP and port of the registry:

    $ oc get svc docker-registry
NAME              LABELS                                    SELECTOR                  IP(S)            PORT(S)
docker-registry   docker-registry=default                   docker-registry=default   172.30.124.220   5000/TCP

  3. You can use an existing server certificate, or create a key and server certificate valid for specified IPs and host names, signed by a specified CA. To create a server certificate for the registry service IP and the docker-registry.default.svc.cluster.local host name:

    $ oadm ca create-server-cert --signer-cert=ca.crt \
    --signer-key=ca.key --signer-serial=ca.serial.txt \
    --hostnames='docker-registry.default.svc.cluster.local,172.30.124.220' \
    --cert=registry.crt --key=registry.key

  4. Create the secret for the registry certificates:

    $ oc secrets new registry-secret registry.crt registry.key

  5. Add the secret to the registry pod’s service account (i.e., the default service account):

    $ oc secrets add serviceaccounts/default secrets/registry-secret

  6. Add the secret volume to the registry deployment configuration:

    $ oc volume dc/docker-registry --add --type=secret \
    --secret-name=registry-secret -m /etc/secrets

  7. Enable TLS by adding the following environment variables to the registry deployment configuration:

    $ oc env dc/docker-registry \
    REGISTRY_HTTP_TLS_CERTIFICATE=/etc/secrets/registry.crt \
    REGISTRY_HTTP_TLS_KEY=/etc/secrets/registry.key

    See more details on overriding registry options.

  8. Validate the registry is running in TLS mode. Wait until the docker-registry pod status changes to Running and verify the Docker logs for the registry container. You should find an entry for listening on :5000, tls.

    $ oc get pods
POD                       IP           CONTAINER(S)   IMAGE(S)                                  HOST                           LABELS                                                                                  STATUS    CREATED    MESSAGE
docker-registry-1-da73t   172.17.0.1                                                            openshiftdev.local/127.0.0.1   deployment=docker-registry-4,deploymentconfig=docker-registry,docker-registry=default   Running   38 hours

$ oc logs docker-registry-1-da73t | grep tls
time="2015-05-27T05:05:53Z" level=info msg="listening on :5000, tls" instance.id=deeba528-c478-41f5-b751-dc48e4935fc2

  9. Copy the CA certificate to the Docker certificates directory. This must be done on all nodes in the cluster:

    $ sudo mkdir -p /etc/docker/certs.d/172.30.124.220:5000
$ sudo cp ca.crt /etc/docker/certs.d/172.30.124.220:5000

$ sudo mkdir -p /etc/docker/certs.d/docker-registry.default.svc.cluster.local:5000
$ sudo cp ca.crt /etc/docker/certs.d/docker-registry.default.svc.cluster.local:5000

  10. Remove the --insecure-registry option only for this particular registry in the /etc/sysconfig/docker file. Then, reload the daemon and restart the docker service to reflect this configuration change:

    $ sudo systemctl daemon-reload
$ sudo systemctl restart docker

  11. Validate the docker client connection. Running docker push to the registry or docker pull from the registry should succeed. Make sure you have logged into the registry.

    $ docker tag|push <registry/image> <internal_registry/project/image>

    For example:

    $ docker pull busybox
$ docker tag docker.io/busybox 172.30.124.220:5000/openshift/busybox
$ docker push 172.30.124.220:5000/openshift/busybox
...
cf2616975b4a: Image successfully pushed
Digest: sha256:3662dd821983bc4326bee12caec61367e7fb6f6a3ee547cbaff98f77403cab55

Exposing the Registry

To expose your internal registry externally, it is recommended that you run a secure registry. To expose the registry you must first have deployed a router.

  1. Deploy the registry.

  2. Secure the registry.

  3. Deploy a router.

  4. Create your passthrough route with oc create -f <filename>.json. The passthrough route will point to the registry service that you have created.

    apiVersion: v1
kind: Route
metadata:
  name: registry
spec:
  host: <host> (1)
  to:
    kind: Service
    name: docker-registry (2)
  tls:
    termination: passthrough (3)
    1 The host for your route. You must be able to resolve this name externally via DNS to the router’s IP address.
    2 The service name for your registry.
    3 Specify this route as a passthrough route.

    Passthrough is currently the only type of route supported for exposing the secure registry.

  5. Next, you must trust the certificates being used for the registry on your host system. The certificates referenced were created when you secured your registry.

    $ sudo mkdir -p /etc/docker/certs.d/<host>
$ sudo cp <ca certificate file> /etc/docker/certs.d/<host>
$ sudo systemctl restart docker

  6. Log in to the registry using the information from securing the registry. However, this time point to the host name used in the route rather than your service IP. You should now be able to tag and push images using the route host.

    $ oc get imagestreams -n test
NAME      DOCKER REPO   TAGS      UPDATED

$ docker pull busybox
$ docker tag busybox <host>/test/busybox
$ docker push <host>/test/busybox
The push refers to a repository [<host>/test/busybox] (len: 1)
8c2e06607696: Image already exists
6ce2e90b0bc7: Image successfully pushed
cf2616975b4a: Image successfully pushed
Digest: sha256:6c7e676d76921031532d7d9c0394d0da7c2906f4cb4c049904c4031147d8ca31

$ docker pull <host>/test/busybox
latest: Pulling from <host>/test/busybox
cf2616975b4a: Already exists
6ce2e90b0bc7: Already exists
8c2e06607696: Already exists
Digest: sha256:6c7e676d76921031532d7d9c0394d0da7c2906f4cb4c049904c4031147d8ca31
Status: Image is up to date for <host>/test/busybox:latest

$ oc get imagestreams -n test
NAME      DOCKER REPO                       TAGS      UPDATED
busybox   172.30.11.215:5000/test/busybox   latest    2 seconds ago

    Your image streams will have the IP address and port of the registry service, not the route name and port. See oc get imagestreams for details.

    In the <host>/test/busybox example above, test refers to the project name.

What’s Next?

After you have a registry deployed, you can: