$ oc get svc/docker-registry -o yaml | grep clusterIP:
OpenShift Container Platform refers to the integrated registry by its service IP address, so if you decide to delete and recreate the docker-registry service, you can ensure a completely transparent transition by arranging to re-use the old IP address in the new service. If a new IP address cannot be avoided, you can minimize cluster disruption by rebooting only the masters.
To re-use the IP address, you must save the IP address of the old docker-registry service prior to deleting it, and arrange to replace the newly assigned IP address with the saved one in the new docker-registry service.
Make a note of the
clusterIP for the service:
$ oc get svc/docker-registry -o yaml | grep clusterIP:
Delete the service:
$ oc delete svc/docker-registry dc/docker-registry
Create the registry definition in registry.yaml, replacing
with, for example, those used in step 3 of the instructions in the
Non-Production Use section:
$ oadm registry <options> -o yaml > registry.yaml
Edit registry.yaml, find the
and change its
clusterIP to the address noted in step 1.
Create the registry using the modified registry.yaml:
$ oc create -f registry.yaml
If you are unable to re-use the IP address, any operation that uses a pull specification that includes the old IP address will fail. To minimize cluster disruption, you must reboot the masters:
# systemctl restart atomic-openshift-master
This ensures that the old registry URL, which includes the old IP address, is cleared from the cache.
|We recommend against rebooting the entire cluster because that incurs unnecessary downtime for pods and does not actually clear the cache.|
You can specify a whitelist of docker registries, allowing you to curate a set of images and templates that are available for download by OpenShift Container Platform users. This curated set can be placed in one or more docker registries, and then added to the whitelist. When using a whitelist, only the specified registries are accessible within OpenShift Container Platform, and all other registries are denied access by default.
To configure a whitelist:
Edit the /etc/sysconfig/docker file to block all registries:
You may need to uncomment the
In the same file, add registries to which you want to allow access:
This example would restrict access to images available on the Red Hat Customer Portal.
Once the whitelist is configured, if a user tries to pull from a docker registry that is not on the whitelist, they will receive an error message stating that this registry is not allowed.
You can override the integrated registry’s default configuration, found by default at /config.yml in a running registry’s container, with your own custom configuration.
Upstream configuration options in this file may also be overridden using environment variables. The middleware section is an exception as there are just a few options that can be overridden using environment variables. Learn how to override specific configuration options.
To enable management of the registry configuration file directly and deploy an updated configuration using
Edit the registry configuration file locally as needed. The initial YAML file deployed on the registry is provided below. Review supported options.
version: 0.1 log: level: debug http: addr: :5000 storage: cache: blobdescriptor: inmemory filesystem: rootdirectory: /registry delete: enabled: true auth: openshift: realm: openshift middleware: registry: - name: openshift repository: - name: openshift options: acceptschema2: false pullthrough: true enforcequota: false projectcachettl: 1m blobrepositorycachettl: 10m storage: - name: openshift
ConfigMap holding the content of each file in this directory:
$ oc create configmap registry-config \ --from-file=</path/to/custom/registry/config.yml>/
Add the registry-config ConfigMap as a volume to the registry’s deployment configuration to mount the custom configuration file at /etc/docker/registry/:
$ oc volume dc/docker-registry --add --type=configmap \ --configmap-name=registry-config -m /etc/docker/registry/
Update the registry to reference the configuration path from the previous step by adding the following environment variable to the registry’s deployment configuration:
$ oc set env dc/docker-registry \ REGISTRY_CONFIGURATION_PATH=/etc/docker/registry/config.yml
This may be performed as an iterative process to achieve the desired configuration. For example, during troubleshooting, the configuration may be temporarily updated to put it in debug mode.
To update an existing configuration:
This procedure will overwrite the currently deployed registry configuration.
Edit the local registry configuration file, config.yml.
Delete the registry-config secret:
$ oc delete secret registry-config
Recreate the secret to reference the updated configuration file:
$ oc secrets new registry-config config.yml=</path/to/custom/registry/config.yml>
Redeploy the registry to read the updated configuration:
$ oc rollout latest docker-registry
Maintain configuration files in a source control repository.
There are many configuration options available in the upstream docker distribution library. Not all configuration options are supported or enabled. Use this section as a reference when overriding the registry configuration.
Upstream configuration options in this file may also be overridden using environment variables. However, the middleware section may not be overridden using environment variables. Learn how to override specific configuration options.
Upstream options are supported.
log: level: debug formatter: text fields: service: registry environment: staging
This section lists the supported registry storage drivers.
The following list includes storage drivers that need to be configured in the registry’s configuration file:
General registry storage configuration options are supported.
The following storage options need to be configured through the filesystem driver:
storage: delete: enabled: true (1) redirect: disable: false cache: blobdescriptor: inmemory maintenance: uploadpurging: enabled: true age: 168h interval: 24h dryrun: false readonly: enabled: false
|1||This entry is mandatory for image pruning to work properly.|
Auth options should not be altered. The openshift extension is the only supported option.
auth: openshift: realm: openshift
The repository middleware extension allows to configure OpenShift Container Platform middleware responsible for interaction with OpenShift Container Platform and image proxying.
middleware: registry: - name: openshift (1) repository: - name: openshift (1) options: acceptschema2: false (2) pullthrough: true (3) enforcequota: false (4) projectcachettl: 1m (5) blobrepositorycachettl: 10m (6) storage: - name: openshift (1)
|1||These entries are mandatory. Their presence ensures required components get loaded. These values shouldn’t be changed.|
|2||Allow to store manifest schema v2 during a push to the registry. See below for more details.|
|3||Let the registry act as a proxy for remote blobs. See below for more details.|
|4||Prevent blob uploads exceeding size limit defined in targeted project.|
|5||An expiration timeout for limits cached in the registry. The lower the value, the less time will it take for the limit changes to propagate to the registry. However, the registry will query limits from the server more frequently and, as a consequence, pushes will be slower.|
|6||An expiration timeout for remembered associations between blob and repository. The higher the value, the higher probability of fast lookup and more efficient registry operation. On the other hand, memory usage will raise as well as a risk of serving image layer to user, who is no longer authorized to access it.|
The CloudFront middleware extension can be added to support AWS, CloudFront CDN storage provider. CloudFront middleware speeds up distribution of image content internationally. The blobs are distributed to several edge locations around the world. The client is always directed to the edge with the lowest latency.
The following is an example of minimal configuration of S3 storage driver with a CloudFront middleware:
version: 0.1 log: level: debug http: addr: :5000 storage: cache: blobdescriptor: inmemory delete: enabled: true s3: (1) accesskey: BJKMSZBRESWJQXRWMAEQ secretkey: 5ah5I91SNXbeoUXXDasFtadRqOdy62JzlnOW1goS region: us-east-1 bucket: docker.myregistry.com auth: openshift: realm: openshift middleware: registry: - name: openshift repository: - name: openshift storage: - name: cloudfront (2) options: baseurl: https://jrpbyn0k5k88bi.cloudfront.net/ (3) privatekey: /etc/docker/cloudfront-ABCEDFGHIJKLMNOPQRST.pem (4) keypairid: ABCEDFGHIJKLMNOPQRST (5) - name: openshift
|1||The S3 storage must be configured the same way regardless of CloudFront middleware.|
|2||The CloudFront storage middleware needs to be listed before OpenShift middleware.|
|3||The CloudFront base URL. In the AWS management console, this is listed as Domain Name of CloudFront distribution.|
|4||The location of your AWS private key on the filesystem. This must be not confused with Amazon EC2 key pair. Please refer to AWS documentation on creating CloudFront key pairs for your trusted signers. The file needs to be mounted as a secret secret into the registry pod.|
|5||The ID of your Cloudfront key pair.|
The middleware section cannot be overridden using environment variables. There are a few exceptions, however. For example:
middleware: repository: - name: openshift options: acceptschema2: false (1) enforcequota: false (2) projectcachettl: 1m (3) blobrepositorycachettl: 10m (4)
|1||A configuration option that can be overridden by the boolean environment
|2||A configuration option that can be overridden by the boolean environment
|3||A configuration option that can be overridden by the environment variable
|4||A configuration option that can be overridden by the environment variable
If enabled, the registry will attempt to fetch requested blob from a remote registry unless the blob exists locally. The remote candidates are calculated from DockerImage entries stored in status of the image stream, a client pulls from. All the unique remote registry references in such entries will be tried in turn until the blob is found.
This feature is on by default. However, it can be disabled using a configuration option.
Each image has a manifest describing its blobs, instructions for running it and additional metadata. The manifest is versioned, with each version having different structure and fields as it evolves over time. The same image can be represented by multiple manifest versions. Each version will have different digest though.
You should be wary of compatibility issues with various Docker clients:
Docker clients of version 1.9 or older support only schema1. Any manifest this client pulls or pushes will be of this legacy schema.
Docker clients of version 1.10 support both schema1 and schema2. And by default, it will push the latter to the registry if it supports newer schema.
The registry, storing an image with schema1 will always return it unchanged to the client. Schema2 will be transferred unchanged only to newer Docker client. For the older one, it will be converted on-the-fly to schema1.
This has significant consequences. For example an image pushed to the registry by a newer Docker client cannot be pulled by the older Docker by its digest. That’s because the stored image’s manifest is of schema2 and its digest can be used to pull only this version of manifest.
For this reason, the registry is configured by default not to store schema2. This ensures that any docker client will be able to pull from the registry any image pushed there regardless of client’s version.
Once you’re confident that all the registry clients support schema2, you’ll be safe to enable its support in the registry. See the middleware configuration reference above for particular option.
Upstream options are supported. Learn how to alter these settings via environment variables. Only the tls section should be altered. For example:
http: addr: :5000 tls: certificate: /etc/secrets/registry.crt key: /etc/secrets/registry.key
notifications: endpoints: - name: registry disabled: false url: https://url:port/path headers: Accept: - text/plain timeout: 500 threshold: 5 backoff: 1000
Upstream options are supported. The registry deployment configuration provides an integrated health check at /healthz.
Proxy configuration should not be enabled. This functionality is provided by the OpenShift Container Platform repository middleware extension, pullthrough: true.