Image registry removed during installation

On platforms that do not provide shareable object storage, the OpenShift Image Registry Operator bootstraps itself as Removed. This allows openshift-installer to complete installations on these platform types.

After installation, you must edit the Image Registry Operator configuration to switch the ManagementState from Removed to Managed.

The Prometheus console provides an ImageRegistryRemoved alert, for example:

"Image Registry has been removed. ImageStreamTags, BuildConfigs and DeploymentConfigs which reference ImageStreamTags may not work as expected. Please configure storage and update the config to Managed state by editing configs.imageregistry.operator.openshift.io."

Change image registry ManagementState

To start the image registry, you must change ManagementState Image Registry Operator configuration from Removed to Managed.

Procedure
  • Change ManagementState Image Registry Operator configuration from Removed to Managed. For example:

    apiVersion: imageregistry.operator.openshift.io/v1
    kind: Config
    metadata:
      creationTimestamp: <time>
      finalizers:
        - imageregistry.operator.openshift.io/finalizer
      generation: 3
      name: cluster
      resourceVersion:  <version>
      selfLink: <link>
    spec:
      readOnly: false
      disableRedirect: false
      requests:
        read:
          maxInQueue: 0
          maxRunning: 0
          maxWaitInQueue: 0s
        write:
          maxInQueue: 0
          maxRunning: 0
          maxWaitInQueue: 0s
      defaultRoute: true
      managementState: Managed

Image registry storage configuration

The image-registry Operator is not initially available for platforms that do not provide default storage. After installation, you must configure your registry to use storage so the Registry Operator is made available.

Instructions for both configuring a PersistentVolume, which is required for production clusters, and for configuring an empty directory as the storage location, which is available for only non-production clusters, are shown.

Configuring registry storage for VMware vSphere

As a cluster administrator, following installation you must configure your registry to use storage.

Prerequisites
  • Cluster administrator permissions.

  • A cluster on VMware vSphere.

  • Provision persistent storage for your cluster. To deploy a private image registry, your storage must provide ReadWriteMany access mode.

    vSphere volumes do not support the ReadWriteMany access mode. You must use a different storage backend, such as NFS, to configure the registry storage.

  • Must have "100Gi" capacity.

Procedure
  1. To configure your registry to use storage, change the spec.storage.pvc in the configs.imageregistry/cluster resource.

    When using shared storage such as NFS, it is strongly recommended to use the supplementalGroups strategy, which dictates the allowable supplemental groups for the Security Context, rather than the fsGroup ID. Refer to the NFS Group IDs documentation for details.

  2. Verify you do not have a registry Pod:

    $ oc get pod -n openshift-image-registry
    • If the storage type is emptyDIR, the replica number cannot be greater than 1.

    • If the storage type is NFS, you must enable the no_wdelay and root_squash mount options. For example:

      # cat /etc/exports
      /mnt/data *(rw,sync,no_wdelay,root_squash,insecure,fsid=0)
      sh-4.2# exportfs -rv
      exporting *:/mnt/data
  3. Check the registry configuration:

    $ oc edit configs.imageregistry.operator.openshift.io
    
    storage:
      pvc:
        claim:

    Leave the claim field blank to allow the automatic creation of an image-registry-storage PVC.

  4. Optional: Add a new storage class to a PV:

    1. Create the PV:

      $ oc create -f -
      apiVersion: v1
      kind: PersistentVolume
      metadata:
        name: image-registry-pv
      spec:
        accessModes:
          - ReadWriteMany
        capacity:
            storage: 100Gi
        nfs:
          path: /registry
          server: 172.16.231.181
        persistentVolumeReclaimPolicy: Retain
        storageClassName: nfs01
      $ oc get pv
    2. Create the PVC:

      $ oc create -n openshift-image-registry -f -
      apiVersion: "v1"
      kind: "PersistentVolumeClaim"
      metadata:
        name: "image-registry-pvc"
      spec:
        accessModes:
          - ReadWriteMany
        resources:
          requests:
            storage: 100Gi
        storageClassName: nfs01
        volumeMode: Filesystem
      $ oc get pvc -n openshift-image-registry

      Finally, add the name of your PVC:

      $ oc edit configs.imageregistry.operator.openshift.io -o yaml
      storage:
        pvc:
          claim: image-registry-pvc (1)
      1 Creating a custom PVC allows you to leave the claim field blank for default automatic creation of an image-registry-storage PVC.
  5. Check the clusteroperator status:

    $ oc get clusteroperator image-registry

See Group IDs for additional details about using supplemental groups to handle NFS access.

Configuring storage for the image registry in non-production clusters

You must configure storage for the image registry Operator. For non-production clusters, you can set the image registry to an empty directory. If you do so, all images are lost if you restart the registry.

Procedure
  • To set the image registry storage to an empty directory:

    $ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"storage":{"emptyDir":{}}}}'

    Configure this option for only non-production clusters.

    If you run this command before the Image Registry Operator initializes its components, the oc patch command fails with the following error:

    Error from server (NotFound): configs.imageregistry.operator.openshift.io "cluster" not found

    Wait a few minutes and run the command again.

Configuring block registry storage for VMware vSphere

To allow the image registry to use block storage types such as vSphere Virtual Machine Disk (VMDK) during upgrades as a cluster administrator, you can use the Recreate rollout strategy.

Procedure
  1. To set the image registry storage as a block storage type, patch the registry so that it uses the Recreate rollout strategy and runs with only 1 replica:

    $ oc patch config.imageregistry.operator.openshift.io/cluster --type=merge -p '{"spec":{"rolloutStrategy":"Recreate","replicas":1}}'
  2. Provision the PV for the block storage device, and create a PVC for that volume. The requested block volume uses the ReadWriteOnce (RWO) access mode.

  3. Edit the registry configuration so that it references the correct PVC.

For instructions about configuring registry storage so that it references the correct PVC, see Configuring the registry for vSphere.

Additional resources

For more details about configuring registry storage for vSphere, see Recommended configurable storage technology.