Chapter 17. Migrating a standalone Red Hat Quay deployment to a Red Hat Quay Operator deployment

Procedure

1. Back up the config.yaml of your standalone Red Hat Quay deployment:

   $ mkdir /tmp/quay-backup
   $ cp /path/to/Quay/config/directory/config.yaml /tmp/quay-backup

2. Create a backup of the database that your standalone Red Hat Quay deployment is using:

   $ pg_dump -h DB_HOST -p 5432 -d QUAY_DATABASE_NAME -U QUAY_DATABASE_USER -W -O > /tmp/quay-backup/quay-database-backup.sql

3. Install the AWS CLI if you do not have it already.

4. Create an ~/.aws/ directory:

   $ mkdir ~/.aws/

5. Obtain the access_key and secret_key from the config.yaml of your standalone deployment:

   $ grep -i DISTRIBUTED_STORAGE_CONFIG -A10 /tmp/quay-backup/config.yaml

   Example output:

   DISTRIBUTED_STORAGE_CONFIG:
       minio-1:
           - RadosGWStorage
           - access_key: ##########
             bucket_name: quay
             hostname: 172.24.10.50
             is_secure: false
             port: "9000"
             secret_key: ##########
             storage_path: /datastorage/registry

6. Store the access_key and secret_key from the config.yaml file in your ~/.aws directory:

   $ touch ~/.aws/credentials

7. Optional: Check that your access_key and secret_key are stored:

   $ cat > ~/.aws/credentials << EOF
   [default]
   aws_access_key_id = ACCESS_KEY_FROM_QUAY_CONFIG
   aws_secret_access_key = SECRET_KEY_FROM_QUAY_CONFIG
   EOF

   Example output:

   aws_access_key_id = ACCESS_KEY_FROM_QUAY_CONFIG
   aws_secret_access_key = SECRET_KEY_FROM_QUAY_CONFIG

   Note

   If the AWS CLI does not automatically collect the access_key and secret_key from the ~/.aws/credentials file, you can configure these by running aws configure and manually entering the credentials.

8. In your quay-backup directory, create a bucket_backup directory:

   $ mkdir /tmp/quay-backup/bucket-backup

9. Back up all blobs from the S3 storage:

   $ aws s3 sync --no-verify-ssl --endpoint-url https://PUBLIC_S3_ENDPOINT:PORT s3://QUAY_BUCKET/ /tmp/quay-backup/bucket-backup/

   Note

   The PUBLIC_S3_ENDPOINT can be read from the Red Hat Quay config.yaml file under hostname in the DISTRIBUTED_STORAGE_CONFIG. If the endpoint is insecure, use http instead of https in the endpoint URL.

Procedure

1. Scale down the Red Hat Quay Operator:

   $ oc scale --replicas=0 deployment quay-operator.v3.6.2 -n openshift-operators

2. Scale down the application and mirror deployments:

   $ oc scale --replicas=0 deployment QUAY_MAIN_APP_DEPLOYMENT QUAY_MIRROR_DEPLOYMENT

3. Copy the database SQL backup to the Quay PostgreSQL database instance:

   $ oc cp /tmp/user/quay-backup/quay-database-backup.sql quay-enterprise/quayregistry-quay-database-54956cdd54-p7b2w:/var/lib/pgsql/data/userdata

4. Obtain the database password from the Operator-created config.yaml file:

   $ oc get deployment quay-quay-app -o json | jq '.spec.template.spec.volumes[].projected.sources' | grep -i config-secret

   Example output:

         "name": "QUAY_CONFIG_SECRET_NAME"

   $ oc get secret quay-quay-config-secret-9t77hb84tb -o json | jq '.data."config.yaml"' | cut -d '"' -f2 | base64 -d -w0 > /tmp/quay-backup/operator-quay-config-yaml-backup.yaml

   $ cat /tmp/quay-backup/operator-quay-config-yaml-backup.yaml | grep -i DB_URI

   Example output:

   postgresql://QUAY_DATABASE_OWNER:PASSWORD@DATABASE_HOST/QUAY_DATABASE_NAME

5. Execute a shell inside of the database pod:

   # oc exec -it quay-postgresql-database-pod -- /bin/bash

6. Enter psql:

   bash-4.4$ psql

7. Drop the database:

   postgres=# DROP DATABASE "example-restore-registry-quay-database";

   Example output:

   DROP DATABASE

8. Create a new database and set the owner as the same name:

   postgres=# CREATE DATABASE "example-restore-registry-quay-database" OWNER "example-restore-registry-quay-database";

   Example output:

   CREATE DATABASE

9. Connect to the database:

   postgres=# \c "example-restore-registry-quay-database";

   Example output:

   You are now connected to database "example-restore-registry-quay-database" as user "postgres".

10. Create a pg_trgm extension of your Quay database:

    example-restore-registry-quay-database=# CREATE EXTENSION IF NOT EXISTS pg_trgm ;

    Example output:

    CREATE EXTENSION

11. Exit the postgres CLI to re-enter bash-4.4:

    \q

12. Set the password for your PostgreSQL deployment:

    bash-4.4$ psql -h localhost -d "QUAY_DATABASE_NAME" -U QUAY_DATABASE_OWNER -W < /var/lib/pgsql/data/userdata/quay-database-backup.sql

    Example output:

    SET
    SET
    SET
    SET
    SET

13. Exit bash mode:

    bash-4.4$ exit

14. Create a new configuration bundle for the Red Hat Quay Operator.

    $ touch config-bundle.yaml

15. In your new config-bundle.yaml, include all of the information that the registry requires, such as LDAP configuration, keys, and other modifications that your old registry had. Run the following command to move the secret_key to your config-bundle.yaml:

    $ cat /tmp/quay-backup/config.yaml | grep SECRET_KEY > /tmp/quay-backup/config-bundle.yaml

    Note

    You must manually copy all the LDAP, OIDC, and other information and add it to the /tmp/quay-backup/config-bundle.yaml file.

16. Create a configuration bundle secret inside of your OpenShift cluster:

    $ oc create secret generic new-custom-config-bundle --from-file=config.yaml=/tmp/quay-backup/config-bundle.yaml

17. Scale up the Quay pods:

    $ oc scale --replicas=1 deployment quayregistry-quay-app

    Example output:

    deployment.apps/quayregistry-quay-app scaled

18. Scale up the mirror pods:

    $ oc scale --replicas=1 deployment quayregistry-quay-mirror

    Example output:

    deployment.apps/quayregistry-quay-mirror scaled

19. Patch the QuayRegistry CRD so that it contains the reference to the new custom configuration bundle:

    $ oc patch quayregistry QUAY_REGISTRY_NAME --type=merge -p '{"spec":{"configBundleSecret":"new-custom-config-bundle"}}'

    Note

    If Red Hat Quay returns a 500 internal server error, you might have to update the location of your DISTRIBUTED_STORAGE_CONFIG to default.

20. Create a new AWS credentials.yaml in your /.aws/ directory and include the access_key and secret_key from the Operator-created config.yaml file:

    $ touch credentials.yaml

    $ grep -i DISTRIBUTED_STORAGE_CONFIG -A10 /tmp/quay-backup/operator-quay-config-yaml-backup.yaml

    $ cat > ~/.aws/credentials << EOF
    [default]
    aws_access_key_id = ACCESS_KEY_FROM_QUAY_CONFIG
    aws_secret_access_key = SECRET_KEY_FROM_QUAY_CONFIG
    EOF

    Note

    If the AWS CLI does not automatically collect the access_key and secret_key from the ~/.aws/credentials file, you can configure these by running aws configure and manually entering the credentials.

21. Record the NooBaa’s publicly available endpoint:

    $ oc get route s3 -n openshift-storage -o yaml -o jsonpath="{.spec.host}{'\n'}"

22. Sync the backup data to the NooBaa backend storage:

    $ aws s3 sync --no-verify-ssl --endpoint-url https://NOOBAA_PUBLIC_S3_ROUTE /tmp/quay-backup/bucket-backup/* s3://QUAY_DATASTORE_BUCKET_NAME

23. Scale the Operator back up to 1 pod:

    $ oc scale --replicas=1 deployment quay-operator.v3.6.4 -n openshift-operators

    The Operator uses the custom configuration bundle provided and reconciles all secrets and deployments. Your new Red Hat Quay deployment on OpenShift Container Platform contains all of the information that the old deployment had. You can pull all images.