MTC custom resources

This section describes the custom resources (CRs) that are used by the Migration Toolkit for Containers (MTC).

About MTC custom resources

The Migration Toolkit for Containers (MTC) creates the following custom resources (CRs):

migration architecture diagram

20 MigCluster (configuration, MTC cluster): Cluster definition

20 MigStorage (configuration, MTC cluster): Storage definition

20 MigPlan (configuration, MTC cluster): Migration plan

The MigPlan CR describes the source and target clusters, replication repository, and namespaces being migrated. It is associated with 0, 1, or many MigMigration CRs.

Deleting a MigPlan CR deletes the associated MigMigration CRs.

20 BackupStorageLocation (configuration, MTC cluster): Location of Velero backup objects

20 VolumeSnapshotLocation (configuration, MTC cluster): Location of Velero volume snapshots

20 MigMigration (action, MTC cluster): Migration, created every time you stage or migrate data. Each MigMigration CR is associated with a MigPlan CR.

20 Backup (action, source cluster): When you run a migration plan, the MigMigration CR creates two Velero backup CRs on each source cluster:

  • Backup CR #1 for Kubernetes objects

  • Backup CR #2 for PV data

20 Restore (action, target cluster): When you run a migration plan, the MigMigration CR creates two Velero restore CRs on the target cluster:

  • Restore CR #1 (using Backup CR #2) for PV data

  • Restore CR #2 (using Backup CR #1) for Kubernetes objects

MTC custom resource manifests

Migration Toolkit for Containers (MTC) uses the following custom resource (CR) manifests for migrating applications.

DirectImageMigration

The DirectImageMigration CR copies images directly from the source cluster to the destination cluster.

apiVersion: migration.openshift.io/v1alpha1
kind: DirectImageMigration
metadata:
  labels:
    controller-tools.k8s.io: "1.0"
  name: <direct_image_migration>
spec:
  srcMigClusterRef:
    name: <source_cluster>
    namespace: openshift-migration
  destMigClusterRef:
    name: <destination_cluster>
    namespace: openshift-migration
  namespaces: (1)
    - <source_namespace_1>
    - <source_namespace_2>:<destination_namespace_3> (2)
1 One or more namespaces containing images to be migrated. By default, the destination namespace has the same name as the source namespace.
2 Source namespace mapped to a destination namespace with a different name.

DirectImageStreamMigration

The DirectImageStreamMigration CR copies image stream references directly from the source cluster to the destination cluster.

apiVersion: migration.openshift.io/v1alpha1
kind: DirectImageStreamMigration
metadata:
  labels:
    controller-tools.k8s.io: "1.0"
  name: <direct_image_stream_migration>
spec:
  srcMigClusterRef:
    name: <source_cluster>
    namespace: openshift-migration
  destMigClusterRef:
    name: <destination_cluster>
    namespace: openshift-migration
  imageStreamRef:
    name: <image_stream>
    namespace: <source_image_stream_namespace>
  destNamespace: <destination_image_stream_namespace>

DirectVolumeMigration

The DirectVolumeMigration CR copies persistent volumes (PVs) directly from the source cluster to the destination cluster.

apiVersion: migration.openshift.io/v1alpha1
kind: DirectVolumeMigration
metadata:
  name: <direct_volume_migration>
  namespace: openshift-migration
spec:
  createDestinationNamespaces: false (1)
  deleteProgressReportingCRs: false (2)
  destMigClusterRef:
    name: <host_cluster> (3)
    namespace: openshift-migration
  persistentVolumeClaims:
  - name: <pvc> (4)
    namespace: <pvc_namespace>
  srcMigClusterRef:
    name: <source_cluster>
    namespace: openshift-migration
1 Set to true to create namespaces for the PVs on the destination cluster.
2 Set to true to delete DirectVolumeMigrationProgress CRs after migration. The default is false so that DirectVolumeMigrationProgress CRs are retained for troubleshooting.
3 Update the cluster name if the destination cluster is not the host cluster.
4 Specify one or more PVCs to be migrated.

DirectVolumeMigrationProgress

The DirectVolumeMigrationProgress CR shows the progress of the DirectVolumeMigration CR.

apiVersion: migration.openshift.io/v1alpha1
kind: DirectVolumeMigrationProgress
metadata:
  labels:
    controller-tools.k8s.io: "1.0"
  name: <direct_volume_migration_progress>
spec:
  clusterRef:
    name: <source_cluster>
    namespace: openshift-migration
  podRef:
    name: <rsync_pod>
    namespace: openshift-migration

MigAnalytic

The MigAnalytic CR collects the number of images, Kubernetes resources, and the persistent volume (PV) capacity from an associated MigPlan CR.

You can configure the data that it collects.

apiVersion: migration.openshift.io/v1alpha1
kind: MigAnalytic
metadata:
  annotations:
    migplan: <migplan>
  name: <miganalytic>
  namespace: openshift-migration
  labels:
    migplan: <migplan>
spec:
  analyzeImageCount: true (1)
  analyzeK8SResources: true (2)
  analyzePVCapacity: true (3)
  listImages: false (4)
  listImagesLimit: 50 (5)
  migPlanRef:
    name: <migplan>
    namespace: openshift-migration
1 Optional: Returns the number of images.
2 Optional: Returns the number, kind, and API version of the Kubernetes resources.
3 Optional: Returns the PV capacity.
4 Returns a list of image names. The default is false so that the output is not excessively long.
5 Optional: Specify the maximum number of image names to return if listImages is true.

MigCluster

The MigCluster CR defines a host, local, or remote cluster.

apiVersion: migration.openshift.io/v1alpha1
kind: MigCluster
metadata:
  labels:
    controller-tools.k8s.io: "1.0"
  name: <host_cluster> (1)
  namespace: openshift-migration
spec:
  isHostCluster: true (2)
# The 'azureResourceGroup' parameter is relevant only for Microsoft Azure.
  azureResourceGroup: <azure_resource_group> (3)
  caBundle: <ca_bundle_base64> (4)
  insecure: false (5)
  refresh: false (6)
# The 'restartRestic' parameter is relevant for a source cluster.
  restartRestic: true (7)
# The following parameters are relevant for a remote cluster.
  exposedRegistryPath: <registry_route> (8)
  url: <destination_cluster_url> (9)
  serviceAccountSecretRef:
    name: <source_secret> (10)
    namespace: openshift-config
1 Update the cluster name if the migration-controller pod is not running on this cluster.
2 The migration-controller pod runs on this cluster if true.
3 Microsoft Azure only: Specify the resource group.
4 Optional: If you created a certificate bundle for self-signed CA certificates and if the insecure parameter value is false, specify the base64-encoded certificate bundle.
5 Set to true to disable SSL verification.
6 Set to true to validate the cluster.
7 Set to true to restart the Restic pods on the source cluster after the Stage pods are created.
8 Remote cluster and direct image migration only: Specify the exposed secure registry path.
9 Remote cluster only: Specify the URL.
10 Remote cluster only: Specify the name of the Secret CR.

MigHook

The MigHook CR defines a migration hook that runs custom code at a specified stage of the migration. You can create up to four migration hooks. Each hook runs during a different phase of the migration.

You can configure the hook name, runtime duration, a custom image, and the cluster where the hook will run.

The migration phases and namespaces of the hooks are configured in the MigPlan CR.

apiVersion: migration.openshift.io/v1alpha1
kind: MigHook
metadata:
  generateName: <hook_name_prefix> (1)
  name: <mighook> (2)
  namespace: openshift-migration
spec:
  activeDeadlineSeconds: 1800 (3)
  custom: false (4)
  image: <hook_image> (5)
  playbook: <ansible_playbook_base64> (6)
  targetCluster: source (7)
1 Optional: A unique hash is appended to the value for this parameter so that each migration hook has a unique name. You do not need to specify the value of the name parameter.
2 Specify the migration hook name, unless you specify the value of the generateName parameter.
3 Optional: Specify the maximum number of seconds that a hook can run. The default is 1800.
4 The hook is a custom image if true. The custom image can include Ansible or it can be written in a different programming language.
5 Specify the custom image, for example, quay.io/konveyor/hook-runner:latest. Required if custom is true.
6 Base64-encoded Ansible playbook. Required if custom is false.
7 Specify the cluster on which the hook will run. Valid values are source or destination.

MigMigration

The MigMigration CR runs a MigPlan CR.

You can configure a Migmigration CR to run a stage or incremental migration, to cancel a migration in progress, or to roll back a completed migration.

apiVersion: migration.openshift.io/v1alpha1
kind: MigMigration
metadata:
  labels:
    controller-tools.k8s.io: "1.0"
  name: <migmigration>
  namespace: openshift-migration
spec:
  canceled: false (1)
  rollback: false (2)
  stage: false (3)
  quiescePods: true (4)
  keepAnnotations: true (5)
  verify: false (6)
  migPlanRef:
    name: <migplan>
    namespace: openshift-migration
1 Set to true to cancel a migration in progress.
2 Set to true to roll back a completed migration.
3 Set to true to run a stage migration. Data is copied incrementally and the pods on the source cluster are not stopped.
4 Set to true to stop the application during migration. The pods on the source cluster are scaled to 0 after the Backup stage.
5 Set to true to retain the labels and annotations applied during the migration.
6 Set to true to check the status of the migrated pods on the destination cluster are checked and to return the names of pods that are not in a Running state.

MigPlan

The MigPlan CR defines the parameters of a migration plan.

You can configure destination namespaces, hook phases, and direct or indirect migration.

By default, a destination namespace has the same name as the source namespace. If you configure a different destination namespace, you must ensure that the namespaces are not duplicated on the source or the destination clusters because the UID and GID ranges are copied during migration.

apiVersion: migration.openshift.io/v1alpha1
kind: MigPlan
metadata:
  labels:
    controller-tools.k8s.io: "1.0"
  name: <migplan>
  namespace: openshift-migration
spec:
  closed: false (1)
  srcMigClusterRef:
    name: <source_cluster>
    namespace: openshift-migration
  destMigClusterRef:
    name: <destination_cluster>
    namespace: openshift-migration
  hooks: (2)
    - executionNamespace: <namespace> (3)
      phase: <migration_phase> (4)
      reference:
        name: <hook> (5)
        namespace: <hook_namespace> (6)
      serviceAccount: <service_account> (7)
  indirectImageMigration: true (8)
  indirectVolumeMigration: false (9)
  migStorageRef:
    name: <migstorage>
    namespace: openshift-migration
  namespaces:
    - <source_namespace_1> (10)
    - <source_namespace_2>
    - <source_namespace_3>:<destination_namespace_4> (11)
  refresh: false  (12)
1 The migration has completed if true. You cannot create another MigMigration CR for this MigPlan CR.
2 Optional: You can specify up to four migration hooks. Each hook must run during a different migration phase.
3 Optional: Specify the namespace in which the hook will run.
4 Optional: Specify the migration phase during which a hook runs. One hook can be assigned to one phase. Valid values are PreBackup, PostBackup, PreRestore, and PostRestore.
5 Optional: Specify the name of the MigHook CR.
6 Optional: Specify the namespace of MigHook CR.
7 Optional: Specify a service account with cluster-admin privileges.
8 Direct image migration is disabled if true. Images are copied from the source cluster to the replication repository and from the replication repository to the destination cluster.
9 Direct volume migration is disabled if true. PVs are copied from the source cluster to the replication repository and from the replication repository to the destination cluster.
10 Specify one or more source namespaces. If you specify only the source namespace, the destination namespace is the same.
11 Specify the destination namespace if it is different from the source namespace.
12 The MigPlan CR is validated if true.

MigStorage

The MigStorage CR describes the object storage for the replication repository.

Amazon Web Services (AWS), Microsoft Azure, Google Cloud Storage, Multi-Cloud Object Gateway, and generic S3-compatible cloud storage are supported.

AWS and the snapshot copy method have additional parameters.

apiVersion: migration.openshift.io/v1alpha1
kind: MigStorage
metadata:
  labels:
    controller-tools.k8s.io: "1.0"
  name: <migstorage>
  namespace: openshift-migration
spec:
  backupStorageProvider: <backup_storage_provider> (1)
  volumeSnapshotProvider: <snapshot_storage_provider> (2)
  backupStorageConfig:
    awsBucketName: <bucket> (3)
    awsRegion: <region> (4)
    credsSecretRef:
      namespace: openshift-config
      name: <storage_secret> (5)
    awsKmsKeyId: <key_id> (6)
    awsPublicUrl: <public_url> (7)
    awsSignatureVersion: <signature_version> (8)
  volumeSnapshotConfig:
    awsRegion: <region> (9)
    credsSecretRef:
      namespace: openshift-config
      name: <storage_secret> (10)
  refresh: false (11)
1 Specify the storage provider.
2 Snapshot copy method only: Specify the storage provider.
3 AWS only: Specify the bucket name.
4 AWS only: Specify the bucket region, for example, us-east-1.
5 Specify the name of the Secret CR that you created for the storage.
6 AWS only: If you are using the AWS Key Management Service, specify the unique identifier of the key.
7 AWS only: If you granted public access to the AWS bucket, specify the bucket URL.
8 AWS only: Specify the AWS signature version for authenticating requests to the bucket, for example, 4.
9 Snapshot copy method only: Specify the geographical region of the clusters.
10 Snapshot copy method only: Specify the name of the Secret CR that you created for the storage.
11 Set to true to validate the cluster.

Migrating your applications with the MTC API

This section describes how to migrate your applications with the MTC API from the command line interface (CLI).

About migrating applications

You can migrate applications from a local cluster to a remote cluster, from a remote cluster to a local cluster, and between remote clusters.

Terminology

Source cluster
  • Cluster from which the applications are migrated.

Destination cluster
  • Cluster to which the applications are migrated.

Replication repository
  • Object storage.

  • Requires network access to all clusters.

Indirect migration
  • Images, volumes, and Kubernetes objects are copied from the source cluster to the replication repository and then from the replication repository to the destination cluster.

Direct volume migration
  • Volumes are copied directly from the source cluster to the destination cluster.

  • Significantly faster than indirect migration.

Direct image migration
  • Images are copied directly from the source cluster to the destination cluster.

  • Significantly faster than indirect migration.

Host cluster
  • Cluster on which the migration-controller pod and the web console run.

  • Usually the same as the destination cluster and the local cluster but this is not a requirement.

  • Does not require an exposed secure registry route for direct image migration.

Remote cluster
  • Usually the same as the source cluster but this is not a requirement.

  • Requires an exposed secure registry route for direct image migration.

  • Requires a Secret CR containing the migration-controller service account token.

Mapping destination namespaces in the MigPlan custom resource (CR)

If you map destination namespaces in the MigPlan CR, you must ensure that the namespaces are not duplicated on the source or the destination clusters because the UID and GID ranges of the namespaces are copied during migration.

Two source namespaces mapped to the same destination namespace
spec:
  namespaces:
    - namespace_2
    - namespace_1:namespace_2

If you want the source namespace to be mapped to a namespace of the same name, you do not need to create a mapping. By default, a source namespace and a target namespace have the same name.

Incorrect namespace mapping
spec:
  namespaces:
    - namespace_1:namespace_1
Correct namespace reference
spec:
  namespaces:
    - namespace_1

Stage migration, migration cancellation, and migration rollback

You can create and associate multiple MigMigration custom resources (CRs) with the same MigPlan CR for the following use cases:

  • To perform a stage migration, which copies all available data without stopping the application pods. Running a stage migration reduces the cutover time.

  • To cancel a migration in progress.

  • To roll back a completed migration.

Creating a registry route for direct image migration

For direct image migration, you must create a route to the exposed internal registry on all remote clusters.

Prerequisites
  • The internal registry must be exposed to external traffic on all remote clusters.

    The OpenShift Container Platform 4 registry is exposed by default.

    The OpenShift Container Platform 3 registry must be exposed manually.

Procedure
  • To create a route to an OpenShift Container Platform 3 registry, run the following command:

    $ oc create route passthrough --service=docker-registry --port=5000 -n default
  • To create a route to an OpenShift Container Platform 4 registry, run the following command:

    $ oc create route passthrough --service=image-registry --port=5000 -n openshift-image-registry

Migration prerequisites

  • You must be logged in as a user with cluster-admin privileges on all clusters.

Direct image migration
  • You must ensure that the secure internal registry of the source cluster is exposed.

  • You must create a route to the exposed registry.

Direct volume migration
  • If your clusters use proxies, you must configure an Stunnel TCP proxy.

Internal images
  • If your application uses internal images from the openshift namespace, you must ensure that the required versions of the images are present on the target cluster.

    You can manually update an image stream tag in order to use a deprecated OpenShift Container Platform 3 image on an OpenShift Container Platform 4.6 cluster.

Clusters
  • The source cluster must be upgraded to the latest MTC z-stream release.

  • The MTC version must be the same on all clusters.

Network
  • The clusters have unrestricted network access to each other and to the replication repository.

  • If you copy the persistent volumes with move, the clusters must have unrestricted network access to the remote volumes.

  • You must enable the following ports on an OpenShift Container Platform 3 cluster:

    • 8443 (API server)

    • 443 (routes)

    • 53 (DNS)

  • You must enable the following ports on an OpenShift Container Platform 4 cluster:

    • 6443 (API server)

    • 443 (routes)

    • 53 (DNS)

  • You must enable port 443 on the replication repository if you are using TLS.

Persistent volumes (PVs)
  • The PVs must be valid.

  • The PVs must be bound to persistent volume claims.

  • If you use snapshots to copy the PVs, the following additional prerequisites apply:

    • The cloud provider must support snapshots.

    • The PVs must have the same cloud provider.

    • The PVs must be located in the same geographic region.

    • The PVs must have the same storage class.

Configuring an Stunnel proxy for direct volume migration

If you are performing direct volume migration from a source cluster behind a proxy, you must configure an Stunnel proxy in the MigrationController custom resource (CR). Stunnel creates a transparent tunnel between the source and target clusters for the TCP connection without changing the certificates.

Direct volume migration supports only one proxy. The source cluster cannot access the route of the target cluster if the target cluster is also behind a proxy.

Prerequisites
  • You must be logged in as a user with cluster-admin privileges on all clusters.

Procedure
  1. Log in to the cluster on which the MigrationController pod runs.

  2. Get the MigrationController CR manifest:

    $ oc get migrationcontroller <migration_controller> -n openshift-migration
  3. Add the stunnel_tcp_proxy parameter:

    apiVersion: migration.openshift.io/v1alpha1
    kind: MigrationController
    metadata:
      name: <migration-controller>
      namespace: openshift-migration
    ...
    spec:
      stunnel_tcp_proxy: <stunnel_proxy> (1)
    1 Specify the Stunnel proxy: http://<user>:<password>@<ip_address>:<port>.
  4. Save the manifest as migration-controller.yaml.

  5. Apply the updated manifest:

    $ oc replace -f migration-controller.yaml -n openshift-migration

Migrating your applications from the command line

You can migrate your applications from the command line with the Migration Toolkit for Containers (MTC) API.

Procedure
  1. Create a MigCluster CR manifest for the host cluster:

    $ cat << EOF | oc apply -f -
    apiVersion: migration.openshift.io/v1alpha1
    kind: MigCluster
    metadata:
      name: <host_cluster>
      namespace: openshift-migration
    spec:
      isHostCluster: true
    EOF
  2. Create a Secret CR manifest for each remote cluster:

    $ cat << EOF | oc apply -f -
    apiVersion: v1
    kind: Secret
    metadata:
      name: <cluster_secret>
      namespace: openshift-config
    type: Opaque
    data:
      saToken: <sa_token> (1)
    EOF
    1 Specify the base64-encoded migration-controller service account (SA) token of the remote cluster. You can obtain the token by running the following command:
    $ oc sa get-token migration-controller -n openshift-migration | base64 -w 0
  3. Create a MigCluster CR manifest for each remote cluster:

    $ cat << EOF | oc apply -f -
    apiVersion: migration.openshift.io/v1alpha1
    kind: MigCluster
    metadata:
      name: <remote_cluster> (1)
      namespace: openshift-migration
    spec:
      exposedRegistryPath: <exposed_registry_route> (2)
      insecure: false (3)
      isHostCluster: false
      serviceAccountSecretRef:
        name: <remote_cluster_secret> (4)
        namespace: openshift-config
      url: <remote_cluster_url> (5)
    EOF
    1 Specify the Cluster CR of the remote cluster.
    2 Optional: For direct image migration, specify the exposed registry route.
    3 SSL verification is enabled if false. CA certificates are not required or checked if true.
    4 Specify the Secret CR of the remote cluster.
    5 Specify the URL of the remote cluster.
  4. Verify that all clusters are in a Ready state:

    $ oc describe cluster <cluster>
  5. Create a Secret CR manifest for the replication repository:

    $ cat << EOF | oc apply -f -
    apiVersion: v1
    kind: Secret
    metadata:
      namespace: openshift-config
      name: <migstorage_creds>
    type: Opaque
    data:
      aws-access-key-id: <key_id_base64> (1)
      aws-secret-access-key: <secret_key_base64> (2)
    EOF
    1 Specify the key ID in base64 format.
    2 Specify the secret key in base64 format.

    AWS credentials are base64-encoded by default. For other storage providers, you must encode your credentials by running the following command with each key:

    $ echo -n "<key>" | base64 -w 0 (1)
    1 Specify the key ID or the secret key. Both keys must be base64-encoded.
  6. Create a MigStorage