When using a canary deployment, the rollout deploys an initial canary version of the application. You can verify it either manually or programmatically. After you verify the canary version and promote it to stable, the new stable version is made available to all users.
However, sometimes bugs, errors, or deployment issues are discovered in the canary version, and you might want to abort the canary rollout and rollback to a stable version of your application.
Aborting a canary rollout deletes the resources of the new canary version and restores the previous stable version of your application. All network traffic such as ingress, route, or virtual service that was being directed to the canary returns to the original stable version.
The following example procedure deploys a new red
canary version of your application, and then aborts it before it is fully promoted to stable.
Procedure
-
Update the container image version and and modify the .spec.template.spec.containers.image
value from argoproj/rollouts-demo:yellow
to argoproj/rollouts-demo:red
by running the following command in the Argo Rollouts CLI:
$ oc argo rollouts set image rollouts-demo rollouts-demo=argoproj/rollouts-demo:red -n <namespace> (1)
1 |
Specify the namespace where the Rollout custom resource (CR) is defined. |
Example output
rollout "rollouts-demo" image updated
The container image deployed in the rollout is modified and the rollout initiates a new canary deployment.
-
Wait for the rollout to reach the paused status.
-
Verify that the rollout deploys the rollouts-demo:red
canary version and reaches the paused status by running the following command:
$ oc argo rollouts get rollout rollouts-demo --watch -n <namespace> (1)
1 |
Specify the namespace where the Rollout CR is defined. |
Example output
Name: rollouts-demo
Namespace: spring-petclinic
Status: ॥ Paused
Message: CanaryPauseStep
Strategy: Canary
Step: 1/8
SetWeight: 20
ActualWeight: 20
Images: argoproj/rollouts-demo:red (canary)
argoproj/rollouts-demo:yellow (stable)
Replicas:
Desired: 5
Current: 5
Updated: 1
Ready: 5
Available: 5
NAME KIND STATUS AGE INFO
⟳ rollouts-demo Rollout ॥ Paused 17m
├──# revision:3
│ └──⧉ rollouts-demo-5747959bdb ReplicaSet ✔ Healthy 75s canary
│ └──□ rollouts-demo-5747959bdb-fdrsg Pod ✔ Running 75s ready:1/1
├──# revision:2
│ └──⧉ rollouts-demo-6cf78c66c5 ReplicaSet ✔ Healthy 9m45s stable
│ ├──□ rollouts-demo-6cf78c66c5-zrgd4 Pod ✔ Running 9m44s ready:1/1
│ ├──□ rollouts-demo-6cf78c66c5-2ptpp Pod ✔ Running 4m58s ready:1/1
│ ├──□ rollouts-demo-6cf78c66c5-tmk6c Pod ✔ Running 4m38s ready:1/1
│ └──□ rollouts-demo-6cf78c66c5-zv6lx Pod ✔ Running 4m27s ready:1/1
└──# revision:1
└──⧉ rollouts-demo-687d76d795 ReplicaSet • ScaledDown 17m
-
Abort the update of the rollout by running the following command:
$ oc argo rollouts abort rollouts-demo -n <namespace> (1)
1 |
Specify the namespace where the Rollout CR is defined. |
Example output
rollout 'rollouts-demo' aborted
The Argo Rollouts controller deletes the canary resources of the application, and rolls back to the stable version.
-
Verify that after aborting the rollout, now the canary ReplicaSet
is scaled to 0 replicas by running the following command:
$ oc argo rollouts get rollout rollouts-demo --watch -n <namespace> (1)
1 |
Specify the namespace where the Rollout CR is defined. |
Example output
Name: rollouts-demo
Namespace: spring-petclinic
Status: ✖ Degraded
Message: RolloutAborted: Rollout aborted update to revision 3
Strategy: Canary
Step: 0/8
SetWeight: 0
ActualWeight: 0
Images: argoproj/rollouts-demo:yellow (stable)
Replicas:
Desired: 5
Current: 5
Updated: 0
Ready: 5
Available: 5
NAME KIND STATUS AGE INFO
⟳ rollouts-demo Rollout ✖ Degraded 24m
├──# revision:3
│ └──⧉ rollouts-demo-5747959bdb ReplicaSet • ScaledDown 7m38s canary
├──# revision:2
│ └──⧉ rollouts-demo-6cf78c66c5 ReplicaSet ✔ Healthy 16m stable
│ ├──□ rollouts-demo-6cf78c66c5-zrgd4 Pod ✔ Running 16m ready:1/1
│ ├──□ rollouts-demo-6cf78c66c5-2ptpp Pod ✔ Running 11m ready:1/1
│ ├──□ rollouts-demo-6cf78c66c5-tmk6c Pod ✔ Running 11m ready:1/1
│ ├──□ rollouts-demo-6cf78c66c5-zv6lx Pod ✔ Running 10m ready:1/1
│ └──□ rollouts-demo-6cf78c66c5-mlbsh Pod ✔ Running 4m47s ready:1/1
└──# revision:1
└──⧉ rollouts-demo-687d76d795 ReplicaSet • ScaledDown 24m
The rollout status is marked as Degraded
indicating that even though the application has rolled back to the previous stable version, yellow
, the rollout is not currently at the wanted version, red
, that was set within the .spec.template.spec.containers.image
field.
|
The Degraded status does not reflect the health of the application. It only indicates that there is a mismatch between the wanted and running container image versions.
|
-
Update the container image version to the previous stable version, yellow
, and modify the .spec.template.spec.containers.image
value by running the following command:
$ oc argo rollouts set image rollouts-demo rollouts-demo=argoproj/rollouts-demo:yellow -n <namespace> (1)
1 |
Specify the namespace where the Rollout CR is defined. |
Example output
rollout "rollouts-demo" image updated
The rollout skips the analysis and promotion steps, rolls back to the previous stable version, yellow
, and fast-tracks the deployment of the stable ReplicaSet
.
-
Verify that the rollout status is immediately marked as Healthy
by running the following command:
$ oc argo rollouts get rollout rollouts-demo --watch -n <namespace> (1)
1 |
Specify the namespace where the Rollout CR is defined. |
Example output
Name: rollouts-demo
Namespace: spring-petclinic
Status: ✔ Healthy
Strategy: Canary
Step: 8/8
SetWeight: 100
ActualWeight: 100
Images: argoproj/rollouts-demo:yellow (stable)
Replicas:
Desired: 5
Current: 5
Updated: 5
Ready: 5
Available: 5
NAME KIND STATUS AGE INFO
⟳ rollouts-demo Rollout ✔ Healthy 63m
├──# revision:4
│ └──⧉ rollouts-demo-6cf78c66c5 ReplicaSet ✔ Healthy 55m stable
│ ├──□ rollouts-demo-6cf78c66c5-zrgd4 Pod ✔ Running 55m ready:1/1
│ ├──□ rollouts-demo-6cf78c66c5-2ptpp Pod ✔ Running 50m ready:1/1
│ ├──□ rollouts-demo-6cf78c66c5-tmk6c Pod ✔ Running 50m ready:1/1
│ ├──□ rollouts-demo-6cf78c66c5-zv6lx Pod ✔ Running 50m ready:1/1
│ └──□ rollouts-demo-6cf78c66c5-mlbsh Pod ✔ Running 44m ready:1/1
├──# revision:3
│ └──⧉ rollouts-demo-5747959bdb ReplicaSet • ScaledDown 46m
└──# revision:1
└──⧉ rollouts-demo-687d76d795 ReplicaSet • ScaledDown 63m