Automatic rollback when an OS update fails
When an OS update fails health checks after reboot, the device automatically rolls back to the previous working operating system version without manual intervention.
Red Hat Edge Manager uses greenboot integration to provide autonomous failed upgrade recovery. When the new OS image fails health checks (for example, the agent not becoming active in time or entering a crash loop), the device reverts to the previous working OS version.
greenboot rollback operations
When an OS update fails health checks after reboot, greenboot performs rollback through the following sequence:
- Deployment and reboot: The agent deploys the new OS image to the device and triggers a reboot.
- Health check initiation: After the device comes back up,
greenbootruns the flightctl health check. This check has two phases: it waits up to 150 seconds forflightctl-agent.serviceto become active, then monitors service stability for 60 seconds to detect crash loops. A rollback is triggered if the agent does not become active in time or if it restarts (crash-loops) during the stability window. - Retries and configuration: If health checks fail,
greenbootreboots and retries. By default, three retries occur before a rollback is performed. To change this threshold, update theGREENBOOT_MAX_BOOTSvalue in/etc/greenboot/greenboot.conf. - Triggering the rollback: After all retry attempts are exhausted,
greenbootreverts to the previous OS deployment and reboots the device. - Reporting: The device starts on the previous OS version, and the agent reports the rollback to the Red Hat Edge Manager service.
- Version exclusion: That specific OS version is recorded as failed, ensuring the agent skips it during all future reconciliations.
Red Hat Edge Manager automatically disables third-party greenboot health check scripts to ensure that only the Red Hat Edge Manager agent health check controls rollback decisions. The flightctl-configure-greenboot service runs before greenboot-healthcheck on every boot and disables scripts such as MicroShift’s 40_microshift_running_check.sh. Only core greenboot scripts and the flightctl health check are preserved. Do not manually edit DISABLED_HEALTHCHECKS in /etc/greenboot/greenboot.conf; it is replaced on every boot.
During the retry phase, the device might reboot multiple times before the final rollback. This is expected behavior, not a separate problem.
This keeps devices operational and avoids repeated failed update attempts.
When automatic rollback applies
Automatic rollback requires image-mode (bootc) devices with greenboot enabled.
Package-mode devices do not support transactional OS updates or greenboot-based rollback.
For more information about image-based OS and transactional updates, see Operating system images for the Red Hat Edge Manager in the Additional resources section.
Observing rollback and update status
You can see whether a device rolled back or is in a failed-update state by viewing the device status. The status.updated.status field indicates whether the device is up to date, updating, or out of date. When an update fails and rollback completes, the update state is reported as Error. The device returns to the previous OS version. For details, see Device update status and update state in the Additional resources section.
If a device has rolled back or you need to diagnose a failed update, see Troubleshooting OS update rollback in the Additional resources section.