Tighten CMC kernel upgrade test checklist

This commit is contained in:
Cirrus Codex
2026-05-20 21:11:32 -04:00
parent 17ff34d0b7
commit 3c87b2e0b2

View File

@@ -89,170 +89,81 @@ Validate CMC behavior across staged kernel upgrades on a cloned VM, including re
- If any checklist item cannot be checked, stop the test and record the blocker. - If any checklist item cannot be checked, stop the test and record the blocker.
- [ ] 0. Source `/home/cirrus/cds/.env.credentials.local` and verify required credential variables are present without printing secret values. - [ ] 0. Source `/home/cirrus/cds/.env.credentials.local` and verify required credential variables are present without printing secret values.
- [ ] 1. Confirm the requested source host is not a SUSE/SLES machine. - [ ] 1. Confirm the requested source host is not a SUSE/SLES machine; if it is SUSE/SLES, hard stop before source power-on or clone creation.
- [ ] 2. Remove offline hosts in `skidamarink` using MCP offline-host cleanup. - [ ] 2. Remove offline hosts in `skidamarink` using MCP offline-host cleanup.
- [ ] 3. Confirm source host is powered on for the inspection phase. - [ ] 3. From vCenter, confirm source host is powered on for the inspection phase; power it on if it is not already powered on.
- [ ] 3a. Before listing available kernel builds, inspect source repository files and hard stop if any enabled/source repo points at `192.168.3.199`. - [ ] 4. From vCenter, query guest-tools for the live source host IP address.
- [ ] 4. SSH to the source host and check available kernel versions on the source before cloning. - [ ] 5. SSH to the source host IP address found in step 4 using credentials from `/home/cirrus/cds/.env.credentials.local`.
- [ ] 5. Build source-host kernel candidate list from all available versions after a fresh package metadata refresh. On Ubuntu, inspect the generic track first, then confirm candidate availability with alternate package listing methods if needed before deciding whether the generic track is usable. - [ ] 6. On the source host, inspect distro repository files before listing available kernel builds and hard stop if any enabled/source repo points at `192.168.3.199` (`/etc/yum.repos.d/*.repo`, `/etc/apt/sources.list`, `/etc/apt/sources.list.d/*`, `/etc/zypp/repos.d/*.repo`, or equivalent files present on the host).
- [ ] 6. Apply the candidate scope rule: same major OS family only, with same minor stream preferred. - [ ] 7. On the source host, record the current OS version and running kernel version before cloning.
- [ ] 7. Verify at least 2 upgrade candidates exist in the filtered candidate list. - [ ] 8. On the source host, refresh package metadata and build the kernel candidate list from all available versions using the distro command set: RHEL/Oracle/Rocky/Alma: `dnf makecache; dnf list --showduplicates kernel kernel-devel kernel-headers`; older RHEL/CentOS: `yum makecache; yum list --showduplicates kernel kernel-devel kernel-headers`; Debian/Ubuntu: `rm -rf /var/lib/apt/lists/* && apt-get clean && apt-get update; apt-cache madison linux-image-generic linux-headers-generic; apt list -a linux-image-generic linux-headers-generic`. Do not run this test for SUSE/SLES; step 1 must stop those hosts before this point. On Ubuntu, inspect the generic track first, then confirm candidate availability with alternate package listing methods if needed before deciding whether the generic track is usable.
- [ ] 8. If fewer than 2 candidates, hard stop and end run before clone creation. - [ ] 9. Apply the candidate scope rule: same major OS family only, with same minor stream preferred.
- [ ] 9. Perform the gate check before continuing. - [ ] 10. Verify at least 2 upgrade candidates exist in the filtered candidate list.
- [ ] 10. Issue the source-host power-off request and wait for `poweredOff`. - [ ] 11. If fewer than 2 candidates exist, hard stop and end the run before clone creation.
- [ ] 11. Confirm the source host is still `poweredOff` immediately before cloning. - [ ] 12. Confirm steps 6-11 passed; if any stop condition was hit, do not clone.
- [ ] 12. Determine the base clone name `aw999-[source-without-atvmxxx-]`. - [ ] 13. From vCenter, issue the source-host power-off request and wait for `poweredOff`.
- [ ] 13. Check whether the base clone name already exists in vCenter. - [ ] 14. From vCenter, confirm the source host is still `poweredOff` immediately before cloning.
- [ ] 14. If needed, choose the next available suffixed clone name. - [ ] 15. Determine the base clone name `aw999-[source-without-atvmxxx-]`.
- [ ] 15. Clone the source VM on `AutomatedTest-UnitTesting` and default it to `CDS1-ESX165` / `192.168.1.165` unless overridden. - [ ] 16. From vCenter, check whether the base clone name already exists.
- [ ] 16. Pass only the clone VM name to the clone command destination. - [ ] 17. If needed, choose the next available suffixed clone name using `aw999-[source-without-atvmxxx-]-1`, then `-2`, then `-N` as needed.
- [ ] 17. Detach the 2 FC PCI adapters from the cloned VM. - [ ] 18. From vCenter, clone the source VM on `AutomatedTest-UnitTesting` using the resolved clone VM name from steps 15-17, pass only the clone VM name as the destination, and default it to `CDS1-ESX165` / `192.168.1.165` unless overridden.
- [ ] 18. Verify both FC passthrough devices are no longer present on the clone. - [ ] 19. From vCenter, detach the 2 FC PCI adapters from the cloned VM.
- [ ] 19. Power on the clone. - [ ] 20. From vCenter, verify both FC passthrough devices are no longer present on the clone.
- [ ] 20. Query vCenter guest-tools for the live clone IP. - [ ] 21. From vCenter, power on the clone.
- [ ] 21. SSH to the live clone IP using credentials from `/home/cirrus/cds/.env.credentials.local`. - [ ] 22. From vCenter, query guest-tools for the live clone IP.
- [ ] 22. Change the OS hostname to the clone name with `.` replaced by `-`. - [ ] 23. SSH to the live clone IP found in step 22 using credentials from `/home/cirrus/cds/.env.credentials.local`.
- [ ] 23. Convert networking from static IP to DHCP. - [ ] 24. On the clone, change the OS hostname to the clone name with `.` replaced by `-`.
- [ ] 24. Remove/clean static IP configuration references. - [ ] 25. On the clone, convert networking from static IP to DHCP.
- [ ] 25. Reboot the clone. - [ ] 26. On the clone, remove/clean static IP configuration references.
- [ ] 26. Query vCenter guest-tools again for the new live clone IP. - [ ] 27. On the clone, reboot the machine.
- [ ] 27. SSH to the new live clone IP and verify DHCP state. - [ ] 28. From vCenter, query guest-tools again for the new live clone IP.
- [ ] 28. If the clone still reports the previous static IP, fix config cleanup and repeat reboot/verify. - [ ] 29. SSH to the new live clone IP found in step 28.
- [ ] 29. Continue all remaining steps using the live DHCP IP from vCenter. - [ ] 30. On the clone, verify DHCP state.
- [ ] 30. Wipe `/dev/sdb` once and verify no filesystem or partition signatures remain. - [ ] 31. If the clone still reports the previous static IP, fix config cleanup and repeat steps 26-30.
- [ ] 31. Reinstall CMC on the clone with `-no-prebuilt-mtdi-nexus`. - [ ] 32. Continue all remaining steps using the live DHCP IP confirmed in step 30.
- [ ] 32. Create the first local migration from 10 GB to 11 GB. - [ ] 33. On the clone, wipe `/dev/sdb` once and verify no filesystem or partition signatures remain.
- [ ] 33. If migration session creation fails, hard stop as blocker-fail. - [ ] 34. Using the cirrusdata skill, reinstall CMC on the clone in the `skidamarink` project with `-no-prebuilt-mtdi-nexus`.
- [ ] 34. Wait for initial sync completion. - [ ] 35. Using the cirrusdata skill, create the first local migration from the 10 GB source disk to the 11 GB destination disk in the `skidamarink` project.
- [ ] 35. Check available kernels again using full candidate listing. - [ ] 36. If migration session creation fails, hard stop as blocker-fail.
- [ ] 36. Select the first-upgrade target from the filtered candidate list. - [ ] 37. Using the cirrusdata skill, wait for initial sync completion in the `skidamarink` project.
- [ ] 37. Verify matching dev/header packages are available for the first-upgrade target. - [ ] 38. SSH to the live DHCP clone IP confirmed in step 30, refresh package metadata, and check available kernels again using the full distro candidate listing: RHEL/Oracle/Rocky/Alma: `dnf makecache; dnf list --showduplicates kernel kernel-devel kernel-headers`; older RHEL/CentOS: `yum makecache; yum list --showduplicates kernel kernel-devel kernel-headers`; Debian/Ubuntu: `rm -rf /var/lib/apt/lists/* && apt-get clean && apt-get update; apt-cache madison linux-image-generic linux-headers-generic; apt list -a linux-image-generic linux-headers-generic`.
- [ ] 38. Install the first-upgrade kernel and matching dev/header packages, then reboot. - [ ] 39. Select the first-upgrade target from the filtered candidate list; it must stay in the same major OS family and must not be the latest candidate. If no valid non-latest first-upgrade target exists, hard stop as blocker-fail.
- [ ] 38a. On Red Hat-family systems with `grubby` including Oracle Linux, set the first-upgrade kernel as the grubby default and verify `grubby --default-kernel` returns the selected `/boot/vmlinuz-<target>` path before reboot. - [ ] 40. On the clone, verify matching dev/header packages are available for the exact first-upgrade target.
- [ ] 39. Query vCenter guest-tools again for the live clone IP after reboot. - [ ] 41. On the clone, install the first-upgrade kernel and matching dev/header packages without rebooting yet.
- [ ] 40. SSH to the rebooted clone and verify kernel plus dev/header package versions match the selected first-upgrade version. - [ ] 42. On Red Hat-family systems with `grubby` including Oracle Linux, set the first-upgrade kernel as the grubby default and verify `grubby --default-kernel` returns the selected `/boot/vmlinuz-<target>` path before reboot.
- [ ] 41. If versions do not match exactly, stop as blocker-fail. - [ ] 43. On the clone, reboot into the first-upgrade kernel.
- [ ] 42. Verify the clone is online in `skidamarink` using `cirrusdata`. - [ ] 44. From vCenter, query guest-tools again for the live clone IP after reboot.
- [ ] 43. SSH to the clone and verify MTDI and Galaxy Migrate services/driver are up. - [ ] 45. SSH to the rebooted clone IP found in step 44.
- [ ] 44. Write sample data to the source 10 GB disk. - [ ] 46. On the clone, verify kernel plus dev/header package versions match the selected first-upgrade version.
- [ ] 45. Trigger sync and confirm tracking status using `cirrusdata`. - [ ] 47. If versions do not match exactly, stop as blocker-fail.
- [ ] 46. Uninstall CMC. - [ ] 48. Using the cirrusdata skill, verify the clone is online in the `skidamarink` project.
- [ ] 47. Run MCP host cleanup for `skidamarink` and remove the cloned host entry for this test clone only, regardless of online/offline status. - [ ] 49. On the clone, verify MTDI and Galaxy Migrate services/driver are up.
- [ ] 48. Check available kernels again. - [ ] 50. On the clone, write sample data to the source 10 GB disk.
- [ ] 49. Select the latest-upgrade target kernel from the filtered candidate list. - [ ] 51. Using the cirrusdata skill, trigger sync and confirm tracking status in the `skidamarink` project.
- [ ] 50. Verify matching dev/header packages are available for the latest-upgrade target. - [ ] 52. Using the cirrusdata skill, uninstall CMC from the clone in the `skidamarink` project.
- [ ] 51. Install the latest-upgrade kernel and matching dev/header packages, then reboot. - [ ] 53. Using MCP, run host cleanup for `skidamarink` and remove the cloned host entry for this test clone only, regardless of online/offline status.
- [ ] 51a. On Red Hat-family systems with `grubby` including Oracle Linux, set the latest-upgrade kernel as the grubby default and verify `grubby --default-kernel` returns the selected `/boot/vmlinuz-<target>` path before reboot. - [ ] 54. Using MCP, verify the cloned host entry and all migration sessions for the cloned host are gone from `skidamarink` before continuing.
- [ ] 52. Query vCenter guest-tools again for the live clone IP after reboot. - [ ] 55. SSH to the live DHCP clone IP confirmed in step 30, refresh package metadata, and check available kernels again using the full distro candidate listing: RHEL/Oracle/Rocky/Alma: `dnf makecache; dnf list --showduplicates kernel kernel-devel kernel-headers`; older RHEL/CentOS: `yum makecache; yum list --showduplicates kernel kernel-devel kernel-headers`; Debian/Ubuntu: `rm -rf /var/lib/apt/lists/* && apt-get clean && apt-get update; apt-cache madison linux-image-generic linux-headers-generic; apt list -a linux-image-generic linux-headers-generic`.
- [ ] 53. SSH to the rebooted clone and verify kernel plus dev/header package versions match the selected latest-upgrade version. - [ ] 56. Select the latest-upgrade target kernel from the filtered candidate list; it must stay in the same major OS family and should use the latest available candidate in that scope. If no valid latest-upgrade target exists, hard stop as blocker-fail.
- [ ] 54. If versions do not match exactly, stop as blocker-fail. - [ ] 57. On the clone, verify matching dev/header packages are available for the exact latest-upgrade target.
- [ ] 55. Reinstall CMC via `cirrusdata` with `-no-prebuilt-mtdi-nexus` on the latest kernel. - [ ] 58. On the clone, install the latest-upgrade kernel and matching dev/header packages without rebooting yet.
- [ ] 56. Create the second local migration from 10 GB to 11 GB and wait for initial sync completion. - [ ] 59. On Red Hat-family systems with `grubby` including Oracle Linux, set the latest-upgrade kernel as the grubby default and verify `grubby --default-kernel` returns the selected `/boot/vmlinuz-<target>` path before reboot.
- [ ] 57. If migration session creation fails, hard stop as blocker-fail. - [ ] 60. On the clone, reboot into the latest-upgrade kernel.
- [ ] 58. Confirm the machine is online in `skidamarink` using `cirrusdata`. - [ ] 61. From vCenter, query guest-tools again for the live clone IP after reboot.
- [ ] 59. SSH and verify MTDI and Galaxy Migrate services/driver are up. - [ ] 62. SSH to the rebooted clone IP found in step 61.
- [ ] 60. Only after steps 55-59 all pass, begin success-path cleanup. - [ ] 63. On the clone, verify kernel plus dev/header package versions match the selected latest-upgrade version.
- [ ] 61. Power off the cloned machine. - [ ] 64. If versions do not match exactly, stop as blocker-fail.
- [ ] 62. Delete the cloned VM and its disks from vCenter inventory. - [ ] 65. Using the cirrusdata skill, reinstall CMC on the clone in the `skidamarink` project with `-no-prebuilt-mtdi-nexus` on the latest kernel.
- [ ] 63. Run final MCP host cleanup for `skidamarink` and remove the cloned host entry for this test clone only. - [ ] 66. Using the cirrusdata skill, create the second local migration from the 10 GB source disk to the 11 GB destination disk in the `skidamarink` project and wait for initial sync completion.
- [ ] 64. If a blocker-fail occurred after clone creation, leave the cloned VM powered on and present in inventory for manual inspection. - [ ] 67. If migration session creation fails, hard stop as blocker-fail.
- [ ] 65. Append the current run to the summary and results files with the required host metadata, kernel progression, execution summary, final outcome, and total test duration. - [ ] 68. Using the cirrusdata skill, confirm the machine is online in the `skidamarink` project.
- [ ] 69. SSH to the live clone IP currently reported by vCenter and verify MTDI and Galaxy Migrate services/driver are up.
## Test Procedure - [ ] 70. Only after steps 65-69 all pass, begin success-path cleanup.
The `Execution Checklist` above is the authoritative run ledger. Use the procedure below as the detailed action reference for each checklist item. - [ ] 71. From vCenter, power off the cloned machine.
0. Credential gate: - [ ] 72. From vCenter, delete the cloned VM and its disks from inventory.
- Before any MCP, vCenter, SSH, Red Hat subscription, or CMC action, load credentials with `set -a; source /home/cirrus/cds/.env.credentials.local; set +a`. - [ ] 73. Using MCP, run final host cleanup for `skidamarink`, remove the cloned host entry for this test clone only, and verify the cloned host entry plus all migration sessions for the cloned host are gone.
- Verify required variables are present without printing secret values: `VCENTER_HOST`, `VCENTER_USER`, `VCENTER_PASSWORD`, `ATVM_TARGET_USER`, `ATVM_TARGET_PASSWORD`, `REDHAT_SUBSCRIPTION_USER`, `REDHAT_SUBSCRIPTION_PASSWORD`, `CMC_GCSTAGE_URL`, `CMC_GCSTAGE_REGISTRATION_CODE`, and `CIRRUS_API_TOKEN`. - [ ] 74. Blocker-fail path after clone creation, as an alternate to steps 70-73: leave the cloned VM powered on and present in inventory for manual inspection.
- If any required variable is missing, hard stop before powering on or modifying any VM. - [ ] 75. Append the current run to the summary and results files with the required host metadata, kernel progression, execution summary, final outcome, and total test duration.
- Do not parse the credential file with `grep`/`awk` as the authority; source it and inspect the environment because entries may use `export KEY=...`.
1. Confirm the requested source host is not a SUSE/SLES machine. If it is SUSE/SLES, hard stop immediately and do not power on, inspect, or clone the machine.
2. Remove offline hosts in `skidamarink` using MCP offline-host cleanup.
3. Confirm source host is powered on for the inspection phase. If it is powered off, power it on.
3a. Before listing available kernel builds, inspect the source host's package repository configuration files.
- Check the distro-appropriate repo sources, such as `/etc/yum.repos.d/*.repo`, `/etc/apt/sources.list`, `/etc/apt/sources.list.d/*`, `/etc/zypp/repos.d/*.repo`, and equivalent package-manager source files that exist on the machine.
- If any enabled/source repository contains `192.168.3.199`, treat the machine as using the local offline DVD/vault repository.
- Hard stop before kernel candidate discovery, fail the test, and do not clone the machine.
- Power off the source VM after inspection and confirm vCenter reports `poweredOff`.
- Append the FAIL result to both result files and explicitly note that the run was skipped because the source package repositories point at the local offline DVD/vault repository at `192.168.3.199`.
4. SSH to the source host and check available kernel versions on the source before cloning.
5. Build source-host kernel candidate list from all available versions (include intermediate versions, not just the latest from `check-update`).
- On package-managed distros, first force a fresh metadata refresh using the package manager's normal refresh command, then use the distro-appropriate tooling to list all available kernel candidates before selecting a target.
- On APT-based distros, use `rm -rf /var/lib/apt/lists/* && apt-get clean && apt-get update` instead of a plain `apt update` so stale or empty package-list files are rebuilt before kernel discovery.
- On Ubuntu, inspect the generic kernel track first after the hard APT refresh, using more than one discovery method before concluding what is available. Use `apt-cache madison`, `apt list -a`, and `apt-cache policy` for `linux-image-generic` and `linux-headers-generic`; if needed, repeat those checks for `linux-generic` and the relevant HWE meta packages.
- If the refreshed generic-track view still returns no usable upgrade candidates, pause and ask the operator whether to continue with alternate Ubuntu kernel tracks.
- When prompting for alternate Ubuntu tracks, display the other available kernel candidates, including `linux-image-generic-hwe-24.04` / `linux-headers-generic-hwe-24.04` when present, and wait for explicit operator confirmation before proceeding.
6. Candidate scope rule:
- Include only kernels in the same major OS family as the current machine (no major-version upgrades).
- Prefer candidates within the same minor stream as current OS/kernel when available.
7. Verify at least 2 upgrade candidates exist in the filtered candidate list.
8. If fewer than 2 candidates: hard stop and end run before clone creation.
9. Gate check:
- If step 7 triggered a stop condition, execute no further steps.
- If no stop condition was triggered, continue with the next step.
10. After source-host inspection is complete, issue the power-off request and wait for vCenter to report the source host as `poweredOff`.
11. Confirm the source host is still `poweredOff` in vCenter immediately before cloning. Do not start the clone while the source VM is transitioning or pending power-off.
12. Determine base clone name: `aw999-[source-without-atvmxxx-]`.
13. Before cloning, check whether that clone name already exists in vCenter.
14. If the name exists, choose the next available suffixed name: `aw999-[source-without-atvmxxx-]-1`, then `-2`, then `-N` as needed.
15. Clone source VM using the resolved unique clone name on datastore `AutomatedTest-UnitTesting` only, and place the clone on `CDS1-ESX165` / `192.168.1.165` by default unless the operator explicitly specifies a different ESXi host.
16. For the clone command destination name, pass only the VM name (for example `aw999-ubuntu24.04-1`), not an inventory path like `/CDSHQ-Eng/vm/...`; set folder separately if needed.
17. Detach the 2 FC PCI adapters from the cloned VM.
18. Verify in vCenter that both FC passthrough devices are no longer present on the clone.
19. Power on clone.
20. Query vCenter guest-tools for the live clone IP.
21. SSH to the live clone IP using credentials from `/home/cirrus/cds/.env.credentials.local`.
22. Change OS hostname to clone name, replacing `.` with `-`.
23. Convert networking from static IP to DHCP.
24. Remove/clean static IP configuration references.
25. Reboot clone.
26. Query vCenter guest-tools again for the new live clone IP.
27. SSH to the new live clone IP and verify the DHCP state.
28. If the clone still reports the previous static IP, fix static config cleanup and repeat reboot/verify.
29. Continue all remaining steps using the live DHCP IP from vCenter and credentials from `/home/cirrus/cds/.env.credentials.local`.
30. Before the first CMC install, wipe the 10GB source disk with `dd if=/dev/zero of=/dev/sdb bs=1M count=32 status=progress conv=fsync`, then verify that no filesystem or partition signatures remain (`wipefs -n /dev/sdb`, `blkid /dev/sdb`, `file -s /dev/sdb`, `lsblk -f /dev/sdb`). This disk prep is one-time only and must not be repeated in later stages of the test.
31. Using `cirrusdata` (`gcstage`, project `skidamarink`), reinstall CMC on clone, always adding `-no-prebuilt-mtdi-nexus`.
32. Create local migration from 10GB source disk to 11GB destination disk using `cirrusdata`.
33. If migration session creation fails (including API/service errors such as 5xx), hard stop as blocker-fail.
34. Wait for initial sync completion.
35. Check available kernels again using full candidate listing (not latest-only output). Refresh package metadata first; on APT-based distros, use the hard APT refresh command from step 5.
36. Select first-upgrade target from filtered candidate list (same major; same minor preferred), ensuring it is not the latest candidate.
37. Verify matching dev/header packages for the selected first-upgrade target are available.
38. Install selected first-upgrade kernel and matching dev/header packages, then reboot.
- On Red Hat-family systems with `grubby`, run `grubby --set-default /boot/vmlinuz-<selected-kernel>` and verify `grubby --default-kernel` returns that exact path before rebooting.
39. Query vCenter guest-tools again for the live clone IP after reboot.
40. SSH to the rebooted clone via the live vCenter IP and verify running kernel and installed dev/header package versions match the selected first-upgrade version.
41. If versions do not match exactly, stop as blocker-fail.
42. After reboot, verify clone is online in `skidamarink` using `cirrusdata`.
43. SSH to clone and verify MTDI, Galaxy Migrate services/driver are up.
44. Write sample data to source 10GB disk.
45. Trigger sync and confirm tracking status using `cirrusdata`.
46. Uninstall CMC.
47. Post-uninstall cleanup checkpoint:
- Run MCP host cleanup for `skidamarink`.
- Remove the cloned VM host entry specifically via MCP (target only this test clone host), regardless of whether CDC currently reports it as online or offline.
48. Check available kernels. Refresh package metadata first; on APT-based distros, use the hard APT refresh command from step 5.
49. Select latest-upgrade target kernel from the filtered candidate list (same major required; same minor preferred).
50. Verify matching dev/header packages for the selected latest-upgrade target are available.
51. Install selected latest-upgrade kernel and matching dev/header packages, then reboot.
- On Red Hat-family systems with `grubby`, run `grubby --set-default /boot/vmlinuz-<selected-kernel>` and verify `grubby --default-kernel` returns that exact path before rebooting.
52. Query vCenter guest-tools again for the live clone IP after reboot.
53. SSH to the rebooted clone via the live vCenter IP and verify running kernel and installed dev/header package versions match the selected latest-upgrade version.
54. If versions do not match exactly, stop as blocker-fail.
55. Reinstall CMC via `cirrusdata` (`gcstage`, `skidamarink`), always adding `-no-prebuilt-mtdi-nexus`.
56. Create a local migration (10GB -> 11GB) via `cirrusdata` and wait for initial sync completion.
57. If migration session creation fails (including API/service errors such as 5xx), hard stop as blocker-fail.
58. Confirm machine is online in `skidamarink` using `cirrusdata`.
59. SSH and verify MTDI, Galaxy Migrate services/driver are up.
60. Only after steps 55-59 all pass, begin success-path cleanup.
61. Success-path cleanup only: power off cloned machine.
62. Success-path cleanup only: delete cloned VM and its disks from vCenter inventory.
63. Success-path final cleanup checkpoint:
- Run MCP host cleanup for `skidamarink`.
- Remove the cloned VM host entry specifically via MCP (target only this test clone host), regardless of whether CDC currently reports it as online or offline.
64. Blocker-fail path after clone creation:
- Stop test immediately after recording failure details.
- Leave cloned VM powered on and present in inventory for manual inspection.
- Do not run clone power-off/delete steps in blocker-fail path.
## Stop Conditions ## Stop Conditions
- Requested source host is a SUSE/SLES machine. - Requested source host is a SUSE/SLES machine.