Tighten step timing and cleanup rules; record test duration

This commit is contained in:
Cirrus Codex
2026-05-15 12:39:33 -04:00
parent da0f83d3c8
commit 142387ce5e

View File

@@ -52,6 +52,8 @@ Validate CMC behavior across staged kernel upgrades on a cloned VM, including re
- Do not pause for additional operator prompts between steps.
- Keep monitoring and continue automatically until the test reaches a terminal outcome (`PASS` or `FAIL`) and all required cleanup/reporting steps are completed.
- Only stop early if a true blocker prevents safe continuation, and still complete required cleanup/reporting before returning control.
- Time every step explicitly.
- If any single step takes longer than 10 minutes, hard stop the test and treat it as a blocker-fail.
## Naming Rule
- Base clone VM name in vCenter: `aw999-[source hostname without atvmxxx- prefix]`
@@ -126,9 +128,8 @@ Validate CMC behavior across staged kernel upgrades on a cloned VM, including re
44. Trigger sync and confirm tracking status using `cirrusdata`.
45. Uninstall CMC.
46. Post-uninstall cleanup checkpoint:
- Run MCP offline-host cleanup for `skidamarink`.
- If the cloned VM is still marked online after uninstall, remove that cloned VM host entry specifically via MCP (target only this test clone host).
- Because CMC status can lag behind VM state, poll briefly for status transition; if still online, perform targeted MCP host removal for the tested clone.
- Run MCP host cleanup for `skidamarink`.
- Remove the cloned VM host entry specifically via MCP (target only this test clone host), regardless of whether CDC currently reports it as online or offline.
47. Check available kernels.
48. Select latest-upgrade target kernel from the filtered candidate list (same major required; same minor preferred).
49. Verify matching dev/header packages for the selected latest-upgrade target are available.
@@ -144,9 +145,8 @@ Validate CMC behavior across staged kernel upgrades on a cloned VM, including re
59. Success-path cleanup only: power off cloned machine.
60. Success-path cleanup only: delete cloned VM and its disks from vCenter inventory.
61. Success-path final cleanup checkpoint:
- Run MCP offline-host cleanup for `skidamarink`.
- If the cloned VM is still marked online at the end of the test, remove that cloned VM host entry specifically via MCP (target only this test clone host).
- Because CMC status can lag behind VM deletion/power-off, wait/poll briefly first; if still online, perform targeted MCP host removal for the tested clone.
- Run MCP host cleanup for `skidamarink`.
- Remove the cloned VM host entry specifically via MCP (target only this test clone host), regardless of whether CDC currently reports it as online or offline.
62. Blocker-fail path after clone creation:
- Stop test immediately after recording failure details.
- Leave cloned VM powered on and present in inventory for manual inspection.
@@ -167,6 +167,7 @@ Use one cumulative results file and append one new section per tested host.
### Host Metadata
- Test date/time (UTC):
- Test duration:
- Operator:
- Source VM:
- Cloned VM name:
@@ -231,10 +232,13 @@ Also generate a run summary file in the same directory:
- Always append the latest run outcome to the results file and summary file at the end of each run.
- Do this for `PASS`, `FAIL`, and `PARTIAL` outcomes.
- Do not leave a completed test run only in conversation; the artifact files are the source of record.
- Include the total test runtime in both artifact files for every run.
- If a run is still in progress when first recorded, update the runtime once the run reaches its terminal outcome.
Summary file requirements:
- Title: `CMC Upgrade Kernel Test Summary`
- Include UTC date/time for the run
- Include total test duration for the run
- Include a short workflow summary (current kernel -> install CMC -> kernel upgrade -> uninstall CMC -> kernel upgrade -> install CMC)
- Include host tested, kernel progression (start, step-up, latest), and overall result
- Start each run section with a `##` heading that includes the OS family and the final outcome, for example: `## Amazon Linux 2023 - PASS`.