From cc41d26e9ffe410ef6e9fd2402ea23b6d268a9af Mon Sep 17 00:00:00 2001 From: "anthony.wen" Date: Tue, 12 May 2026 14:17:12 -0400 Subject: [PATCH] docs(test): enforce continuous execution and clarify cmc-upgrade outcome/reporting rules --- tests/cmc-upgrade-test.md | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/tests/cmc-upgrade-test.md b/tests/cmc-upgrade-test.md index 91767c1..61204c5 100644 --- a/tests/cmc-upgrade-test.md +++ b/tests/cmc-upgrade-test.md @@ -25,6 +25,12 @@ Validate CMC behavior across staged kernel upgrades on a cloned VM, including re - Exception: offline-host cleanup is not handled by that skill yet; use the MCP connection for offline-host removal. - Apply this rule to every relevant step in this procedure. +## Execution Mode (Global) +- Run this test in continuous execution mode. +- Do not pause for additional operator prompts between steps. +- Keep monitoring and continue automatically until the test reaches a terminal outcome (`PASS` or `FAIL`) and all required cleanup/reporting steps are completed. +- Only stop early if a true blocker prevents safe continuation, and still complete required cleanup/reporting before returning control. + ## Naming Rule - Base clone VM name in vCenter: `aw999-[source hostname without atvmxxx- prefix]` - Before cloning, verify the clone VM name is not already in use. @@ -79,7 +85,7 @@ Validate CMC behavior across staged kernel upgrades on a cloned VM, including re 35. Check available kernels. 36. Upgrade to latest kernel and reboot. 37. Reinstall CMC via `cirrusdata` (`gcstage`, `skidamarink`). -38. Recreate local migration (10GB -> 11GB) via `cirrusdata` and wait for initial sync completion. +38. Create a local migration (10GB -> 11GB) via `cirrusdata` and wait for initial sync completion. 39. Confirm machine is online in `skidamarink` using `cirrusdata`. 40. SSH and verify MTDI, Galaxy Migrate services/driver are up. 41. Power off cloned machine. @@ -136,6 +142,10 @@ Use one cumulative results file and append one new section per tested host. ### Final Outcome - Overall result: `PASS|FAIL|PARTIAL` +- Outcome interpretation: + - `PASS`: full planned test flow completed and core validation goals passed (CMC install/uninstall/reinstall, kernel step-up/latest upgrade, and post-upgrade service/driver health checks), even if non-blocking warnings occurred. + - `FAIL`: a true blocker prevented completion of required validation goals. + - `PARTIAL`: use only when execution stops early by operator choice or scope is intentionally reduced, not for non-blocking warnings in a completed run. - Blocking issue summary: - Follow-up actions: @@ -146,3 +156,12 @@ Use one cumulative results file and append one new section per tested host. ## Result Storage Location Store and append all per-host results in: - `/home/aw/code/cds/tmp/tests/cmc upgrade test/cmc-upgrade-kernel-test-results.md` + +Also generate a run summary file in the same directory: +- `/home/aw/code/cds/tmp/tests/cmc upgrade test/cmc-upgrade-kernel-test-summary.md` + +Summary file requirements: +- Title: `CMC Upgrade Kernel Test Summary` +- Include UTC date/time for the run +- Include a short workflow summary (current kernel -> install CMC -> kernel upgrade -> uninstall CMC -> kernel upgrade -> install CMC) +- Include host tested, kernel progression (start, step-up, latest), and overall result