anthony.wen/cds-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Tighten step timing and cleanup rules; record test duration

Browse Source
This commit is contained in:
Cirrus Codex
2026-05-15 12:39:33 -04:00
parent da0f83d3c8
commit 142387ce5e
1 changed files with 10 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
tests/cmc-upgrade-kernel-test.md
View File

@@ -52,6 +52,8 @@ Validate CMC behavior across staged kernel upgrades on a cloned VM, including re
- Do not pause for additional operator prompts between steps. - Do not pause for additional operator prompts between steps.
- Keep monitoring and continue automatically until the test reaches a terminal outcome (`PASS` or `FAIL`) and all required cleanup/reporting steps are completed. - Keep monitoring and continue automatically until the test reaches a terminal outcome (`PASS` or `FAIL`) and all required cleanup/reporting steps are completed.
- Only stop early if a true blocker prevents safe continuation, and still complete required cleanup/reporting before returning control. - Only stop early if a true blocker prevents safe continuation, and still complete required cleanup/reporting before returning control.
- Time every step explicitly.
- If any single step takes longer than 10 minutes, hard stop the test and treat it as a blocker-fail.
## Naming Rule ## Naming Rule
- Base clone VM name in vCenter: `aw999-[source hostname without atvmxxx- prefix]` - Base clone VM name in vCenter: `aw999-[source hostname without atvmxxx- prefix]`
@@ -126,9 +128,8 @@ Validate CMC behavior across staged kernel upgrades on a cloned VM, including re
44. Trigger sync and confirm tracking status using `cirrusdata`. 44. Trigger sync and confirm tracking status using `cirrusdata`.
45. Uninstall CMC. 45. Uninstall CMC.
46. Post-uninstall cleanup checkpoint: 46. Post-uninstall cleanup checkpoint:
- Run MCP offline-host cleanup for `skidamarink`. - Run MCP host cleanup for `skidamarink`.
- If the cloned VM is still marked online after uninstall, remove that cloned VM host entry specifically via MCP (target only this test clone host). - Remove the cloned VM host entry specifically via MCP (target only this test clone host), regardless of whether CDC currently reports it as online or offline.
- Because CMC status can lag behind VM state, poll briefly for status transition; if still online, perform targeted MCP host removal for the tested clone.
47. Check available kernels. 47. Check available kernels.
48. Select latest-upgrade target kernel from the filtered candidate list (same major required; same minor preferred). 48. Select latest-upgrade target kernel from the filtered candidate list (same major required; same minor preferred).
49. Verify matching dev/header packages for the selected latest-upgrade target are available. 49. Verify matching dev/header packages for the selected latest-upgrade target are available.
@@ -144,9 +145,8 @@ Validate CMC behavior across staged kernel upgrades on a cloned VM, including re
59. Success-path cleanup only: power off cloned machine. 59. Success-path cleanup only: power off cloned machine.
60. Success-path cleanup only: delete cloned VM and its disks from vCenter inventory. 60. Success-path cleanup only: delete cloned VM and its disks from vCenter inventory.
61. Success-path final cleanup checkpoint: 61. Success-path final cleanup checkpoint:
- Run MCP offline-host cleanup for `skidamarink`. - Run MCP host cleanup for `skidamarink`.
- If the cloned VM is still marked online at the end of the test, remove that cloned VM host entry specifically via MCP (target only this test clone host). - Remove the cloned VM host entry specifically via MCP (target only this test clone host), regardless of whether CDC currently reports it as online or offline.
- Because CMC status can lag behind VM deletion/power-off, wait/poll briefly first; if still online, perform targeted MCP host removal for the tested clone.
62. Blocker-fail path after clone creation: 62. Blocker-fail path after clone creation:
- Stop test immediately after recording failure details. - Stop test immediately after recording failure details.
- Leave cloned VM powered on and present in inventory for manual inspection. - Leave cloned VM powered on and present in inventory for manual inspection.
@@ -167,6 +167,7 @@ Use one cumulative results file and append one new section per tested host.
### Host Metadata ### Host Metadata
- Test date/time (UTC): - Test date/time (UTC):
- Test duration:
- Operator: - Operator:
- Source VM: - Source VM:
- Cloned VM name: - Cloned VM name:
@@ -231,10 +232,13 @@ Also generate a run summary file in the same directory:
- Always append the latest run outcome to the results file and summary file at the end of each run. - Always append the latest run outcome to the results file and summary file at the end of each run.
- Do this for `PASS`, `FAIL`, and `PARTIAL` outcomes. - Do this for `PASS`, `FAIL`, and `PARTIAL` outcomes.
- Do not leave a completed test run only in conversation; the artifact files are the source of record. - Do not leave a completed test run only in conversation; the artifact files are the source of record.
- Include the total test runtime in both artifact files for every run.
- If a run is still in progress when first recorded, update the runtime once the run reaches its terminal outcome.
Summary file requirements: Summary file requirements:
- Title: `CMC Upgrade Kernel Test Summary` - Title: `CMC Upgrade Kernel Test Summary`
- Include UTC date/time for the run - Include UTC date/time for the run
- Include total test duration for the run
- Include a short workflow summary (current kernel -> install CMC -> kernel upgrade -> uninstall CMC -> kernel upgrade -> install CMC) - Include a short workflow summary (current kernel -> install CMC -> kernel upgrade -> uninstall CMC -> kernel upgrade -> install CMC)
- Include host tested, kernel progression (start, step-up, latest), and overall result - Include host tested, kernel progression (start, step-up, latest), and overall result
- Start each run section with a `##` heading that includes the OS family and the final outcome, for example: `## Amazon Linux 2023 - PASS`. - Start each run section with a `##` heading that includes the OS family and the final outcome, for example: `## Amazon Linux 2023 - PASS`.