Fix duplicate watcher posts and restore 22-step E2E flow

This commit is contained in:
2026-03-27 11:28:48 -04:00
parent cc551a6922
commit 4b04f9b49d
4 changed files with 52 additions and 32 deletions

View File

@@ -237,7 +237,7 @@ Status-report expectations:
- In `COVERAGE:`, describe the template, datastore/config family, migration style, and plugin/integration path, but do not list target hosts there.
- In `TEST FLOW:`, show the template-specific numbered run flow once for the whole test, not per host.
- Resolve the flow from the run template name.
- `cmc-e2e` currently uses the 18-step migration flow documented in `/home/aw/code/cds/atvm/docs/automation/status-template.md`.
- `cmc-e2e` currently uses the 22-step migration flow documented in `/home/aw/code/cds/atvm/docs/automation/status-template.md`.
- For the `Kernel` column, cross-reference the host name against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`.
- If the hostname is not present in `vm-inventory.md`, report the kernel value as `unknown`.
- Treat references to the "ATVM automation run" or "automation run" as referring to this ATVM folder workflow and the automation VM at `192.168.3.190`, not to Cirrus project operations such as the `atvm - cypress` project.

View File

@@ -205,21 +205,25 @@ This file stores run-specific examples only when a run produced a new learning r
- `1. Verifying set up`
- `2. Power on and obtain ip address and host name`
- `3. Uninstall CMC if still exists`
- `4. Setting up disk`
- `4. Setting up disk on the host`
- `5. Copy CMC install command from GUI`
- `6. Install CMC`
- `7. Create migration session`
- `8. Tracking Changes`
- `9. Trigger cmotion and do I/O test before actual cutover`
- `10. Verify migration remains healthy during I/O activity`
- `11. Prepare for cutover`
- `12. Stop application / stop test I/O`
- `13. Run final sync`
- `14. Confirm destination is fully up to date`
- `15. Perform cutover`
- `16. Validate destination host / disk state`
- `17. Run post-cutover checks`
- `18. Power off`
- `10. Verify data for cmotion`
- `11. Trigger revert cmotion and do I/O test before and during cmotion`
- `12. Verify data for revert cmotion`
- `13. Trigger cmotion again`
- `14. Finalize cutover`
- `15. Create migration report`
- `16. Delete migration session`
- `17. Verify local destination disk`
- `18. Remove enabled FC integration`
- `19. Remove host and volumes`
- `20. Uninstall CMC`
- `21. Clean up iSCSI targets`
- `22. Power off`
## Run Learning: 2026-03-27 (Start watcher before runner when watcher is requested)
- Observed failure mode:
@@ -240,3 +244,11 @@ This file stores run-specific examples only when a run produced a new learning r
- Do not rely only on `/tmp/<build-name>.log` for non-categorized watcher result recovery.
- When final `check-xml-files.ts` validation is present but host XML is absent, recover host completion from the latest matching per-host reporter artifact within the run window.
- Keep non-categorized watcher notes accurate; do not describe that failure as a categorized sub-run issue.
## Run Learning: 2026-03-27 (Non-categorized watcher runs must post once and show the full 22-step E2E flow)
- Observed failure mode:
- A non-categorized watcher run for `cmc-e2e` sent two Mattermost posts for the same build.
- The posted `TEST FLOW:` list only showed 18 steps even though the current `cmc-e2e` ATVM flow has 22 steps.
- Action for future runs:
- For non-categorized runs, post only the parent run status and do not also post the single synthetic subrun.
- Keep the static `cmc-e2e` watcher flow aligned with the current 22-step ATVM E2E sequence.

View File

@@ -78,21 +78,25 @@ Use this as the default ATVM automation run-status template for:
- `1. Verifying set up`
- `2. Power on and obtain ip address and host name`
- `3. Uninstall CMC if still exists`
- `4. Setting up disk`
- `4. Setting up disk on the host`
- `5. Copy CMC install command from GUI`
- `6. Install CMC`
- `7. Create migration session`
- `8. Tracking Changes`
- `9. Trigger cmotion and do I/O test before actual cutover`
- `10. Verify migration remains healthy during I/O activity`
- `11. Prepare for cutover`
- `12. Stop application / stop test I/O`
- `13. Run final sync`
- `14. Confirm destination is fully up to date`
- `15. Perform cutover`
- `16. Validate destination host / disk state`
- `17. Run post-cutover checks`
- `18. Power off`
- `10. Verify data for cmotion`
- `11. Trigger revert cmotion and do I/O test before and during cmotion`
- `12. Verify data for revert cmotion`
- `13. Trigger cmotion again`
- `14. Finalize cutover`
- `15. Create migration report`
- `16. Delete migration session`
- `17. Verify local destination disk`
- `18. Remove enabled FC integration`
- `19. Remove host and volumes`
- `20. Uninstall CMC`
- `21. Clean up iSCSI targets`
- `22. Power off`
- See `/home/aw/code/cds/atvm/docs/automation/examples.md` for `cmc-e2e` examples.
- Resolve kernel values by cross-referencing hostnames against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`.
- If no kernel value can be verified from `vm-inventory.md`, use `unknown`.

View File

@@ -44,21 +44,25 @@ TEMPLATE_TEST_FLOWS = {
"1. Verifying set up",
"2. Power on and obtain ip address and host name",
"3. Uninstall CMC if still exists",
"4. Setting up disk",
"4. Setting up disk on the host",
"5. Copy CMC install command from GUI",
"6. Install CMC",
"7. Create migration session",
"8. Tracking Changes",
"9. Trigger cmotion and do I/O test before actual cutover",
"10. Verify migration remains healthy during I/O activity",
"11. Prepare for cutover",
"12. Stop application / stop test I/O",
"13. Run final sync",
"14. Confirm destination is fully up to date",
"15. Perform cutover",
"16. Validate destination host / disk state",
"17. Run post-cutover checks",
"18. Power off",
"10. Verify data for cmotion",
"11. Trigger revert cmotion and do I/O test before and during cmotion",
"12. Verify data for revert cmotion",
"13. Trigger cmotion again",
"14. Finalize cutover",
"15. Create migration report",
"16. Delete migration session",
"17. Verify local destination disk",
"18. Remove enabled FC integration",
"19. Remove host and volumes",
"20. Uninstall CMC",
"21. Clean up iSCSI targets",
"22. Power off",
],
}
@@ -1271,7 +1275,7 @@ if __name__ == "__main__":
"started_at": subrun["start_ts"].isoformat() if subrun["start_ts"] else None,
"ended_at": subrun["end_ts"].isoformat() if subrun["end_ts"] else None,
}
if subrun["state"] in {"COMPLETED", "FAILED"} and not subrun_posted_marker.exists():
if metadata.get("categorized") and subrun["state"] in {"COMPLETED", "FAILED"} and not subrun_posted_marker.exists():
status_text = build_status_markdown(
build_name=subrun["display_name"],
metadata=metadata,