anthony.wen/cds-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Tighten ATVM categorized run status verification

Browse Source 
require categorized PASS results to be confirmed from the child run log or Cloud Run Finished summary

forbid treating watcher host_results, grouped XML, or check-xml-files.ts alone as proof of PASS

record the 2026-04-24 false-PASS incident as a run learning
This commit is contained in:
anthony.wen
2026-04-24 08:46:37 -04:00
parent 86e7830d63
commit eb0e171e41
3 changed files with 15 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
atvm/docs/automation/run-learnings.md
View File

@@ -9,6 +9,16 @@ This file stores run-specific examples only when a run produced a new learning r
## Current State
- No run-learning entries recorded yet from `guide.md` source material.
## Run Learning: 2026-04-24 (Categorized watcher false-PASS guardrail)
- Observed failure mode:
- A categorized compute-migration run was incorrectly reported as `PASS` for `atvm121-ubuntu24.04` even though the actual Ubuntu grouped sub-run failed.
- The false `PASS` came from cached watcher `host_results` plus a grouped XML that only contained `check-xml-files.ts` with `failures="0"`.
- The authoritative launch log and `Cloud Run Finished` summary for that same child run showed `1 failing`.
- Action for future runs:
- Do not report a categorized grouped sub-run as `PASS` from watcher state, grouped XML, or `check-xml-files.ts` alone.
- Before reporting a categorized grouped sub-run as `PASS`, confirm that the matching child batch also passed in the live launch log or the final `Cloud Run Finished` summary for that child run.
- If watcher state or grouped XML disagrees with the launch log or child-run summary, treat the cached/grouped result as stale and report from the launch log plus per-host artifacts instead.
## Run Learning: 2026-03-08 (E2E redhat9.7, pure/fc)
- Request:
- template: `cmc-e2e`