anthony.wen/cds-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update ATVM watcher for categorized sub-run posting

Browse Source 
- update the watcher design and automation guide to treat --categorize as sequential ATVM sub-runs rather than one parent run with internal phases
- document that categorized runs should send one Mattermost status per completed grouped sub-run instead of one parent-only final post
- add a --categorize option to the watcher start helper so categorized mode is explicit in watcher startup
- update the watcher implementation to track categorized sub-runs separately, write per-subrun state, and post each completed grouped run once
This commit is contained in:
anthony.wen
2026-03-26 11:00:39 -04:00
parent 68cd428733
commit d60b8b9b18
6 changed files with 399 additions and 89 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
atvm/docs/automation/guide.md
View File

@@ -43,6 +43,9 @@ Run ATVM CMC automation tests on the designated automation VM without unintended
- Execute ATVM run commands only after explicit approval.
- Treat `approve` as approval to run without the watcher service.
- Treat `approve with watcher` as approval to run and also start the per-run watcher service for that build.
- When `--categorize` is used with watcher enabled, treat the watcher as a sequential grouped-run watcher:
- it must post one final Mattermost status per completed categorized group/sub-run
- it must not wait and replace those with one single parent-only post
- After execution, report immediate success/failure only.
- Do not actively monitor completion unless explicitly requested.
- If monitoring is requested, allow long runtime windows (15-30+ minutes) and continue until completion unless operator instructs otherwise.
@@ -154,13 +157,14 @@ Before any new automation request:
4. When the watcher is available, present the watcher-start command separately from the core run commands.
5. Treat `approve` as approval to execute the ATVM run without starting the watcher.
6. Treat `approve with watcher` as approval to execute the ATVM run and start the watcher for that build.
7. Run only approved command(s), no extra options and no silent substitutions.
8. When both template generation and the Cypress runner are requested, run them sequentially, not in parallel.
9. Do not launch `run-sorry-cypress.py` until `cmc-templates.py` has exited successfully and finished updating the intended config/spec files.
10. Treat displayed commands as a review gate: do not execute either command until the operator has had a chance to review them and explicitly approve.
11. If the operator asks to change plugin, config, filters, build name, Gold Disk, or scope after commands are shown, discard the old plan, show the revised commands, and wait for new approval.
12. If monitoring was not requested, report immediate success/failure for each command.
13. If monitoring was requested, keep monitoring until completion and report final outcome.
7. If the run uses `--categorize` and the watcher is requested, include `--categorize` on the watcher start command too so the watcher tracks sequential categorized sub-runs correctly.
8. Run only approved command(s), no extra options and no silent substitutions.
9. When both template generation and the Cypress runner are requested, run them sequentially, not in parallel.
10. Do not launch `run-sorry-cypress.py` until `cmc-templates.py` has exited successfully and finished updating the intended config/spec files.
11. Treat displayed commands as a review gate: do not execute either command until the operator has had a chance to review them and explicitly approve.
12. If the operator asks to change plugin, config, filters, build name, Gold Disk, or scope after commands are shown, discard the old plan, show the revised commands, and wait for new approval.
13. If monitoring was not requested, report immediate success/failure for each command.
14. If monitoring was requested, keep monitoring until completion and report final outcome.
## Requested Test Style
When asked for one VM or a VM set:
@@ -193,6 +197,7 @@ When asked for one VM or a VM set:
- Use the same ATVM status layout that would be shown to the operator locally when posting to Mattermost.
- Default status template: `/home/aw/code/cds/atvm/docs/automation/status-template.md`
- Do not post to Mattermost unless the operator explicitly asks for the run status to be sent there.
- For categorized execution with watcher enabled, send one Mattermost status per completed categorized sub-run/group after that grouped run fully finishes.
## Status Reporting Format
When the operator asks for the status of an ATVM automation run, report in this order: