anthony.wen/cds-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update ATVM watcher for categorized sub-run posting

Browse Source 
- update the watcher design and automation guide to treat --categorize as sequential ATVM sub-runs rather than one parent run with internal phases
- document that categorized runs should send one Mattermost status per completed grouped sub-run instead of one parent-only final post
- add a --categorize option to the watcher start helper so categorized mode is explicit in watcher startup
- update the watcher implementation to track categorized sub-runs separately, write per-subrun state, and post each completed grouped run once
This commit is contained in:
anthony.wen
2026-03-26 11:00:39 -04:00
parent 68cd428733
commit d60b8b9b18
6 changed files with 399 additions and 89 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
atvm/watcher-service/README.md
View File

@@ -4,10 +4,14 @@ This folder contains a per-run ATVM watcher service package that is intended to
## Purpose
Watch a single ATVM automation run until it reaches a terminal state, then:
Watch an ATVM automation request until it reaches a terminal state, then:
- post the final status to Mattermost if the run state is `COMPLETED` or `FAILED`
- verify the Mattermost post succeeded
- for non-categorized runs:
- post one final status to Mattermost if the run state is `COMPLETED` or `FAILED`
- for categorized runs:
- detect each sequential categorized sub-run
- post one final status per completed categorized sub-run if that grouped run state is `COMPLETED` or `FAILED`
- verify each Mattermost post succeeded
- write durable watcher state
- exit cleanly so the service stops
@@ -38,14 +42,14 @@ Do not treat `/root/atvm-watcher-service` as the preferred long-term install loc
## Per-Run Behavior
Each watcher instance is tied to one build name.
Each watcher instance is tied to one requested build name.
Typical workflow:
1. Launch the ATVM run.
2. Start the watcher for that run.
3. The watcher polls the run log, process state, and `cmcReporter` artifacts.
4. When the run reaches a terminal state:
4. For non-categorized runs, when the run reaches a terminal state:
- `COMPLETED` or `FAILED`
- build the final ATVM status
- send the status to Mattermost
@@ -56,6 +60,12 @@ Typical workflow:
- do not post
- mark the final state
- exit
5. For categorized runs:
- detect each grouped sub-run in sequence from the parent run log
- wait for that grouped sub-run to finish
- send one Mattermost post for that grouped sub-run if it reached `COMPLETED` or `FAILED`
- continue to the next grouped sub-run
- exit after the parent request reaches a terminal state
## Required Environment
@@ -71,6 +81,7 @@ Optional metadata for better status formatting:
- `ATVM_WATCHER_MIGRATION_STYLE`
- `ATVM_WATCHER_INTEGRATION_PLUGIN`
- `ATVM_WATCHER_SCOPE_DESCRIPTION`
- `ATVM_WATCHER_CATEGORIZED`
## Start Example
@@ -83,6 +94,7 @@ This helper writes a per-run environment file and starts the matching instance:
--config-family gold \
--migration-style "ATVM end-to-end migration validation" \
--integration-plugin "pure with fc" \
--categorize \
--scope-description "mixed Linux and Windows FC E2E validation on the gold datastore set"
```
@@ -105,5 +117,7 @@ This writes a cancellation marker, updates `state.json` to `CANCELLED`, and stop
- The watcher uses the same ATVM status layout documented in `atvm/docs/automation/status-template.md`.
- Kernel values are resolved from `atvm/inventory/vm-inventory.md`.
- Categorized execution is treated as sequential grouped ATVM sub-runs, not as one parent run with internal phases.
- In categorized mode, the watcher writes per-subrun state under `subruns/` and posts each completed grouped run separately.
- Best-practice controller install path: `/opt/atvm-watcher-service`.
- This package is local-only right now. Nothing here is installed on the controller yet.