Switch ATVM watcher status posts to MS Teams

This commit is contained in:
Anthony Wen
2026-07-27 19:07:50 -04:00
parent 67886174b3
commit 063d13d01a
8 changed files with 107 additions and 90 deletions
+15 -15
View File
@@ -10,9 +10,9 @@ Install the local watcher/runner package so the controller can:
- start one requested ATVM Cypress runner per service instance
- watch one requested ATVM run per watcher instance
- for non-categorized runs, send one final Mattermost status only for `COMPLETED` or `FAILED`
- for categorized runs, send one final Mattermost status per completed categorized sub-run/group
- suppress Mattermost posts for `CANCELLED`, `TERMINATED`, `HUNG`, and `UNKNOWN`
- for non-categorized runs, send one final MS Teams status only for `COMPLETED` or `FAILED`
- for categorized runs, send one final MS Teams status per completed categorized sub-run/group
- suppress MS Teams posts for `CANCELLED`, `TERMINATED`, `HUNG`, and `UNKNOWN`
- stop automatically after the watched run reaches a terminal state
## Controller Target Layout
@@ -64,7 +64,7 @@ The controller must have:
- `python3`
- `systemd`
- outbound network access to the Mattermost webhook
- outbound network access to the MS Teams webhook
- read access to:
- `/root/cdc-e2e-cyp-12.17.4/cypress/cmcReporter`
- `/tmp/<build-name>.log`
@@ -73,8 +73,8 @@ The controller must have:
The controller needs a watcher environment file with:
- `MATTERMOST_ATVM_WEBHOOK`
- `MATTERMOST_ATVM_CHANNEL`
- `MS_TEAMS_ATVM_WEBHOOK`
- `MS_TEAMS_ATVM_CHANNEL`
Recommended file:
@@ -110,7 +110,7 @@ Recommended permissions:
- `cancel-atvm-run-watcher.sh`
4. Create `/etc/atvm-run-watcher.env`.
- add Mattermost webhook/channel
- add MS Teams webhook/channel
- keep permissions restricted
5. Install the `systemd` unit file.
@@ -134,7 +134,7 @@ Recommended permissions:
- launch a real run
- start the watcher for that build name
- if the run uses `--categorize`, also pass `--categorize` to the watcher start helper
- confirm final Mattermost delivery for a completed run
- confirm final MS Teams delivery for a completed run
- confirm categorized execution sends one post per completed grouped sub-run
- confirm the watcher stays alive between categorized grouped runs while the parent request is still active
- confirm reused parent build names do not inherit stale `cancelled.marker`, `posted.marker`, or `subruns/` state from older runs
@@ -242,7 +242,7 @@ The cancel helper should:
- write `cancelled.marker`
- update `state.json` so the final watcher state is `CANCELLED`
- stop the watcher instance
- avoid any Mattermost post for that run
- avoid any MS Teams post for that run
## Operational Notes
@@ -255,7 +255,7 @@ The cancel helper should:
- In categorized execution, the watcher must remain alive until the parent request has actually gone inactive past the grace window, even if one grouped sub-run already completed.
- The watcher exits after the run reaches a terminal state.
- The watcher writes state under `/var/lib/atvm-run-watcher/<build-name>`.
- The watcher prevents duplicate Mattermost posts by writing posted markers.
- The watcher prevents duplicate MS Teams posts by writing posted markers.
- Categorized sub-run state is written under `/var/lib/atvm-run-watcher/<build-name>/subruns/<subrun-key>/`.
## Failure Handling
@@ -263,16 +263,16 @@ The cancel helper should:
Expected terminal behavior:
- `COMPLETED`
- post to Mattermost
- verify `ok`
- post to MS Teams
- verify the webhook returned HTTP 2xx, typically `202 Accepted`
- exit
- `FAILED`
- post to Mattermost
- verify `ok`
- post to MS Teams
- verify the webhook returned HTTP 2xx, typically `202 Accepted`
- exit
- categorized `COMPLETED` / `FAILED`
- post once for that grouped sub-run
- verify `ok`
- verify the webhook returned HTTP 2xx, typically `202 Accepted`
- continue until the parent request finishes
- `CANCELLED`
- write final `CANCELLED` state to `state.json`