Commit Graph

51 Commits

Author SHA1 Message Date
Anthony Wen
5f7824619d Simplify ATVM host detail reporting 2026-06-13 18:41:19 -04:00
8c4985d33a fix(atvm-watcher): synthesize failed host result on hang-kill/nonzero exit; update run learning and vm inventory 2026-05-12 14:42:11 -04:00
65330ee9f8 atvm: fail runs explicitly on hang-kill and runner non-zero exit 2026-05-07 13:34:37 -04:00
e3497111dd atvm: default windows runs to hang_retries 0 2026-05-07 11:13:42 -04:00
28cd8a217f atvm: default compute migrateops runs to set destination static IP
Update automation guide defaults so cmc-migrateops-compute-migration includes --set_static_ip_dest by default unless explicitly overridden. Clarify vmware-specific default as vm_platforms selection, and record the behavior change in run learnings for future runs.
2026-05-07 10:08:23 -04:00
82f285a7b1 atvm: default automation runs to watcher-backed execute mode
Update ATVM run workflow rules to remove the default pre-run approval gate for automation requests while keeping safety checks around live running-state and spec verification. Set watcher-backed execution as the default unless explicitly overridden and require post-execution reporting of the exact template and runner commands used. Record the workflow shift in automation run learnings with a dated entry for future consistency.
2026-05-06 18:40:22 -04:00
2e0acb69c1 fix watcher failure detection for host reporter json
Handle dict-shaped reporter events when deriving watcher host failures.

- parse reporter JSON events with type/message/severity fields
- preserve existing support for list-shaped event records
- record the false-PASS failure mode in ATVM automation run learnings
2026-04-29 12:37:48 -04:00
9673d769e2 fix atvm watcher-backed run launch sequence
Execute the template step before starting watcher-backed ATVM runs.

- run --template-command synchronously in start-atvm-run.sh
- write template output to /tmp/<build>.launch.log
- stop before watcher/runner startup if template generation fails
- document the corrected wrapper behavior in watcher-service docs
- record the stale specPattern failure mode in automation run learnings
2026-04-29 12:14:55 -04:00
eb0e171e41 Tighten ATVM categorized run status verification
require categorized PASS results to be confirmed from the child run log or Cloud Run Finished summary

forbid treating watcher host_results, grouped XML, or check-xml-files.ts alone as proof of PASS

record the 2026-04-24 false-PASS incident as a run learning
2026-04-24 08:46:37 -04:00
86e7830d63 Fix ATVM watcher host parsing for wrapped duration rows
Handle Currents "Cloud Run Finished" table rows where the trailing
"s" in long duration values wraps onto its own continuation line.

Instead of appending that standalone continuation to the end of the
host row, drop the wrapped row and rely on the existing duration
parser to accept values without the trailing "s". This preserves all
host rows in parent summary parsing for completed non-categorized runs.

Also record the failure mode and recovery guidance in the ATVM run
learnings doc.
2026-04-22 14:03:02 -04:00
4f56ff9c4d Fix ATVM watcher install-only test flow extraction
Teach generated-spec TEST FLOW parsing to honor test-install-only runtime gates, including single-quoted Cypress.env checks, and suppress plugin branches when install-only mode disables them.

Document the 2026-04-16 install-only flow mismatch so future watcher updates keep Mattermost TEST FLOW aligned with the actual generated spec path.
2026-04-16 15:29:03 -04:00
37853e56a9 Harden ATVM watcher grouped-run summary recovery
Tighten `Cloud Run Finished` parsing so categorized watcher recovery only starts on real summary headers, stops at the grouped run's `Recorded Run` URL, and parses only actual summary-table host rows.

Also merge grouped per-host reporter artifacts into categorized recovery so completed grouped batches keep the correct host membership and Mattermost posts remain stable for both categorized and non-categorized runs.
2026-04-16 10:31:20 -04:00
70542a13a0 Fix ATVM watcher truncation after Cloud Run Finished
Adjust parent-run summary parsing so the watcher does not stop at the "Recorded Run" detection log line and miss later host rows in Mattermost status output.

Document the 2026-04-15 failure mode in ATVM run learnings so future watcher/reporting work preserves the broader Cloud Run Finished block parsing rule.
2026-04-15 21:11:38 -04:00
a5e30b6acf Use actual ATVM runtime flow steps in watcher output 2026-04-14 19:05:37 -04:00
893fe35524 Clarify ATVM systemOS command defaults 2026-04-14 15:37:53 -04:00
72ef15f308 Fix ATVM watcher test flow plugin filtering 2026-04-14 12:40:21 -04:00
e60bd5e9cd Default ATVM vmware compute migration options 2026-03-31 12:35:44 -04:00
f88f910815 Fix ATVM generated spec test flow fallback 2026-03-31 11:14:38 -04:00
da56c2668e Fix ATVM mochawesome failure extraction 2026-03-31 08:43:11 -04:00
7ab5daeca8 Split ATVM failure notes from general status notes 2026-03-30 22:31:41 -04:00
18dcbc89f9 Improve ATVM failed-host detail recovery 2026-03-30 21:38:59 -04:00
c64a64ca14 Prefer generated specs for ATVM test flow 2026-03-30 20:31:14 -04:00
b1b7e89449 Default ATVM template runs to test partition 2026-03-30 20:26:08 -04:00
e8353dc9a9 Require explicit confirmation for reboot plugin both 2026-03-30 20:20:15 -04:00
dec13a4667 Tighten ATVM completed-run status verification 2026-03-30 20:12:56 -04:00
b45375dbbc Fix false ATVM failures from reporter txt fallback 2026-03-30 17:10:25 -04:00
0b379612a8 Clarify ATVM plugin validation rules 2026-03-30 16:22:59 -04:00
944a13ec26 Tighten ATVM git push approval gate 2026-03-30 15:05:23 -04:00
1405a2e879 Fix categorized ATVM watcher host result recovery 2026-03-30 14:02:32 -04:00
63dd40faab Add detailed ATVM reboot test flow 2026-03-30 11:22:15 -04:00
d383b57ccc Align systemOS watcher output with template behavior 2026-03-27 21:30:36 -04:00
6c7ba5212b Refine categorized timing metrics and coverage output 2026-03-27 18:49:11 -04:00
06b8098499 Align ATVM coverage and notes with template command 2026-03-27 17:47:36 -04:00
708eddd7fa Remove verbose scope text from ATVM coverage 2026-03-27 17:40:32 -04:00
5432240525 Expand ATVM coverage details with run options 2026-03-27 17:32:22 -04:00
9024d5cadb Trim internal fallback details from ATVM notes 2026-03-27 16:12:14 -04:00
86e89ab9f1 Fix categorized watcher grouping and timing reconstruction 2026-03-27 16:08:13 -04:00
20c9ba7178 Fix categorized watcher subrun host recovery 2026-03-27 15:59:04 -04:00
650adb085e Keep ATVM watcher notes operator-focused 2026-03-27 14:54:39 -04:00
4186f2d0ea Persist Currents run URLs for ATVM watcher notes 2026-03-27 13:51:12 -04:00
3b28f877ef Refine explicit-VM blacklist handling for ATVM runs 2026-03-27 12:30:31 -04:00
95eeba60cd Make watcher the default ATVM approval path 2026-03-27 12:24:21 -04:00
c60542e0e5 Reorder ATVM status output and standardize push reminders 2026-03-27 11:57:19 -04:00
4b04f9b49d Fix duplicate watcher posts and restore 22-step E2E flow 2026-03-27 11:28:48 -04:00
cc551a6922 Recover watcher results when run log is missing 2026-03-27 11:00:11 -04:00
53f9e68fcd Show full template test flow in ATVM status 2026-03-27 10:19:02 -04:00
2699651f8f Require watcher startup before ATVM runner launch
- update the ATVM automation guide to make watcher-first launch order explicit whenever the watcher is approved
- update the ATVM AGENTS rules so the runner is never started before the watcher for watcher-backed runs
- add a 2026-03-27 run learning documenting that the watcher helper can delete the live runner log if the runner starts first
2026-03-27 09:25:48 -04:00
6daa83b0c5 Replace FUNCTIONALLY with TEST FLOW in ATVM status output
- update the ATVM status template to replace the FUNCTIONALLY section with a TEST FLOW section that shows the generic numbered run steps once for the whole test
- update the ATVM automation guide to describe TEST FLOW as the standard status-report section instead of FUNCTIONALLY
- update the watcher-generated status output so Mattermost and local status responses use the same TEST FLOW section
- add a 2026-03-27 run learning recording the move from FUNCTIONALLY to TEST FLOW for future ATVM reporting
2026-03-27 08:07:00 -04:00
833225378d Suppress harmless watcher reset-failed noise in ATVM updates
- update the ATVM AGENTS guidance so expected  output is not called out in routine run updates
- update the ATVM automation guide to mention reset-failed output only when it actually blocks watcher startup or matters for debugging
- add a 2026-03-26 run learning recording that this reset-failed output is normal low-signal noise during successful watcher startup
2026-03-26 21:37:35 -04:00
3431c40af7 Document ATVM spec verification lesson in run learnings
- add a 2026-03-26 run learning that explains how cmc-templates.py can generate the requested spec files while a fragile verification step still misses them
- document that shell-escaped regex one-liners over SSH are not a reliable way to validate the controller specPattern
- record the preferred future workflow: verify generated .ts files and the config specPattern directly on the controller before launching run-sorry-cypress.py
2026-03-26 19:48:16 -04:00