diff --git a/atvm/docs/automation/run-learnings.md b/atvm/docs/automation/run-learnings.md index 66d55ec..ad467fa 100644 --- a/atvm/docs/automation/run-learnings.md +++ b/atvm/docs/automation/run-learnings.md @@ -174,3 +174,12 @@ This file stores run-specific examples only when a run produced a new learning r - Do not run `run-sorry-cypress.py` until the operator explicitly approves the displayed commands. - Treat template generation as execution that also requires operator approval. - If any requested option changes after commands are displayed, rebuild and redisplay the commands and wait for fresh approval. + +## Run Learning: 2026-03-26 (Verify generated specs directly on the controller before launching the runner) +- Observed failure mode: + - `cmc-templates.py` can successfully generate the requested `.ts` files, but a subsequent run can still start with an incomplete or stale `specPattern` if the runner is launched too early or the verification step is too fragile. + - Shell-escaped regex one-liners used over SSH can fail even when the controller config is actually correct, which makes the verification gate unreliable. +- Action for future runs: + - After `cmc-templates.py`, verify both the generated `.ts` files and the controller config `specPattern` before launching `run-sorry-cypress.py`. + - Prefer direct controller-side inspection of the config block and file presence rather than fragile shell-escaped regex checks. + - If the requested VM list is not visibly present in both places, stop and report the mismatch instead of starting the runner.