From 3431c40af75f9956b83be1fd05fe12593f64fe0b Mon Sep 17 00:00:00 2001 From: "anthony.wen" Date: Thu, 26 Mar 2026 19:48:16 -0400 Subject: [PATCH] Document ATVM spec verification lesson in run learnings - add a 2026-03-26 run learning that explains how cmc-templates.py can generate the requested spec files while a fragile verification step still misses them - document that shell-escaped regex one-liners over SSH are not a reliable way to validate the controller specPattern - record the preferred future workflow: verify generated .ts files and the config specPattern directly on the controller before launching run-sorry-cypress.py --- atvm/docs/automation/run-learnings.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/atvm/docs/automation/run-learnings.md b/atvm/docs/automation/run-learnings.md index 66d55ec..ad467fa 100644 --- a/atvm/docs/automation/run-learnings.md +++ b/atvm/docs/automation/run-learnings.md @@ -174,3 +174,12 @@ This file stores run-specific examples only when a run produced a new learning r - Do not run `run-sorry-cypress.py` until the operator explicitly approves the displayed commands. - Treat template generation as execution that also requires operator approval. - If any requested option changes after commands are displayed, rebuild and redisplay the commands and wait for fresh approval. + +## Run Learning: 2026-03-26 (Verify generated specs directly on the controller before launching the runner) +- Observed failure mode: + - `cmc-templates.py` can successfully generate the requested `.ts` files, but a subsequent run can still start with an incomplete or stale `specPattern` if the runner is launched too early or the verification step is too fragile. + - Shell-escaped regex one-liners used over SSH can fail even when the controller config is actually correct, which makes the verification gate unreliable. +- Action for future runs: + - After `cmc-templates.py`, verify both the generated `.ts` files and the controller config `specPattern` before launching `run-sorry-cypress.py`. + - Prefer direct controller-side inspection of the config block and file presence rather than fragile shell-escaped regex checks. + - If the requested VM list is not visibly present in both places, stop and report the mismatch instead of starting the runner.