anthony.wen/cds-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document ATVM spec verification lesson in run learnings

Browse Source 
- add a 2026-03-26 run learning that explains how cmc-templates.py can generate the requested spec files while a fragile verification step still misses them
- document that shell-escaped regex one-liners over SSH are not a reliable way to validate the controller specPattern
- record the preferred future workflow: verify generated .ts files and the config specPattern directly on the controller before launching run-sorry-cypress.py
This commit is contained in:
anthony.wen
2026-03-26 19:48:16 -04:00
parent 337a42066f
commit 3431c40af7
1 changed files with 9 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
atvm/docs/automation/run-learnings.md
View File

@@ -174,3 +174,12 @@ This file stores run-specific examples only when a run produced a new learning r
- Do not run `run-sorry-cypress.py` until the operator explicitly approves the displayed commands.
- Treat template generation as execution that also requires operator approval.
- If any requested option changes after commands are displayed, rebuild and redisplay the commands and wait for fresh approval.
## Run Learning: 2026-03-26 (Verify generated specs directly on the controller before launching the runner)
- Observed failure mode:
- `cmc-templates.py` can successfully generate the requested `.ts` files, but a subsequent run can still start with an incomplete or stale `specPattern` if the runner is launched too early or the verification step is too fragile.
- Shell-escaped regex one-liners used over SSH can fail even when the controller config is actually correct, which makes the verification gate unreliable.
- Action for future runs:
- After `cmc-templates.py`, verify both the generated `.ts` files and the controller config `specPattern` before launching `run-sorry-cypress.py`.
- Prefer direct controller-side inspection of the config block and file presence rather than fragile shell-escaped regex checks.
- If the requested VM list is not visibly present in both places, stop and report the mismatch instead of starting the runner.