anthony.wen/cds-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Require ATVM spec verification before starting the runner

Browse Source 
- update the ATVM automation guide to require a post-template verification gate before launching run-sorry-cypress.py
- require verification that both the generated .ts files and the config specPattern include every requested VM after cmc-templates.py finishes
- document that the runner must not be launched when any requested VM is missing from the generated spec set
- update the ATVM AGENTS rules so this template-to-runner verification step is part of the default automation workflow
This commit is contained in:
anthony.wen
2026-03-26 15:55:13 -04:00
parent 35342c1f23
commit 2eed645a85
2 changed files with 10 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
atvm/AGENTS.md
View File

@@ -60,6 +60,8 @@ This file defines how to operate and maintain the ATVM workspace in `/home/aw/co
- Always show exact planned ATVM commands before execution.
- Never execute setup or automation commands that require approval until the operator explicitly approves them.
- For ATVM run approvals, treat `approve` as run-without-watcher and `approve with watcher` as run-with-watcher.
- After `cmc-templates.py`, always verify that the generated spec files and the config `specPattern` still include every requested VM before starting `run-sorry-cypress.py`.
- If any requested VM is missing after template generation, stop and report the mismatch instead of launching the runner.
- For host-level test detail and failed-test investigation, use `/root/cdc-e2e-cyp-12.17.4/cypress/cmcReporter`, especially `logs/`, `xml/`, and `mochawesome/`.
- If the operator asks for ATVM run status without mentioning Mattermost, respond locally only and do not post externally.
- If the operator asks to send ATVM run status to Mattermost, use `MATTERMOST_ATVM_WEBHOOK` and `MATTERMOST_ATVM_CHANNEL` from `/home/aw/code/cds/.env.credentials.local` by default and send the final status only after the run has fully completed, whether the run passed or failed.

13
atvm/docs/automation/guide.md
View File

@@ -78,7 +78,8 @@ Typical sequence:
3. Wait for explicit approval.
4. Run `cmc-templates.py` with the approved options.
5. Wait for `cmc-templates.py` to fully finish and confirm success.
6. Run `run-sorry-cypress.py` with the matching approved config and build name.
6. Verify the generated `.ts` files and the config `specPattern` include every requested VM before starting the runner.
7. Run `run-sorry-cypress.py` with the matching approved config and build name.
## Config File / Gold Disk Mapping
- `cypress.atvm-config-gold.ts` -> Gold Disk 1
@@ -164,10 +165,12 @@ Before any new automation request:
8. Run only approved command(s), no extra options and no silent substitutions.
9. When both template generation and the Cypress runner are requested, run them sequentially, not in parallel.
10. Do not launch `run-sorry-cypress.py` until `cmc-templates.py` has exited successfully and finished updating the intended config/spec files.
11. Treat displayed commands as a review gate: do not execute either command until the operator has had a chance to review them and explicitly approve.
12. If the operator asks to change plugin, config, filters, build name, Gold Disk, or scope after commands are shown, discard the old plan, show the revised commands, and wait for new approval.
13. If monitoring was not requested, report immediate success/failure for each command.
14. If monitoring was requested, keep monitoring until completion and report final outcome.
11. After `cmc-templates.py`, always verify that the generated spec files on disk and the config `specPattern` both contain the full requested VM set before launching `run-sorry-cypress.py`.
12. If any requested VM is missing from the generated files or `specPattern`, stop and report the mismatch instead of launching the runner.
13. Treat displayed commands as a review gate: do not execute either command until the operator has had a chance to review them and explicitly approve.
14. If the operator asks to change plugin, config, filters, build name, Gold Disk, or scope after commands are shown, discard the old plan, show the revised commands, and wait for new approval.
15. If monitoring was not requested, report immediate success/failure for each command.
16. If monitoring was requested, keep monitoring until completion and report final outcome.
## Requested Test Style
When asked for one VM or a VM set: