diff --git a/atvm/AGENTS.md b/atvm/AGENTS.md index a880de6..2eaa5ad 100644 --- a/atvm/AGENTS.md +++ b/atvm/AGENTS.md @@ -62,6 +62,8 @@ This file defines how to operate and maintain the ATVM workspace in `/home/aw/co - For ATVM run approvals, treat `approve` as run-without-watcher and `approve with watcher` as run-with-watcher. - After `cmc-templates.py`, always verify that the generated spec files and the config `specPattern` still include every requested VM before starting `run-sorry-cypress.py`. - If any requested VM is missing after template generation, stop and report the mismatch instead of launching the runner. +- When the watcher is requested, start the watcher before `run-sorry-cypress.py`. +- Do not start the runner before the watcher, because the watcher helper clears stale `/tmp/.log` and can delete the fresh live runner log if the runner starts first. - For host-level test detail and failed-test investigation, use `/root/cdc-e2e-cyp-12.17.4/cypress/cmcReporter`, especially `logs/`, `xml/`, and `mochawesome/`. - If the operator asks for ATVM run status without mentioning Mattermost, respond locally only and do not post externally. - If the operator asks to send ATVM run status to Mattermost, use `MATTERMOST_ATVM_WEBHOOK` and `MATTERMOST_ATVM_CHANNEL` from `/home/aw/code/cds/.env.credentials.local` by default and send the final status only after the run has fully completed, whether the run passed or failed. diff --git a/atvm/docs/automation/guide.md b/atvm/docs/automation/guide.md index 9f579d9..b536e2f 100644 --- a/atvm/docs/automation/guide.md +++ b/atvm/docs/automation/guide.md @@ -81,7 +81,8 @@ Typical sequence: 4. Run `cmc-templates.py` with the approved options. 5. Wait for `cmc-templates.py` to fully finish and confirm success. 6. Verify the generated `.ts` files and the config `specPattern` include every requested VM before starting the runner. -7. Run `run-sorry-cypress.py` with the matching approved config and build name. +7. If the watcher is approved, start the watcher before launching `run-sorry-cypress.py`. +8. Run `run-sorry-cypress.py` with the matching approved config and build name. ## Config File / Gold Disk Mapping - `cypress.atvm-config-gold.ts` -> Gold Disk 1 @@ -173,6 +174,8 @@ Before any new automation request: 14. If the operator asks to change plugin, config, filters, build name, Gold Disk, or scope after commands are shown, discard the old plan, show the revised commands, and wait for new approval. 15. If monitoring was not requested, report immediate success/failure for each command. 16. If monitoring was requested, keep monitoring until completion and report final outcome. +17. When the watcher is requested, launch the watcher before `run-sorry-cypress.py`. +18. Do not start the runner before the watcher, because the watcher helper clears stale `/tmp/.log` and can delete the fresh live runner log if the runner starts first. ## Requested Test Style When asked for one VM or a VM set: diff --git a/atvm/docs/automation/run-learnings.md b/atvm/docs/automation/run-learnings.md index 90d9224..b540edd 100644 --- a/atvm/docs/automation/run-learnings.md +++ b/atvm/docs/automation/run-learnings.md @@ -211,3 +211,13 @@ This file stores run-specific examples only when a run produced a new learning r - `9. Trigger cmotion and do I/O test before actual cutover` - `...` - `22. Power off` + +## Run Learning: 2026-03-27 (Start watcher before runner when watcher is requested) +- Observed failure mode: + - Starting `run-sorry-cypress.py` before the watcher can race with the watcher helper's stale-log cleanup. + - The watcher helper clears stale `/tmp/.log` before startup. + - If the runner has already opened the new log, the helper can delete that live log path, leaving the watcher unable to read the run by filename. +- Action for future runs: + - When the watcher is approved, start the watcher before `run-sorry-cypress.py`. + - Keep the order as: template generation, verification, watcher start, runner start. + - Do not launch the runner first when the watcher is part of the approved command set.