anthony.wen/cds-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix ATVM watcher service restart after failed units

Browse Source
This commit is contained in:
Anthony Wen
2026-08-03 22:50:04 -04:00
parent 556cce9408
commit 4d1be35ed5
4 changed files with 10 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
atvm/watcher-service/README.md
+1
View File
@@ -181,6 +181,7 @@ Runner cancel example:
- The watcher uses the same ATVM status layout documented in `atvm/docs/automation/status-template.md`. - The watcher uses the same ATVM status layout documented in `atvm/docs/automation/status-template.md`.
- Prefer the controller-local `atvm-runner@...` service over ad hoc `nohup` or detached SSH launch patterns for `run-sorry-cypress.py`. - Prefer the controller-local `atvm-runner@...` service over ad hoc `nohup` or detached SSH launch patterns for `run-sorry-cypress.py`.
- Prefer `start-atvm-run.sh` when launching both services together because it prevents the watcher/runner log-path race by enforcing watcher-first ordering. - Prefer `start-atvm-run.sh` when launching both services together because it prevents the watcher/runner log-path race by enforcing watcher-first ordering.
- The start helpers clear stale failed systemd state with `systemctl reset-failed` before starting per-build watcher/runner units.
- Kernel values are resolved from `atvm/inventory/vm-inventory.md`. - Kernel values are resolved from `atvm/inventory/vm-inventory.md`.
- Categorized execution is treated as sequential grouped ATVM sub-runs, not as one parent run with internal phases. - Categorized execution is treated as sequential grouped ATVM sub-runs, not as one parent run with internal phases.
- In categorized mode, the watcher writes per-subrun state under `subruns/` and posts each completed grouped run separately. - In categorized mode, the watcher writes per-subrun state under `subruns/` and posts each completed grouped run separately.
atvm/watcher-service/start-atvm-run-watcher.sh
+1
View File
@@ -63,6 +63,7 @@ fi
RUN_DIR="${STATE_ROOT}/${BUILD_NAME}" RUN_DIR="${STATE_ROOT}/${BUILD_NAME}"
RUN_LOG="/tmp/${BUILD_NAME}.log" RUN_LOG="/tmp/${BUILD_NAME}.log"
systemctl stop "atvm-run-watcher@${BUILD_NAME}.service" >/dev/null 2>&1 || true systemctl stop "atvm-run-watcher@${BUILD_NAME}.service" >/dev/null 2>&1 || true
systemctl reset-failed "atvm-run-watcher@${BUILD_NAME}.service" >/dev/null 2>&1 || true
rm -rf "$RUN_DIR" rm -rf "$RUN_DIR"
rm -f "$RUN_LOG" rm -f "$RUN_LOG"
mkdir -p "$RUN_DIR" mkdir -p "$RUN_DIR"
atvm/watcher-service/start-atvm-runner.sh
+1
View File
@@ -59,5 +59,6 @@ ATVM_RUNNER_LOG=${RUNNER_LOG@Q}
EOF EOF
systemctl stop "atvm-runner@${BUILD_NAME}.service" >/dev/null 2>&1 || true systemctl stop "atvm-runner@${BUILD_NAME}.service" >/dev/null 2>&1 || true
systemctl reset-failed "atvm-runner@${BUILD_NAME}.service" >/dev/null 2>&1 || true
systemctl start "atvm-runner@${BUILD_NAME}.service" systemctl start "atvm-runner@${BUILD_NAME}.service"
systemctl status --no-pager "atvm-runner@${BUILD_NAME}.service" || true systemctl status --no-pager "atvm-runner@${BUILD_NAME}.service" || true
skills/atvm/SKILL.md
+7
View File
@@ -206,6 +206,13 @@ Typical controller-side tools:
- `cmc-templates.py` - `cmc-templates.py`
- `run-sorry-cypress.py` - `run-sorry-cypress.py`
Default controller watcher wrapper:
- `/opt/atvm-watcher-service/start-atvm-run.sh`
Use that wrapper for watcher-backed ATVM automation launches when present. Do
not search only inside the Cypress checkout for the wrapper; on the ATVM
Cypress controller it is installed under `/opt/atvm-watcher-service`.
Treat their exact location as environment-specific unless a local repo or controller path is available. Treat their exact location as environment-specific unless a local repo or controller path is available.
## Artifact Recovery and Reporting ## Artifact Recovery and Reporting