anthony.wen/cds-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Prefer generated specs for ATVM test flow

Browse Source
This commit is contained in:
anthony.wen
2026-03-30 20:31:14 -04:00
parent b1b7e89449
commit c64a64ca14
3 changed files with 16 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
atvm/docs/automation/guide.md
View File

@@ -274,7 +274,11 @@ Status-report expectations:
- If `categorize mode: enabled` is already shown in `COVERAGE:`, do not also repeat `--categorize` under `run options`.
- When grouped categorized timing is reconstructed from host reporter artifacts, derive per-host quickest/longest/average durations from the sequence of recovered host timestamps and the grouped end time instead of leaving those metrics as `n/a`.
- In `TEST FLOW:`, show the template-specific numbered run flow once for the whole test, not per host.
- Resolve the flow from the run template name.
- For `TEST FLOW:`, treat the generated host spec from the actual run as the source of truth whenever it exists.
- Extract the numbered flow steps from the generated `.ts` spec referenced by that run's `specPattern`.
- Do not prefer a static template flow list over a generated spec from the actual run.
- Use template-level or static fallback flow only when the generated spec cannot be found or parsed.
- If fallback is required, resolve it from the run template name before using any generic default flow.
- `cmc-e2e` currently uses the 22-step migration flow documented in `/home/aw/code/cds/atvm/docs/automation/status-template.md`.
- `cmc-systemOS` currently uses the 21-step boot-disk migration flow documented in `/home/aw/code/cds/atvm/docs/automation/status-template.md`.
- Keep `NOTES:` behavior consistent across template types; do not add template-specific internal-source notes such as parent-log-summary recovery details.

8
atvm/docs/automation/run-learnings.md
View File

@@ -443,3 +443,11 @@ This file stores run-specific examples only when a run produced a new learning r
- Action for future runs:
- Add `--test_partition` to `cmc-templates.py` commands by default.
- Omit it only when the operator explicitly asks not to use it.
## Run Learning: 2026-03-30 (Use generated spec as the source of truth for `TEST FLOW:`)
- Observed operator requirement:
- The operator wants the current full workflow steps for the actual test template/run, not a stale hand-maintained flow list.
- Action for future runs:
- Resolve `TEST FLOW:` from the generated `.ts` spec for the actual run whenever that spec exists.
- Extract the numbered `it(...)` steps from the generated spec referenced by the run's `specPattern`.
- Only use template-level or static fallback flow definitions when the generated spec cannot be found or parsed.