anthony.wen/cds-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix ATVM generated spec test flow fallback

Browse Source
This commit is contained in:
anthony.wen
2026-03-31 11:14:38 -04:00
parent 7f87356832
commit f88f910815
2 changed files with 42 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
atvm/docs/automation/run-learnings.md
View File

@@ -497,3 +497,12 @@ This file stores run-specific examples only when a run produced a new learning r
- Parse mochawesome one testcase object at a time and do not cross object boundaries when matching `title`, `fullTitle`, `state`, `message`, and `estack`.
- Only use mochawesome to enrich host detail when it returns a non-empty failure payload.
- If mochawesome and structured reporter artifacts disagree on the step number, keep the structured reporter step as the safer fallback for the host detail.
## Run Learning: 2026-03-31 (Generated-spec `TEST FLOW` must not depend only on log-scoped `specPattern`)
- Observed failure mode:
- A completed `e2e-redhat8.10-both` run posted the static 22-step `cmc-e2e` flow even though the generated spec for that exact run contained a longer flow.
- The watcher only extracted generated-spec flow when it could find `Extracted specPattern:` in the available log text.
- When that log-scoped `specPattern` line was unavailable at final render time, the watcher silently fell back to the static template flow.
- Action for future runs:
- Resolve generated-spec `TEST FLOW` from the active config file's `specPattern` when the required log line is missing.
- Treat the static template flow as a last-resort fallback only after both log-derived and config-derived `specPattern` resolution fail.