anthony.wen/cds-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix ATVM mochawesome failure extraction

Browse Source
This commit is contained in:
anthony.wen
2026-03-31 08:43:11 -04:00
parent 7ab5daeca8
commit da56c2668e
2 changed files with 60 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
atvm/docs/automation/run-learnings.md
View File

@@ -487,3 +487,13 @@ This file stores run-specific examples only when a run produced a new learning r
- Keep `HOSTS` detail to the failing step plus a short error summary only.
- Put richer per-host error excerpts in `FAILURE NOTES:`.
- Reserve `NOTES:` for non-failure context such as template command, Currents URL, and operator-facing caveats.
## Run Learning: 2026-03-31 (Mochawesome failure parsing must stay within one testcase object)
- Observed failure mode:
- A reboot failure post showed step `36` with an empty `FAILURE NOTES:` excerpt even though the real failure remained step `38` and the mochawesome HTML contained the full `sshpass` / `md5sum` error text.
- The parser was scanning beyond the current mochawesome testcase object, so it paired one step title with another step's later failed-state/message fields.
- Empty mochawesome `message` / `estack` values must not be accepted as valid failure detail.
- Action for future runs:
- Parse mochawesome one testcase object at a time and do not cross object boundaries when matching `title`, `fullTitle`, `state`, `message`, and `estack`.
- Only use mochawesome to enrich host detail when it returns a non-empty failure payload.
- If mochawesome and structured reporter artifacts disagree on the step number, keep the structured reporter step as the safer fallback for the host detail.