Fix false ATVM failures from reporter txt fallback
This commit is contained in:
@@ -392,3 +392,12 @@ This file stores run-specific examples only when a run produced a new learning r
|
||||
- Do not treat the presence of plugin-specific strings or code blocks in the generated `.ts` file as proof that those plugin steps will execute.
|
||||
- For plugin-specific questions, determine expected behavior from the template/runtime gate and only call it a mismatch if the runtime logic would execute the wrong plugin path.
|
||||
- Continue verifying that the requested VM set is present in the generated files and `specPattern`, but keep plugin-path validation separate from simple text-presence checks.
|
||||
|
||||
## Run Learning: 2026-03-30 (Do not classify reporter TXT logs as failed from generic `error` words)
|
||||
- Observed failure mode:
|
||||
- A completed `reboot-redhat8.10-iscsi` run actually passed in the launch log and `Cloud Run Finished` table, but the watcher saved it as failed.
|
||||
- The TXT fallback matched generic strings such as `auth error encountered` and treated them as proof of host failure.
|
||||
- Action for future runs:
|
||||
- Do not classify a reporter TXT artifact as failed just because it contains the word `error`.
|
||||
- For TXT fallback, require explicit terminal failure markers such as `cy:command error`, `cy:task error`, or real `Error:`/`AssertionError:`/timeout text.
|
||||
- Prefer the parent run summary when available, because it is less prone to false failure signals than raw per-step console text.
|
||||
|
||||
Reference in New Issue
Block a user