anthony.wen/cds-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update ATVM status reporting to use longer inline failure descriptions

Browse Source 
- show failed machines with a longer failure description on the same status line

- keep Notes for broader context beyond the machine-specific failure reason

- update the ATVM automation guide and AGENTS rules to match

- record the reporting preference in atvm-automation-runs.md
This commit is contained in:
anthony.wen
2026-03-12 21:13:54 -04:00
parent ecae52712f
commit eb66f93432
3 changed files with 13 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
atvm/AGENTS.md
View File

@@ -124,7 +124,8 @@ Status details:
- Show in-progress machines under remaining machines as `RUNNING`. - Show in-progress machines under remaining machines as `RUNNING`.
- Show not-yet-started machines as `NOT STARTED`. - Show not-yet-started machines as `NOT STARTED`.
- Use completed spec results already recorded in the log to determine machine pass/fail state. - Use completed spec results already recorded in the log to determine machine pass/fail state.
- For failed machines, mark the machine `FAIL` in the completed list and include the failure reason in `Notes`. - For failed machines, mark the machine `FAIL` in the completed list and append a longer failure description on the same machine line when the reason materially helps the operator understand the failure.
- Keep `Notes` for broader context, anomalies, or follow-up details that are not part of the machine-specific failure description.
- Include start time in status output when it can be derived from the log. - Include start time in status output when it can be derived from the log.
- Include end time and total runtime for completed runs, or elapsed runtime for active runs. - Include end time and total runtime for completed runs, or elapsed runtime for active runs.
- Include quickest completed test runtime, longest completed test runtime, and average completed test runtime under timing details when they can be derived from the log. - Include quickest completed test runtime, longest completed test runtime, and average completed test runtime under timing details when they can be derived from the log.

3
atvm/atvm-automation-guide.md
View File

@@ -182,6 +182,7 @@ Status-report expectations:
- If a machine is currently in progress, show it under remaining machines as `RUNNING`. - If a machine is currently in progress, show it under remaining machines as `RUNNING`.
- If a machine has not started yet, show it under remaining machines as `NOT STARTED`. - If a machine has not started yet, show it under remaining machines as `NOT STARTED`.
- If no failures are present in completed spec results, report those completed machines as `PASS`. - If no failures are present in completed spec results, report those completed machines as `PASS`.
- If a completed spec result shows a failure, report that machine as `FAIL` in the completed list and put the failure reason in `Notes`. - If a completed spec result shows a failure, report that machine as `FAIL` in the completed list and append a longer same-line failure description when the extra detail is useful to the operator.
- Use `Notes` for extra context beyond the machine-specific same-line failure description.
- Base the completion estimate on the full remaining machine count and recent per-machine runtime visible in the run log. - Base the completion estimate on the full remaining machine count and recent per-machine runtime visible in the run log.
- Make the estimate explicitly refer to completion of the entire remaining run, not only the current machine/spec. - Make the estimate explicitly refer to completion of the entire remaining run, not only the current machine/spec.

9
atvm/atvm-automation-runs.md
View File

@@ -117,3 +117,12 @@ This file stores run-specific examples only when a run produced a new learning r
- Use one consistent ATVM status layout for all automation status responses. - Use one consistent ATVM status layout for all automation status responses.
- Keep the order the same: build name, completed machines, notes, skipped machines, remaining machines, summary, timing, estimated completion time. - Keep the order the same: build name, completed machines, notes, skipped machines, remaining machines, summary, timing, estimated completion time.
- Keep machine entries one per line as `machine-name - STATUS` regardless of test type. - Keep machine entries one per line as `machine-name - STATUS` regardless of test type.
## Run Learning: 2026-03-13 (Put longer failure description on failed machine line)
- Observed requirement:
- Failed machines are easier to scan when the failure description appears directly on the same line as the machine status.
- A longer same-line description works better than a very short label when the extra detail helps explain what actually failed.
- Action for future runs:
- Format failed machine lines as `machine-name - FAIL - <failure description>`.
- Prefer the longer same-line description when it adds useful operator-facing context.
- Keep `Notes` for broader context, anomalies, and extra follow-up detail beyond the machine-specific failure description.