Update ATVM automation blacklist and status formatting guidance

Add atvm144-suse15.0 to the ATVM automation blacklist because it crashes when creating a migration session.

Update maintained exclude examples to include the new blacklist entry.

Tighten status reporting guidance to require one machine per line.

Add a Notes section for failure reasons and operator-facing context.

Record the new run learnings in atvm-automation-runs.md.
This commit is contained in:
2026-03-12 11:03:08 -04:00
parent 48f189792b
commit e1a551cdb2
4 changed files with 45 additions and 16 deletions

View File

@@ -83,6 +83,9 @@ Permanently blacklisted because CMC cannot compile:
- `atvm41-redhat6.0`
- `atvm73-oracle6.0`
Temporarily blacklisted because the run crashes when creating a migration session:
- `atvm144-suse15.0`
Temporarily blacklisted while support requests are waiting:
- `atvm113-debian9.0.0`
- `atvm115-debian9.1.0`
@@ -93,7 +96,7 @@ Temporarily blacklisted until re-created:
- `atvm157-debian13.0.0`
Preferred exclude list:
- `--exclude_partial_match atvm6-centos6.0 atvm41-redhat6.0 atvm73-oracle6.0 atvm113-debian9.0.0 atvm115-debian9.1.0 atvm116-debian9.2.0 atvm156-debian9.3.0 atvm157-debian13.0.0`
- `--exclude_partial_match atvm6-centos6.0 atvm41-redhat6.0 atvm73-oracle6.0 atvm144-suse15.0 atvm113-debian9.0.0 atvm115-debian9.1.0 atvm116-debian9.2.0 atvm156-debian9.3.0 atvm157-debian13.0.0`
## Running-Automation Check (Mandatory)
Before any new automation request:
@@ -134,22 +137,25 @@ When asked for one VM or a VM set:
When the operator asks for the status of an ATVM automation run, report in this order:
1. Heading/title using the run `build_name`.
2. Completed machines with machine name first and status second for each machine.
3. Skipped machines with reason.
4. Remaining machines still to run.
5. Summary counts for finished, passed, failed, and skipped machines.
6. Timing details:
3. Notes.
4. Skipped machines with reason.
5. Remaining machines still to run.
6. Summary counts for finished, passed, failed, and skipped machines.
7. Timing details:
- start time
- end time if complete
- total run time if complete, or elapsed run time if still running
- quickest completed test runtime
- longest completed test runtime
- average completed test runtime
7. Estimated completion time.
8. Estimated completion time.
Status-report expectations:
- Use the live automation VM state when available.
- Derive the heading/title from the run `build_name` when available.
- Format every machine entry as `machine-name - STATUS`.
- Put each machine on its own line; never combine multiple machines into one paragraph or comma-separated line.
- Use a separate `Notes` section for failure reasons, anomalies, or operator-relevant context rather than cramming those details into the completed-machine list.
- For categorized runs, reconstruct the whole run across all category batches; do not treat the current live category batch as the full run scope.
- Derive completed-machine status from completed spec results already written during the same run.
- Parse all same-run `test-result-*.xml` files, not only machine-named `test-result-atvm*.xml` files.
@@ -168,6 +174,6 @@ Status-report expectations:
- If a machine is currently in progress, show it under remaining machines as `RUNNING`.
- If a machine has not started yet, show it under remaining machines as `NOT STARTED`.
- If no failures are present in completed spec results, report those completed machines as `PASS`.
- If a completed spec result shows a failure, report that machine as `FAIL` and include the failure reason from the run log.
- If a completed spec result shows a failure, report that machine as `FAIL` in the completed list and put the failure reason in `Notes`.
- Base the completion estimate on the full remaining machine count and recent per-machine runtime visible in the run log.
- Make the estimate explicitly refer to completion of the entire remaining run, not only the current machine/spec.