anthony.wen/cds-ai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add ATVM coverage and functional sections to status template

Browse Source 
- update the ATVM status template to include COVERAGE: and FUNCTIONALLY: sections ahead of the existing summary tables
- document that COVERAGE: should describe intended run scope without listing target hosts
- document that FUNCTIONALLY: should summarize the intended workflow steps at a high level
- align the ATVM automation guide so local status output and Mattermost posts use the expanded default format
This commit is contained in:
anthony.wen
2026-03-24 22:10:25 -04:00
parent 86b1a0e4a9
commit 0bf07f0e9c
2 changed files with 28 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
atvm/docs/automation/guide.md
View File

@@ -190,24 +190,28 @@ When asked for one VM or a VM set:
## Status Reporting Format
When the operator asks for the status of an ATVM automation run, report in this order:
1. Heading/title using the run `build_name`.
2. Completed machines with machine name first and status second for each machine.
3. Notes.
4. Skipped machines with reason.
5. Remaining machines still to run.
6. Summary counts for finished, passed, failed, and skipped machines.
7. Timing details:
2. `COVERAGE:` section describing what the run was intended to cover, excluding the target-host list.
3. `FUNCTIONALLY:` section describing the intended workflow steps at a high level.
4. Completed machines with machine name first and status second for each machine.
5. Notes.
6. Skipped machines with reason.
7. Remaining machines still to run.
8. Summary counts for finished, passed, failed, and skipped machines.
9. Timing details:
- start time
- end time if complete
- total run time if complete, or elapsed run time if still running
- quickest completed test runtime
- longest completed test runtime
- average completed test runtime
8. Estimated completion time.
10. Estimated completion time.
Status-report expectations:
- Use the same display layout for every ATVM automation status response regardless of test type (`e2e`, `systemOS`, `reboot`, `migrateops`, and others).
- Use `/home/aw/code/cds/atvm/docs/automation/status-template.md` as the default template for both local status output and Mattermost status posts.
- The default ATVM status template uses Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:` and uses `NOTES:` for flat operator-facing notes.
- The default ATVM status template uses flat bullet-list sections for `COVERAGE:` and `FUNCTIONALLY:`, Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`, and uses `NOTES:` for flat operator-facing notes.
- In `COVERAGE:`, describe the template, datastore/config family, migration style, and plugin/integration path, but do not list target hosts there.
- In `FUNCTIONALLY:`, summarize what the run is intended to do at a high level, even when the run failed before reaching those steps.
- Treat references to the "ATVM automation run" or "automation run" as referring to this ATVM folder workflow and the automation VM at `192.168.3.190`, not to Cirrus project operations such as the `atvm - cypress` project.
- Treat a status request as a request for live status by default.
- Unless the operator explicitly asks to send the status to Mattermost, print the status only in the local terminal response.

17
atvm/docs/automation/status-template.md
View File

@@ -10,6 +10,18 @@ Use this as the default ATVM automation run-status template for:
## ATVM Run Status
### <build_name>
**COVERAGE:**
- template: `<template-name>`
- datastore/config family: `<config family>`
- migration style: `<high-level test style>`
- integration/plugin path: `<integration/plugin>`
- scope of this run: `<batch or run scope>`
**FUNCTIONALLY:**
- <intended functional step>
- <intended functional step>
- <intended functional step>
**SUMMARY:**
| Metric | Value |
@@ -45,10 +57,11 @@ Use this as the default ATVM automation run-status template for:
```
## Rules
- Keep `SUMMARY:`, `HOSTS:`, `TIMING:`, and `NOTES:` in that order.
- Keep `COVERAGE:`, `FUNCTIONALLY:`, `SUMMARY:`, `HOSTS:`, `TIMING:`, and `NOTES:` in that order.
- Use the title format:
- `## ATVM Run Status`
- `### <build_name>`
- Use flat bullet lists for `COVERAGE:` and `FUNCTIONALLY:`.
- Use Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`.
- Use one host per row in the `HOSTS:` section.
- For completed hosts, prefer:
@@ -59,4 +72,6 @@ Use this as the default ATVM automation run-status template for:
- `⏭️ SKIP`
- Keep `Detail` concise.
- Put broader context under `NOTES:`, not in the host table.
- `COVERAGE:` should describe what the run was intended to cover without listing target hosts.
- `FUNCTIONALLY:` should describe the intended workflow steps at a high level, even if the run failed before completing them.
- Use the same template for Mattermost and local operator-visible status output.