anthony.wen
6daa83b0c5
- update the ATVM status template to replace the FUNCTIONALLY section with a TEST FLOW section that shows the generic numbered run steps once for the whole test - update the ATVM automation guide to describe TEST FLOW as the standard status-report section instead of FUNCTIONALLY - update the watcher-generated status output so Mattermost and local status responses use the same TEST FLOW section - add a 2026-03-27 run learning recording the move from FUNCTIONALLY to TEST FLOW for future ATVM reporting
89 lines
2.6 KiB
Markdown
89 lines
2.6 KiB
Markdown
# ATVM Status Template
|
|
|
|
Use this as the default ATVM automation run-status template for:
|
|
- local status responses in the terminal
|
|
- Mattermost status posts after a completed run
|
|
|
|
## Layout
|
|
|
|
```md
|
|
## ATVM Run Status
|
|
### <build_name>
|
|
|
|
**COVERAGE:**
|
|
- template: `<template-name>`
|
|
- datastore/config family: `<config family>`
|
|
- migration style: `<high-level test style>`
|
|
- integration/plugin path: `<integration/plugin>`
|
|
- scope of this run: `<batch or run scope>`
|
|
|
|
**TEST FLOW:**
|
|
- 1. Verifying set up
|
|
- 2. Power on and obtain ip address and host name
|
|
- 3. Uninstall cmc if still exist
|
|
- 4. Setting up disk
|
|
- 5. Copy cmc install command from GUI
|
|
- 6. Install cmc
|
|
- 7. Create migration session
|
|
- 8. Tracking Changes
|
|
- 9. Trigger cmotion and do I/O test before actual cutover
|
|
- ...
|
|
- 22. Power off
|
|
|
|
**SUMMARY:**
|
|
|
|
| Metric | Value |
|
|
|---|---:|
|
|
| finished | <n> |
|
|
| passed | <n> |
|
|
| failed | <n> |
|
|
| skipped | <n> |
|
|
|
|
**HOSTS:**
|
|
|
|
| Host | Kernel | Status | Detail |
|
|
|---|---|---|---|
|
|
| <host-name> | <kernel-version or unknown> | ✅ PASS | completed |
|
|
| <host-name> | <kernel-version or unknown> | ⚠️ FAIL | <useful failure description> |
|
|
| <host-name> | <kernel-version or unknown> | ⏳ RUN | in progress |
|
|
| <host-name> | <kernel-version or unknown> | ⏭️ SKIP | <skip reason> |
|
|
|
|
**TIMING:**
|
|
|
|
| Metric | Value |
|
|
|---|---|
|
|
| start | <start time> |
|
|
| end | <end time or n/a> |
|
|
| total | <total or elapsed runtime> |
|
|
| quickest | <host> - <runtime> or n/a |
|
|
| longest | <host> - <runtime> or n/a |
|
|
| average | <runtime> or n/a |
|
|
|
|
**NOTES:**
|
|
- <note>
|
|
- <note>
|
|
```
|
|
|
|
## Rules
|
|
- Keep `COVERAGE:`, `TEST FLOW:`, `SUMMARY:`, `HOSTS:`, `TIMING:`, and `NOTES:` in that order.
|
|
- Use the title format:
|
|
- `## ATVM Run Status`
|
|
- `### <build_name>`
|
|
- Use flat bullet lists for `COVERAGE:` and `TEST FLOW:`.
|
|
- Use Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`.
|
|
- Use one host per row in the `HOSTS:` section.
|
|
- Include a `Kernel` column immediately after `Host` in the `HOSTS:` table.
|
|
- For completed hosts, prefer:
|
|
- `✅ PASS`
|
|
- `⚠️ FAIL`
|
|
- For in-progress or skipped hosts, use:
|
|
- `⏳ RUN`
|
|
- `⏭️ SKIP`
|
|
- Keep `Detail` concise.
|
|
- Put broader context under `NOTES:`, not in the host table.
|
|
- `COVERAGE:` should describe what the run was intended to cover without listing target hosts.
|
|
- `TEST FLOW:` should describe the generic numbered run flow once for the whole test, not per host.
|
|
- Resolve kernel values by cross-referencing hostnames against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`.
|
|
- If no kernel value can be verified from `vm-inventory.md`, use `unknown`.
|
|
- Use the same template for Mattermost and local operator-visible status output.
|