Files

anthony.wen 6daa83b0c5 Replace FUNCTIONALLY with TEST FLOW in ATVM status output

- update the ATVM status template to replace the FUNCTIONALLY section with a TEST FLOW section that shows the generic numbered run steps once for the whole test
- update the ATVM automation guide to describe TEST FLOW as the standard status-report section instead of FUNCTIONALLY
- update the watcher-generated status output so Mattermost and local status responses use the same TEST FLOW section
- add a 2026-03-27 run learning recording the move from FUNCTIONALLY to TEST FLOW for future ATVM reporting

2026-03-27 08:07:00 -04:00

2.6 KiB

Raw Blame History

ATVM Status Template

Use this as the default ATVM automation run-status template for:

local status responses in the terminal
Mattermost status posts after a completed run

Layout

## ATVM Run Status
### <build_name>

**COVERAGE:**
- template: `<template-name>`
- datastore/config family: `<config family>`
- migration style: `<high-level test style>`
- integration/plugin path: `<integration/plugin>`
- scope of this run: `<batch or run scope>`

**TEST FLOW:**
- 1. Verifying set up
- 2. Power on and obtain ip address and host name
- 3. Uninstall cmc if still exist
- 4. Setting up disk
- 5. Copy cmc install command from GUI
- 6. Install cmc
- 7. Create migration session
- 8. Tracking Changes
- 9. Trigger cmotion and do I/O test before actual cutover
- ...
- 22. Power off

**SUMMARY:**

| Metric | Value |
|---|---:|
| finished | <n> |
| passed | <n> |
| failed | <n> |
| skipped | <n> |

**HOSTS:**

| Host | Kernel | Status | Detail |
|---|---|---|---|
| <host-name> | <kernel-version or unknown> | ✅ PASS | completed |
| <host-name> | <kernel-version or unknown> | ⚠️ FAIL | <useful failure description> |
| <host-name> | <kernel-version or unknown> | ⏳ RUN | in progress |
| <host-name> | <kernel-version or unknown> | ⏭️ SKIP | <skip reason> |

**TIMING:**

| Metric | Value |
|---|---|
| start | <start time> |
| end | <end time or n/a> |
| total | <total or elapsed runtime> |
| quickest | <host> - <runtime> or n/a |
| longest | <host> - <runtime> or n/a |
| average | <runtime> or n/a |

**NOTES:**
- <note>
- <note>

Rules

Keep COVERAGE:, TEST FLOW:, SUMMARY:, HOSTS:, TIMING:, and NOTES: in that order.
Use the title format:
- ## ATVM Run Status
- ### <build_name>
Use flat bullet lists for COVERAGE: and TEST FLOW:.
Use Markdown tables for SUMMARY:, HOSTS:, and TIMING:.
Use one host per row in the HOSTS: section.
Include a Kernel column immediately after Host in the HOSTS: table.
For completed hosts, prefer:
- ✅ PASS
- ⚠️ FAIL
For in-progress or skipped hosts, use:
- ⏳ RUN
- ⏭️ SKIP
Keep Detail concise.
Put broader context under NOTES:, not in the host table.
COVERAGE: should describe what the run was intended to cover without listing target hosts.
TEST FLOW: should describe the generic numbered run flow once for the whole test, not per host.
Resolve kernel values by cross-referencing hostnames against /home/aw/code/cds/atvm/inventory/vm-inventory.md.
If no kernel value can be verified from vm-inventory.md, use unknown.
Use the same template for Mattermost and local operator-visible status output.

2.6 KiB Raw Blame History

ATVM Status Template

Layout

Rules

2.6 KiB

Raw Blame History