Replace FUNCTIONALLY with TEST FLOW in ATVM status output

- update the ATVM status template to replace the FUNCTIONALLY section with a TEST FLOW section that shows the generic numbered run steps once for the whole test
- update the ATVM automation guide to describe TEST FLOW as the standard status-report section instead of FUNCTIONALLY
- update the watcher-generated status output so Mattermost and local status responses use the same TEST FLOW section
- add a 2026-03-27 run learning recording the move from FUNCTIONALLY to TEST FLOW for future ATVM reporting
This commit is contained in:
2026-03-27 08:07:00 -04:00
parent 833225378d
commit 6daa83b0c5
4 changed files with 49 additions and 17 deletions

View File

@@ -211,7 +211,7 @@ When asked for one VM or a VM set:
When the operator asks for the status of an ATVM automation run, report in this order:
1. Heading/title using the run `build_name`.
2. `COVERAGE:` section describing what the run was intended to cover, excluding the target-host list.
3. `FUNCTIONALLY:` section describing the intended workflow steps at a high level.
3. `TEST FLOW:` section describing the generic numbered run flow for the test.
4. Completed machines with machine name first and status second for each machine.
5. Notes.
6. Skipped machines with reason.
@@ -229,10 +229,10 @@ When the operator asks for the status of an ATVM automation run, report in this
Status-report expectations:
- Use the same display layout for every ATVM automation status response regardless of test type (`e2e`, `systemOS`, `reboot`, `migrateops`, and others).
- Use `/home/aw/code/cds/atvm/docs/automation/status-template.md` as the default template for both local status output and Mattermost status posts.
- The default ATVM status template uses flat bullet-list sections for `COVERAGE:` and `FUNCTIONALLY:`, Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`, and uses `NOTES:` for flat operator-facing notes.
- The default ATVM status template uses flat bullet-list sections for `COVERAGE:` and `TEST FLOW:`, Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`, and uses `NOTES:` for flat operator-facing notes.
- The `HOSTS:` table includes `Host`, `Kernel`, `Status`, and `Detail` columns in that order.
- In `COVERAGE:`, describe the template, datastore/config family, migration style, and plugin/integration path, but do not list target hosts there.
- In `FUNCTIONALLY:`, summarize what the run is intended to do at a high level, even when the run failed before reaching those steps.
- In `TEST FLOW:`, show the generic numbered run flow once for the whole test, not per host.
- For the `Kernel` column, cross-reference the host name against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`.
- If the hostname is not present in `vm-inventory.md`, report the kernel value as `unknown`.
- Treat references to the "ATVM automation run" or "automation run" as referring to this ATVM folder workflow and the automation VM at `192.168.3.190`, not to Cirrus project operations such as the `atvm - cypress` project.

View File

@@ -192,3 +192,22 @@ This file stores run-specific examples only when a run produced a new learning r
- Action for future runs:
- Do not mention expected, harmless `reset-failed` output in routine run updates.
- Only mention it if it actually prevents watcher startup or becomes relevant to debugging.
## Run Learning: 2026-03-27 (Replace FUNCTIONALLY with TEST FLOW in status output)
- Observed requirement:
- The operator wants the status format to show the generic numbered ATVM test flow rather than a vague high-level `FUNCTIONALLY:` summary.
- The step list should appear once for the whole run, not repeated per host.
- Action for future runs:
- Replace the `FUNCTIONALLY:` section with `TEST FLOW:` in ATVM status output.
- Use the generic numbered run flow:
- `1. Verifying set up`
- `2. Power on and obtain ip address and host name`
- `3. Uninstall cmc if still exist`
- `4. Setting up disk`
- `5. Copy cmc install command from GUI`
- `6. Install cmc`
- `7. Create migration session`
- `8. Tracking Changes`
- `9. Trigger cmotion and do I/O test before actual cutover`
- `...`
- `22. Power off`

View File

@@ -17,10 +17,18 @@ Use this as the default ATVM automation run-status template for:
- integration/plugin path: `<integration/plugin>`
- scope of this run: `<batch or run scope>`
**FUNCTIONALLY:**
- <intended functional step>
- <intended functional step>
- <intended functional step>
**TEST FLOW:**
- 1. Verifying set up
- 2. Power on and obtain ip address and host name
- 3. Uninstall cmc if still exist
- 4. Setting up disk
- 5. Copy cmc install command from GUI
- 6. Install cmc
- 7. Create migration session
- 8. Tracking Changes
- 9. Trigger cmotion and do I/O test before actual cutover
- ...
- 22. Power off
**SUMMARY:**
@@ -57,11 +65,11 @@ Use this as the default ATVM automation run-status template for:
```
## Rules
- Keep `COVERAGE:`, `FUNCTIONALLY:`, `SUMMARY:`, `HOSTS:`, `TIMING:`, and `NOTES:` in that order.
- Keep `COVERAGE:`, `TEST FLOW:`, `SUMMARY:`, `HOSTS:`, `TIMING:`, and `NOTES:` in that order.
- Use the title format:
- `## ATVM Run Status`
- `### <build_name>`
- Use flat bullet lists for `COVERAGE:` and `FUNCTIONALLY:`.
- Use flat bullet lists for `COVERAGE:` and `TEST FLOW:`.
- Use Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`.
- Use one host per row in the `HOSTS:` section.
- Include a `Kernel` column immediately after `Host` in the `HOSTS:` table.
@@ -74,7 +82,7 @@ Use this as the default ATVM automation run-status template for:
- Keep `Detail` concise.
- Put broader context under `NOTES:`, not in the host table.
- `COVERAGE:` should describe what the run was intended to cover without listing target hosts.
- `FUNCTIONALLY:` should describe the intended workflow steps at a high level, even if the run failed before completing them.
- `TEST FLOW:` should describe the generic numbered run flow once for the whole test, not per host.
- Resolve kernel values by cross-referencing hostnames against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`.
- If no kernel value can be verified from `vm-inventory.md`, use `unknown`.
- Use the same template for Mattermost and local operator-visible status output.

View File

@@ -532,13 +532,18 @@ def build_status_markdown(
f"- integration/plugin path: `{metadata['integration_plugin']}`",
f"- scope of this run: {metadata['scope_description']}",
"",
"**FUNCTIONALLY:**",
"- verify VM setup and power state",
"- power on, obtain IP address, and verify hostname reachability",
"- uninstall existing CMC if present",
"- prepare source and destination disks and validate source-side data",
"- install CMC and execute the requested ATVM migration workflow",
"- finalize reporting, cleanup, and the final `check-xml-files.ts` validation step",
"**TEST FLOW:**",
"- 1. Verifying set up",
"- 2. Power on and obtain ip address and host name",
"- 3. Uninstall cmc if still exist",
"- 4. Setting up disk",
"- 5. Copy cmc install command from GUI",
"- 6. Install cmc",
"- 7. Create migration session",
"- 8. Tracking Changes",
"- 9. Trigger cmotion and do I/O test before actual cutover",
"- ...",
"- 22. Power off",
"",
"**SUMMARY:**",
"",