Replace FUNCTIONALLY with TEST FLOW in ATVM status output

- update the ATVM status template to replace the FUNCTIONALLY section with a TEST FLOW section that shows the generic numbered run steps once for the whole test
- update the ATVM automation guide to describe TEST FLOW as the standard status-report section instead of FUNCTIONALLY
- update the watcher-generated status output so Mattermost and local status responses use the same TEST FLOW section
- add a 2026-03-27 run learning recording the move from FUNCTIONALLY to TEST FLOW for future ATVM reporting
This commit is contained in:
2026-03-27 08:07:00 -04:00
parent 833225378d
commit 6daa83b0c5
4 changed files with 49 additions and 17 deletions

View File

@@ -211,7 +211,7 @@ When asked for one VM or a VM set:
When the operator asks for the status of an ATVM automation run, report in this order: When the operator asks for the status of an ATVM automation run, report in this order:
1. Heading/title using the run `build_name`. 1. Heading/title using the run `build_name`.
2. `COVERAGE:` section describing what the run was intended to cover, excluding the target-host list. 2. `COVERAGE:` section describing what the run was intended to cover, excluding the target-host list.
3. `FUNCTIONALLY:` section describing the intended workflow steps at a high level. 3. `TEST FLOW:` section describing the generic numbered run flow for the test.
4. Completed machines with machine name first and status second for each machine. 4. Completed machines with machine name first and status second for each machine.
5. Notes. 5. Notes.
6. Skipped machines with reason. 6. Skipped machines with reason.
@@ -229,10 +229,10 @@ When the operator asks for the status of an ATVM automation run, report in this
Status-report expectations: Status-report expectations:
- Use the same display layout for every ATVM automation status response regardless of test type (`e2e`, `systemOS`, `reboot`, `migrateops`, and others). - Use the same display layout for every ATVM automation status response regardless of test type (`e2e`, `systemOS`, `reboot`, `migrateops`, and others).
- Use `/home/aw/code/cds/atvm/docs/automation/status-template.md` as the default template for both local status output and Mattermost status posts. - Use `/home/aw/code/cds/atvm/docs/automation/status-template.md` as the default template for both local status output and Mattermost status posts.
- The default ATVM status template uses flat bullet-list sections for `COVERAGE:` and `FUNCTIONALLY:`, Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`, and uses `NOTES:` for flat operator-facing notes. - The default ATVM status template uses flat bullet-list sections for `COVERAGE:` and `TEST FLOW:`, Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`, and uses `NOTES:` for flat operator-facing notes.
- The `HOSTS:` table includes `Host`, `Kernel`, `Status`, and `Detail` columns in that order. - The `HOSTS:` table includes `Host`, `Kernel`, `Status`, and `Detail` columns in that order.
- In `COVERAGE:`, describe the template, datastore/config family, migration style, and plugin/integration path, but do not list target hosts there. - In `COVERAGE:`, describe the template, datastore/config family, migration style, and plugin/integration path, but do not list target hosts there.
- In `FUNCTIONALLY:`, summarize what the run is intended to do at a high level, even when the run failed before reaching those steps. - In `TEST FLOW:`, show the generic numbered run flow once for the whole test, not per host.
- For the `Kernel` column, cross-reference the host name against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`. - For the `Kernel` column, cross-reference the host name against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`.
- If the hostname is not present in `vm-inventory.md`, report the kernel value as `unknown`. - If the hostname is not present in `vm-inventory.md`, report the kernel value as `unknown`.
- Treat references to the "ATVM automation run" or "automation run" as referring to this ATVM folder workflow and the automation VM at `192.168.3.190`, not to Cirrus project operations such as the `atvm - cypress` project. - Treat references to the "ATVM automation run" or "automation run" as referring to this ATVM folder workflow and the automation VM at `192.168.3.190`, not to Cirrus project operations such as the `atvm - cypress` project.

View File

@@ -192,3 +192,22 @@ This file stores run-specific examples only when a run produced a new learning r
- Action for future runs: - Action for future runs:
- Do not mention expected, harmless `reset-failed` output in routine run updates. - Do not mention expected, harmless `reset-failed` output in routine run updates.
- Only mention it if it actually prevents watcher startup or becomes relevant to debugging. - Only mention it if it actually prevents watcher startup or becomes relevant to debugging.
## Run Learning: 2026-03-27 (Replace FUNCTIONALLY with TEST FLOW in status output)
- Observed requirement:
- The operator wants the status format to show the generic numbered ATVM test flow rather than a vague high-level `FUNCTIONALLY:` summary.
- The step list should appear once for the whole run, not repeated per host.
- Action for future runs:
- Replace the `FUNCTIONALLY:` section with `TEST FLOW:` in ATVM status output.
- Use the generic numbered run flow:
- `1. Verifying set up`
- `2. Power on and obtain ip address and host name`
- `3. Uninstall cmc if still exist`
- `4. Setting up disk`
- `5. Copy cmc install command from GUI`
- `6. Install cmc`
- `7. Create migration session`
- `8. Tracking Changes`
- `9. Trigger cmotion and do I/O test before actual cutover`
- `...`
- `22. Power off`

View File

@@ -17,10 +17,18 @@ Use this as the default ATVM automation run-status template for:
- integration/plugin path: `<integration/plugin>` - integration/plugin path: `<integration/plugin>`
- scope of this run: `<batch or run scope>` - scope of this run: `<batch or run scope>`
**FUNCTIONALLY:** **TEST FLOW:**
- <intended functional step> - 1. Verifying set up
- <intended functional step> - 2. Power on and obtain ip address and host name
- <intended functional step> - 3. Uninstall cmc if still exist
- 4. Setting up disk
- 5. Copy cmc install command from GUI
- 6. Install cmc
- 7. Create migration session
- 8. Tracking Changes
- 9. Trigger cmotion and do I/O test before actual cutover
- ...
- 22. Power off
**SUMMARY:** **SUMMARY:**
@@ -57,11 +65,11 @@ Use this as the default ATVM automation run-status template for:
``` ```
## Rules ## Rules
- Keep `COVERAGE:`, `FUNCTIONALLY:`, `SUMMARY:`, `HOSTS:`, `TIMING:`, and `NOTES:` in that order. - Keep `COVERAGE:`, `TEST FLOW:`, `SUMMARY:`, `HOSTS:`, `TIMING:`, and `NOTES:` in that order.
- Use the title format: - Use the title format:
- `## ATVM Run Status` - `## ATVM Run Status`
- `### <build_name>` - `### <build_name>`
- Use flat bullet lists for `COVERAGE:` and `FUNCTIONALLY:`. - Use flat bullet lists for `COVERAGE:` and `TEST FLOW:`.
- Use Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`. - Use Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`.
- Use one host per row in the `HOSTS:` section. - Use one host per row in the `HOSTS:` section.
- Include a `Kernel` column immediately after `Host` in the `HOSTS:` table. - Include a `Kernel` column immediately after `Host` in the `HOSTS:` table.
@@ -74,7 +82,7 @@ Use this as the default ATVM automation run-status template for:
- Keep `Detail` concise. - Keep `Detail` concise.
- Put broader context under `NOTES:`, not in the host table. - Put broader context under `NOTES:`, not in the host table.
- `COVERAGE:` should describe what the run was intended to cover without listing target hosts. - `COVERAGE:` should describe what the run was intended to cover without listing target hosts.
- `FUNCTIONALLY:` should describe the intended workflow steps at a high level, even if the run failed before completing them. - `TEST FLOW:` should describe the generic numbered run flow once for the whole test, not per host.
- Resolve kernel values by cross-referencing hostnames against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`. - Resolve kernel values by cross-referencing hostnames against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`.
- If no kernel value can be verified from `vm-inventory.md`, use `unknown`. - If no kernel value can be verified from `vm-inventory.md`, use `unknown`.
- Use the same template for Mattermost and local operator-visible status output. - Use the same template for Mattermost and local operator-visible status output.

View File

@@ -532,13 +532,18 @@ def build_status_markdown(
f"- integration/plugin path: `{metadata['integration_plugin']}`", f"- integration/plugin path: `{metadata['integration_plugin']}`",
f"- scope of this run: {metadata['scope_description']}", f"- scope of this run: {metadata['scope_description']}",
"", "",
"**FUNCTIONALLY:**", "**TEST FLOW:**",
"- verify VM setup and power state", "- 1. Verifying set up",
"- power on, obtain IP address, and verify hostname reachability", "- 2. Power on and obtain ip address and host name",
"- uninstall existing CMC if present", "- 3. Uninstall cmc if still exist",
"- prepare source and destination disks and validate source-side data", "- 4. Setting up disk",
"- install CMC and execute the requested ATVM migration workflow", "- 5. Copy cmc install command from GUI",
"- finalize reporting, cleanup, and the final `check-xml-files.ts` validation step", "- 6. Install cmc",
"- 7. Create migration session",
"- 8. Tracking Changes",
"- 9. Trigger cmotion and do I/O test before actual cutover",
"- ...",
"- 22. Power off",
"", "",
"**SUMMARY:**", "**SUMMARY:**",
"", "",