Replace FUNCTIONALLY with TEST FLOW in ATVM status output
- update the ATVM status template to replace the FUNCTIONALLY section with a TEST FLOW section that shows the generic numbered run steps once for the whole test - update the ATVM automation guide to describe TEST FLOW as the standard status-report section instead of FUNCTIONALLY - update the watcher-generated status output so Mattermost and local status responses use the same TEST FLOW section - add a 2026-03-27 run learning recording the move from FUNCTIONALLY to TEST FLOW for future ATVM reporting
This commit is contained in:
@@ -211,7 +211,7 @@ When asked for one VM or a VM set:
|
|||||||
When the operator asks for the status of an ATVM automation run, report in this order:
|
When the operator asks for the status of an ATVM automation run, report in this order:
|
||||||
1. Heading/title using the run `build_name`.
|
1. Heading/title using the run `build_name`.
|
||||||
2. `COVERAGE:` section describing what the run was intended to cover, excluding the target-host list.
|
2. `COVERAGE:` section describing what the run was intended to cover, excluding the target-host list.
|
||||||
3. `FUNCTIONALLY:` section describing the intended workflow steps at a high level.
|
3. `TEST FLOW:` section describing the generic numbered run flow for the test.
|
||||||
4. Completed machines with machine name first and status second for each machine.
|
4. Completed machines with machine name first and status second for each machine.
|
||||||
5. Notes.
|
5. Notes.
|
||||||
6. Skipped machines with reason.
|
6. Skipped machines with reason.
|
||||||
@@ -229,10 +229,10 @@ When the operator asks for the status of an ATVM automation run, report in this
|
|||||||
Status-report expectations:
|
Status-report expectations:
|
||||||
- Use the same display layout for every ATVM automation status response regardless of test type (`e2e`, `systemOS`, `reboot`, `migrateops`, and others).
|
- Use the same display layout for every ATVM automation status response regardless of test type (`e2e`, `systemOS`, `reboot`, `migrateops`, and others).
|
||||||
- Use `/home/aw/code/cds/atvm/docs/automation/status-template.md` as the default template for both local status output and Mattermost status posts.
|
- Use `/home/aw/code/cds/atvm/docs/automation/status-template.md` as the default template for both local status output and Mattermost status posts.
|
||||||
- The default ATVM status template uses flat bullet-list sections for `COVERAGE:` and `FUNCTIONALLY:`, Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`, and uses `NOTES:` for flat operator-facing notes.
|
- The default ATVM status template uses flat bullet-list sections for `COVERAGE:` and `TEST FLOW:`, Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`, and uses `NOTES:` for flat operator-facing notes.
|
||||||
- The `HOSTS:` table includes `Host`, `Kernel`, `Status`, and `Detail` columns in that order.
|
- The `HOSTS:` table includes `Host`, `Kernel`, `Status`, and `Detail` columns in that order.
|
||||||
- In `COVERAGE:`, describe the template, datastore/config family, migration style, and plugin/integration path, but do not list target hosts there.
|
- In `COVERAGE:`, describe the template, datastore/config family, migration style, and plugin/integration path, but do not list target hosts there.
|
||||||
- In `FUNCTIONALLY:`, summarize what the run is intended to do at a high level, even when the run failed before reaching those steps.
|
- In `TEST FLOW:`, show the generic numbered run flow once for the whole test, not per host.
|
||||||
- For the `Kernel` column, cross-reference the host name against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`.
|
- For the `Kernel` column, cross-reference the host name against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`.
|
||||||
- If the hostname is not present in `vm-inventory.md`, report the kernel value as `unknown`.
|
- If the hostname is not present in `vm-inventory.md`, report the kernel value as `unknown`.
|
||||||
- Treat references to the "ATVM automation run" or "automation run" as referring to this ATVM folder workflow and the automation VM at `192.168.3.190`, not to Cirrus project operations such as the `atvm - cypress` project.
|
- Treat references to the "ATVM automation run" or "automation run" as referring to this ATVM folder workflow and the automation VM at `192.168.3.190`, not to Cirrus project operations such as the `atvm - cypress` project.
|
||||||
|
|||||||
@@ -192,3 +192,22 @@ This file stores run-specific examples only when a run produced a new learning r
|
|||||||
- Action for future runs:
|
- Action for future runs:
|
||||||
- Do not mention expected, harmless `reset-failed` output in routine run updates.
|
- Do not mention expected, harmless `reset-failed` output in routine run updates.
|
||||||
- Only mention it if it actually prevents watcher startup or becomes relevant to debugging.
|
- Only mention it if it actually prevents watcher startup or becomes relevant to debugging.
|
||||||
|
|
||||||
|
## Run Learning: 2026-03-27 (Replace FUNCTIONALLY with TEST FLOW in status output)
|
||||||
|
- Observed requirement:
|
||||||
|
- The operator wants the status format to show the generic numbered ATVM test flow rather than a vague high-level `FUNCTIONALLY:` summary.
|
||||||
|
- The step list should appear once for the whole run, not repeated per host.
|
||||||
|
- Action for future runs:
|
||||||
|
- Replace the `FUNCTIONALLY:` section with `TEST FLOW:` in ATVM status output.
|
||||||
|
- Use the generic numbered run flow:
|
||||||
|
- `1. Verifying set up`
|
||||||
|
- `2. Power on and obtain ip address and host name`
|
||||||
|
- `3. Uninstall cmc if still exist`
|
||||||
|
- `4. Setting up disk`
|
||||||
|
- `5. Copy cmc install command from GUI`
|
||||||
|
- `6. Install cmc`
|
||||||
|
- `7. Create migration session`
|
||||||
|
- `8. Tracking Changes`
|
||||||
|
- `9. Trigger cmotion and do I/O test before actual cutover`
|
||||||
|
- `...`
|
||||||
|
- `22. Power off`
|
||||||
|
|||||||
@@ -17,10 +17,18 @@ Use this as the default ATVM automation run-status template for:
|
|||||||
- integration/plugin path: `<integration/plugin>`
|
- integration/plugin path: `<integration/plugin>`
|
||||||
- scope of this run: `<batch or run scope>`
|
- scope of this run: `<batch or run scope>`
|
||||||
|
|
||||||
**FUNCTIONALLY:**
|
**TEST FLOW:**
|
||||||
- <intended functional step>
|
- 1. Verifying set up
|
||||||
- <intended functional step>
|
- 2. Power on and obtain ip address and host name
|
||||||
- <intended functional step>
|
- 3. Uninstall cmc if still exist
|
||||||
|
- 4. Setting up disk
|
||||||
|
- 5. Copy cmc install command from GUI
|
||||||
|
- 6. Install cmc
|
||||||
|
- 7. Create migration session
|
||||||
|
- 8. Tracking Changes
|
||||||
|
- 9. Trigger cmotion and do I/O test before actual cutover
|
||||||
|
- ...
|
||||||
|
- 22. Power off
|
||||||
|
|
||||||
**SUMMARY:**
|
**SUMMARY:**
|
||||||
|
|
||||||
@@ -57,11 +65,11 @@ Use this as the default ATVM automation run-status template for:
|
|||||||
```
|
```
|
||||||
|
|
||||||
## Rules
|
## Rules
|
||||||
- Keep `COVERAGE:`, `FUNCTIONALLY:`, `SUMMARY:`, `HOSTS:`, `TIMING:`, and `NOTES:` in that order.
|
- Keep `COVERAGE:`, `TEST FLOW:`, `SUMMARY:`, `HOSTS:`, `TIMING:`, and `NOTES:` in that order.
|
||||||
- Use the title format:
|
- Use the title format:
|
||||||
- `## ATVM Run Status`
|
- `## ATVM Run Status`
|
||||||
- `### <build_name>`
|
- `### <build_name>`
|
||||||
- Use flat bullet lists for `COVERAGE:` and `FUNCTIONALLY:`.
|
- Use flat bullet lists for `COVERAGE:` and `TEST FLOW:`.
|
||||||
- Use Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`.
|
- Use Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`.
|
||||||
- Use one host per row in the `HOSTS:` section.
|
- Use one host per row in the `HOSTS:` section.
|
||||||
- Include a `Kernel` column immediately after `Host` in the `HOSTS:` table.
|
- Include a `Kernel` column immediately after `Host` in the `HOSTS:` table.
|
||||||
@@ -74,7 +82,7 @@ Use this as the default ATVM automation run-status template for:
|
|||||||
- Keep `Detail` concise.
|
- Keep `Detail` concise.
|
||||||
- Put broader context under `NOTES:`, not in the host table.
|
- Put broader context under `NOTES:`, not in the host table.
|
||||||
- `COVERAGE:` should describe what the run was intended to cover without listing target hosts.
|
- `COVERAGE:` should describe what the run was intended to cover without listing target hosts.
|
||||||
- `FUNCTIONALLY:` should describe the intended workflow steps at a high level, even if the run failed before completing them.
|
- `TEST FLOW:` should describe the generic numbered run flow once for the whole test, not per host.
|
||||||
- Resolve kernel values by cross-referencing hostnames against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`.
|
- Resolve kernel values by cross-referencing hostnames against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`.
|
||||||
- If no kernel value can be verified from `vm-inventory.md`, use `unknown`.
|
- If no kernel value can be verified from `vm-inventory.md`, use `unknown`.
|
||||||
- Use the same template for Mattermost and local operator-visible status output.
|
- Use the same template for Mattermost and local operator-visible status output.
|
||||||
|
|||||||
@@ -532,13 +532,18 @@ def build_status_markdown(
|
|||||||
f"- integration/plugin path: `{metadata['integration_plugin']}`",
|
f"- integration/plugin path: `{metadata['integration_plugin']}`",
|
||||||
f"- scope of this run: {metadata['scope_description']}",
|
f"- scope of this run: {metadata['scope_description']}",
|
||||||
"",
|
"",
|
||||||
"**FUNCTIONALLY:**",
|
"**TEST FLOW:**",
|
||||||
"- verify VM setup and power state",
|
"- 1. Verifying set up",
|
||||||
"- power on, obtain IP address, and verify hostname reachability",
|
"- 2. Power on and obtain ip address and host name",
|
||||||
"- uninstall existing CMC if present",
|
"- 3. Uninstall cmc if still exist",
|
||||||
"- prepare source and destination disks and validate source-side data",
|
"- 4. Setting up disk",
|
||||||
"- install CMC and execute the requested ATVM migration workflow",
|
"- 5. Copy cmc install command from GUI",
|
||||||
"- finalize reporting, cleanup, and the final `check-xml-files.ts` validation step",
|
"- 6. Install cmc",
|
||||||
|
"- 7. Create migration session",
|
||||||
|
"- 8. Tracking Changes",
|
||||||
|
"- 9. Trigger cmotion and do I/O test before actual cutover",
|
||||||
|
"- ...",
|
||||||
|
"- 22. Power off",
|
||||||
"",
|
"",
|
||||||
"**SUMMARY:**",
|
"**SUMMARY:**",
|
||||||
"",
|
"",
|
||||||
|
|||||||
Reference in New Issue
Block a user