Show full template test flow in ATVM status
This commit is contained in:
@@ -214,7 +214,7 @@ When asked for one VM or a VM set:
|
|||||||
When the operator asks for the status of an ATVM automation run, report in this order:
|
When the operator asks for the status of an ATVM automation run, report in this order:
|
||||||
1. Heading/title using the run `build_name`.
|
1. Heading/title using the run `build_name`.
|
||||||
2. `COVERAGE:` section describing what the run was intended to cover, excluding the target-host list.
|
2. `COVERAGE:` section describing what the run was intended to cover, excluding the target-host list.
|
||||||
3. `TEST FLOW:` section describing the generic numbered run flow for the test.
|
3. `TEST FLOW:` section describing the template-specific numbered run flow for the test.
|
||||||
4. Completed machines with machine name first and status second for each machine.
|
4. Completed machines with machine name first and status second for each machine.
|
||||||
5. Notes.
|
5. Notes.
|
||||||
6. Skipped machines with reason.
|
6. Skipped machines with reason.
|
||||||
@@ -235,7 +235,9 @@ Status-report expectations:
|
|||||||
- The default ATVM status template uses flat bullet-list sections for `COVERAGE:` and `TEST FLOW:`, Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`, and uses `NOTES:` for flat operator-facing notes.
|
- The default ATVM status template uses flat bullet-list sections for `COVERAGE:` and `TEST FLOW:`, Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`, and uses `NOTES:` for flat operator-facing notes.
|
||||||
- The `HOSTS:` table includes `Host`, `Kernel`, `Status`, and `Detail` columns in that order.
|
- The `HOSTS:` table includes `Host`, `Kernel`, `Status`, and `Detail` columns in that order.
|
||||||
- In `COVERAGE:`, describe the template, datastore/config family, migration style, and plugin/integration path, but do not list target hosts there.
|
- In `COVERAGE:`, describe the template, datastore/config family, migration style, and plugin/integration path, but do not list target hosts there.
|
||||||
- In `TEST FLOW:`, show the generic numbered run flow once for the whole test, not per host.
|
- In `TEST FLOW:`, show the template-specific numbered run flow once for the whole test, not per host.
|
||||||
|
- Resolve the flow from the run template name.
|
||||||
|
- `cmc-e2e` currently uses the 18-step migration flow documented in `/home/aw/code/cds/atvm/docs/automation/status-template.md`.
|
||||||
- For the `Kernel` column, cross-reference the host name against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`.
|
- For the `Kernel` column, cross-reference the host name against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`.
|
||||||
- If the hostname is not present in `vm-inventory.md`, report the kernel value as `unknown`.
|
- If the hostname is not present in `vm-inventory.md`, report the kernel value as `unknown`.
|
||||||
- Treat references to the "ATVM automation run" or "automation run" as referring to this ATVM folder workflow and the automation VM at `192.168.3.190`, not to Cirrus project operations such as the `atvm - cypress` project.
|
- Treat references to the "ATVM automation run" or "automation run" as referring to this ATVM folder workflow and the automation VM at `192.168.3.190`, not to Cirrus project operations such as the `atvm - cypress` project.
|
||||||
|
|||||||
@@ -195,22 +195,31 @@ This file stores run-specific examples only when a run produced a new learning r
|
|||||||
|
|
||||||
## Run Learning: 2026-03-27 (Replace FUNCTIONALLY with TEST FLOW in status output)
|
## Run Learning: 2026-03-27 (Replace FUNCTIONALLY with TEST FLOW in status output)
|
||||||
- Observed requirement:
|
- Observed requirement:
|
||||||
- The operator wants the status format to show the generic numbered ATVM test flow rather than a vague high-level `FUNCTIONALLY:` summary.
|
- The operator wants the status format to show the full numbered ATVM test flow for the active template rather than a vague high-level `FUNCTIONALLY:` summary.
|
||||||
|
- Each ATVM template can have its own test-flow step list.
|
||||||
- The step list should appear once for the whole run, not repeated per host.
|
- The step list should appear once for the whole run, not repeated per host.
|
||||||
- Action for future runs:
|
- Action for future runs:
|
||||||
- Replace the `FUNCTIONALLY:` section with `TEST FLOW:` in ATVM status output.
|
- Replace the `FUNCTIONALLY:` section with `TEST FLOW:` in ATVM status output.
|
||||||
- Use the generic numbered run flow:
|
- Resolve `TEST FLOW:` from the ATVM template name instead of hardcoding one shared list for every template.
|
||||||
|
- For `cmc-e2e`, use this numbered run flow:
|
||||||
- `1. Verifying set up`
|
- `1. Verifying set up`
|
||||||
- `2. Power on and obtain ip address and host name`
|
- `2. Power on and obtain ip address and host name`
|
||||||
- `3. Uninstall cmc if still exist`
|
- `3. Uninstall CMC if still exists`
|
||||||
- `4. Setting up disk`
|
- `4. Setting up disk`
|
||||||
- `5. Copy cmc install command from GUI`
|
- `5. Copy CMC install command from GUI`
|
||||||
- `6. Install cmc`
|
- `6. Install CMC`
|
||||||
- `7. Create migration session`
|
- `7. Create migration session`
|
||||||
- `8. Tracking Changes`
|
- `8. Tracking Changes`
|
||||||
- `9. Trigger cmotion and do I/O test before actual cutover`
|
- `9. Trigger cmotion and do I/O test before actual cutover`
|
||||||
- `...`
|
- `10. Verify migration remains healthy during I/O activity`
|
||||||
- `22. Power off`
|
- `11. Prepare for cutover`
|
||||||
|
- `12. Stop application / stop test I/O`
|
||||||
|
- `13. Run final sync`
|
||||||
|
- `14. Confirm destination is fully up to date`
|
||||||
|
- `15. Perform cutover`
|
||||||
|
- `16. Validate destination host / disk state`
|
||||||
|
- `17. Run post-cutover checks`
|
||||||
|
- `18. Power off`
|
||||||
|
|
||||||
## Run Learning: 2026-03-27 (Start watcher before runner when watcher is requested)
|
## Run Learning: 2026-03-27 (Start watcher before runner when watcher is requested)
|
||||||
- Observed failure mode:
|
- Observed failure mode:
|
||||||
|
|||||||
@@ -18,17 +18,7 @@ Use this as the default ATVM automation run-status template for:
|
|||||||
- scope of this run: `<batch or run scope>`
|
- scope of this run: `<batch or run scope>`
|
||||||
|
|
||||||
**TEST FLOW:**
|
**TEST FLOW:**
|
||||||
- 1. Verifying set up
|
- <template-specific numbered steps>
|
||||||
- 2. Power on and obtain ip address and host name
|
|
||||||
- 3. Uninstall cmc if still exist
|
|
||||||
- 4. Setting up disk
|
|
||||||
- 5. Copy cmc install command from GUI
|
|
||||||
- 6. Install cmc
|
|
||||||
- 7. Create migration session
|
|
||||||
- 8. Tracking Changes
|
|
||||||
- 9. Trigger cmotion and do I/O test before actual cutover
|
|
||||||
- ...
|
|
||||||
- 22. Power off
|
|
||||||
|
|
||||||
**SUMMARY:**
|
**SUMMARY:**
|
||||||
|
|
||||||
@@ -82,7 +72,28 @@ Use this as the default ATVM automation run-status template for:
|
|||||||
- Keep `Detail` concise.
|
- Keep `Detail` concise.
|
||||||
- Put broader context under `NOTES:`, not in the host table.
|
- Put broader context under `NOTES:`, not in the host table.
|
||||||
- `COVERAGE:` should describe what the run was intended to cover without listing target hosts.
|
- `COVERAGE:` should describe what the run was intended to cover without listing target hosts.
|
||||||
- `TEST FLOW:` should describe the generic numbered run flow once for the whole test, not per host.
|
- `TEST FLOW:` should describe the template-specific numbered run flow once for the whole test, not per host.
|
||||||
|
- The watcher resolves `TEST FLOW:` from the run template name.
|
||||||
|
- `cmc-e2e` currently uses this flow:
|
||||||
|
- `1. Verifying set up`
|
||||||
|
- `2. Power on and obtain ip address and host name`
|
||||||
|
- `3. Uninstall CMC if still exists`
|
||||||
|
- `4. Setting up disk`
|
||||||
|
- `5. Copy CMC install command from GUI`
|
||||||
|
- `6. Install CMC`
|
||||||
|
- `7. Create migration session`
|
||||||
|
- `8. Tracking Changes`
|
||||||
|
- `9. Trigger cmotion and do I/O test before actual cutover`
|
||||||
|
- `10. Verify migration remains healthy during I/O activity`
|
||||||
|
- `11. Prepare for cutover`
|
||||||
|
- `12. Stop application / stop test I/O`
|
||||||
|
- `13. Run final sync`
|
||||||
|
- `14. Confirm destination is fully up to date`
|
||||||
|
- `15. Perform cutover`
|
||||||
|
- `16. Validate destination host / disk state`
|
||||||
|
- `17. Run post-cutover checks`
|
||||||
|
- `18. Power off`
|
||||||
|
- See `/home/aw/code/cds/atvm/docs/automation/examples.md` for `cmc-e2e` examples.
|
||||||
- Resolve kernel values by cross-referencing hostnames against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`.
|
- Resolve kernel values by cross-referencing hostnames against `/home/aw/code/cds/atvm/inventory/vm-inventory.md`.
|
||||||
- If no kernel value can be verified from `vm-inventory.md`, use `unknown`.
|
- If no kernel value can be verified from `vm-inventory.md`, use `unknown`.
|
||||||
- Use the same template for Mattermost and local operator-visible status output.
|
- Use the same template for Mattermost and local operator-visible status output.
|
||||||
|
|||||||
@@ -30,6 +30,39 @@ RUN_STATES = {
|
|||||||
LOCAL_TZ = datetime.now().astimezone().tzinfo or timezone.utc
|
LOCAL_TZ = datetime.now().astimezone().tzinfo or timezone.utc
|
||||||
|
|
||||||
|
|
||||||
|
DEFAULT_TEST_FLOW = [
|
||||||
|
"1. Test flow is template-specific for this run",
|
||||||
|
"2. Verifying set up",
|
||||||
|
"3. Execute the requested template workflow",
|
||||||
|
"4. Validate expected run outcome",
|
||||||
|
"5. Power off",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
TEMPLATE_TEST_FLOWS = {
|
||||||
|
"cmc-e2e": [
|
||||||
|
"1. Verifying set up",
|
||||||
|
"2. Power on and obtain ip address and host name",
|
||||||
|
"3. Uninstall CMC if still exists",
|
||||||
|
"4. Setting up disk",
|
||||||
|
"5. Copy CMC install command from GUI",
|
||||||
|
"6. Install CMC",
|
||||||
|
"7. Create migration session",
|
||||||
|
"8. Tracking Changes",
|
||||||
|
"9. Trigger cmotion and do I/O test before actual cutover",
|
||||||
|
"10. Verify migration remains healthy during I/O activity",
|
||||||
|
"11. Prepare for cutover",
|
||||||
|
"12. Stop application / stop test I/O",
|
||||||
|
"13. Run final sync",
|
||||||
|
"14. Confirm destination is fully up to date",
|
||||||
|
"15. Perform cutover",
|
||||||
|
"16. Validate destination host / disk state",
|
||||||
|
"17. Run post-cutover checks",
|
||||||
|
"18. Power off",
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class HostResult:
|
class HostResult:
|
||||||
host: str
|
host: str
|
||||||
@@ -492,6 +525,12 @@ def format_timestamp_local(ts: Optional[datetime]) -> str:
|
|||||||
return local.strftime("%Y-%m-%d %H:%M:%S %Z")
|
return local.strftime("%Y-%m-%d %H:%M:%S %Z")
|
||||||
|
|
||||||
|
|
||||||
|
def get_test_flow(template_name: object) -> List[str]:
|
||||||
|
if not isinstance(template_name, str):
|
||||||
|
return DEFAULT_TEST_FLOW
|
||||||
|
return TEMPLATE_TEST_FLOWS.get(template_name, DEFAULT_TEST_FLOW)
|
||||||
|
|
||||||
|
|
||||||
def build_status_markdown(
|
def build_status_markdown(
|
||||||
build_name: str,
|
build_name: str,
|
||||||
metadata: Dict[str, object],
|
metadata: Dict[str, object],
|
||||||
@@ -527,6 +566,7 @@ def build_status_markdown(
|
|||||||
notes = notes + [f"Currents recorded run: `{currents_url}`"]
|
notes = notes + [f"Currents recorded run: `{currents_url}`"]
|
||||||
|
|
||||||
notes_block = "\n".join(f"- {note}" for note in notes) if notes else "- none"
|
notes_block = "\n".join(f"- {note}" for note in notes) if notes else "- none"
|
||||||
|
test_flow_lines = [f"- {step}" for step in get_test_flow(metadata.get("template"))]
|
||||||
|
|
||||||
lines = [
|
lines = [
|
||||||
"## ATVM Run Status",
|
"## ATVM Run Status",
|
||||||
@@ -540,17 +580,7 @@ def build_status_markdown(
|
|||||||
f"- scope of this run: {metadata['scope_description']}",
|
f"- scope of this run: {metadata['scope_description']}",
|
||||||
"",
|
"",
|
||||||
"**TEST FLOW:**",
|
"**TEST FLOW:**",
|
||||||
"- 1. Verifying set up",
|
*test_flow_lines,
|
||||||
"- 2. Power on and obtain ip address and host name",
|
|
||||||
"- 3. Uninstall cmc if still exist",
|
|
||||||
"- 4. Setting up disk",
|
|
||||||
"- 5. Copy cmc install command from GUI",
|
|
||||||
"- 6. Install cmc",
|
|
||||||
"- 7. Create migration session",
|
|
||||||
"- 8. Tracking Changes",
|
|
||||||
"- 9. Trigger cmotion and do I/O test before actual cutover",
|
|
||||||
"- ...",
|
|
||||||
"- 22. Power off",
|
|
||||||
"",
|
"",
|
||||||
"**SUMMARY:**",
|
"**SUMMARY:**",
|
||||||
"",
|
"",
|
||||||
|
|||||||
Reference in New Issue
Block a user