Align ATVM coverage and notes with template command

This commit is contained in:
2026-03-27 17:47:36 -04:00
parent 708eddd7fa
commit 06b8098499
5 changed files with 24 additions and 3 deletions

View File

@@ -238,10 +238,11 @@ Status-report expectations:
- The default ATVM status template uses flat bullet-list sections for `COVERAGE:` and `TEST FLOW:`, Markdown tables for `SUMMARY:`, `HOSTS:`, and `TIMING:`, and uses `NOTES:` for flat operator-facing notes.
- Order the status sections as `SUMMARY:`, `HOSTS:`, `TIMING:`, `COVERAGE:`, `TEST FLOW:`, then `NOTES:`.
- Keep `NOTES:` focused on operator-facing value such as the Currents run URL, real anomalies, failure context, or material fallback behavior.
- Include the exact `cmc-templates.py` command used to trigger the ATVM automation run in `NOTES:`, without the outer `sshpass`/`ssh` wrapper and without trimming it.
- Do not include generic watcher bookkeeping messages in `NOTES:` such as artifact-detection confirmations.
- Do not include internal watcher fallback notes in `NOTES:` such as `check-xml-files.ts` validation confirmations or reporter-artifact recovery details.
- The `HOSTS:` table includes `Host`, `Kernel`, `Status`, and `Detail` columns in that order.
- In `COVERAGE:`, describe the template, datastore/config family, config filename, migration style, plugin/integration path, and other operator-relevant run options, but do not list target hosts there or include verbose prose scope descriptions.
- In `COVERAGE:`, describe the important `cmc-templates.py` command inputs such as template, categorize mode, datastore/config family, config filename, migration style, plugin/integration path, and other operator-relevant run options, but do not list target hosts there or include verbose prose scope descriptions.
- In `TEST FLOW:`, show the template-specific numbered run flow once for the whole test, not per host.
- Resolve the flow from the run template name.
- `cmc-e2e` currently uses the 22-step migration flow documented in `/home/aw/code/cds/atvm/docs/automation/status-template.md`.

View File

@@ -311,6 +311,15 @@ This file stores run-specific examples only when a run produced a new learning r
- Include important operator-relevant run options such as `--ignore_force_shutdown` in `COVERAGE:`.
- Keep `COVERAGE:` focused on run intent and options, not the explicit target-host list.
- Do not include verbose prose lines such as `scope of this run: ...` in `COVERAGE:`.
- Treat `COVERAGE:` as a concise reflection of the important `cmc-templates.py` command inputs.
## Run Learning: 2026-03-27 (Log the exact template command in NOTES)
- Observed requirement:
- The operator wants `NOTES:` to include the exact `cmc-templates.py` command that triggered the ATVM run.
- The outer `sshpass`/`ssh` wrapper should be omitted, but the command itself should not be trimmed even when long.
- Action for future runs:
- Store and display the exact `cmc-templates.py` command in `NOTES:`.
- Omit only the outer remote-execution wrapper.
## Run Learning: 2026-03-27 (Do not auto-add blacklist excludes for explicitly specified VMs)
- Observed requirement:

View File

@@ -41,11 +41,12 @@ Use this as the default ATVM automation run-status template for:
**COVERAGE:**
- template: `<template-name>`
- categorize mode: `enabled` or `disabled`
- datastore/config family: `<config family>`
- config file: `<config-file-name>`
- migration style: `<high-level test style>`
- integration/plugin path: `<integration/plugin>`
- run options: `<operator-relevant options such as --ignore_force_shutdown>`
- run options: `<operator-relevant template flags such as --ignore_force_shutdown>`
**TEST FLOW:**
- <template-specific numbered steps>
@@ -73,11 +74,12 @@ Use this as the default ATVM automation run-status template for:
- Keep `Detail` concise.
- Put broader context under `NOTES:`, not in the host table.
- When available, put the persistent Currents run URL in `NOTES:` so operators can open the exact recorded run directly.
- Include the exact `cmc-templates.py` command used to trigger the run in `NOTES:`, without the outer `sshpass`/`ssh` wrapper.
- Keep `NOTES:` limited to meaningful operator-facing items such as the Currents link, real anomalies, failure context, or important fallback behavior.
- Do not include generic watcher bookkeeping lines in `NOTES:` such as "run artifacts were detected" or "final reporting artifacts were detected."
- Do not include internal fallback notes in `NOTES:` such as "`check-xml-files.ts` validation passed" or "host details were derived from reporter artifacts."
- `COVERAGE:` should describe what the run was intended to cover without listing target hosts.
- `COVERAGE:` should also include operator-relevant run options such as the config filename and important flags like `--ignore_force_shutdown`.
- `COVERAGE:` should mostly mirror the important `cmc-templates.py` command inputs such as template, categorize mode, config filename, integration/plugin path, and important flags like `--ignore_force_shutdown`.
- `TEST FLOW:` should describe the template-specific numbered run flow once for the whole test, not per host.
- The watcher resolves `TEST FLOW:` from the run template name.
- `cmc-e2e` currently uses this flow:

View File

@@ -697,6 +697,7 @@ def infer_metadata() -> Dict[str, object]:
extra_options = [value for value in extra_options if isinstance(value, str) and value]
return {
"template": os.environ.get("ATVM_WATCHER_TEMPLATE", "unknown"),
"template_command": os.environ.get("ATVM_WATCHER_TEMPLATE_COMMAND", ""),
"config_family": os.environ.get("ATVM_WATCHER_CONFIG_FAMILY", "unknown"),
"config_file": os.environ.get("ATVM_WATCHER_CONFIG_FILE", "unknown"),
"migration_style": os.environ.get("ATVM_WATCHER_MIGRATION_STYLE", "ATVM automation validation"),
@@ -765,6 +766,9 @@ def build_status_markdown(
if currents_url:
notes = notes + [f"Currents recorded run: `{currents_url}`"]
template_command = metadata.get("template_command")
if isinstance(template_command, str) and template_command:
notes = notes + [f"Template command: `{template_command}`"]
notes_block = "\n".join(f"- {note}" for note in notes) if notes else "- none"
test_flow_lines = [f"- {step}" for step in get_test_flow(metadata.get("template"))]
@@ -799,6 +803,7 @@ def build_status_markdown(
"",
"**COVERAGE:**",
f"- template: `{metadata['template']}`",
f"- categorize mode: `{'enabled' if metadata.get('categorized') else 'disabled'}`",
f"- datastore/config family: `{metadata['config_family']}`",
f"- config file: `{metadata.get('config_file', 'unknown')}`",
f"- migration style: {metadata['migration_style']}",

View File

@@ -9,6 +9,7 @@ Usage:
Options:
--build-name <name>
--template <name>
--template-command <text>
--config-family <name>
--config-file <path>
--migration-style <text>
@@ -22,6 +23,7 @@ EOF
BUILD_NAME=""
TEMPLATE=""
TEMPLATE_COMMAND=""
CONFIG_FAMILY=""
CONFIG_FILE=""
MIGRATION_STYLE=""
@@ -35,6 +37,7 @@ while [[ $# -gt 0 ]]; do
case "$1" in
--build-name) BUILD_NAME="${2:-}"; shift 2 ;;
--template) TEMPLATE="${2:-}"; shift 2 ;;
--template-command) TEMPLATE_COMMAND="${2:-}"; shift 2 ;;
--config-family) CONFIG_FAMILY="${2:-}"; shift 2 ;;
--config-file) CONFIG_FILE="${2:-}"; shift 2 ;;
--migration-style) MIGRATION_STYLE="${2:-}"; shift 2 ;;
@@ -70,6 +73,7 @@ PY
cat >"${RUN_DIR}/watch.env" <<EOF
ATVM_WATCHER_TEMPLATE=${TEMPLATE@Q}
ATVM_WATCHER_TEMPLATE_COMMAND=${TEMPLATE_COMMAND@Q}
ATVM_WATCHER_CONFIG_FAMILY=${CONFIG_FAMILY@Q}
ATVM_WATCHER_CONFIG_FILE=${CONFIG_FILE@Q}
ATVM_WATCHER_MIGRATION_STYLE=${MIGRATION_STYLE@Q}