Fix ATVM watcher install-only test flow extraction

Teach generated-spec TEST FLOW parsing to honor test-install-only runtime gates, including single-quoted Cypress.env checks, and suppress plugin branches when install-only mode disables them.

Document the 2026-04-16 install-only flow mismatch so future watcher updates keep Mattermost TEST FLOW aligned with the actual generated spec path.
This commit is contained in:
2026-04-16 15:29:03 -04:00
parent 37853e56a9
commit 4f56ff9c4d
2 changed files with 22 additions and 0 deletions

View File

@@ -507,6 +507,15 @@ This file stores run-specific examples only when a run produced a new learning r
- Resolve generated-spec `TEST FLOW` from the active config file's `specPattern` when the required log line is missing.
- Treat the static template flow as a last-resort fallback only after both log-derived and config-derived `specPattern` resolution fail.
## Run Learning: 2026-04-16 (Generated-spec `TEST FLOW` must honor `test-install-only` gates)
- Observed failure mode:
- An install-only `cmc-e2e` run for `atvm5-ubuntu22.04` posted the full 22-step `TEST FLOW:` to Mattermost even though the generated spec for that run only executed the shorter install-only path.
- The watcher already used the generated spec as the source of truth, but its gate evaluator did not understand `if (Cypress.env("test-install-only") == true/false)`.
- That left both the install-only branch and the normal post-install branch visible to the flow extractor.
- Action for future runs:
- When extracting `TEST FLOW:` from a generated spec, evaluate `test-install-only` gates the same way plugin and cutover gates are evaluated.
- For install-only runs, exclude the normal post-install branch and report only the actual numbered install-only steps from the generated spec.
## Run Learning: 2026-03-31 (Default vmware compute-migration options for ATVM)
- Observed operator requirement:
- For `cmc-migrateops-compute-migration` runs to VMware, the operator wants a stable default option set instead of having to restate the same platform flags each time.

View File

@@ -1276,6 +1276,9 @@ def extract_test_flow_from_generated_spec(
runtime_settings[key] = match.group(1) == "true"
def selected_plugin_gates() -> Optional[set[str]]:
install_only = runtime_settings.get("test-install-only")
if install_only is True:
return set()
pure_plugin_type = runtime_settings.get("pure_plugin_type")
if isinstance(pure_plugin_type, str):
lowered = pure_plugin_type.lower()
@@ -1310,6 +1313,16 @@ def extract_test_flow_from_generated_spec(
if normalized.startswith('if(Cypress.env("test-unaligned-fio")==true)'):
value = runtime_settings.get("test-unaligned-fio")
return True, value if isinstance(value, bool) else None
install_only_match = re.match(
r"""if\(Cypress\.env\((["'])test-install-only\1\)==(true|false)\)""",
normalized,
)
if install_only_match:
value = runtime_settings.get("test-install-only")
expected = install_only_match.group(2) == "true"
if isinstance(value, bool):
return True, value is expected
return True, None
if normalized.startswith('if(Cypress.env("isRegularCutover")==false)'):
value = runtime_settings.get("isRegularCutover")
return True, (value is False) if isinstance(value, bool) else None