From 0b379612a8106d6cda35610144e1986c23ad3eab Mon Sep 17 00:00:00 2001
From: "anthony.wen" <awen01@gmail.com>
Date: Mon, 30 Mar 2026 16:22:59 -0400
Subject: [PATCH] Clarify ATVM plugin validation rules

---
 atvm/AGENTS.md                        | 2 ++
 atvm/docs/automation/guide.md         | 2 ++
 atvm/docs/automation/run-learnings.md | 9 +++++++++
 3 files changed, 13 insertions(+)

diff --git a/atvm/AGENTS.md b/atvm/AGENTS.md
index 700c4eb..8fe3ee2 100644
--- a/atvm/AGENTS.md
+++ b/atvm/AGENTS.md
@@ -69,6 +69,8 @@ This file defines how to operate and maintain the ATVM workspace in `/home/aw/co
 - Do not treat `approve` after a commit as permission to push; pushing requires separate explicit wording and still remains manual-reference-only unless the operator explicitly overrides this workspace rule.
 - After `cmc-templates.py`, always verify that the generated spec files and the config `specPattern` still include every requested VM before starting `run-sorry-cypress.py`.
 - If any requested VM is missing after template generation, stop and report the mismatch instead of launching the runner.
+- Do not infer plugin behavior from the mere presence of plugin-specific strings or code blocks in a generated spec file.
+- For plugin selection questions, determine expected behavior from the template/runtime gate such as `Cypress.env(...)` conditions or other execution guards, and only treat it as a mismatch if the runtime logic would execute the wrong plugin path.
 - When the watcher is requested, start the watcher before `run-sorry-cypress.py`.
 - Do not start the runner before the watcher, because the watcher helper clears stale `/tmp/<build-name>.log` and can delete the fresh live runner log if the runner starts first.
 - For host-level test detail and failed-test investigation, use `/root/cdc-e2e-cyp-12.17.4/cypress/cmcReporter`, especially `logs/`, `xml/`, and `mochawesome/`.
diff --git a/atvm/docs/automation/guide.md b/atvm/docs/automation/guide.md
index 9bcd9ac..944ff24 100644
--- a/atvm/docs/automation/guide.md
+++ b/atvm/docs/automation/guide.md
@@ -118,6 +118,8 @@ python3 ./run-sorry-cypress.py --config_file <config-file> --build_name <hyphena
 - Keep this guide focused on run-control rules and workflow constraints.
 - Use examples as reference material only, not as default intent for new operator requests.
 - Keep `examples.md` limited to reusable example commands; keep workflow rules, defaults, blacklist policy, and reporting rules in this guide or `run-learnings.md`.
+- When validating plugin-specific ATVM requests, do not treat the presence of plugin-specific code blocks or step labels in the generated `.ts` file as proof that those steps will execute.
+- Determine plugin behavior from the template/runtime gating logic, for example `Cypress.env(...)`-driven conditionals, and only call it a mismatch when that runtime logic would execute the wrong plugin path.
 
 ## Example Option Patterns (Guide-Only)
 - Distro-scoped VM selection:
diff --git a/atvm/docs/automation/run-learnings.md b/atvm/docs/automation/run-learnings.md
index c0c5ea8..901c13b 100644
--- a/atvm/docs/automation/run-learnings.md
+++ b/atvm/docs/automation/run-learnings.md
@@ -383,3 +383,12 @@ This file stores run-specific examples only when a run produced a new learning r
   - Never execute `git push` for this workspace unless the operator explicitly overrides the workspace rule.
   - After creating a local commit, provide the manual push command reference only, defaulting to `git push origin main` unless the operator explicitly asks for a different remote or branch.
   - Do not interpret a generic `approve` after a commit as push approval.
+
+## Run Learning: 2026-03-30 (Do not infer plugin execution from generated spec text alone)
+- Observed failure mode:
+  - A generated reboot spec for Pure still contained both iSCSI and FC code blocks, and that was incorrectly treated as proof that both plugin paths would run.
+  - In this template, the generated file includes both branches, but runtime execution is gated by `Cypress.env("pure_plugin_type")`.
+- Action for future runs:
+  - Do not treat the presence of plugin-specific strings or code blocks in the generated `.ts` file as proof that those plugin steps will execute.
+  - For plugin-specific questions, determine expected behavior from the template/runtime gate and only call it a mismatch if the runtime logic would execute the wrong plugin path.
+  - Continue verifying that the requested VM set is present in the generated files and `specPattern`, but keep plugin-path validation separate from simple text-presence checks.