Clarify ATVM plugin validation rules

This commit is contained in:
2026-03-30 16:22:59 -04:00
parent 6000fa0e78
commit 0b379612a8
3 changed files with 13 additions and 0 deletions

View File

@@ -69,6 +69,8 @@ This file defines how to operate and maintain the ATVM workspace in `/home/aw/co
- Do not treat `approve` after a commit as permission to push; pushing requires separate explicit wording and still remains manual-reference-only unless the operator explicitly overrides this workspace rule.
- After `cmc-templates.py`, always verify that the generated spec files and the config `specPattern` still include every requested VM before starting `run-sorry-cypress.py`.
- If any requested VM is missing after template generation, stop and report the mismatch instead of launching the runner.
- Do not infer plugin behavior from the mere presence of plugin-specific strings or code blocks in a generated spec file.
- For plugin selection questions, determine expected behavior from the template/runtime gate such as `Cypress.env(...)` conditions or other execution guards, and only treat it as a mismatch if the runtime logic would execute the wrong plugin path.
- When the watcher is requested, start the watcher before `run-sorry-cypress.py`.
- Do not start the runner before the watcher, because the watcher helper clears stale `/tmp/<build-name>.log` and can delete the fresh live runner log if the runner starts first.
- For host-level test detail and failed-test investigation, use `/root/cdc-e2e-cyp-12.17.4/cypress/cmcReporter`, especially `logs/`, `xml/`, and `mochawesome/`.

View File

@@ -118,6 +118,8 @@ python3 ./run-sorry-cypress.py --config_file <config-file> --build_name <hyphena
- Keep this guide focused on run-control rules and workflow constraints.
- Use examples as reference material only, not as default intent for new operator requests.
- Keep `examples.md` limited to reusable example commands; keep workflow rules, defaults, blacklist policy, and reporting rules in this guide or `run-learnings.md`.
- When validating plugin-specific ATVM requests, do not treat the presence of plugin-specific code blocks or step labels in the generated `.ts` file as proof that those steps will execute.
- Determine plugin behavior from the template/runtime gating logic, for example `Cypress.env(...)`-driven conditionals, and only call it a mismatch when that runtime logic would execute the wrong plugin path.
## Example Option Patterns (Guide-Only)
- Distro-scoped VM selection:

View File

@@ -383,3 +383,12 @@ This file stores run-specific examples only when a run produced a new learning r
- Never execute `git push` for this workspace unless the operator explicitly overrides the workspace rule.
- After creating a local commit, provide the manual push command reference only, defaulting to `git push origin main` unless the operator explicitly asks for a different remote or branch.
- Do not interpret a generic `approve` after a commit as push approval.
## Run Learning: 2026-03-30 (Do not infer plugin execution from generated spec text alone)
- Observed failure mode:
- A generated reboot spec for Pure still contained both iSCSI and FC code blocks, and that was incorrectly treated as proof that both plugin paths would run.
- In this template, the generated file includes both branches, but runtime execution is gated by `Cypress.env("pure_plugin_type")`.
- Action for future runs:
- Do not treat the presence of plugin-specific strings or code blocks in the generated `.ts` file as proof that those plugin steps will execute.
- For plugin-specific questions, determine expected behavior from the template/runtime gate and only call it a mismatch if the runtime logic would execute the wrong plugin path.
- Continue verifying that the requested VM set is present in the generated files and `specPattern`, but keep plugin-path validation separate from simple text-presence checks.