257 lines
13 KiB
Markdown
257 lines
13 KiB
Markdown
---
|
|
name: atvm
|
|
description: Portable ATVM operations skill for setup/bootstrap, ATVM Cypress automation planning and execution, watcher-aware run control, live and historical run-status reporting, Mattermost posting on request, and maintenance of ATVM workflow docs and inventory. Use when working with an ATVM target host, the ATVM automation controller, or ATVM-specific workflow rules, defaults, and artifacts.
|
|
---
|
|
|
|
# ATVM
|
|
|
|
Use this skill for ATVM setup, ATVM Cypress automation, ATVM run monitoring, watcher-aware launch flow, run-status reporting, and maintenance of ATVM operational documentation.
|
|
|
|
This is a portable packaged skill. Prefer the bundled `references/` docs first. If a live repo copy of the ATVM workspace exists, use it as an execution source when appropriate, but do not assume it exists.
|
|
|
|
## Read First
|
|
|
|
Pick only the files needed for the current task.
|
|
|
|
### Setup/bootstrap
|
|
- `references/setup-guide.md`
|
|
- `references/setup-run-learnings.md`
|
|
- `references/atvm-prep-reference.md`
|
|
|
|
### Automation execution
|
|
- `references/automation-guide.md`
|
|
- `references/automation-examples.md`
|
|
- `references/automation-run-learnings.md`
|
|
- `references/automation-status-template.md`
|
|
|
|
### Workspace maintenance conventions
|
|
- `references/workflow-conventions.md`
|
|
|
|
### Environment reference
|
|
- `references/inventory-overview.md`
|
|
- `references/infrastructure.md`
|
|
- `references/reserved-ips.md`
|
|
- `references/specialized-reference.md`
|
|
- `references/vm-inventory.md`
|
|
|
|
### Credentials and local environment contract
|
|
- `references/credentials-contract.md`
|
|
|
|
### Historical long-form notes
|
|
- `references/imported-notes-cypress-automation-for-cmc.md`
|
|
|
|
## Operating Model
|
|
|
|
This skill is execute-by-default for ATVM automation runs.
|
|
|
|
Always:
|
|
- treat git/commit approval as separate from ATVM run execution
|
|
- never execute `git push` on the operator's behalf unless a different workspace policy explicitly overrides that rule
|
|
- default ATVM automation runs to watcher-backed execution unless the operator explicitly requests no watcher
|
|
- after run launch, report the exact executed `cmc-templates.py` and `run-sorry-cypress.py` commands
|
|
|
|
Only when explicitly requested by the operator:
|
|
- show planned ATVM commands before execution as a review step
|
|
|
|
## Core Defaults
|
|
|
|
### Setup track
|
|
Default ATVM target host values:
|
|
- target host reference: `192.168.3.191`
|
|
- static IP target: `192.168.3.191/22`
|
|
- gateway: `192.168.0.1`
|
|
- DNS: `8.8.8.8`, `8.8.4.4`
|
|
- client log file: `atvm_setup_script.log`
|
|
|
|
Default Linux target credentials:
|
|
- `ATVM_TARGET_USER`
|
|
- `ATVM_TARGET_PASSWORD`
|
|
|
|
Default Windows target credentials:
|
|
- `ATVM_WINDOWS_TARGET_USER`
|
|
- `ATVM_WINDOWS_TARGET_PASSWORD`
|
|
|
|
For SSH to the default target host, ignore host key mismatch by default:
|
|
- `-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null`
|
|
|
|
Use `references/setup-guide.md` for the hardware-prep trigger phrase:
|
|
- `finish prepping the ATVM VM hardware for <vm-name>`
|
|
|
|
That procedure adds the standard two ATVM data disks, two FC passthrough ports, and full guest-memory reservation for the exact VM name supplied by the operator.
|
|
|
|
### Automation track
|
|
Default controller values:
|
|
- controller host: `atvm-cypres-vm-1`
|
|
- controller IP: `192.168.3.190`
|
|
|
|
Default controller credentials:
|
|
- `ATVM_CONTROLLER_USER`
|
|
- `ATVM_CONTROLLER_PASSWORD`
|
|
|
|
Detailed controller artifact root:
|
|
- `/root/cdc-e2e-cyp-12.17.4/cypress/cmcReporter`
|
|
|
|
Automation defaults:
|
|
- default plugin-bearing template plugin: `--use_specified_plugin iscsi`
|
|
- default config family: `gold`
|
|
- do not include Gold Disk identifiers in `--build_name`
|
|
- `--build_name` must not contain spaces; use `-` between words
|
|
- for multiple VMs in the same distro, prefer distro-scoped filtering (`--containsVm`) over long explicit VM lists
|
|
- always include `--ignore_force_shutdown` unless explicitly told not to
|
|
- always include `--test_partition` unless explicitly told not to
|
|
- for `cmc-migrateops-compute-migration` to VMware, default to:
|
|
- `--vm_platforms vmware`
|
|
- `--set_static_ip_dest`
|
|
- for vCenter inspection and placement checks, prefer `govc` and raw vCenter REST calls before alternate wrappers when available
|
|
- treat `cmc-systemOS` as not using plugin or integration-type arguments; do not add `--use_specified_plugin`, `--integration_type`, or watcher integration/plugin metadata for that template
|
|
- do not auto-add a maintained `--exclude_partial_match` blacklist when the operator explicitly targets named VMs with `--specify_vms`
|
|
- even for explicit `--specify_vms` requests, first check whether any requested VM is on the maintained blacklist and stop if it is
|
|
|
|
## Mandatory Safety Rules
|
|
|
|
### Setup
|
|
- never run setup without both `--expected-ip` and `--expected-hostname`
|
|
- keep static IP configuration as the final setup step
|
|
- do not weaken host identity validation
|
|
- for ATVM hardware prep, operate only on the exact VM name, verify the VM is powered off before hardware changes, and never delete, destroy, remove from inventory, or unregister the VM
|
|
|
|
### Automation
|
|
- before any automation run, always check whether automation is already running
|
|
- treat any previous controller running-state result as stale once a new ATVM request arrives; perform a fresh live controller check at request time instead of relying on the immediately previous result
|
|
- always report whether automation is already running
|
|
- if automation is already running, terminate only with explicit approval
|
|
- after `cmc-templates.py`, always verify that generated spec files and the config `specPattern` still include every requested VM before launching `run-sorry-cypress.py`
|
|
- if any requested VM is missing, stop and report the mismatch
|
|
- when watcher mode is in use (default), start the watcher before `run-sorry-cypress.py`
|
|
- when watcher mode is in use (default), build the watcher-start command so it automatically includes the exact executed `cmc-templates.py` command via `--template-command` and the exact executed `run-sorry-cypress.py` command via `--runner-command`; do not require the operator to restate them separately
|
|
- when watcher mode is in use (default), prefer the controller-local `atvm-runner@...` systemd service over detached SSH background launch patterns for `run-sorry-cypress.py`
|
|
- do not start the runner before the watcher, because watcher startup may clear stale `/tmp/<build>.log` files and can delete a fresh live runner log if the runner starts first
|
|
- when starting both watcher and runner, prefer the combined `start-atvm-run.sh` wrapper so they are not launched in parallel against the same `/tmp/<build>.log`
|
|
- do not infer plugin behavior from stray strings in generated specs; determine it from runtime gates such as `Cypress.env(...)`
|
|
- for `cmc-reboot`, treat `--use_specified_plugin both` as a separate confirmation gate and warn about the FC+iSCSI timing risk unless the operator explicitly reconfirms `both`
|
|
- for vCenter VM snapshot requests, default the snapshot name to `VM Snapshot [mm/dd/yyyy:hh:mm:ss AM/PM]` in the local `America/New_York` timezone unless the operator explicitly requests a different name
|
|
- never delete, destroy, remove from inventory, or unregister any ATVM VM from vCenter
|
|
- treat a vCenter VM as protected ATVM inventory if its name starts with `atvm`, if it resides on an `AutomatedTest-*` ATVM datastore, or if it is an ATVM platform/controller VM, even when it is not listed in `inventory/vm-inventory.md`
|
|
- preserve ATVM vCenter VM inventory records even when power, maintenance, datastore, cleanup, or prep work is requested; for cleanup, only power off, detach/unmount supporting resources, or report the needed manual action
|
|
|
|
### Red Hat manual tasks (non-automation)
|
|
- for manually requested tasks on Red Hat machines, run the subscription reset/register sequence before the requested task
|
|
- do not apply this step to ATVM automation runs where template/runtime scripts already handle the subscription flow
|
|
- use credentials from the local environment source:
|
|
- `REDHAT_SUBSCRIPTION_USER`
|
|
- `REDHAT_SUBSCRIPTION_PASSWORD`
|
|
- required sequence:
|
|
- `subscription-manager remove --all`
|
|
- `subscription-manager unregister`
|
|
- `subscription-manager clean`
|
|
- `subscription-manager register --username "$REDHAT_SUBSCRIPTION_USER" --password "$REDHAT_SUBSCRIPTION_PASSWORD"`
|
|
|
|
### Status confirmation
|
|
Treat watcher state as cached output only:
|
|
- `/var/lib/atvm-run-watcher/<build>/state.json`
|
|
|
|
Before confirming a completed run, verify in this order:
|
|
1. launch log
|
|
2. matching reporter artifacts
|
|
3. `Cloud Run Finished` summary / Currents URL
|
|
4. watcher state only as comparison
|
|
|
|
Never confirm a completed run from watcher state alone.
|
|
|
|
For categorized runs:
|
|
- never report a grouped sub-run as `PASS` from watcher `host_results`, grouped XML, or a lone `check-xml-files.ts` result by itself
|
|
- before reporting a grouped sub-run as `PASS`, confirm that the matching child batch also passed in the live launch log or the final `Cloud Run Finished` summary for that child run
|
|
|
|
### Mattermost
|
|
- if the operator asks for ATVM run status without mentioning Mattermost, respond locally only
|
|
- if the operator explicitly asks to send status to Mattermost, use the locally configured:
|
|
- `MATTERMOST_ATVM_WEBHOOK`
|
|
- `MATTERMOST_ATVM_CHANNEL`
|
|
- send the final status only after the run fully completes
|
|
|
|
## Credentials Contract
|
|
|
|
Do not bundle real credentials inside the skill.
|
|
|
|
Use this resolution order:
|
|
1. operator-provided env vars in the current shell
|
|
2. operator-provided local env file
|
|
3. machine-local default env file if one is known and present
|
|
|
|
Expected variables:
|
|
- `ATVM_TARGET_USER`
|
|
- `ATVM_TARGET_PASSWORD`
|
|
- `ATVM_WINDOWS_TARGET_USER`
|
|
- `ATVM_WINDOWS_TARGET_PASSWORD`
|
|
- `ATVM_CONTROLLER_USER`
|
|
- `ATVM_CONTROLLER_PASSWORD`
|
|
- `MATTERMOST_ATVM_WEBHOOK`
|
|
- `MATTERMOST_ATVM_CHANNEL`
|
|
|
|
If credentials are missing, stop and ask for the local source or the missing variable names. Do not invent fallback secrets.
|
|
|
|
## Execution Entry Points
|
|
|
|
Prefer documented scripts and commands over ad hoc rewrites.
|
|
|
|
### Setup entry points
|
|
- client setup script: `scripts/atvm-setup-script.sh` when available in the current workspace
|
|
- controller wrapper: `scripts/run-atvm-setup-and-collect-log.sh` when available in the current workspace
|
|
- controller prep script: `/root/atvm_prep/atvm_prep.py` on `192.168.3.190`; show the exact planned command and wait for explicit operator approval before execution
|
|
|
|
If the local ATVM repo is not present, use the procedural rules in the bundled references and ask the operator for the actual script location before execution.
|
|
|
|
### Automation entry points
|
|
Typical controller-side tools:
|
|
- `cmc-templates.py`
|
|
- `run-sorry-cypress.py`
|
|
|
|
Treat their exact location as environment-specific unless a local repo or controller path is available.
|
|
|
|
## Artifact Recovery and Reporting
|
|
|
|
For detailed host-level failure analysis, prefer:
|
|
- controller run log
|
|
- `cmcReporter/logs/`
|
|
- `cmcReporter/mochawesome/`
|
|
- structured reporter artifacts such as JSON and XML
|
|
- plain text reporter artifacts last
|
|
|
|
Use the status layout from `references/automation-status-template.md`.
|
|
|
|
Keep the `HOSTS` detail compact.
|
|
Put richer failure excerpts in `FAILURE NOTES:`.
|
|
Reserve `NOTES:` for non-failure context such as template command, Currents URL, and operator-facing caveats.
|
|
Apply failed-host detail recovery consistently for every ATVM template run, not just `cmc-reboot`.
|
|
For failed hosts, recover detail in this order when available: consolidated run log, `mochawesome`, structured reporter artifacts such as JSON and XML, then plain text artifacts.
|
|
When reporting `TEST FLOW:`, prefer the numbered steps extracted from the generated spec for that exact run.
|
|
If the generated spec exists, do not rely on a static template flow list.
|
|
Only fall back to template-level or static flow definitions when the generated spec cannot be located or parsed.
|
|
|
|
## Maintenance Rules
|
|
|
|
When changing workflow behavior:
|
|
- update the relevant guide content
|
|
|
|
When adding reusable command patterns:
|
|
- update the automation examples content
|
|
|
|
When a run produces a new lasting lesson:
|
|
- update the relevant run learnings content
|
|
|
|
Keep durable environment reference in the inventory reference files.
|
|
Do not remove detailed inventory or credential contract information unless explicitly instructed.
|
|
|
|
## Practical Use Pattern
|
|
|
|
1. Classify the task as setup, automation execution, status/reporting, watcher behavior, or documentation maintenance.
|
|
2. Read only the smallest relevant reference set.
|
|
3. Build exact run commands and execute by default for automation requests.
|
|
4. If the operator explicitly asks to review commands first, show planned commands before execution.
|
|
5. For any new run request, re-check live controller running state before deciding whether the previous run is still active.
|
|
6. If running automation, verify generated specs and `specPattern` before runner launch.
|
|
7. Default to watcher-backed launch unless explicitly told not to.
|
|
8. After launch, report immediate execution outcome and the exact executed template/runner commands.
|
|
9. If reporting status, use live evidence and reporter artifacts before cached watcher state.
|
|
10. If editing docs, keep rules in guide content, reusable commands in examples content, dated lessons in run learnings, and durable environment data in inventory content.
|