Update ATVM status reporting and credential handling docs
- change ATVM status formatting to the approved Markdown-table template with SUMMARY:, HOSTS:, TIMING:, and NOTES: - document that normal status requests print locally only unless explicitly asked to send to Mattermost - document Mattermost defaults and posting rules, including only sending after full run completion - document the controller-side systemd watcher design for future automation - add the secrets migration/cleanup review doc - ignore .env.credentials.local in git and reflect the move toward using that local credentials file instead of hardcoded secrets
This commit is contained in:
188
atvm/docs/workflow/secrets-migration-and-cleanup.md
Normal file
188
atvm/docs/workflow/secrets-migration-and-cleanup.md
Normal file
@@ -0,0 +1,188 @@
|
||||
# Secrets Migration And Cleanup
|
||||
|
||||
## Purpose
|
||||
This document explains:
|
||||
- whether the workspace can be cleaned up to stop storing credentials and tokens in tracked files
|
||||
- how `.env.credentials.local` should be used
|
||||
- what has to happen to remove already-committed secrets from git history and the remote repository
|
||||
|
||||
## 1. Can the workspace be cleaned up to stop referencing raw secrets in tracked files?
|
||||
Yes.
|
||||
|
||||
The intended cleanup is:
|
||||
- remove hardcoded credentials, API tokens, webhook URLs, and similar secrets from tracked docs and files
|
||||
- replace those values with references to `/home/aw/code/cds/.env.credentials.local`
|
||||
- keep only non-secret metadata in tracked files, such as:
|
||||
- hostnames
|
||||
- IPs
|
||||
- usernames when acceptable
|
||||
- variable names
|
||||
- usage instructions
|
||||
|
||||
Examples of what tracked docs should say instead of storing raw values:
|
||||
- `Use ATVM_CONTROLLER_PASSWORD from /home/aw/code/cds/.env.credentials.local`
|
||||
- `Use VCENTER_USER and VCENTER_PASSWORD from /home/aw/code/cds/.env.credentials.local`
|
||||
- `Use MATTERMOST_ATVM_WEBHOOK from /home/aw/code/cds/.env.credentials.local`
|
||||
|
||||
Recommended scope of cleanup:
|
||||
- `atvm/inventory/accounts-and-credentials.md`
|
||||
- `atvm/inventory/infrastructure.md`
|
||||
- any other tracked docs or scripts that contain:
|
||||
- passwords
|
||||
- API tokens
|
||||
- TOTP secrets
|
||||
- webhook URLs
|
||||
- install codes or secrets that should not remain in git
|
||||
|
||||
## 2. What do I need to do for the assistant to use `.env.credentials.local`?
|
||||
The file exists on disk, but the assistant does not automatically import shell environment files unless one of the following is done:
|
||||
|
||||
### Option A: Explicitly source it in the shell session
|
||||
Example:
|
||||
|
||||
```bash
|
||||
source /home/aw/code/cds/.env.credentials.local
|
||||
```
|
||||
|
||||
This is the simplest and most reliable option for interactive terminal work.
|
||||
|
||||
### Option B: Scripts explicitly read it
|
||||
A script can do:
|
||||
|
||||
```bash
|
||||
source /home/aw/code/cds/.env.credentials.local
|
||||
```
|
||||
|
||||
before using any secret-backed variables.
|
||||
|
||||
### Option C: The workflow documentation tells the assistant to load it
|
||||
The workspace docs can instruct the assistant to use `/home/aw/code/cds/.env.credentials.local` when credentials are required, but the assistant still needs an execution path that actually loads those variables into the shell or reads them directly from the file.
|
||||
|
||||
## Practical rule
|
||||
If you want the assistant to reliably use these values during execution, the safest approach is:
|
||||
- either explicitly source the file first
|
||||
- or instruct the assistant to source it as part of the command/script it runs
|
||||
|
||||
## Important limitation
|
||||
The existence of `.env.credentials.local` does not automatically make every shell command aware of those variables.
|
||||
|
||||
The assistant needs one of these:
|
||||
- the current shell environment already contains the exported variables
|
||||
- the command explicitly sources the file
|
||||
- the script being executed explicitly sources the file
|
||||
|
||||
## 3. What do I need to do if secrets were already committed and pushed to the remote repository?
|
||||
If secrets were already committed to git history and pushed, `.gitignore` does not fix that.
|
||||
|
||||
You need to treat those secrets as exposed.
|
||||
|
||||
## Required response
|
||||
Do these in this order:
|
||||
|
||||
### Step 1: Rotate or revoke the exposed secrets
|
||||
This is the most important step.
|
||||
|
||||
Examples:
|
||||
- regenerate Mattermost webhook URLs
|
||||
- replace API tokens
|
||||
- rotate passwords
|
||||
- regenerate TOTP/shared secrets if applicable
|
||||
- replace any service registration or install tokens that should be considered exposed
|
||||
|
||||
Even if you later remove them from git history, assume they were already copied.
|
||||
|
||||
### Step 2: Remove secrets from the current tracked files
|
||||
Edit the tracked docs and scripts so they no longer contain raw secrets.
|
||||
|
||||
Replace them with:
|
||||
- references to `.env.credentials.local`
|
||||
- redacted placeholders
|
||||
- variable names
|
||||
|
||||
### Step 3: Rewrite git history to remove the secrets from all commits
|
||||
This is a history-rewrite operation.
|
||||
|
||||
Typical tools:
|
||||
- `git filter-repo` (preferred)
|
||||
- BFG Repo-Cleaner
|
||||
|
||||
High-level workflow:
|
||||
1. identify all tracked files and literal secrets that must be removed
|
||||
2. rewrite repository history to remove or replace them
|
||||
3. verify the secrets no longer exist in any commit
|
||||
4. force-push the rewritten history to the remote
|
||||
|
||||
### Step 4: Force-push the cleaned history
|
||||
After rewriting history, the remote must be updated with a force push.
|
||||
|
||||
That usually means:
|
||||
- `git push --force-with-lease origin <branch>`
|
||||
|
||||
### Step 5: Coordinate with anyone else using the repo
|
||||
Anyone with an old clone will still have the old history unless they reset or reclone.
|
||||
|
||||
They need instructions to:
|
||||
- stop using the old history
|
||||
- fetch the rewritten branch
|
||||
- hard reset or reclone as appropriate
|
||||
|
||||
## Important caution about remote cleanup
|
||||
Cleaning the git remote history does not guarantee that every copy is gone.
|
||||
|
||||
Secrets may still exist in:
|
||||
- old clones
|
||||
- forks
|
||||
- CI logs
|
||||
- code review systems
|
||||
- backups
|
||||
- screenshots or pasted chat logs
|
||||
|
||||
That is why secret rotation must happen first.
|
||||
|
||||
## Recommended cleanup policy for this workspace
|
||||
For this workspace, the correct policy should be:
|
||||
- keep real secrets only in `/home/aw/code/cds/.env.credentials.local`
|
||||
- keep that file gitignored
|
||||
- remove raw secrets from tracked docs
|
||||
- document variable names and usage instead of values
|
||||
- rotate any secrets that were ever committed
|
||||
- rewrite history if the repository should no longer retain those secret values
|
||||
|
||||
## Proposed next implementation work
|
||||
When approved, the cleanup work would likely be:
|
||||
1. inventory all tracked files containing secrets
|
||||
2. patch those files to reference `.env.credentials.local`
|
||||
3. update docs so the credential source is explicit
|
||||
4. prepare a history-rewrite plan
|
||||
5. prepare exact git commands for review before any destructive git action
|
||||
|
||||
## Git-history cleanup note
|
||||
History rewriting is disruptive and should not be done casually.
|
||||
|
||||
Before doing it, prepare:
|
||||
- the list of files and secrets to purge
|
||||
- the exact rewrite tool and command
|
||||
- the exact verification commands
|
||||
- the exact force-push command
|
||||
- the operator communication plan for other users of the repo
|
||||
|
||||
## Summary
|
||||
Answers to the three direct questions:
|
||||
|
||||
### Question 1
|
||||
Yes, the workspace can be cleaned up to stop storing secrets in tracked files and instead reference `/home/aw/code/cds/.env.credentials.local`.
|
||||
|
||||
### Question 2
|
||||
To have the assistant reliably use `.env.credentials.local`, either:
|
||||
- explicitly source it
|
||||
- or ensure the script/command being run sources it
|
||||
|
||||
The assistant does not automatically inherit its contents just because the file exists.
|
||||
|
||||
### Question 3
|
||||
If secrets were already committed and pushed:
|
||||
- rotate them first
|
||||
- remove them from current files
|
||||
- rewrite git history
|
||||
- force-push the cleaned history
|
||||
- coordinate with anyone else who has a clone
|
||||
Reference in New Issue
Block a user