Document the Deployment folder
It was the only folder without docs, and it holds the only versioned record of what configuration exists — the confs themselves are gitignored.
This commit is contained in:
Gmer4Lfe
@@ -0,0 +1,225 @@
|
||||
# ━━━━━ DEPLOYMENT — Manual ━━━━━
|
||||
|
||||
Procedures and flag reference for the schema layer.
|
||||
For overview see README-Deployment.md. For per-script detail see the script headers.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ PROCEDURES ━━━
|
||||
|
||||
### Adding a New Configuration Variable
|
||||
|
||||
The single most common task here, and the one with the quietest failure mode if done wrong.
|
||||
|
||||
```bash
|
||||
# 1. Add it to the template — this is the versioned schema
|
||||
# Shared threshold/toggle → Deployment/master.conf.template
|
||||
# Per-host credential/path → Deployment/host.conf.template (use the HOSTN_ prefix)
|
||||
|
||||
# 2. Add it to your own live conf so you can test immediately
|
||||
# Configurations/master.conf or Configurations/host1.conf
|
||||
|
||||
# 3. Use it in the script, with a safe default
|
||||
# [[ "${NEW_THRESHOLD:-50}" -gt ... ]]
|
||||
|
||||
# 4. Commit the script change and the template change TOGETHER
|
||||
git add Deployment/master.conf.template Watchdogs/System/storage_watchdog.sh
|
||||
git commit -m "..."
|
||||
```
|
||||
|
||||
**Step 1 is the one that gets skipped.** A script merged without its template entry works on
|
||||
the machine it was written on and nowhere else — the variable is simply empty on every other
|
||||
node, and the script takes whatever branch an empty value produces. Nothing errors.
|
||||
|
||||
Placement rule, same as everywhere in the ecosystem:
|
||||
|
||||
| Kind | Goes in |
|
||||
|------|---------|
|
||||
| Threshold, toggle, profile, job list | `master.conf.template` |
|
||||
| Credential, path, container name, per-host identity | `host.conf.template` |
|
||||
|
||||
---
|
||||
|
||||
### Previewing a Schema Change Before It Lands
|
||||
|
||||
```bash
|
||||
bash Deployment/conf_upgrade.sh \
|
||||
--template Deployment/master.conf.template \
|
||||
--target Configurations/master.conf \
|
||||
--dry-run
|
||||
```
|
||||
|
||||
Prints the full ADDED / REMOVED / KEPT report and writes nothing. No root required — safe to
|
||||
run as any user, on any target, at any time.
|
||||
|
||||
Read the **REMOVED** list carefully. A key showing as REMOVED means it is in your conf but no
|
||||
longer in the template — either genuinely deprecated, or someone forgot step 1 above and the
|
||||
next pull will drop a setting you still rely on.
|
||||
|
||||
---
|
||||
|
||||
### Applying a Schema Change by Hand
|
||||
|
||||
Normally automatic via `git_pull_execute.sh`. To run it manually:
|
||||
|
||||
```bash
|
||||
bash Deployment/conf_upgrade.sh \
|
||||
--template Deployment/master.conf.template \
|
||||
--target Configurations/master.conf \
|
||||
--backup
|
||||
```
|
||||
|
||||
`--backup` writes `master.conf.bak` first. Use it — see The Recovery Gap in the README for
|
||||
why the backup matters more here than it looks.
|
||||
|
||||
For a host conf, the template must have its prefix resolved first, exactly as
|
||||
`git_pull_execute.sh` does it:
|
||||
|
||||
```bash
|
||||
TMPL=$(mktemp)
|
||||
sed "s/HOSTN_/HOST1_/g; s/REMOTE_ID/HOST2/g" Deployment/host.conf.template > "$TMPL"
|
||||
bash Deployment/conf_upgrade.sh --template "$TMPL" --target Configurations/host1.conf --backup
|
||||
rm -f "$TMPL"
|
||||
```
|
||||
|
||||
Merging the raw template without substituting `HOSTN_` would add 140 new `HOSTN_*` keys
|
||||
alongside your real `HOST1_*` ones, and mark every real key as REMOVED.
|
||||
|
||||
---
|
||||
|
||||
### Populating Credentials on a New Host
|
||||
|
||||
```bash
|
||||
bash Deployment/conf_populate.sh --dry-run # always first
|
||||
bash Deployment/conf_populate.sh
|
||||
```
|
||||
|
||||
Reads from the services actually running on this host and fills **empty** fields only.
|
||||
Existing values are never touched without `--overwrite`.
|
||||
|
||||
What it detects: arr API keys and ports from each `config.xml`, root folders from the arr
|
||||
rootFolder API, path maps from docker volume mounts, SABnzbd/slskd/qBittorrent credentials
|
||||
from their own config files, Emby/Jellyfin containers and ports, the boot device transport,
|
||||
and the default-gateway NIC.
|
||||
|
||||
Then pushes the updated conf to partners via `conf_sync.sh` so they hold the fresh keys
|
||||
immediately. `--no-push` skips that.
|
||||
|
||||
**If a field stays empty after a run,** check the output for an ambiguity warning:
|
||||
|
||||
```
|
||||
WARN: Container prefix 'authelia' is ambiguous — matches: Authelia-Secondary Authelia
|
||||
WARN: Refusing to guess. Set the container name manually in host*.conf.
|
||||
```
|
||||
|
||||
That is working as intended. Set it by hand and re-run.
|
||||
|
||||
---
|
||||
|
||||
### After Rotating an API Key
|
||||
|
||||
```bash
|
||||
bash Deployment/conf_populate.sh --overwrite --dry-run
|
||||
bash Deployment/conf_populate.sh --overwrite
|
||||
```
|
||||
|
||||
`--overwrite` replaces detected fields even when already set. This is the intended use for it
|
||||
— rotating an arr key, rebuilding a container, or repointing at a moved service.
|
||||
|
||||
Note it overwrites **every** detected field, not just the rotated one. Run the dry-run first
|
||||
and read the list.
|
||||
|
||||
---
|
||||
|
||||
### Rebuilding a Wiped Node
|
||||
|
||||
The templates are what make this possible without copying another machine's credentials.
|
||||
|
||||
```bash
|
||||
# 1. Clone the repo — templates come with it, confs do not (gitignored)
|
||||
# 2. Seed the confs from the templates
|
||||
cp Deployment/master.conf.template Configurations/master.conf
|
||||
sed "s/HOSTN_/HOST2_/g; s/REMOTE_ID/HOST1/g" Deployment/host.conf.template > Configurations/host2.conf
|
||||
|
||||
# 3. Fill in what can be detected automatically
|
||||
bash Deployment/conf_populate.sh
|
||||
|
||||
# 4. Fill in the rest by hand — anything conf_populate cannot see:
|
||||
# hostnames, DDNS containers, fallback tier lists, sync share lists,
|
||||
# partner credentials, Discord webhook
|
||||
```
|
||||
|
||||
From then on, `git_pull_execute.sh` keeps the conf in step with the template automatically.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ FLAG REFERENCE ━━━
|
||||
|
||||
### `conf_upgrade.sh`
|
||||
|
||||
| Flag | Required | What it does |
|
||||
|------|----------|-------------|
|
||||
| `--template <file>` | yes | Source of structure and new keys |
|
||||
| `--target <file>` | yes | Existing conf — source of real values, always preserved |
|
||||
| `--dry-run` | | Print the change report, write nothing. No root needed. |
|
||||
| `--backup` | | Write `<target>.bak` before installing |
|
||||
|
||||
Takes no other flags. It does not source `load_config.sh`, so `--log` and `--status` do not
|
||||
exist here — see the script header for why that is deliberate.
|
||||
|
||||
### `conf_populate.sh`
|
||||
|
||||
| Flag | What it does |
|
||||
|------|-------------|
|
||||
| `--dry-run` | Show what would be written, truncated. Changes nothing. |
|
||||
| `--overwrite` | Replace detected fields even when already set |
|
||||
| `--no-push` | Skip pushing the updated conf to partners |
|
||||
| `--log` | Verbose per-field output |
|
||||
|
||||
---
|
||||
|
||||
## ━━━ TROUBLESHOOTING ━━━
|
||||
|
||||
### A variable is empty on one node but set on another
|
||||
|
||||
The template entry is missing. Confirm:
|
||||
|
||||
```bash
|
||||
grep -n "MY_VARIABLE" Deployment/master.conf.template Deployment/host.conf.template
|
||||
```
|
||||
|
||||
No hit means the variable was added to a conf directly and never to the template, so it has
|
||||
never reached any other node. Add it to the template; the next pull propagates it.
|
||||
|
||||
### `conf_upgrade` reports a key as REMOVED that is still in use
|
||||
|
||||
Same cause, opposite direction — the key is in your conf and in a script, but not in the
|
||||
template. Add it to the template before the next pull drops it.
|
||||
|
||||
### The conf came back with different permissions
|
||||
|
||||
It should not — the target's mode and owner are copied onto the staged file before the
|
||||
rename. If they did change, check that the target existed before the run: `chmod --reference`
|
||||
silently no-ops against a missing file.
|
||||
|
||||
### A conf edit disappeared after a pull
|
||||
|
||||
Expected if the key is not in the template. `conf_upgrade` keeps values for keys that exist in
|
||||
both; a key present only in your conf is classified REMOVED and dropped. Add it to the
|
||||
template.
|
||||
|
||||
### `conf_populate` skipped a field
|
||||
|
||||
Either the service is not running, its config file was unreadable, or the container name
|
||||
prefix was ambiguous. The last case prints an explicit warning with the full match list —
|
||||
set that value by hand.
|
||||
|
||||
---
|
||||
|
||||
## ━━━ WHAT THIS FOLDER DOES NOT DO ━━━
|
||||
|
||||
- **It does not create `Configurations/`.** `conf_upgrade.sh` aborts if the target does not
|
||||
exist rather than creating a partial conf. Seeding a new node is the manual step above.
|
||||
- **It does not sync confs between hosts.** That is `System_Essentials/conf_sync.sh`.
|
||||
- **It does not validate values.** It reconciles *structure*. A threshold set to nonsense
|
||||
merges through untouched — the consuming script owns validation.
|
||||
Reference in New Issue
Block a user