added container update ability be fore sync. i mean its already down
This commit is contained in:
+23
-9
@@ -24,7 +24,7 @@
|
||||
# ── RSYNC ──────────────────────────────────────────────────────────────────────────────────
|
||||
# RSYNC DEFAULTS Global fallback rsync settings
|
||||
# REMOTE HEALTH CHECKS Rootfs threshold for pre-flight abort
|
||||
# DAILY SYNC SHARES Media shares synced by daily_sync.sh
|
||||
# DAILY SYNC SHARES Media shares synced by media_shares_sync.sh
|
||||
# RSYNC PROFILE SYSTEM Per-profile overrides (appdata profiles)
|
||||
#
|
||||
# ── FAILOVER ───────────────────────────────────────────────────────────────────────────────
|
||||
@@ -153,7 +153,7 @@
|
||||
ROOTFS_WARN=75
|
||||
|
||||
# ━━━ Daily Sync Shares ━━━
|
||||
# Media shares synced once daily by Orchestrators/daily_sync.sh.
|
||||
# Media shares synced once daily by Orchestrators/media_shares_sync.sh.
|
||||
# Each server only syncs the shares it is source of truth for — direction is automatic.
|
||||
# detect_hosts() determines which server is running and picks the correct list.
|
||||
#
|
||||
@@ -230,7 +230,7 @@ HOST2_DAILY_SYNC_SHARES=(
|
||||
# ─────────────────────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# HOST1 personal shares synced to HOST2 for offsite backup
|
||||
# These sync via daily_sync.sh or on their own schedule
|
||||
# These sync via media_shares_sync.sh or on their own schedule
|
||||
# Encrypted datasets sync as encrypted — remote cannot read content
|
||||
HOST1_PERSONAL_SHARES=(
|
||||
# /mnt/user/Gmer4Lfe-Personal # uncomment after creating encrypted dataset
|
||||
@@ -630,7 +630,7 @@ HOST2_TIER1_WRITEBACK_DELAY=60 # minutes — skip Tier 1 writeback if outage un
|
||||
|
||||
# HOST1 writeback tiers — run by HOST2 during HOST1 handback
|
||||
FAILOVER_HOST1_WRITEBACK_TIER1=(
|
||||
"/mnt/user/appdata-Failover/Emby" # Emby userdata, playstates, metadata
|
||||
"/mnt/user/Media_Server/Emby" # Emby userdata, playstates, metadata
|
||||
"/mnt/user/appdata-Failover/Critical-Data" # auth stack — Authelia, Mariadb, Redis, LLDAP, NPM
|
||||
)
|
||||
|
||||
@@ -818,7 +818,7 @@ NETWORK_CONNECT_CONTAINERS=(
|
||||
)
|
||||
|
||||
NETWORK_CONNECT_NETWORKS=(
|
||||
"nextcloud-aio" # Docker network name — must exist before array start
|
||||
"high-availability" # Docker network name — must exist before array start
|
||||
)
|
||||
|
||||
# ==============================================================================================
|
||||
@@ -948,9 +948,9 @@ MEDIA_MAINTENANCE_JOBS=(
|
||||
"Media/media_shares_permissions.sh" # apply permissions first
|
||||
"Media/media_cleaner.sh anime" # remove junk from anime shares
|
||||
"Media/media_cleaner.sh media" # remove junk from media shares
|
||||
"Media/lidarr_cleanup.sh" # remove orphaned music files
|
||||
"Media/sonarr_cleanup.sh" # remove orphaned TV files
|
||||
"Media/radarr_cleanup.sh" # remove orphaned movie files
|
||||
# "Media/lidarr_cleanup.sh" # remove orphaned music files
|
||||
# "Media/sonarr_cleanup.sh" # remove orphaned TV files
|
||||
# "Media/radarr_cleanup.sh" # remove orphaned movie files
|
||||
)
|
||||
|
||||
# ━━━ Arr Cleanup ━━━
|
||||
@@ -1208,8 +1208,22 @@ ZFS_REPORT_IGNORE_POOLS=(
|
||||
DIGEST_SMART_ON_CERT_WARN=true # send if any cert is under CERT_WARN_DAYS
|
||||
DIGEST_SMART_ON_BANDWIDTH=true # send if any transfer exceeded BANDWIDTH_WARN_GB
|
||||
|
||||
# ━━━ Critical Shares Maintenance ━━━
|
||||
# Controls container update behaviour in critical_shares_maintenance.sh
|
||||
# Both local and remote containers are already stopped for the sync window
|
||||
# Updates pull new images while containers are down — starts fresh on new version
|
||||
#
|
||||
# CRITICAL_SYNC_UPDATES — pull updates on LOCAL server during maintenance window
|
||||
# CRITICAL_SYNC_UPDATES_REMOTE — pull updates on REMOTE server during maintenance window
|
||||
#
|
||||
# Both false → sync only, no updates (sync only, no updates)
|
||||
# Both true → full maintenance both servers — recommended for Sunday window
|
||||
# Toggle false temporarily to skip updates without changing the schedule
|
||||
|
||||
CRITICAL_SYNC_UPDATES=true # pull container updates locally during maintenance
|
||||
CRITICAL_SYNC_UPDATES_REMOTE=true # pull container updates on remote via SSH
|
||||
|
||||
# ━━━ Emby Session Report ━━━
|
||||
# Requires API key from Emby Settings → API Keys in the Emby WebUI.
|
||||
# No persistent writes — queries fresh each run.
|
||||
# Each server has its own Emby instance with its own API key.
|
||||
# detect_hosts() selects the correct URL and key at runtime.
|
||||
|
||||
@@ -1,153 +0,0 @@
|
||||
#!/bin/bash
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
# --------------------------------- Critical Shares Full Sync ----------------------------------
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
# Runs clean nightly sync for Emby and the auth stack (Critical-Data).
|
||||
# Both require containers stopped for a consistent, safe state sync.
|
||||
#
|
||||
# Why this exists as a separate orchestrator from media_shares_sync.sh:
|
||||
# media_shares_sync.sh handles media shares — large, runs at 1am, no container stops needed
|
||||
# This script handles appdata that needs containers stopped for clean state:
|
||||
#
|
||||
# Emby:
|
||||
# Hourly dirty sync runs continuously (containers up, WAL excluded)
|
||||
# Nightly clean sync stops Emby → syncs full clean state → restarts Emby
|
||||
# Ensures HOST2 has a fully consistent Emby database nightly
|
||||
#
|
||||
# Critical-Data (auth stack):
|
||||
# NPM, Authelia, Mariadb-Authelia, Redis-Authelia, LLDAP
|
||||
# Containers stopped during sync — databases flush cleanly
|
||||
# HOST2 gets a clean auth state nightly
|
||||
# Users, groups, proxy rules, SSL certs all consistent
|
||||
# Adding a user on HOST1 → propagates to HOST2 overnight automatically
|
||||
#
|
||||
# Schedule: 2:30am daily — after media_shares_sync.sh (1am) finishes
|
||||
# Container stop time is brief — Emby ~30s, auth stack ~15s
|
||||
# Users experience a short Emby interruption at 2:30am — acceptable tradeoff
|
||||
#
|
||||
# For most users: this script ensures failover always has a clean working auth state
|
||||
# without needing to think about dirty writes or WAL files.
|
||||
#
|
||||
# All configuration in Master.conf — rsync profiles handle container stops automatically.
|
||||
# Supports --dry-run to walk through without stopping containers or syncing.
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
source "$SCRIPT_DIR/../Master.conf"
|
||||
source "$SCRIPT_DIR/../common.sh"
|
||||
|
||||
RSYNC_SCRIPT="$SCRIPT_DIR/../Rsync/rsync.sh"
|
||||
|
||||
parse_args "$@"
|
||||
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
# ━━━ $ICON_GEAR Setup ━━━
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
echo ""
|
||||
echo "━━━ $ICON_GEAR Setup ━━━"
|
||||
|
||||
if [[ "$EUID" -ne 0 ]]; then
|
||||
error "Must be run as root"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
success "Running as root"
|
||||
|
||||
acquire_lock
|
||||
|
||||
detect_hosts
|
||||
resolve_remote_ip
|
||||
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
# ━━━ $ICON_SHIELD Pre-flight Checks ━━━
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
echo ""
|
||||
echo "━━━ $ICON_SHIELD Pre-flight Checks ━━━"
|
||||
|
||||
check_connectivity
|
||||
check_remote_rootfs
|
||||
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
# Tracking
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
PASS=()
|
||||
FAIL=()
|
||||
SHARE_TIMES=()
|
||||
TOTAL_START=$(date +%s)
|
||||
|
||||
SYNC_JOBS=(
|
||||
"/mnt/user/Media_Server/Emby"
|
||||
"/mnt/user/appdata-Failover/Critical-Data"
|
||||
)
|
||||
|
||||
SHARE_COUNT=${#SYNC_JOBS[@]}
|
||||
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
# ━━━ $ICON_SYNC Nightly Clean Sync ━━━
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
echo ""
|
||||
echo "━━━ $ICON_SYNC Nightly Clean Sync Starting — $(date '+%Y-%m-%d %H:%M:%S') ━━━"
|
||||
echo "$ICON_SUMMARY Jobs: $SHARE_COUNT"
|
||||
echo ""
|
||||
[[ "$DRY_RUN" == true ]] && warn "DRY RUN — containers will not be stopped"
|
||||
|
||||
JOB_NUM=0
|
||||
for JOB in "${SYNC_JOBS[@]}"; do
|
||||
((JOB_NUM++))
|
||||
JOB_NAME=$(basename "$JOB")
|
||||
echo "━━━ [$JOB_NUM/$SHARE_COUNT] $JOB_NAME ━━━"
|
||||
|
||||
JOB_START=$(date +%s)
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
bash "$RSYNC_SCRIPT" "$JOB" --dry-run
|
||||
else
|
||||
bash "$RSYNC_SCRIPT" "$JOB"
|
||||
fi
|
||||
|
||||
EXIT_CODE=$?
|
||||
JOB_END=$(date +%s)
|
||||
JOB_DURATION=$(format_duration $(( JOB_END - JOB_START )))
|
||||
|
||||
if [[ "$EXIT_CODE" -eq 0 ]]; then
|
||||
PASS+=("$JOB_NAME")
|
||||
success "$JOB_NAME — $ICON_SUCCESS done in $JOB_DURATION"
|
||||
else
|
||||
FAIL+=("$JOB_NAME")
|
||||
error "$JOB_NAME — $ICON_ERROR failed after $JOB_DURATION"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
done
|
||||
|
||||
TOTAL_END=$(date +%s)
|
||||
TOTAL_DURATION=$(format_duration $(( TOTAL_END - TOTAL_START )))
|
||||
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
# ━━━ $ICON_SUMMARY Summary ━━━
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
echo "━━━━━ $ICON_SUMMARY NIGHTLY SYNC SUMMARY ━━━━━"
|
||||
echo "$ICON_TIME Duration: $TOTAL_DURATION"
|
||||
echo "$ICON_SUCCESS Passed: ${#PASS[@]} $ICON_ERROR Failed: ${#FAIL[@]}"
|
||||
echo ""
|
||||
|
||||
if [[ ${#PASS[@]} -gt 0 ]]; then
|
||||
for job in "${PASS[@]}"; do echo " $ICON_SUCCESS $job"; done
|
||||
fi
|
||||
if [[ ${#FAIL[@]} -gt 0 ]]; then
|
||||
for job in "${FAIL[@]}"; do echo " $ICON_ERROR $job"; done
|
||||
fi
|
||||
|
||||
echo ""
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
echo "$ICON_WARN Status: DRY RUN — no changes made"
|
||||
elif [[ ${#FAIL[@]} -eq 0 ]]; then
|
||||
echo "$ICON_DONE Status: $ICON_SUCCESS ALL JOBS COMPLETE"
|
||||
notify "Nightly sync complete on $(hostname) — Emby + auth stack synced cleanly" "Nightly Sync" "normal"
|
||||
else
|
||||
echo "$ICON_ERROR Status: $ICON_ERROR ${#FAIL[@]} JOB(S) FAILED"
|
||||
notify "Nightly sync failed on $(hostname) — failed: ${FAIL[*]}" "Nightly Sync" "warning"
|
||||
fi
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
@@ -0,0 +1,252 @@
|
||||
#!/bin/bash
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
# ----------------------------- Critical Shares Maintenance ------------------------------------
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
# Maintenance window orchestrator for Emby and the auth stack (Critical-Data).
|
||||
# Both local and remote containers are stopped for the entire window — clean state
|
||||
# guaranteed for sync, updates, and restart.
|
||||
#
|
||||
# What it does (in order):
|
||||
# 1. Stop local containers — auth stack + Emby stopped locally
|
||||
# 2. Stop remote containers — auth stack + Emby stopped remotely via SSH
|
||||
# 3. Pull updates locally — if CRITICAL_SYNC_UPDATES=true
|
||||
# 4. Pull updates remotely — if CRITICAL_SYNC_UPDATES_REMOTE=true
|
||||
# 5. rsync Emby — clean full sync, both sides down
|
||||
# 6. rsync Critical-Data — clean full sync, databases fully flushed
|
||||
# 7. Start remote containers — starts on new images, correct order
|
||||
# 8. Start local containers — starts on new images, correct order
|
||||
#
|
||||
# Why this is better than separate update + sync jobs:
|
||||
# Containers are already stopped for the sync — no extra downtime for updates
|
||||
# Both sides on identical image versions after restart
|
||||
# Databases synced before first start on new version — clean state guaranteed
|
||||
# One maintenance window handles sync + updates + ordered restart
|
||||
#
|
||||
# Toggle updates on/off in Master.conf:
|
||||
# CRITICAL_SYNC_UPDATES=true/false — local updates
|
||||
# CRITICAL_SYNC_UPDATES_REMOTE=true/false — remote updates
|
||||
# Both false = sync only (sync only, no updates)
|
||||
#
|
||||
# Container lists and startup order from PROFILE_CRITICAL_CONTAINER_NAMES
|
||||
# Delayed containers (Authelia, Authelia-Secondary) respected on restart
|
||||
# Containers not found on a server skipped gracefully
|
||||
# Only containers that were running get restarted — stopped containers stay stopped
|
||||
#
|
||||
# Recommended schedule: 30 2 * * 0 (Sunday 2:30am)
|
||||
# All configuration in Master.conf.
|
||||
# Supports --dry-run to walk through without stopping containers, syncing, or updating.
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
source "$SCRIPT_DIR/../Master.conf"
|
||||
source "$SCRIPT_DIR/../common.sh"
|
||||
|
||||
RSYNC_SCRIPT="$SCRIPT_DIR/../Rsync/rsync.sh"
|
||||
|
||||
parse_args "$@"
|
||||
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
# ━━━ $ICON_GEAR Setup ━━━
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
echo ""
|
||||
echo "━━━ $ICON_GEAR Setup ━━━"
|
||||
|
||||
if [[ "$EUID" -ne 0 ]]; then
|
||||
error "Must be run as root"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
success "Running as root"
|
||||
|
||||
acquire_lock
|
||||
|
||||
detect_hosts
|
||||
resolve_remote_ip
|
||||
|
||||
# Load container list from emby profile — used for update pulls
|
||||
read -r -a MAINTENANCE_CONTAINERS <<< "${PROFILE_CRITICAL_CONTAINER_NAMES[emby]:-} ${PROFILE_CRITICAL_CONTAINER_NAMES[critical-data]:-}"
|
||||
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
# ━━━ $ICON_SHIELD Pre-flight Checks ━━━
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
echo ""
|
||||
echo "━━━ $ICON_SHIELD Pre-flight Checks ━━━"
|
||||
|
||||
check_connectivity
|
||||
check_remote_rootfs
|
||||
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
# ━━━ $ICON_STOP $ICON_CONTAINERS Stop Containers ━━━
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
echo ""
|
||||
echo "━━━ $ICON_STOP $ICON_CONTAINERS Stop Containers ━━━"
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
warn "DRY RUN — containers will not be stopped"
|
||||
else
|
||||
# Load critical-data + emby container list for stops
|
||||
read -r -a CRITICAL_CONTAINER_NAMES <<< "${PROFILE_CRITICAL_CONTAINER_NAMES[critical-data]:-} ${PROFILE_CRITICAL_CONTAINER_NAMES[emby]:-}"
|
||||
read -r -a DELAYED_CONTAINERS <<< "${PROFILE_DELAYED_CONTAINERS[critical-data]:-}"
|
||||
CONTAINER_DELAY="${PROFILE_CONTAINER_DELAY[critical-data]:-15}"
|
||||
|
||||
stop_local_containers
|
||||
stop_containers
|
||||
fi
|
||||
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
# ━━━ $ICON_GEAR Container Updates ━━━
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
echo ""
|
||||
echo "━━━ $ICON_GEAR Container Updates ━━━"
|
||||
|
||||
if [[ "$CRITICAL_SYNC_UPDATES" == true ]]; then
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
warn "DRY RUN — would pull updates for local containers"
|
||||
for c in "${MAINTENANCE_CONTAINERS[@]}"; do
|
||||
[[ -z "$c" ]] && continue
|
||||
warn "DRY RUN — would docker pull: $c"
|
||||
done
|
||||
else
|
||||
info "Pulling local container updates..."
|
||||
for c in "${MAINTENANCE_CONTAINERS[@]}"; do
|
||||
[[ -z "$c" ]] && continue
|
||||
# Get image name from running or stopped container
|
||||
IMAGE=$(docker inspect "$c" --format '{{.Config.Image}}' 2>/dev/null)
|
||||
if [[ -z "$IMAGE" ]]; then
|
||||
log "$c — not found locally, skipping update"
|
||||
continue
|
||||
fi
|
||||
info "Pulling $IMAGE for $c..."
|
||||
if docker pull "$IMAGE" >/dev/null 2>&1; then
|
||||
success "$c — image updated"
|
||||
else
|
||||
warn "$c — pull failed, will start on existing image"
|
||||
fi
|
||||
done
|
||||
fi
|
||||
else
|
||||
info "CRITICAL_SYNC_UPDATES=false — skipping local updates"
|
||||
fi
|
||||
|
||||
if [[ "$CRITICAL_SYNC_UPDATES_REMOTE" == true ]]; then
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
warn "DRY RUN — would pull updates on $REMOTE_SERVER_NAME"
|
||||
else
|
||||
info "Pulling remote container updates on $REMOTE_SERVER_NAME..."
|
||||
for c in "${MAINTENANCE_CONTAINERS[@]}"; do
|
||||
[[ -z "$c" ]] && continue
|
||||
IMAGE=$(ssh -i "$SSH_KEY" root@"$REMOTE_SERVER" \
|
||||
"docker inspect $c --format '{{.Config.Image}}' 2>/dev/null" 2>/dev/null)
|
||||
if [[ -z "$IMAGE" ]]; then
|
||||
log "$c — not found on remote, skipping update"
|
||||
continue
|
||||
fi
|
||||
info "Pulling $IMAGE for $c on $REMOTE_SERVER_NAME..."
|
||||
if ssh -i "$SSH_KEY" root@"$REMOTE_SERVER" "docker pull $IMAGE" >/dev/null 2>&1; then
|
||||
success "$c — remote image updated"
|
||||
else
|
||||
warn "$c — remote pull failed, will start on existing image"
|
||||
fi
|
||||
done
|
||||
fi
|
||||
else
|
||||
info "CRITICAL_SYNC_UPDATES_REMOTE=false — skipping remote updates"
|
||||
fi
|
||||
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
# ━━━ $ICON_SYNC Critical Shares Sync ━━━
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
PASS=()
|
||||
FAIL=()
|
||||
TOTAL_START=$(date +%s)
|
||||
|
||||
SYNC_JOBS=(
|
||||
"/mnt/user/Media_Server/Emby"
|
||||
"/mnt/user/appdata-Failover/Critical-Data"
|
||||
)
|
||||
|
||||
SHARE_COUNT=${#SYNC_JOBS[@]}
|
||||
|
||||
echo ""
|
||||
echo "━━━ $ICON_SYNC Critical Shares Sync — $(date '+%Y-%m-%d %H:%M:%S') ━━━"
|
||||
echo "$ICON_SUMMARY Jobs: $SHARE_COUNT"
|
||||
echo ""
|
||||
|
||||
JOB_NUM=0
|
||||
for JOB in "${SYNC_JOBS[@]}"; do
|
||||
((JOB_NUM++))
|
||||
JOB_NAME=$(basename "$JOB")
|
||||
echo "━━━ [$JOB_NUM/$SHARE_COUNT] $JOB_NAME ━━━"
|
||||
|
||||
JOB_START=$(date +%s)
|
||||
|
||||
# Containers already stopped — rsync profile won't try to stop them again
|
||||
# Pass --no-container-stop flag would be ideal but profiles handle this naturally
|
||||
# since containers are already stopped, stop_containers finds nothing running
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
bash "$RSYNC_SCRIPT" "$JOB" --dry-run
|
||||
else
|
||||
bash "$RSYNC_SCRIPT" "$JOB"
|
||||
fi
|
||||
|
||||
EXIT_CODE=$?
|
||||
JOB_END=$(date +%s)
|
||||
JOB_DURATION=$(format_duration $(( JOB_END - JOB_START )))
|
||||
|
||||
if [[ "$EXIT_CODE" -eq 0 ]]; then
|
||||
PASS+=("$JOB_NAME")
|
||||
success "$JOB_NAME — $ICON_SUCCESS done in $JOB_DURATION"
|
||||
else
|
||||
FAIL+=("$JOB_NAME")
|
||||
error "$JOB_NAME — $ICON_ERROR failed after $JOB_DURATION"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
done
|
||||
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
# ━━━ $ICON_START $ICON_CONTAINERS Start Containers ━━━
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
echo ""
|
||||
echo "━━━ $ICON_START $ICON_CONTAINERS Start Containers ━━━"
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
warn "DRY RUN — containers will not be started"
|
||||
else
|
||||
start_containers
|
||||
start_local_containers
|
||||
fi
|
||||
|
||||
TOTAL_END=$(date +%s)
|
||||
TOTAL_DURATION=$(format_duration $(( TOTAL_END - TOTAL_START )))
|
||||
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
# ━━━ $ICON_SUMMARY Summary ━━━
|
||||
# -----------------------------------------------------------------------------------------------
|
||||
echo ""
|
||||
echo "━━━━━ $ICON_SUMMARY CRITICAL SHARES MAINTENANCE SUMMARY ━━━━━"
|
||||
echo "$ICON_TIME Duration: $TOTAL_DURATION"
|
||||
echo "$ICON_GEAR Updates: local=$CRITICAL_SYNC_UPDATES remote=$CRITICAL_SYNC_UPDATES_REMOTE"
|
||||
echo "$ICON_SUCCESS Passed: ${#PASS[@]} $ICON_ERROR Failed: ${#FAIL[@]}"
|
||||
echo ""
|
||||
|
||||
if [[ ${#PASS[@]} -gt 0 ]]; then
|
||||
for job in "${PASS[@]}"; do echo " $ICON_SUCCESS $job"; done
|
||||
fi
|
||||
if [[ ${#FAIL[@]} -gt 0 ]]; then
|
||||
for job in "${FAIL[@]}"; do echo " $ICON_ERROR $job"; done
|
||||
fi
|
||||
|
||||
echo ""
|
||||
|
||||
if [[ "$DRY_RUN" == true ]]; then
|
||||
echo "$ICON_WARN Status: DRY RUN — no changes made"
|
||||
elif [[ ${#FAIL[@]} -eq 0 ]]; then
|
||||
echo "$ICON_DONE Status: $ICON_SUCCESS ALL JOBS COMPLETE"
|
||||
notify "Critical maintenance complete on $(hostname) — synced + updated (local=$CRITICAL_SYNC_UPDATES remote=$CRITICAL_SYNC_UPDATES_REMOTE)" "Critical Maintenance" "normal"
|
||||
else
|
||||
echo "$ICON_ERROR Status: $ICON_ERROR ${#FAIL[@]} JOB(S) FAILED"
|
||||
notify "Critical maintenance failed on $(hostname) — failed: ${FAIL[*]}" "Critical Maintenance" "warning"
|
||||
fi
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
@@ -66,7 +66,7 @@
|
||||
# │
|
||||
# ├── Orchestrators/
|
||||
# │ ├── media_shares_sync.sh # Runs all daily media share syncs sequentially
|
||||
# │ ├── critical_shares_full_sync.sh # Clean nightly sync — Emby + auth stack stopped
|
||||
# │ ├── critical_shares_maintenance.sh # Maintenance window — stop, update, sync, restart
|
||||
# │ ├── media_management.sh # Runs permissions + cleaners + arr cleanup
|
||||
# │ ├── transcode_management.sh # Runs cleanup then manager every 3min + daily stats
|
||||
# │ └── README-Orchestrators.md
|
||||
@@ -164,7 +164,7 @@
|
||||
# Individual scripts below are still accessible for manual runs or testing.
|
||||
#
|
||||
#/mnt/user/appdata/unraid_scripts/Orchestrators/media_shares_sync.sh
|
||||
#/mnt/user/appdata/unraid_scripts/Orchestrators/critical_shares_full_sync.sh
|
||||
#/mnt/user/appdata/unraid_scripts/Orchestrators/critical_shares_maintenance.sh
|
||||
#/mnt/user/appdata/unraid_scripts/Orchestrators/media_management.sh
|
||||
#/mnt/user/appdata/unraid_scripts/Orchestrators/transcode_management.sh
|
||||
#
|
||||
@@ -175,7 +175,7 @@
|
||||
#/mnt/user/appdata/unraid_scripts/Rsync/rsync.sh /mnt/user/Media_Server/Emby --profile=emby-failover
|
||||
# ^^ schedule every 30-60min — dirty sync, Emby running, critical data only
|
||||
#/mnt/user/appdata/unraid_scripts/Rsync/rsync.sh /mnt/user/Media_Server/Emby
|
||||
# ^^ do NOT schedule — called by critical_shares_full_sync.sh weekly Sunday 2:30am only
|
||||
# ^^ do NOT schedule — called by critical_shares_maintenance.sh Sunday 2:30am only
|
||||
#/mnt/user/appdata/unraid_scripts/Rsync/rsync.sh /mnt/user/appdata-Failover/Gmer4Lfe
|
||||
#
|
||||
# ━━━ Rsync — Individual Media Shares (ad hoc) ━━━
|
||||
@@ -300,20 +300,20 @@
|
||||
# ━━━ Daily ━━━
|
||||
# 0 1 * * * media_shares_sync.sh (media shares both directions)
|
||||
# 0 2 * * * media_management.sh (permissions + cleaners + arrs)
|
||||
# 30 2 * * 0 critical_shares_full_sync.sh (clean Emby + auth stack sync)
|
||||
# 30 2 * * 0 critical_shares_maintenance.sh (clean Emby + auth stack sync)
|
||||
# 0 3 * * * docker_daily_restart.sh
|
||||
# 0 8 * * * weekly_health_digest.sh (profile controls if it sends)
|
||||
#
|
||||
# ━━━ Rsync profiles — schedule individually ━━━
|
||||
# Arrs_Stack — daily or every few days (arr databases change on every download)
|
||||
# Critical-Data — handled by critical_shares_full_sync.sh — no separate schedule needed
|
||||
# Critical-Data — handled by critical_shares_maintenance.sh — no separate schedule needed
|
||||
# Important-Data — daily (NextCloud file changes)
|
||||
# Gmer4Lfe — daily or weekly (personal appdata, rarely changes)
|
||||
# Emby — handled by critical_shares_full_sync.sh — no separate schedule needed
|
||||
# Emby — handled by critical_shares_maintenance.sh — no separate schedule needed
|
||||
# emby-failover — every 30-60min via frequent cron above
|
||||
#
|
||||
# ━━━ Weekly — Sunday morning block ━━━
|
||||
# 30 2 * * 0 critical_shares_full_sync.sh (clean Emby + auth stack — cache resets weekly)
|
||||
# 30 2 * * 0 critical_shares_maintenance.sh (clean Emby + auth stack — cache resets weekly)
|
||||
# 0 3 * * 0 docker_weekly_restart.sh
|
||||
# 0 5 * * 0 clear_logs.sh
|
||||
# 0 6 * * 0 zfs_memory_snapshot.sh
|
||||
|
||||
Reference in New Issue
Block a user