━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
━━━ 🔄 Play State Sync — 2026-06-11 17:37:58 ━━━
⚠  [WARN]  Emby-Jayred365 (unRAID-Jayred365) — unreachable, skipping
⚠  [WARN]  Jellyfin (unRAID-Jayred365) — unreachable, skipping

── User: gmer4lfe (2 server(s)) ──
⚠  [WARN]    ✗  tvdb:ep:5618559:s7e2 → Emby (unRAID-Gmer4Lfe) failed (HTTP 500)
🏁  [OK]      ✓  tvdb:ep:167594:s5e4 → Jellyfin (unRAID-Gmer4Lfe) marked played
⚠  [WARN]    ✗  tvdb:ep:11818886:s18e1 → Emby (unRAID-Gmer4Lfe) failed (HTTP 500)
⚠  [WARN]    ✗  tvdb:ep:11759792:s9e2 → Emby (unRAID-Gmer4Lfe) failed (HTTP 500)
⚠  [WARN]    ✗  tvdb:ep:11766619:s10e12 → Emby (unRAID-Gmer4Lfe) failed (HTTP 500)
⚠  [WARN]    ✗  imdb:tt8750122 → Emby (unRAID-Gmer4Lfe) failed (HTTP 500)
🏁  [OK]      ✓  tvdb:ep:3279811:s1e2 → Emby (unRAID-Gmer4Lfe) resume synced (370s)
⚠  [WARN]    ✗  tvdb:ep:11493564:s8e11 → Emby (unRAID-Gmer4Lfe) failed (HTTP 500)
⚠  [WARN]    ✗  tvdb:ep:11426444:s1e3 → Emby (unRAID-Gmer4Lfe) failed (HTTP 500)
⚠  [WARN]    ✗  tvdb:ep:10355686:s9e6 → Emby (unRAID-Gmer4Lfe) failed (HTTP 500)
⚠  [WARN]    ✗  tvdb:ep:10355684:s9e7 → Emby (unRAID-Gmer4Lfe) failed (HTTP 500)
⚠  [WARN]    ✗  imdb:tt0437863 → Emby (unRAID-Gmer4Lfe) failed (HTTP 500)
🏁  [OK]      ✓  tvdb:ep:7547570:s1e1 → Jellyfin (unRAID-Gmer4Lfe) marked played
⚠  [WARN]    ✗  tvdb:ep:1555111:s1e1 → Emby (unRAID-Gmer4Lfe) failed (HTTP 500)
⚠  [WARN]    ✗  tvdb:ep:5553405:s4e13 → Emby (unRAID-Gmer4Lfe) failed (HTTP 500)
⚠  [WARN]    ✗  tvdb:ep:893201:s1e5 → Emby (unRAID-Gmer4Lfe) failed (HTTP 500)
⚠  [WARN]    ✗  tvdb:ep:2861921:s1e1 → Emby (unRAID-Gmer4Lfe) failed (HTTP 500)
⚠  [WARN]    ✗  tvdb:ep:11759791:s9e1 → Emby (unRAID-Gmer4Lfe) failed (HTTP 500)


- update-config: Use this skill to configure the Claude Code harness via settings.json. Automated behaviors ("from now on when X", "each time X", "whenever X", "before/after X") require hooks configured in settings.json - the harness executes these, not Claude, so memory/preferences cannot fulfill them. Also use for: permissions ("allow X", "add permission", "move permission to"), env vars ("set X=Y"), hook troubleshooting, or any changes to settings.json/settings.local.json files. Examples: "allow npm commands", "add bq permission to global settings", "move permission to user settings", "set DEBUG=true", "when claude stops show X". For simple settings like theme/model, suggest the /config command.
- keybindings-help: Use when the user wants to customize keyboard shortcuts, rebind keys, add chord bindings, or modify ~/.claude/keybindings.json. Examples: "rebind ctrl+s", "add a chord shortcut", "change the submit key", "customize keybindings".
- verify: Verify that a code change actually does what it's supposed to by running the app and observing behavior. Use when asked to verify a PR, confirm a fix works, test a change manually, check that a feature works, or validate local changes before pushing.
- code-review: Review the current diff for correctness bugs and reuse/simplification/efficiency cleanups at the given effort level (low/medium: fewer, high-confidence findings; high→max: broader coverage, may include uncertain findings; ultra: deep multi-agent review in the cloud). Pass --comment to post findings as inline PR comments, or --fix to apply the findings to the working tree after the review.
- simplify: Review the changed code for reuse, simplification, efficiency, and altitude cleanups, then apply the fixes. Quality only — it does not hunt for bugs; use /code-review for that.
- fewer-permission-prompts: Scan your transcripts for common read-only Bash and MCP tool calls, then add a prioritized allowlist to project .claude/settings.json to reduce permission prompts.
- loop: Run a prompt or slash command on a recurring interval (e.g. /loop 5m /foo). Omit the interval to let the model self-pace. - When the user wants to set up a recurring task, poll for status, or run something repeatedly on an interval (e.g. "check the deploy every 5 minutes", "keep running /babysit-prs"). Do NOT invoke for one-off tasks.
- schedule: Create, update, list, or run scheduled cloud agents (routines) that execute on a cron schedule. - When the user wants to schedule a recurring cloud agent, set up automated tasks, create a cron job for Claude Code, or manage their scheduled agents/routines. Also use when the user wants a one-time scheduled run ("run this once at 3pm", "remind me to check X tomorrow").
- claude-api: Reference for the Claude API / Anthropic SDK — model ids, pricing, params, streaming, tool use, MCP, agents, caching, token counting, model migration.
TRIGGER — read BEFORE opening the target file; don't skip because it "looks like a one-liner" — whenever: the prompt names Claude/Anthropic in any form (Claude, Anthropic, Fable, Opus, Sonnet, Haiku, `anthropic`, `@anthropic-ai`, `claude-*`, `us.anthropic.*`, `[1m]`); the user asks about an LLM (pricing/model choice/limits/caching) — never answer from memory; OR the task is LLM-shaped with provider unstated (agent/MCP/tool-definition/multi-agent/RAG/LLM-judge/computer-use; generate/summarize/extract/classify/rewrite/converse over NL; debugging refusals/cutoffs/streaming/tool-calls/tokens).
SKIP only when another provider is being worked on (overrides all triggers): OpenAI/GPT/Gemini/Llama/Mistral/Cohere/Ollama named in the query; OR `grep -rE 'openai|langchain_openai|google.generativeai|genai|mistralai|cohere|ollama'` over the project hits (run this grep FIRST if no provider named — don't Read the file).
- run: Launch and drive this project's app to see a change working. Use when asked to run, start, or screenshot the app, or to confirm a change works in the real app (not just tests). First looks for a project skill that already covers launching the app; otherwise falls back to built-in patterns per project type (CLI, server, TUI, Electron, browser-driven, library).
- init: Initialize a new CLAUDE.md file with codebase documentation
- review: Review a pull request
- security-review: Complete a security review of the pending changes on the current branch

update-config keybindings-help verify code-review simplify fewer-permission-prompts loop schedule claude-api run init review security-review


Debug Emby play state sync HTTP 500 errors


Let me look at the play state sync script to understand what's causing these HTTP 500s from Emby.


Bash

find /boot/config/plugins/varaverk -name "*play_state*" -o -name "*playstate*" 2>/dev/null

Find play state sync scripts


1	#!/bin/bash
2	# ==============================================================================================
3	# ================================= Play State Sync ============================================
4	# ==============================================================================================
5	#
6	# PURPOSE
7	# ─────────────────────────────────────────────────────────────────────────────
8	# Syncs watched/played state and resume positions across all configured Emby
9	# and Jellyfin servers. Newest timestamp wins — no data is ever lost.
10	#
11	# Users are matched by name (case-insensitive). If a user exists on some servers
12	# but not others, those servers are skipped for that user — no errors, no partial
13	# syncs from unrelated accounts.
14	#
15	# Items are matched by external provider IDs:
16	#   Movies   → IMDb ID, then TMDB ID
17	#   Episodes → TVDB ID + season + episode number
18	#   Audio    → MusicBrainz Track ID
19	#
20	# ==============================================================================================
21	# SYNC LOGIC
22	# ==============================================================================================
23	#
24	# For each matched item across ≥2 servers:
25	#   1. Compare LastPlayedDate across all servers that have a play record.
26	#   2. The server with the newest LastPlayedDate is authoritative.
27	#   3. Push that server's state (Played, PlayCount, LastPlayedDate,
28	#      PlaybackPositionTicks) to every other server.
29	#   4. Servers with no record for that item also receive the state.
30	#
31	# Resume positions (partial plays, not marked Played):
32	#   Synced by comparing PlaybackPositionTicks when LastPlayedDate is absent.
33	#   The higher tick count wins.
34	#
35	# ==============================================================================================
36	# CONFIGURATION (host*.conf, aliased by detect_hosts)
37	# ==============================================================================================
38	#
39	#   HOST*_TRANSCODE_SERVERS  "Name|URL|APIKey|type" entries per host (emby/jellyfin)
40	#                            All hosts are discovered automatically — no extra config needed.
41	#                            Remote host URLs have localhost rewritten to their Tailscale IP.
42	#
43	# master.conf
44	#
45	#   PLAY_SYNC_ENABLED        Master toggle (default: true)
46	#   PLAY_SYNC_REMOTE         Sync across all hosts via Tailscale (default: true)
47	#                            false = local servers only (this host's Emby + Jellyfin)
48	#   PLAY_SYNC_DAYS           How many days back to check for played items (default: 90)
49	#                            Use 0 to sync all played items (slow on large libraries).
50	#   PLAY_SYNC_TYPES          Comma-separated item types to sync (default: Movie,Episode,Audio)
51	#
52	# ==============================================================================================
53	# RUNTIME MODES
54	# ==============================================================================================
55	#
56	# play_state_sync.sh
57	#     Sync all matched users across all configured servers.
58	#
59	# play_state_sync.sh --dry-run
60	#     Show what would be synced without writing any state.
61	#
62	# play_state_sync.sh --status
63	#     Show configured servers, reachability, and user counts.
64	#
65	# play_state_sync.sh --full
66	#     Ignore PLAY_SYNC_DAYS — sync all played items (may be slow).
67	#
68	# play_state_sync.sh --log
69	#     Verbose output — show each item comparison.
70	#
71	# ==============================================================================================
72
73	SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
74	source "$SCRIPT_DIR/../load_config.sh"
75
76	# ── Handle --full before parse_args ───────────────────────────────────────────
77	FULL_SYNC=false
78	_FILTERED=()
79	for _a in "$@"; do
80	    [[ "$_a" == "--full" ]] && FULL_SYNC=true || _FILTERED+=("$_a")
81	done
82	parse_args "${_FILTERED[@]}"
83
84	# ==============================================================================================
85	# ━━━ Setup ━━━
86	# ==============================================================================================
87	[[ "${PLAY_SYNC_ENABLED:-true}" != "true" ]] && echo "Play state sync disabled" && exit 0
88
89	SYNC_DAYS="${PLAY_SYNC_DAYS:-90}"
90	SYNC_TYPES="${PLAY_SYNC_TYPES:-Movie,Episode,Audio}"
91	[[ "$FULL_SYNC" == true ]] && SYNC_DAYS=0
92	log "$ICON_GEAR Config: days=${SYNC_DAYS} types=${SYNC_TYPES} remote=${PLAY_SYNC_REMOTE:-true}"
93
94	command -v jq >/dev/null 2>&1 || { error "jq is required but not installed"; exit 1; }
95
96	acquire_lock
97
98	# ── Build server list across ALL hosts ────────────────────────────────────────
99	# All HOST*_TRANSCODE_SERVERS arrays are loaded into env by load_config.sh.
100	# For remote hosts, localhost in the URL is rewritten to their Tailscale IP.
101	declare -a SRV_NAME SRV_URL SRV_KEY SRV_TYPE
102	_srv_count=0
103	_my_hostname=$(hostname -s)
104
105	_add_server() {
106	    local name="$1" url="$2" key="$3" type="$4"
107	    [[ -z "$url" || -z "$key" ]] && return
108	    [[ "$key" == "YOUR_API_KEY"* || "$key" == "placeholder"* ]] && return
109	    SRV_NAME[$_srv_count]="$name"
110	    SRV_URL[$_srv_count]="$url"
111	    SRV_KEY[$_srv_count]="$key"
112	    SRV_TYPE[$_srv_count]="$type"
113	    (( _srv_count++ ))
114	}
115
116	for _varname in $(compgen -v | grep -E '^HOST[0-9]+$' | sort); do
117	    _num="${_varname//[^0-9]/}"
118	    _host_hostname="${!_varname}"
119	    [[ -z "$_host_hostname" ]] && continue
120
121	    _is_me=false
122	    [[ "${_host_hostname,,}" == "${_my_hostname,,}" ]] && _is_me=true
123
124	    # Skip remote hosts when PLAY_SYNC_REMOTE=false
125	    if [[ "$_is_me" == false && "${PLAY_SYNC_REMOTE:-true}" != "true" ]]; then
126	        log "$_host_hostname — remote sync disabled (PLAY_SYNC_REMOTE=false), skipping"
127	        continue
128	    fi
129
130	    # Resolve Tailscale IP for remote hosts
131	    _ts_ip=""
132	    if [[ "$_is_me" == false ]]; then
133	        _ts_ip=$(resolve_tailscale_ip "$_host_hostname")
134	        if [[ -z "$_ts_ip" ]]; then
135	            log "$_host_hostname — Tailscale IP not found, skipping"
136	            continue
137	        fi
138	    fi
139
140	    # Load this host's TRANSCODE_SERVERS array
141	    _srv_arr_name="HOST${_num}_TRANSCODE_SERVERS"
142	    eval "_host_entries=(\"\${${_srv_arr_name}[@]}\")"
143	    [[ "${#_host_entries[@]}" -eq 0 ]] && continue
144
145	    for _entry in "${_host_entries[@]}"; do
146	        IFS='|' read -r _name _url _key _type <<< "$_entry"
147	        [[ "$_type" == "emby" || "$_type" == "jellyfin" ]] || continue
148	        # For remote hosts rewrite localhost/127.0.0.1 → Tailscale IP
149	        if [[ "$_is_me" == false ]]; then
150	            _url="${_url//localhost/$_ts_ip}"
151	            _url="${_url//127.0.0.1/$_ts_ip}"
152	        fi
153	        _add_server "${_name} (${_host_hostname})" "$_url" "$_key" "$_type"
154	    done
155	done
156
157	if [[ "$_srv_count" -lt 2 ]]; then
158	    error "Need at least 2 media servers configured — found $_srv_count"
159	    exit 1
160	fi
161
162	# ==============================================================================================
163	# ━━━ API Helpers ━━━
164	# ==============================================================================================
165	_api_get() {
166	    local url="$1" key="$2" endpoint="$3"
167	    curl -sf --max-time 30 \
168	        -H "X-Emby-Token: $key" \
169	        "${url%/}/${endpoint}" 2>/dev/null
170	}
171
172	_api_post() {
173	    local url="$1" key="$2" endpoint="$3" data="${4:-}"
174	    if [[ -n "$data" ]]; then
175	        curl -sf --max-time 30 -s -o /dev/null -w "%{http_code}" -X POST \
176	            -H "X-Emby-Token: $key" \
177	            -H "Content-Type: application/json" \
178	            -d "$data" \
179	            "${url%/}/${endpoint}" 2>/dev/null
180	    else
181	        curl -sf --max-time 30 -s -o /dev/null -w "%{http_code}" -X POST \
182	            -H "X-Emby-Token: $key" \
183	            "${url%/}/${endpoint}" 2>/dev/null
184	    fi
185	}
186
187	# ISO 8601 date → unix seconds (portable, no date -d on BusyBox)
188	_iso_to_epoch() {
189	    local dt="$1"
190	    [[ -z "$dt" || "$dt" == "null" ]] && echo 0 && return
191	    # Strip fractional seconds and Z, convert to seconds
192	    dt="${dt%.*}"       # remove .NNNNNNN
193	    dt="${dt%Z}"        # remove trailing Z
194	    dt="${dt/T/ }"      # T → space
195	    date -u -d "$dt UTC" +%s 2>/dev/null || echo 0
196	}
197
198	# Ticks → seconds (1 tick = 100ns, 10_000_000 ticks = 1s)
199	_ticks_to_sec() {
200	    echo $(( ${1:-0} / 10000000 ))
201	}
202
203	# ==============================================================================================
204	# ━━━ Status ━━━
205	# ==============================================================================================
206	if [[ "$SHOW_STATUS" == true ]]; then
207	    echo ""
208	    echo "━━━━━ $ICON_SUMMARY PLAY STATE SYNC STATUS ━━━━━"
209	    echo "$ICON_GEAR  Remote sync: ${PLAY_SYNC_REMOTE:-true}"
210	    echo "$ICON_GEAR  Sync days:   ${SYNC_DAYS:-all}"
211	    echo "$ICON_GEAR  Item types:  $SYNC_TYPES"
212	    echo ""
213	    for i in $(seq 0 $(( _srv_count - 1 ))); do
214	        echo "$ICON_HOST  [${SRV_TYPE[$i]}] ${SRV_NAME[$i]}  (${SRV_URL[$i]})"
215	        _users=$(_api_get "${SRV_URL[$i]}" "${SRV_KEY[$i]}" "Users" 2>/dev/null | jq -r '.[].Name' 2>/dev/null | wc -l)
216	        if [[ "$_users" -gt 0 ]]; then
217	            echo "    $ICON_DONE Reachable — $_users user(s)"
218	            _api_get "${SRV_URL[$i]}" "${SRV_KEY[$i]}" "Users" 2>/dev/null \
219	                | jq -r '.[].Name' 2>/dev/null | while read -r n; do echo "      · $n"; done
220	        else
221	            echo "    $ICON_ERROR Unreachable or no users"
222	        fi
223	    done
224	    echo "━━━━━━━━━━━━━━━━━━━━━━━"
225	    exit 0
226	fi
227
228	# ==============================================================================================
229	# ━━━ Main Sync ━━━
230	# ==============================================================================================
231	echo "━━━ $ICON_SYNC Play State Sync — $(date '+%Y-%m-%d %H:%M:%S') ━━━"
232	[[ "$DRY_RUN" == true ]]  && warn "DRY RUN — no state will be written"
233	[[ "$FULL_SYNC" == true ]] && log "Full sync mode — ignoring PLAY_SYNC_DAYS"
234
235	START=$(date +%s)
236	TOTAL_SYNCED=0
237	TOTAL_SKIPPED=0
238	TOTAL_ERRORS=0
239
240	# ── Step 1: Fetch users from each server ─────────────────────────────────────
241	declare -A SRV_USERS   # idx → JSON array string of users
242
243	log "Fetching users..."
244	for i in $(seq 0 $(( _srv_count - 1 ))); do
245	    _resp=$(_api_get "${SRV_URL[$i]}" "${SRV_KEY[$i]}" "Users")
246	    if [[ -z "$_resp" ]]; then
247	        warn "${SRV_NAME[$i]} — unreachable, skipping"
248	        SRV_USERS[$i]=""
249	        continue
250	    fi
251	    SRV_USERS[$i]="$_resp"
252	    _count=$(echo "$_resp" | jq 'length' 2>/dev/null || echo 0)
253	    log "${SRV_NAME[$i]} — $_count user(s)"
254	done
255
256	# ── Step 2: Build cross-server user map ──────────────────────────────────────
257	# lowercase_name → "server_idx:user_id server_idx:user_id ..."
258	declare -A USER_MAP
259
260	for i in $(seq 0 $(( _srv_count - 1 ))); do
261	    [[ -z "${SRV_USERS[$i]}" ]] && continue
262	    while IFS=$'\t' read -r uid uname; do
263	        [[ -z "$uid" || -z "$uname" ]] && continue
264	        lname="${uname,,}"
265	        if [[ -n "${USER_MAP[$lname]}" ]]; then
266	            USER_MAP[$lname]+=" ${i}:${uid}"
267	        else
268	            USER_MAP[$lname]="${i}:${uid}"
269	        fi
270	    done < <(echo "${SRV_USERS[$i]}" | jq -r '.[] | [.Id, .Name] | @tsv' 2>/dev/null)
271	done
272
273	# ── Step 3: Sync per matched user ────────────────────────────────────────────
274	_date_filter=""
275	if [[ "$SYNC_DAYS" -gt 0 ]]; then
276	    _cutoff=$(date -u -d "$SYNC_DAYS days ago" '+%Y-%m-%dT%H:%M:%SZ' 2>/dev/null || \
277	              date -u -v "-${SYNC_DAYS}d" '+%Y-%m-%dT%H:%M:%SZ' 2>/dev/null)
278	    [[ -n "$_cutoff" ]] && _date_filter="&MinDateLastSaved=$_cutoff"
279	fi
280
281	for lname in "${!USER_MAP[@]}"; do
282	    read -ra _pairs <<< "${USER_MAP[$lname]}"
283
284	    # Skip users only on one server
285	    [[ "${#_pairs[@]}" -lt 2 ]] && log "  $lname — only on 1 server, skipping" && continue
286
287	    echo ""
288	    echo "── User: $lname (${#_pairs[@]} server(s)) ──"
289
290	    # Build per-server user context
291	    declare -A U_IDX U_UID
292	    for _pair in "${_pairs[@]}"; do
293	        IFS=':' read -r _si _ui <<< "$_pair"
294	        U_IDX["$_si"]="$_si"
295	        U_UID["$_si"]="$_ui"
296	    done
297
298	    # ── Fetch played items from each server for this user ────────────────────
299	    # Key: provider_id_string → sorted list of (epoch, srv_idx, item_id, play_count, ticks, played)
300	    declare -A ITEM_MAP   # provider_key → JSON per-server data
301
302	    for _si in "${!U_IDX[@]}"; do
303	        _uid="${U_UID[$_si]}"
304	        _endpoint="Users/${_uid}/Items?Recursive=true&Fields=ProviderIds,UserData,Type,ParentIndexNumber,IndexNumber,SeriesName&IncludeItemTypes=${SYNC_TYPES}&Filters=IsPlayed${_date_filter}&Limit=5000"
305	        _resp=$(_api_get "${SRV_URL[$_si]}" "${SRV_KEY[$_si]}" "$_endpoint")
306	        if [[ -z "$_resp" ]]; then
307	            warn "  ${SRV_NAME[$_si]} — failed to fetch items for $lname"
308	            continue
309	        fi
310
311	        # Also fetch items with resume position (not yet marked played)
312	        _endpoint2="Users/${_uid}/Items?Recursive=true&Fields=ProviderIds,UserData,Type,ParentIndexNumber,IndexNumber,SeriesName&IncludeItemTypes=${SYNC_TYPES}&SortBy=DatePlayed&SortOrder=Descending&Filters=IsResumable${_date_filter}&Limit=500"
313	        _resp2=$(_api_get "${SRV_URL[$_si]}" "${SRV_KEY[$_si]}" "$_endpoint2")
314
315	        # Combine and deduplicate by Id
316	        if [[ -n "$_resp2" ]]; then
317	            _combined=$(printf '%s\n%s' "$_resp" "$_resp2" | jq -s \
318	                '[.[0].Items // [], .[1].Items // []] | add // [] | unique_by(.Id)' 2>/dev/null)
319	        else
320	            _combined=$(echo "$_resp" | jq '.Items // []' 2>/dev/null)
321	        fi
322
323	        _count=$(echo "$_combined" | jq 'length' 2>/dev/null || echo 0)
324	        log "  ${SRV_NAME[$_si]} — $_count item(s) with state for $lname"
325
326	        # Build item lookup by provider key
327	        while IFS=$'\t' read -r iid itype season ep imdb tmdb tvdb mbtrack played ticks lplayed pcount; do
328	            # Build canonical provider key
329	            _pkey=""
330	            case "$itype" in
331	                Movie)
332	                    [[ "$imdb" != "null" && -n "$imdb" ]] && _pkey="imdb:${imdb}"
333	                    [[ -z "$_pkey" && "$tmdb" != "null" && -n "$tmdb" ]] && _pkey="tmdb:movie:${tmdb}"
334	                    ;;
335	                Episode)
336	                    [[ "$tvdb" != "null" && -n "$tvdb" && "$season" != "null" && "$ep" != "null" ]] && \
337	                        _pkey="tvdb:ep:${tvdb}:s${season}e${ep}"
338	                    ;;
339	                Audio)
340	                    [[ "$mbtrack" != "null" && -n "$mbtrack" ]] && _pkey="mb:track:${mbtrack}"
341	                    ;;
342	            esac
343	            [[ -z "$_pkey" ]] && continue
344
345	            _epoch=$(_iso_to_epoch "$lplayed")
346	            _entry="${_si}|${iid}|${played}|${pcount}|${ticks}|${_epoch}|${lplayed}"
347
348	            if [[ -n "${ITEM_MAP[$_pkey]}" ]]; then
349	                ITEM_MAP[$_pkey]+=$'\n'"$_entry"
350	            else
351	                ITEM_MAP[$_pkey]="$_entry"
352	            fi
353
354	        done < <(echo "$_combined" | jq -r '.[] | [
355	            .Id,
356	            .Type,
357	            (.ParentIndexNumber // "null" | tostring),
358	            (.IndexNumber // "null" | tostring),
359	            (.ProviderIds.Imdb // "null"),
360	            (.ProviderIds.Tmdb // "null"),
361	            (.ProviderIds.Tvdb // "null"),
362	            (.ProviderIds.MusicBrainzTrackId // "null"),
363	            (.UserData.Played // false | tostring),
364	            (.UserData.PlaybackPositionTicks // 0 | tostring),
365	            (.UserData.LastPlayedDate // "null"),
366	            (.UserData.PlayCount // 0 | tostring)
367	        ] | @tsv' 2>/dev/null)
368	    done
369
370	    # ── Compare and sync ─────────────────────────────────────────────────────
371	    for _pkey in "${!ITEM_MAP[@]}"; do
372	        # Collect all server entries for this item
373	        declare -A E_EPOCH E_PLAYED E_PCOUNT E_TICKS E_IID E_LPLAYED E_SIDX
374	        _has_entries=false
375
376	        while IFS='|' read -r _si _iid _played _pcount _ticks _epoch _lplayed; do
377	            [[ -z "$_si" ]] && continue
378	            E_SIDX[$_si]="$_si"
379	            E_IID[$_si]="$_iid"
380	            E_PLAYED[$_si]="$_played"
381	            E_PCOUNT[$_si]="$_pcount"
382	            E_TICKS[$_si]="$_ticks"
383	            E_EPOCH[$_si]="$_epoch"
384	            E_LPLAYED[$_si]="$_lplayed"
385	            _has_entries=true
386	        done <<< "${ITEM_MAP[$_pkey]}"
387
388	        [[ "$_has_entries" == false ]] && continue
389
390	        # Find the authoritative server: newest LastPlayedDate epoch
391	        # Tie-break: higher PlayCount, then higher Ticks
392	        _auth_si=""
393	        _auth_epoch=0
394	        _auth_pcount=0
395	        _auth_ticks=0
396
397	        for _si in "${!E_SIDX[@]}"; do
398	            _e="${E_EPOCH[$_si]:-0}"
399	            _pc="${E_PCOUNT[$_si]:-0}"
400	            _tk="${E_TICKS[$_si]:-0}"
401	            if [[ "$_e" -gt "$_auth_epoch" ]] || \
402	               [[ "$_e" -eq "$_auth_epoch" && "$_pc" -gt "$_auth_pcount" ]] || \
403	               [[ "$_e" -eq "$_auth_epoch" && "$_pc" -eq "$_auth_pcount" && "$_tk" -gt "$_auth_ticks" ]]; then
404	                _auth_si="$_si"
405	                _auth_epoch="$_e"
406	                _auth_pcount="$_pc"
407	                _auth_ticks="$_tk"
408	            fi
409	        done
410
411	        [[ -z "$_auth_si" ]] && continue
412
413	        _auth_played="${E_PLAYED[$_auth_si]}"
414	        _auth_lplayed="${E_LPLAYED[$_auth_si]}"
415	        _auth_pcount="${E_PCOUNT[$_auth_si]}"
416	        _auth_ticks="${E_TICKS[$_auth_si]}"
417
418	        # Push to servers with older state OR no state at all
419	        for _si in "${!U_IDX[@]}"; do
420	            [[ "$_si" == "$_auth_si" ]] && continue
421	            _their_epoch="${E_EPOCH[$_si]:-0}"
422	            _their_played="${E_PLAYED[$_si]:-false}"
423
424	            # Skip if they already have the same/newer state
425	            if [[ "$_their_epoch" -ge "$_auth_epoch" ]] && \
426	               [[ "$_their_played" == "$_auth_played" ]]; then
427	                log "  SKIP  $_pkey → ${SRV_NAME[$_si]} already up to date"
428	                (( TOTAL_SKIPPED++ ))
429	                continue
430	            fi
431
432	            _uid="${U_UID[$_si]}"
433	            _iid="${E_IID[$_si]:-}"   # might not exist on this server yet
434
435	            # Find item ID on target server by provider key if not in our map
436	            if [[ -z "$_iid" ]]; then
437	                _ptype="${_pkey%%:*}"
438	                _pval="${_pkey##*:}"
439	                case "$_ptype" in
440	                    imdb)   _search_field="imdb.${_pval}" ;;
441	                    tmdb)   _search_field="tmdb.${_pval##movie:}" ;;
442	                    tvdb)
443	                        # _pkey format: tvdb:ep:{tvdb_id}:s{season}e{ep}
444	                        # ##*: gives "s7e2" (wrong); strip prefix then first :
445	                        _tvdb_num="${_pkey#tvdb:ep:}"; _tvdb_num="${_tvdb_num%%:*}"
446	                        _search_field="tvdb.${_tvdb_num}"
447	                        ;;
448	                    mb)     _search_field="" ;;  # skip music if not found
449	                esac
450
451	                if [[ -n "$_search_field" ]]; then
452	                    _iid=$(_api_get "${SRV_URL[$_si]}" "${SRV_KEY[$_si]}" \
453	                        "Items?AnyProviderIdEquals=${_search_field}&Recursive=true&Fields=ProviderIds&ExcludeLocationTypes=Virtual&Limit=1" 2>/dev/null \
454	                        | jq -r '.Items[0].Id // empty' 2>/dev/null)
455	                fi
456	                [[ -z "$_iid" ]] && log "  SKIP  $_pkey → ${SRV_NAME[$_si]} item not found on server" && continue
457	            fi
458
459	            log "  SYNC  $_pkey → ${SRV_NAME[$_si]} (auth: ${SRV_NAME[$_auth_si]}, epoch: $_auth_epoch)"
460
461	            if [[ "$DRY_RUN" == true ]]; then
462	                echo "  DRY RUN: would sync $_pkey → ${SRV_NAME[$_si]} user=$lname played=$_auth_played date=$_auth_lplayed"
463	                (( TOTAL_SYNCED++ ))
464	                continue
465	            fi
466
467	            # Write state to target server
468	            if [[ "$_auth_played" == "true" ]]; then
469	                # Mark as played with date
470	                _date_param=""
471	                if [[ "$_auth_lplayed" != "null" && -n "$_auth_lplayed" ]]; then
472	                    # Emby's PlayedItems endpoint rejects 7-digit fractional seconds (.0000000)
473	                    # with HTTP 500; strip to whole seconds before encoding
474	                    if [[ "$_auth_lplayed" == *.* ]]; then
475	                        _lp="${_auth_lplayed%.*}"
476	                        [[ "$_auth_lplayed" == *Z ]] && _lp+="Z"
477	                    else
478	                        _lp="$_auth_lplayed"
479	                    fi
480	                    _date_param="?DatePlayed=${_lp//[: ]/%3A}"
481	                fi
482	                _http=$(_api_post "${SRV_URL[$_si]}" "${SRV_KEY[$_si]}" \
483	                    "Users/${_uid}/PlayedItems/${_iid}${_date_param}")
484	                if [[ "$_http" == "200" || "$_http" == "201" ]]; then
485	                    success "  ✓  $_pkey → ${SRV_NAME[$_si]} marked played"
486	                    (( TOTAL_SYNCED++ ))
487	                else
488	                    warn "  ✗  $_pkey → ${SRV_NAME[$_si]} failed (HTTP ${_http:-err})"
489	                    (( TOTAL_ERRORS++ ))
490	                fi
491	            else
492	                # Sync resume position only
493	                _ticks_int=$(( ${_auth_ticks:-0} ))
494	                if [[ "$_ticks_int" -gt 0 ]]; then
495	                    _payload="{\"PlaybackPositionTicks\":${_ticks_int}}"
496	                    _http=$(_api_post "${SRV_URL[$_si]}" "${SRV_KEY[$_si]}" \
497	                        "Users/${_uid}/Items/${_iid}/UserData" "$_payload")
498	                    if [[ "$_http" == "200" || "$_http" == "204" ]]; then
499	                        success "  ✓  $_pkey → ${SRV_NAME[$_si]} resume synced ($(_ticks_to_sec "$_ticks_int")s)"
500	                        (( TOTAL_SYNCED++ ))
501	                    else
502	                        warn "  ✗  $_pkey → ${SRV_NAME[$_si]} resume sync failed (HTTP ${_http:-err})"
503	                        (( TOTAL_ERRORS++ ))
504	                    fi
505	                fi
506	            fi
507	        done
508
509	        unset E_EPOCH E_PLAYED E_PCOUNT E_TICKS E_IID E_LPLAYED E_SIDX
510	    done
511
512	    unset ITEM_MAP U_IDX U_UID
513	done
514
515	# ==============================================================================================
516	# ━━━ Summary ━━━
517	# ==============================================================================================
518	END=$(date +%s)
519	echo ""
520	echo "━━━━━ $ICON_SUMMARY PLAY STATE SYNC SUMMARY ━━━━━"
521	echo "$ICON_TIME  Duration:  $(format_duration $(( END - START )))"
522	echo "$ICON_DONE  Synced:    $TOTAL_SYNCED"
523	[[ "$TOTAL_SKIPPED" -gt 0 ]] && echo "$ICON_RUNNING  Skipped:   $TOTAL_SKIPPED (already current)"
524	[[ "$TOTAL_ERRORS"  -gt 0 ]] && echo "$ICON_ERROR  Errors:    $TOTAL_ERRORS"
525
526	if [[ "$DRY_RUN" == true ]]; then
527	    warn "DRY RUN — no changes written"
528	elif [[ "$TOTAL_ERRORS" -eq 0 ]]; then
529	    success "Done ✅"
530	else
531	    warn "Done with $TOTAL_ERRORS error(s)"
532	fi
533

#!/bin/bash
# ==============================================================================================
# ================================= Play State Sync ============================================
# ==============================================================================================
#
# PURPOSE
# ─────────────────────────────────────────────────────────────────────────────
# Syncs watched/played state and resume positions across all configured Emby
# and Jellyfin servers. Newest timestamp wins — no data is ever lost.
#
# Users are matched by name (case-insensitive). If a user exists on some servers
# but not others, those servers are skipped for that user — no errors, no partial
# syncs from unrelated accounts.
#
# Items are matched by external provider IDs:
#   Movies   → IMDb ID, then TMDB ID
#   Episodes → TVDB ID + season + episode number
#   Audio    → MusicBrainz Track ID
#
# ==============================================================================================
# SYNC LOGIC
# ==============================================================================================
#
# For each matched item across ≥2 servers:
#   1. Compare LastPlayedDate across all servers that have a play record.
#   2. The server with the newest LastPlayedDate is authoritative.
#   3. Push that server's state (Played, PlayCount, LastPlayedDate,
#      PlaybackPositionTicks) to every other server.
#   4. Servers with no record for that item also receive the state.
#
# Resume positions (partial plays, not marked Played):
#   Synced by comparing PlaybackPositionTicks when LastPlayedDate is absent.
#   The higher tick count wins.
#
# ==============================================================================================
# CONFIGURATION (host*.conf, aliased by detect_hosts)
# ==============================================================================================
#
#   HOST*_TRANSCODE_SERVERS  "Name|URL|APIKey|type" entries per host (emby/jellyfin)
#                            All hosts are discovered automatically — no extra config needed.
#                            Remote host URLs have localhost rewritten to their Tailscale IP.
#
# master.conf
#
#   PLAY_SYNC_ENABLED        Master toggle (default: true)
#   PLAY_SYNC_REMOTE         Sync across all hosts via Tailscale (default: true)
#                            false = local servers only (this host's Emby + Jellyfin)
#   PLAY_SYNC_DAYS           How many days back to check for played items (default: 90)
#                            Use 0 to sync all played items (slow on large libraries).
#   PLAY_SYNC_TYPES          Comma-separated item types to sync (default: Movie,Episode,Audio)
#
# ==============================================================================================
# RUNTIME MODES
# ==============================================================================================
#
# play_state_sync.sh
#     Sync all matched users across all configured servers.
#
# play_state_sync.sh --dry-run
#     Show what would be synced without writing any state.
#
# play_state_sync.sh --status
#     Show configured servers, reachability, and user counts.
#
# play_state_sync.sh --full
#     Ignore PLAY_SYNC_DAYS — sync all played items (may be slow).
#
# play_state_sync.sh --log
#     Verbose output — show each item comparison.
#
# ==============================================================================================

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/../load_config.sh"

# ── Handle --full before parse_args ───────────────────────────────────────────
FULL_SYNC=false
_FILTERED=()
for _a in "$@"; do
    [[ "$_a" == "--full" ]] && FULL_SYNC=true || _FILTERED+=("$_a")
done
parse_args "${_FILTERED[@]}"

# ==============================================================================================
# ━━━ Setup ━━━
# ==============================================================================================
[[ "${PLAY_SYNC_ENABLED:-true}" != "true" ]] && echo "Play state sync disabled" && exit 0

SYNC_DAYS="${PLAY_SYNC_DAYS:-90}"
SYNC_TYPES="${PLAY_SYNC_TYPES:-Movie,Episode,Audio}"
[[ "$FULL_SYNC" == true ]] && SYNC_DAYS=0
log "$ICON_GEAR Config: days=${SYNC_DAYS} types=${SYNC_TYPES} remote=${PLAY_SYNC_REMOTE:-true}"

command -v jq >/dev/null 2>&1 || { error "jq is required but not installed"; exit 1; }

acquire_lock

# ── Build server list across ALL hosts ────────────────────────────────────────
# All HOST*_TRANSCODE_SERVERS arrays are loaded into env by load_config.sh.
# For remote hosts, localhost in the URL is rewritten to their Tailscale IP.
declare -a SRV_NAME SRV_URL SRV_KEY SRV_TYPE
_srv_count=0
_my_hostname=$(hostname -s)

_add_server() {
    local name="$1" url="$2" key="$3" type="$4"
    [[ -z "$url" || -z "$key" ]] && return
    [[ "$key" == "YOUR_API_KEY"* || "$key" == "placeholder"* ]] && return
    SRV_NAME[$_srv_count]="$name"
    SRV_URL[$_srv_count]="$url"
    SRV_KEY[$_srv_count]="$key"
    SRV_TYPE[$_srv_count]="$type"
    (( _srv_count++ ))
}

for _varname in $(compgen -v | grep -E '^HOST[0-9]+$' | sort); do
    _num="${_varname//[^0-9]/}"
    _host_hostname="${!_varname}"
    [[ -z "$_host_hostname" ]] && continue

    _is_me=false
    [[ "${_host_hostname,,}" == "${_my_hostname,,}" ]] && _is_me=true

    # Skip remote hosts when PLAY_SYNC_REMOTE=false
    if [[ "$_is_me" == false && "${PLAY_SYNC_REMOTE:-true}" != "true" ]]; then
        log "$_host_hostname — remote sync disabled (PLAY_SYNC_REMOTE=false), skipping"
        continue
    fi

    # Resolve Tailscale IP for remote hosts
    _ts_ip=""
    if [[ "$_is_me" == false ]]; then
        _ts_ip=$(resolve_tailscale_ip "$_host_hostname")
        if [[ -z "$_ts_ip" ]]; then
            log "$_host_hostname — Tailscale IP not found, skipping"
            continue
        fi
    fi

    # Load this host's TRANSCODE_SERVERS array
    _srv_arr_name="HOST${_num}_TRANSCODE_SERVERS"
    eval "_host_entries=(\"\${${_srv_arr_name}[@]}\")"
    [[ "${#_host_entries[@]}" -eq 0 ]] && continue

    for _entry in "${_host_entries[@]}"; do
        IFS='|' read -r _name _url _key _type <<< "$_entry"
        [[ "$_type" == "emby" || "$_type" == "jellyfin" ]] || continue
        # For remote hosts rewrite localhost/127.0.0.1 → Tailscale IP
        if [[ "$_is_me" == false ]]; then
            _url="${_url//localhost/$_ts_ip}"
            _url="${_url//127.0.0.1/$_ts_ip}"
        fi
        _add_server "${_name} (${_host_hostname})" "$_url" "$_key" "$_type"
    done
done

if [[ "$_srv_count" -lt 2 ]]; then
    error "Need at least 2 media servers configured — found $_srv_count"
    exit 1
fi

# ==============================================================================================
# ━━━ API Helpers ━━━
# ==============================================================================================
_api_get() {
    local url="$1" key="$2" endpoint="$3"
    curl -sf --max-time 30 \
        -H "X-Emby-Token: $key" \
        "${url%/}/${endpoint}" 2>/dev/null
}

_api_post() {
    local url="$1" key="$2" endpoint="$3" data="${4:-}"
    if [[ -n "$data" ]]; then
        curl -sf --max-time 30 -s -o /dev/null -w "%{http_code}" -X POST \
            -H "X-Emby-Token: $key" \
            -H "Content-Type: application/json" \
            -d "$data" \
            "${url%/}/${endpoint}" 2>/dev/null
    else
        curl -sf --max-time 30 -s -o /dev/null -w "%{http_code}" -X POST \
            -H "X-Emby-Token: $key" \
            "${url%/}/${endpoint}" 2>/dev/null
    fi
}

# ISO 8601 date → unix seconds (portable, no date -d on BusyBox)
_iso_to_epoch() {
    local dt="$1"
    [[ -z "$dt" || "$dt" == "null" ]] && echo 0 && return
    # Strip fractional seconds and Z, convert to seconds
    dt="${dt%.*}"       # remove .NNNNNNN
    dt="${dt%Z}"        # remove trailing Z
    dt="${dt/T/ }"      # T → space
    date -u -d "$dt UTC" +%s 2>/dev/null || echo 0
}

# Ticks → seconds (1 tick = 100ns, 10_000_000 ticks = 1s)
_ticks_to_sec() {
    echo $(( ${1:-0} / 10000000 ))
}

# ==============================================================================================
# ━━━ Status ━━━
# ==============================================================================================
if [[ "$SHOW_STATUS" == true ]]; then
    echo ""
    echo "━━━━━ $ICON_SUMMARY PLAY STATE SYNC STATUS ━━━━━"
    echo "$ICON_GEAR  Remote sync: ${PLAY_SYNC_REMOTE:-true}"
    echo "$ICON_GEAR  Sync days:   ${SYNC_DAYS:-all}"
    echo "$ICON_GEAR  Item types:  $SYNC_TYPES"
    echo ""
    for i in $(seq 0 $(( _srv_count - 1 ))); do
        echo "$ICON_HOST  [${SRV_TYPE[$i]}] ${SRV_NAME[$i]}  (${SRV_URL[$i]})"
        _users=$(_api_get "${SRV_URL[$i]}" "${SRV_KEY[$i]}" "Users" 2>/dev/null | jq -r '.[].Name' 2>/dev/null | wc -l)
        if [[ "$_users" -gt 0 ]]; then
            echo "    $ICON_DONE Reachable — $_users user(s)"
            _api_get "${SRV_URL[$i]}" "${SRV_KEY[$i]}" "Users" 2>/dev/null \
                | jq -r '.[].Name' 2>/dev/null | while read -r n; do echo "      · $n"; done
        else
            echo "    $ICON_ERROR Unreachable or no users"
        fi
    done
    echo "━━━━━━━━━━━━━━━━━━━━━━━"
    exit 0
fi

# ==============================================================================================
# ━━━ Main Sync ━━━
# ==============================================================================================
echo "━━━ $ICON_SYNC Play State Sync — $(date '+%Y-%m-%d %H:%M:%S') ━━━"
[[ "$DRY_RUN" == true ]]  && warn "DRY RUN — no state will be written"
[[ "$FULL_SYNC" == true ]] && log "Full sync mode — ignoring PLAY_SYNC_DAYS"

START=$(date +%s)
TOTAL_SYNCED=0
TOTAL_SKIPPED=0
TOTAL_ERRORS=0

# ── Step 1: Fetch users from each server ─────────────────────────────────────
declare -A SRV_USERS   # idx → JSON array string of users

log "Fetching users..."
for i in $(seq 0 $(( _srv_count - 1 ))); do
    _resp=$(_api_get "${SRV_URL[$i]}" "${SRV_KEY[$i]}" "Users")
    if [[ -z "$_resp" ]]; then
        warn "${SRV_NAME[$i]} — unreachable, skipping"
        SRV_USERS[$i]=""
        continue
    fi
    SRV_USERS[$i]="$_resp"
    _count=$(echo "$_resp" | jq 'length' 2>/dev/null || echo 0)
    log "${SRV_NAME[$i]} — $_count user(s)"
done

# ── Step 2: Build cross-server user map ──────────────────────────────────────
# lowercase_name → "server_idx:user_id server_idx:user_id ..."
declare -A USER_MAP

for i in $(seq 0 $(( _srv_count - 1 ))); do
    [[ -z "${SRV_USERS[$i]}" ]] && continue
    while IFS=$'\t' read -r uid uname; do
        [[ -z "$uid" || -z "$uname" ]] && continue
        lname="${uname,,}"
        if [[ -n "${USER_MAP[$lname]}" ]]; then
            USER_MAP[$lname]+=" ${i}:${uid}"
        else
            USER_MAP[$lname]="${i}:${uid}"
        fi
    done < <(echo "${SRV_USERS[$i]}" | jq -r '.[] | [.Id, .Name] | @tsv' 2>/dev/null)
done

# ── Step 3: Sync per matched user ────────────────────────────────────────────
_date_filter=""
if [[ "$SYNC_DAYS" -gt 0 ]]; then
    _cutoff=$(date -u -d "$SYNC_DAYS days ago" '+%Y-%m-%dT%H:%M:%SZ' 2>/dev/null || \
              date -u -v "-${SYNC_DAYS}d" '+%Y-%m-%dT%H:%M:%SZ' 2>/dev/null)
    [[ -n "$_cutoff" ]] && _date_filter="&MinDateLastSaved=$_cutoff"
fi

for lname in "${!USER_MAP[@]}"; do
    read -ra _pairs <<< "${USER_MAP[$lname]}"

    # Skip users only on one server
    [[ "${#_pairs[@]}" -lt 2 ]] && log "  $lname — only on 1 server, skipping" && continue

    echo ""
    echo "── User: $lname (${#_pairs[@]} server(s)) ──"

    # Build per-server user context
    declare -A U_IDX U_UID
    for _pair in "${_pairs[@]}"; do
        IFS=':' read -r _si _ui <<< "$_pair"
        U_IDX["$_si"]="$_si"
        U_UID["$_si"]="$_ui"
    done

    # ── Fetch played items from each server for this user ────────────────────
    # Key: provider_id_string → sorted list of (epoch, srv_idx, item_id, play_count, ticks, played)
    declare -A ITEM_MAP   # provider_key → JSON per-server data

    for _si in "${!U_IDX[@]}"; do
        _uid="${U_UID[$_si]}"
        _endpoint="Users/${_uid}/Items?Recursive=true&Fields=ProviderIds,UserData,Type,ParentIndexNumber,IndexNumber,SeriesName&IncludeItemTypes=${SYNC_TYPES}&Filters=IsPlayed${_date_filter}&Limit=5000"
        _resp=$(_api_get "${SRV_URL[$_si]}" "${SRV_KEY[$_si]}" "$_endpoint")
        if [[ -z "$_resp" ]]; then
            warn "  ${SRV_NAME[$_si]} — failed to fetch items for $lname"
            continue
        fi

        # Also fetch items with resume position (not yet marked played)
        _endpoint2="Users/${_uid}/Items?Recursive=true&Fields=ProviderIds,UserData,Type,ParentIndexNumber,IndexNumber,SeriesName&IncludeItemTypes=${SYNC_TYPES}&SortBy=DatePlayed&SortOrder=Descending&Filters=IsResumable${_date_filter}&Limit=500"
        _resp2=$(_api_get "${SRV_URL[$_si]}" "${SRV_KEY[$_si]}" "$_endpoint2")

        # Combine and deduplicate by Id
        if [[ -n "$_resp2" ]]; then
            _combined=$(printf '%s\n%s' "$_resp" "$_resp2" | jq -s \
                '[.[0].Items // [], .[1].Items // []] | add // [] | unique_by(.Id)' 2>/dev/null)
        else
            _combined=$(echo "$_resp" | jq '.Items // []' 2>/dev/null)
        fi

        _count=$(echo "$_combined" | jq 'length' 2>/dev/null || echo 0)
        log "  ${SRV_NAME[$_si]} — $_count item(s) with state for $lname"

        # Build item lookup by provider key
        while IFS=$'\t' read -r iid itype season ep imdb tmdb tvdb mbtrack played ticks lplayed pcount; do
            # Build canonical provider key
            _pkey=""
            case "$itype" in
                Movie)
                    [[ "$imdb" != "null" && -n "$imdb" ]] && _pkey="imdb:${imdb}"
                    [[ -z "$_pkey" && "$tmdb" != "null" && -n "$tmdb" ]] && _pkey="tmdb:movie:${tmdb}"
                    ;;
                Episode)
                    [[ "$tvdb" != "null" && -n "$tvdb" && "$season" != "null" && "$ep" != "null" ]] && \
                        _pkey="tvdb:ep:${tvdb}:s${season}e${ep}"
                    ;;
                Audio)
                    [[ "$mbtrack" != "null" && -n "$mbtrack" ]] && _pkey="mb:track:${mbtrack}"
                    ;;
            esac
            [[ -z "$_pkey" ]] && continue

            _epoch=$(_iso_to_epoch "$lplayed")
            _entry="${_si}|${iid}|${played}|${pcount}|${ticks}|${_epoch}|${lplayed}"

            if [[ -n "${ITEM_MAP[$_pkey]}" ]]; then
                ITEM_MAP[$_pkey]+=$'\n'"$_entry"
            else
                ITEM_MAP[$_pkey]="$_entry"
            fi

        done < <(echo "$_combined" | jq -r '.[] | [
            .Id,
            .Type,
            (.ParentIndexNumber // "null" | tostring),
            (.IndexNumber // "null" | tostring),
            (.ProviderIds.Imdb // "null"),
            (.ProviderIds.Tmdb // "null"),
            (.ProviderIds.Tvdb // "null"),
            (.ProviderIds.MusicBrainzTrackId // "null"),
            (.UserData.Played // false | tostring),
            (.UserData.PlaybackPositionTicks // 0 | tostring),
            (.UserData.LastPlayedDate // "null"),
            (.UserData.PlayCount // 0 | tostring)
        ] | @tsv' 2>/dev/null)
    done

    # ── Compare and sync ─────────────────────────────────────────────────────
    for _pkey in "${!ITEM_MAP[@]}"; do
        # Collect all server entries for this item
        declare -A E_EPOCH E_PLAYED E_PCOUNT E_TICKS E_IID E_LPLAYED E_SIDX
        _has_entries=false

        while IFS='|' read -r _si _iid _played _pcount _ticks _epoch _lplayed; do
            [[ -z "$_si" ]] && continue
            E_SIDX[$_si]="$_si"
            E_IID[$_si]="$_iid"
            E_PLAYED[$_si]="$_played"
            E_PCOUNT[$_si]="$_pcount"
            E_TICKS[$_si]="$_ticks"
            E_EPOCH[$_si]="$_epoch"
            E_LPLAYED[$_si]="$_lplayed"
            _has_entries=true
        done <<< "${ITEM_MAP[$_pkey]}"

        [[ "$_has_entries" == false ]] && continue

        # Find the authoritative server: newest LastPlayedDate epoch
        # Tie-break: higher PlayCount, then higher Ticks
        _auth_si=""
        _auth_epoch=0
        _auth_pcount=0
        _auth_ticks=0

        for _si in "${!E_SIDX[@]}"; do
            _e="${E_EPOCH[$_si]:-0}"
            _pc="${E_PCOUNT[$_si]:-0}"
            _tk="${E_TICKS[$_si]:-0}"
            if [[ "$_e" -gt "$_auth_epoch" ]] || \
               [[ "$_e" -eq "$_auth_epoch" && "$_pc" -gt "$_auth_pcount" ]] || \
               [[ "$_e" -eq "$_auth_epoch" && "$_pc" -eq "$_auth_pcount" && "$_tk" -gt "$_auth_ticks" ]]; then
                _auth_si="$_si"
                _auth_epoch="$_e"
                _auth_pcount="$_pc"
                _auth_ticks="$_tk"
            fi
        done

        [[ -z "$_auth_si" ]] && continue

        _auth_played="${E_PLAYED[$_auth_si]}"
        _auth_lplayed="${E_LPLAYED[$_auth_si]}"
        _auth_pcount="${E_PCOUNT[$_auth_si]}"
        _auth_ticks="${E_TICKS[$_auth_si]}"

        # Push to servers with older state OR no state at all
        for _si in "${!U_IDX[@]}"; do
            [[ "$_si" == "$_auth_si" ]] && continue
            _their_epoch="${E_EPOCH[$_si]:-0}"
            _their_played="${E_PLAYED[$_si]:-false}"

            # Skip if they already have the same/newer state
            if [[ "$_their_epoch" -ge "$_auth_epoch" ]] && \
               [[ "$_their_played" == "$_auth_played" ]]; then
                log "  SKIP  $_pkey → ${SRV_NAME[$_si]} already up to date"
                (( TOTAL_SKIPPED++ ))
                continue
            fi

            _uid="${U_UID[$_si]}"
            _iid="${E_IID[$_si]:-}"   # might not exist on this server yet

            # Find item ID on target server by provider key if not in our map
            if [[ -z "$_iid" ]]; then
                _ptype="${_pkey%%:*}"
                _pval="${_pkey##*:}"
                case "$_ptype" in
                    imdb)   _search_field="imdb.${_pval}" ;;
                    tmdb)   _search_field="tmdb.${_pval##movie:}" ;;
                    tvdb)
                        # _pkey format: tvdb:ep:{tvdb_id}:s{season}e{ep}
                        # ##*: gives "s7e2" (wrong); strip prefix then first :
                        _tvdb_num="${_pkey#tvdb:ep:}"; _tvdb_num="${_tvdb_num%%:*}"
                        _search_field="tvdb.${_tvdb_num}"
                        ;;
                    mb)     _search_field="" ;;  # skip music if not found
                esac

                if [[ -n "$_search_field" ]]; then
                    _iid=$(_api_get "${SRV_URL[$_si]}" "${SRV_KEY[$_si]}" \
                        "Items?AnyProviderIdEquals=${_search_field}&Recursive=true&Fields=ProviderIds&ExcludeLocationTypes=Virtual&Limit=1" 2>/dev/null \
                        | jq -r '.Items[0].Id // empty' 2>/dev/null)
                fi
                [[ -z "$_iid" ]] && log "  SKIP  $_pkey → ${SRV_NAME[$_si]} item not found on server" && continue
            fi

            log "  SYNC  $_pkey → ${SRV_NAME[$_si]} (auth: ${SRV_NAME[$_auth_si]}, epoch: $_auth_epoch)"

            if [[ "$DRY_RUN" == true ]]; then
                echo "  DRY RUN: would sync $_pkey → ${SRV_NAME[$_si]} user=$lname played=$_auth_played date=$_auth_lplayed"
                (( TOTAL_SYNCED++ ))
                continue
            fi

            # Write state to target server
            if [[ "$_auth_played" == "true" ]]; then
                # Mark as played with date
                _date_param=""
                if [[ "$_auth_lplayed" != "null" && -n "$_auth_lplayed" ]]; then
                    # Emby's PlayedItems endpoint rejects 7-digit fractional seconds (.0000000)
                    # with HTTP 500; strip to whole seconds before encoding
                    if [[ "$_auth_lplayed" == *.* ]]; then
                        _lp="${_auth_lplayed%.*}"
                        [[ "$_auth_lplayed" == *Z ]] && _lp+="Z"
                    else
                        _lp="$_auth_lplayed"
                    fi
                    _date_param="?DatePlayed=${_lp//[: ]/%3A}"
                fi
                _http=$(_api_post "${SRV_URL[$_si]}" "${SRV_KEY[$_si]}" \
                    "Users/${_uid}/PlayedItems/${_iid}${_date_param}")
                if [[ "$_http" == "200" || "$_http" == "201" ]]; then
                    success "  ✓  $_pkey → ${SRV_NAME[$_si]} marked played"
                    (( TOTAL_SYNCED++ ))
                else
                    warn "  ✗  $_pkey → ${SRV_NAME[$_si]} failed (HTTP ${_http:-err})"
                    (( TOTAL_ERRORS++ ))
                fi
            else
                # Sync resume position only
                _ticks_int=$(( ${_auth_ticks:-0} ))
                if [[ "$_ticks_int" -gt 0 ]]; then
                    _payload="{\"PlaybackPositionTicks\":${_ticks_int}}"
                    _http=$(_api_post "${SRV_URL[$_si]}" "${SRV_KEY[$_si]}" \
                        "Users/${_uid}/Items/${_iid}/UserData" "$_payload")
                    if [[ "$_http" == "200" || "$_http" == "204" ]]; then
                        success "  ✓  $_pkey → ${SRV_NAME[$_si]} resume synced ($(_ticks_to_sec "$_ticks_int")s)"
                        (( TOTAL_SYNCED++ ))
                    else
                        warn "  ✗  $_pkey → ${SRV_NAME[$_si]} resume sync failed (HTTP ${_http:-err})"
                        (( TOTAL_ERRORS++ ))
                    fi
                fi
            fi
        done

        unset E_EPOCH E_PLAYED E_PCOUNT E_TICKS E_IID E_LPLAYED E_SIDX
    done

    unset ITEM_MAP U_IDX U_UID
done

# ==============================================================================================
# ━━━ Summary ━━━
# ==============================================================================================
END=$(date +%s)
echo ""
echo "━━━━━ $ICON_SUMMARY PLAY STATE SYNC SUMMARY ━━━━━"
echo "$ICON_TIME  Duration:  $(format_duration $(( END - START )))"
echo "$ICON_DONE  Synced:    $TOTAL_SYNCED"
[[ "$TOTAL_SKIPPED" -gt 0 ]] && echo "$ICON_RUNNING  Skipped:   $TOTAL_SKIPPED (already current)"
[[ "$TOTAL_ERRORS"  -gt 0 ]] && echo "$ICON_ERROR  Errors:    $TOTAL_ERRORS"

if [[ "$DRY_RUN" == true ]]; then
    warn "DRY RUN — no changes written"
elif [[ "$TOTAL_ERRORS" -eq 0 ]]; then
    success "Done ✅"
else
    warn "Done with $TOTAL_ERRORS error(s)"
fi


━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ━━━ 🔄 Play State Sync — 2026-06-11 17:37:58 ━━━ ⚠  [WARN]  Emby-Jayred365 (unRAID-Jayred365) — unreachable, skipping ⚠  [WARN]  Jellyfin (unRAID-Jayred365…


The issue is in the date format being sent to Emby's `PlayedItems` endpoint. The code already has a comment about Emby rejecting fractional seconds, but it's still sending ISO 8601 (`2026-01-15T10:30:45Z` → `2026-01-15T10%3A30%3A45Z`). Emby's API spec requires `yyyyMMddHHmmss` format (`20260115103045`). Jellyfin marks-played are succeeding likely because those items don't have a date (null `_date_param`).

The fix is at lines 473-480 — convert the date format rather than just URL-encoding ISO 8601:


Edit

                    # Emby's PlayedItems endpoint rejects 7-digit fractional seconds (.0000000)
                    # with HTTP 500; strip to whole seconds before encoding
                    if [[ "$_auth_lplayed" == *.* ]]; then
                        _lp="${_auth_lplayed%.*}"
                        [[ "$_auth_lplayed" == *Z ]] && _lp+="Z"
                    else
                        _lp="$_auth_lplayed"
                    fi
                    _date_param="?DatePlayed=${_lp//[: ]/%3A}"

                    # PlayedItems expects DatePlayed in yyyyMMddHHmmss (no separators, no timezone)
                    _lp="${_auth_lplayed%.*}"
                    _lp="${_lp%Z}"
                    _date_param="?DatePlayed=${_lp//[-T:]/}"


The file /boot/config/plugins/varaverk/Media/play_state_sync.sh has been updated successfully. (file state is current in your context — no need to Read it back)

                    # Emby's PlayedItems endpoint rejects 7-digit fractional seconds (.0000000)
                    # with HTTP 500; strip to whole seconds before encoding
                    if [[ "$_auth_lplayed" == *.* ]]; then
                        _lp="${_auth_lplayed%.*}"
                        [[ "$_auth_lplayed" == *Z ]] && _lp+="Z"
                    else
                        _lp="$_auth_lplayed"
                    fi
                    _date_param="?DatePlayed=${_lp//[: ]/%3A}"

                    # PlayedItems expects DatePlayed in yyyyMMddHHmmss (no separators, no timezone)
                    _lp="${_auth_lplayed%.*}"
                    _lp="${_lp%Z}"
                    _date_param="?DatePlayed=${_lp//[-T:]/}"

if [[ "$_auth_lplayed" != "null" && -n "$_auth_lplayed" ]]; then

-                    # Emby's PlayedItems endpoint rejects 7-digit fractional seconds (.0000000)

-                    # with HTTP 500; strip to whole seconds before encoding

-                    if [[ "$_auth_lplayed" == *.* ]]; then

-                        _lp="${_auth_lplayed%.*}"

-                        _lp="$_auth_lplayed"

-                    _date_param="?DatePlayed=${_lp//[: ]/%3A}"

+                    # PlayedItems expects DatePlayed in yyyyMMddHHmmss (no separators, no timezone)

+                    _lp="${_auth_lplayed%.*}"

+                    _date_param="?DatePlayed=${_lp//[-T:]/}"

_http=$(_api_post "${SRV_URL[$_si]}" "${SRV_KEY[$_si]}" \


The root cause: `%.*` strips the fractional part but the previous code then re-appended `Z` and URL-encoded the colons, resulting in `2026-01-15T10%3A30%3A45Z` — which Emby's API rejects. The fix strips fractional seconds and the `Z`, then removes all separators to produce `20260115103045`, which is what the API actually expects.

Next run of play_state_sync should clear those 500s.


lets run it


Bash

Run play state sync


━━━ 🔄 Play State Sync — 2026-06-11 18:04:23 ━━━
⚠  [WARN]  Emby-Jayred365 (unRAID-Jayred365) — unreachable, skipping
⚠  [WARN]  Jellyfin (unRAID-Jayred365) — unreachable, skipping

── User: gmer4lfe (2 server(s)) ──
🏁  [OK]      ✓  tvdb:ep:5618559:s7e2 → Emby (unRAID-Gmer4Lfe) marked played
🏁  [OK]      ✓  tvdb:ep:167594:s5e4 → Jellyfin (unRAID-Gmer4Lfe) marked played
🏁  [OK]      ✓  tvdb:ep:11818886:s18e1 → Emby (unRAID-Gmer4Lfe) marked played
🏁  [OK]      ✓  tvdb:ep:11759792:s9e2 → Emby (unRAID-Gmer4Lfe) marked played
🏁  [OK]      ✓  tvdb:ep:11766619:s10e12 → Emby (unRAID-Gmer4Lfe) marked played
🏁  [OK]      ✓  imdb:tt8750122 → Emby (unRAID-Gmer4Lfe) marked played
🏁  [OK]      ✓  tvdb:ep:3279811:s1e2 → Emby (unRAID-Gmer4Lfe) resume synced (370s)
🏁  [OK]      ✓  tvdb:ep:11493564:s8e11 → Emby (unRAID-Gmer4Lfe) marked played
🏁  [OK]      ✓  tvdb:ep:11426444:s1e3 → Emby (unRAID-Gmer4Lfe) marked played
🏁  [OK]      ✓  tvdb:ep:10355686:s9e6 → Emby (unRAID-Gmer4Lfe) marked played
🏁  [OK]      ✓  tvdb:ep:10355684:s9e7 → Emby (unRAID-Gmer4Lfe) marked played
🏁  [OK]      ✓  imdb:tt0437863 → Emby (unRAID-Gmer4Lfe) marked played
🏁  [OK]      ✓  tvdb:ep:7547570:s1e1 → Jellyfin (unRAID-Gmer4Lfe) marked played
🏁  [OK]      ✓  tvdb:ep:1555111:s1e1 → Emby (unRAID-Gmer4Lfe) marked played
🏁  [OK]      ✓  tvdb:ep:5553405:s4e13 → Emby (unRAID-Gmer4Lfe) marked played
🏁  [OK]      ✓  tvdb:ep:893201:s1e5 → Emby (unRAID-Gmer4Lfe) marked played
🏁  [OK]      ✓  tvdb:ep:2861921:s1e1 → Emby (unRAID-Gmer4Lfe) marked played
🏁  [OK]      ✓  tvdb:ep:11759791:s9e1 → Emby (unRAID-Gmer4Lfe) marked played

── User: failedproxy (2 server(s)) ──

━━━━━ 📋 PLAY STATE SYNC SUMMARY ━━━━━
⏱  Duration:  43s
🏁  Synced:    18
🟢  Skipped:   35 (already current)
🏁  [OK]    Done ✅


All 18 previously-failing items synced clean — zero errors. The `yyyyMMddHHmmss` format is what Emby needed.


🔄  Shares (14):
  ❌ Books — 0s
  ❌ Intros — 0s
  ❌ Kids_Movies — 0s
  ❌ Kids_Tv_Shows — 0s
  ❌ Movies — 0s
  ❌ Music — 0s
  ❌ Music_Videos — 0s
  ❌ Nextcloud — 0s
  ❌ stand-up_comedy — 0s
  ❌ Sports — 0s
  ❌ Anime_Shows-Old — 0s
  ❌ Anime_Movies-Old — 0s
  ❌ Anime_Movies — 0s
  ❌ Anime_Shows — 0s
  Passed: 0/14  Failed: 14/14


Let me find what's generating this output and check the rsync script.


grep -r "Passed:.*Failed:" /boot/config/plugins/varaverk --include="*.sh" -l

Find script producing Passed/Failed output


grep -r "Shares (" /boot/config/plugins/varaverk --include="*.sh" -l

Find script producing Shares output


/boot/config/plugins/varaverk/Orchestrators/daily_sync_maintenance.sh
/boot/config/plugins/varaverk/Orchestrators/intermediate_sync_maintenance.sh


find /boot/config/plugins/varaverk/Rsync -name "*.sh" | sort

List rsync scripts


1	#!/bin/bash
2	# ==============================================================================================
3	# ============================= Rsync Core Script ==============================================
4	# ==============================================================================================
5	#
6	# PURPOSE
7	# ─────────────────────────────────────────────────────────────────────────────
8	# Core rsync engine for the two-server ecosystem. Called per share or per
9	# appdata profile by orchestrators (daily_sync_maintenance, weekly_sync_maintenance,
10	# critical_sync_maintenance) and directly for manual or scheduled dirty syncs.
11	#
12	# Profile is inferred from the directory basename (lowercased). Override with
13	# --profile=name for explicit selection. If no profile matches, global defaults
14	# from master.conf apply and no containers are stopped.
15	#
16	# After each sync, logs transfer data to bandwidth_monitor.sh for the weekly
17	# bandwidth report. Silent on success — only failures produce visible output.
18	#
19	# ==============================================================================================
20	# OPERATIONAL MODEL
21	# ==============================================================================================
22	#
23	# Profiles define per-share behavior:
24	#   PROFILE_CRITICAL_CONTAINER_NAMES  — containers stopped on both servers before sync
25	#   PROFILE_DELAYED_CONTAINERS        — containers with a delay before restart after sync
26	#   PROFILE_CONTAINER_DELAY           — seconds before delayed containers start
27	#   PROFILE_RSYNC_OPTS                — rsync flags (does not inherit DEFAULT_RSYNC_OPTS)
28	#   PROFILE_BW_LIMIT                  — bandwidth limit in KB/s
29	#   PROFILE_RETRY_COUNT               — retry attempts on failure
30	#   PROFILE_SLEEP                     — seconds between retry attempts
31	#   PROFILE_EXCLUDE_DIRS              — paths excluded from transfer
32	#   PROFILE_REMOTE_RESTART_CONTAINERS — containers restarted on remote after dirty sync
33	#                                       Was running → restart. Was stopped → leave stopped.
34	#
35	# Two-tier rsync enable/disable:
36	#   Tier 1: RSYNC_ENABLED=false → all rsync stops immediately (checked by this script)
37	#   Tier 2: per-orchestrator flag (DAILY_RSYNC_ENABLED etc.) → checked by caller
38	#
39	# Bandwidth logging: after each sync, logs profile/duration/status/bytes to
40	# bandwidth_monitor.sh --log-transfer. Bytes captured from rsync --stats via awk
41	# using version-stable field names.
42	#
43	# ==============================================================================================
44	# OPERATIONAL SAFEGUARDS
45	# ==============================================================================================
46	#
47	# Global Rsync Gate
48	#   check_rsync_enabled() — RSYNC_ENABLED=false exits cleanly before any operation.
49	#
50	# Partnership Blocklist
51	#   Refuses to sync if REMOTE_SERVER_NAME appears in the partnership blocklist.
52	#   Written at offboard — prevents stale access after a partnership ends.
53	#
54	# Version Parity
55	#   check_unraid_version_parity — refuses sync if servers on incompatible unRAID versions.
56	#
57	# Remote Health Pre-flights
58	#   check_connectivity() — Tailscale IP reachable before any SSH
59	#   check_remote_rootfs() — aborts if remote rootfs exceeds ROOTFS_WARN_PCT
60	#   check_remote_share() — aborts if target directory missing or empty on remote
61	#   check_remote_disks() — verifies all backing disks online on remote
62	#
63	# Drive Temperature Check
64	#   check_local_disk_temps() — runs before any transfer. Exit 1 = skip this profile,
65	#   exit 2 = abort all remaining syncs (CRITICAL temperature).
66	#
67	# Remote Docker Daemon Check
68	#   check_remote_docker_daemon — verified before any container stop/start operations.
69	#   If daemon unresponsive: container operations skipped, rsync proceeds without stopping.
70	#
71	# Per-Profile Concurrency Lock
72	#   acquire_rsync_lock() — per-profile lock prevents parallel runs of the same profile.
73	#   Global concurrent limit prevents too many simultaneous rsync processes.
74	#
75	# Notification Validated
76	#   platform_require_cmd confirms the notify script is present before use.
77	#
78	# ==============================================================================================
79	# CONFIGURATION
80	# ==============================================================================================
81	#
82	# master.conf
83	#
84	#   RSYNC_ENABLED
85	#       Global on/off toggle for all rsync operations. (default: true)
86	#
87	#   DEFAULT_RSYNC_OPTS
88	#       Base rsync flags for unproiled shares. Note: --delete is intentionally absent —
89	#       media shares spread files only, arr cleanup scripts own deletions. Profile-specific
90	#       opts set --delete explicitly where needed.
91	#
92	#   BW_LIMIT
93	#       Default bandwidth cap in KB/s when no PROFILE_BW_LIMIT is set. (default: 0 = unlimited)
94	#
95	#   RETRY_COUNT
96	#       Default retry attempts on rsync failure. (default: 3)
97	#
98	#   SLEEP
99	#       Default seconds between retry attempts. (default: 60)
100	#
101	#   ROOTFS_WARN_PCT
102	#       Abort threshold for remote rootfs percentage full. (default: 75)
103	#
104	#   PROFILES["profile_KEY"]
105	#       Profile definitions — one entry per PROFILE_* key per profile name.
106	#       See OPERATIONAL MODEL above for all supported keys.
107	#
108	#   BANDWIDTH_LOG / BANDWIDTH_WARN_GB
109	#       Shared with bandwidth_monitor.sh — set once, used by both.
110	#
111	# ==============================================================================================
112	# RUNTIME MODES
113	# ==============================================================================================
114	#
115	# rsync.sh /path/to/share
116	#     Sync the given path to the remote. Profile inferred from directory basename.
117	#
118	# rsync.sh /path/to/share --profile=name
119	#     Sync with explicit profile override — bypasses basename inference.
120	#
121	# rsync.sh /path/to/share --dry-run
122	#     Run all pre-flight checks and show what rsync would transfer. No transfer,
123	#     no container stops.
124	#
125	# rsync.sh /path/to/share --status
126	#     Show resolved profile, remote identity, and configuration. Then exit.
127	#
128	# rsync.sh /path/to/share --log
129	#     Verbose output throughout — every decision logged.
130	#
131	# ==============================================================================================
132
133	SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
134
135	source "$SCRIPT_DIR/../load_config.sh"
136
137	# ── Separate positional directory arg from flags ───────────────────────────────────────────────
138	DIRECTORY=""
139	PROFILE_OVERRIDE=""
140	RAW_ARGS=()
141
142	for ARG in "$@"; do
143	    case "$ARG" in
144	        --profile=*) PROFILE_OVERRIDE="${ARG#--profile=}" ;;
145	        --*|*=*)     RAW_ARGS+=("$ARG") ;;
146	        *)           [[ -z "$DIRECTORY" ]] && DIRECTORY="$ARG" || RAW_ARGS+=("$ARG") ;;
147	    esac
148	done
149
150	parse_args "${RAW_ARGS[@]}"
151
152	[[ -z "$DIRECTORY" ]] && {
153	    error "No directory specified"
154	    error "Usage: rsync.sh <dir> [--dry-run] [--log] [--profile=name]"
155	    exit 1
156	}
157
158	# ==============================================================================================
159	# ━━━ Setup ━━━
160	# ==============================================================================================
161	if [[ "$EUID" -ne 0 ]]; then
162	    error "Must be run as root"
163	    exit 1
164	fi
165
166	acquire_lock
167
168	if ! command -v docker &>/dev/null; then
169	    error "Docker command not found"
170	    exit 1
171	fi
172
173	platform_require_cmd \
174	    "/usr/local/emhttp/plugins/dynamix/scripts/notify" \
175	    "" "" \
176	    "unRAID notify script" || warn "unRAID notify script not found — native notifications disabled"
177
178	detect_hosts
179
180	# Tier 1 global gate — Tier 2 (per-orchestrator) checked by caller
181	if ! check_rsync_enabled; then
182	    warn "RSYNC_ENABLED=false — exiting cleanly"
183	    exit 0
184	fi
185
186	# Blocklist gate — refuse to sync with a partner blocked after offboard
187	BLOCKLIST_FILE="${PARTNERSHIP_BLOCKLIST_FILE:-${STATE_DIR:-/boot/config}/partnership_blocklist.db}"
188	if [[ -f "$BLOCKLIST_FILE" ]] && grep -q "^${REMOTE_SERVER_NAME}|" "$BLOCKLIST_FILE" 2>/dev/null; then
189	    error "Rsync blocked — $REMOTE_SERVER_NAME is on the partnership blocklist"
190	    error "Re-onboard the partnership to restore access: partnership_manager.sh --onboard"
191	    exit 1
192	fi
193
194	resolve_remote_ip
195
196	# ── Profile inference ─────────────────────────────────────────────────────────────────────────
197	if [[ -n "$PROFILE_OVERRIDE" ]]; then
198	    PROFILE_NAME="$PROFILE_OVERRIDE"
199	    log "Profile override: $PROFILE_NAME"
200	else
201	    PROFILE_NAME=$(basename "$DIRECTORY" | tr '[:upper:]' '[:lower:]')
202	    log "Profile inferred: $PROFILE_NAME"
203	fi
204
205	# Acquire per-profile lock and check global concurrent limit
206	acquire_rsync_lock "$PROFILE_NAME"
207
208	# Tee all output to a live log file for the Varaverk UI
209	VV_LIVE_LOG="/tmp/unraid_locks/rsync_${PROFILE_NAME}.log"
210	VV_LAST_LOG="/tmp/unraid_locks/rsync_${PROFILE_NAME}.last.log"
211	: > "$VV_LIVE_LOG"
212	exec 1> >(tee -a "$VV_LIVE_LOG") 2>&1
213
214	# ── Load profile settings ─────────────────────────────────────────────────────────────────────
215	BW_LIMIT=${PROFILE_BW_LIMIT[$PROFILE_NAME]:-$BW_LIMIT}
216	RETRY_COUNT=${PROFILE_RETRY_COUNT[$PROFILE_NAME]:-$RETRY_COUNT}
217	SLEEP=${PROFILE_SLEEP[$PROFILE_NAME]:-$SLEEP}
218	CONTAINER_DELAY=${PROFILE_CONTAINER_DELAY[$PROFILE_NAME]:-$CONTAINER_DELAY}
219
220	read -r -a CRITICAL_CONTAINER_NAMES       <<< "${PROFILE_CRITICAL_CONTAINER_NAMES[$PROFILE_NAME]:-}"
221	read -r -a DELAYED_CONTAINERS             <<< "${PROFILE_DELAYED_CONTAINERS[$PROFILE_NAME]:-}"
222	read -r -a EXCLUDE_DIRS                   <<< "${PROFILE_EXCLUDE_DIRS[$PROFILE_NAME]:-}"
223	read -r -a REMOTE_RESTART_CONTAINERS      <<< "${PROFILE_REMOTE_RESTART_CONTAINERS[$PROFILE_NAME]:-}"
224
225	# Local containers use same names as remote (mirrored naming scheme)
226	LOCAL_CRITICAL_CONTAINER_NAMES=("${CRITICAL_CONTAINER_NAMES[@]}")
227
228	log "$ICON_GEAR Config: profile=${PROFILE_NAME} bw-limit=${BW_LIMIT}KB/s retry=${RETRY_COUNT} sleep=${SLEEP}s container-delay=${CONTAINER_DELAY}s"
229	log "$ICON_GEAR Containers: critical=${CRITICAL_CONTAINER_NAMES[*]:-none} delayed=${DELAYED_CONTAINERS[*]:-none} remote-restart=${REMOTE_RESTART_CONTAINERS[*]:-none}"
230
231	[[ "$SHOW_STATUS" == true ]] && show_status && exit 0
232	[[ "$DRY_RUN" == true ]] && warn "DRY RUN — no changes will be made"
233
234	# ==============================================================================================
235	# ━━━ Pre-flight Checks ━━━
236	# ==============================================================================================
237	echo ""
238	echo "━━━ $ICON_SHIELD Pre-flight ━━━"
239
240	# Disk temp — before touching remote or moving data
241	# Exit 1 = skip this profile | Exit 2 = abort all remaining profiles
242	check_local_disk_temps
243	TEMP_RESULT=$?
244	if [[ "$TEMP_RESULT" -eq 2 ]]; then
245	    error "Drive temps CRITICAL — aborting all remaining syncs"
246	    exit 2
247	elif [[ "$TEMP_RESULT" -eq 1 ]]; then
248	    warn "Drive temps high — skipping profile [$PROFILE_NAME]"
249	    exit 1
250	else
251	    log "Drive temps OK — $TEMP_CHECK_RESULT"
252	fi
253
254	# Version parity — refuse if servers on incompatible unRAID versions
255	check_unraid_version_parity || exit 1
256
257	check_connectivity
258	check_remote_rootfs
259	check_remote_share "$DIRECTORY"
260	check_remote_disks "$DIRECTORY"
261
262	# Remote Docker daemon — check before attempting container operations
263	if [[ ${#CRITICAL_CONTAINER_NAMES[@]} -gt 0 ]] || [[ ${#REMOTE_RESTART_CONTAINERS[@]} -gt 0 ]]; then
264	    check_remote_docker_daemon || {
265	        warn "Remote Docker daemon unresponsive — skipping container operations"
266	        warn "Proceeding with rsync only — containers will not be stopped or restarted"
267	        CRITICAL_CONTAINER_NAMES=()
268	        LOCAL_CRITICAL_CONTAINER_NAMES=()
269	        REMOTE_RESTART_CONTAINERS=()
270	    }
271	fi
272
273	# ==============================================================================================
274	# ━━━ Stop Containers ━━━
275	# ==============================================================================================
276	if [[ ${#CRITICAL_CONTAINER_NAMES[@]} -gt 0 || ${#LOCAL_CRITICAL_CONTAINER_NAMES[@]} -gt 0 ]]; then
277	    echo ""
278	    echo "━━━ $ICON_STOP $ICON_CONTAINERS Stop Containers ━━━"
279	    if [[ "$DRY_RUN" == true ]]; then
280	        for c in "${LOCAL_CRITICAL_CONTAINER_NAMES[@]:-}"; do
281	            [[ -n "$c" ]] && warn "DRY RUN — would stop local: $c"
282	        done
283	        for c in "${CRITICAL_CONTAINER_NAMES[@]:-}"; do
284	            [[ -n "$c" ]] && warn "DRY RUN — would stop remote: $c"
285	        done
286	    else
287	        # Local first — flush local databases before pushing
288	        stop_local_containers
289	        # Remote next — prevent writes while receiving
290	        stop_containers
291	    fi
292	fi
293
294	# ==============================================================================================
295	# ━━━ Transfer ━━━
296	# ==============================================================================================
297	echo ""
298	echo "━━━ $ICON_SYNC Transfer ━━━"
299	echo "$ICON_RUN  Source:   $DIRECTORY"
300	echo "$ICON_NET  Remote:   $REMOTE_SERVER:$DIRECTORY"
301	echo "$ICON_GEAR Profile:  $PROFILE_NAME"
302	echo "$ICON_HOST Identity: $MY_ID → $REMOTE_ID"
303	echo ""
304
305	get_rsync_opts
306
307	# Append profile excludes
308	for ex in "${EXCLUDE_DIRS[@]:-}"; do
309	    [[ -n "$ex" ]] && RSYNC_OPTS+=(--exclude="$ex")
310	done
311
312	# Add --stats to capture bytes transferred for bandwidth logging
313	RSYNC_OPTS+=(--stats)
314
315	[[ "$DRY_RUN" == true ]] && RSYNC_OPTS+=("--dry-run")
316
317	START=$(date +%s)
318	RSYNC_SUCCESS=false
319	BYTES_TRANSFERRED=0
320	ATTEMPT=0
321
322	for (( ATTEMPT=1; ATTEMPT<=RETRY_COUNT; ATTEMPT++ )); do
323	    log "$ICON_RETRY Attempt $ATTEMPT of $RETRY_COUNT..."
324	    echo "$ICON_SYNC  Rsync running — this may take a while..."
325
326	    RSYNC_OUTPUT=$(rsync "${RSYNC_OPTS[@]}" \
327	        -e "ssh -i $SSH_KEY -T -o Compression=no -o IPQoS=throughput" \
328	        "$DIRECTORY" "root@${REMOTE_SERVER}:$(dirname "$DIRECTORY")/" 2>&1)
329
330	    RSYNC_EXIT=$?
331
332	    if [[ "$RSYNC_EXIT" -eq 0 ]]; then
333	        # Parse bytes transferred from --stats output
334	        BYTES_TRANSFERRED=$(echo "$RSYNC_OUTPUT" | \
335	            awk '/Total transferred file size:/{gsub(/,/,"",$NF); gsub(/[^0-9]/,"",$NF); print $NF+0}')
336	        BYTES_TRANSFERRED="${BYTES_TRANSFERRED:-0}"
337
338	        log "$ICON_DONE Rsync complete — $BYTES_TRANSFERRED bytes transferred"
339	        RSYNC_SUCCESS=true
340	        break
341	    else
342	        warn "$ICON_RETRY Rsync failed (attempt $ATTEMPT/$RETRY_COUNT)"
343	        log "Exit code: $RSYNC_EXIT"
344	        if [[ "$ATTEMPT" -lt "$RETRY_COUNT" ]]; then
345	            log "Retrying in ${SLEEP}s..."
346	            sleep "$SLEEP"
347	        fi
348	    fi
349	done
350
351	# ==============================================================================================
352	# ━━━ Start Containers ━━━
353	# ==============================================================================================
354	if [[ ${#CRITICAL_CONTAINER_NAMES[@]} -gt 0 || ${#LOCAL_CRITICAL_CONTAINER_NAMES[@]} -gt 0 ]]; then
355	    echo ""
356	    echo "━━━ $ICON_START $ICON_CONTAINERS Start Containers ━━━"
357	    if [[ "$DRY_RUN" == true ]]; then
358	        for c in "${CRITICAL_CONTAINER_NAMES[@]:-}"; do
359	            [[ -n "$c" ]] && warn "DRY RUN — would start remote: $c"
360	        done
361	        for c in "${LOCAL_CRITICAL_CONTAINER_NAMES[@]:-}"; do
362	            [[ -n "$c" ]] && warn "DRY RUN — would start local: $c"
363	        done
364	    else
365	        # Remote first — can be coming up while local restarts
366	        start_containers
367	        # Local next
368	        start_local_containers
369	    fi
370	fi
371
372	# ==============================================================================================
373	# ━━━ Remote Restart (dirty sync profiles) ━━━
374	# ==============================================================================================
375	# For dirty sync profiles (critical-fallback, emby-fallback) — restart containers on remote
376	# that were running before sync so they pick up config changes from the dirty sync window.
377	# Was running → restart. Was stopped → leave stopped.
378	if [[ ${#REMOTE_RESTART_CONTAINERS[@]} -gt 0 ]]; then
379	    echo ""
380	    echo "━━━ $ICON_START $ICON_CONTAINERS Remote Restart (post dirty sync) ━━━"
381	    log "Restarting configured containers on $REMOTE_SERVER_NAME..."
382
383	    for container in "${REMOTE_RESTART_CONTAINERS[@]}"; do
384	        [[ -z "$container" ]] && continue
385
386	        # Check if container was running before sync (still tracked via RUNNING_CONTAINERS)
387	        WAS_RUNNING=false
388	        for prev in "${RUNNING_CONTAINERS[@]:-}"; do
389	            [[ "$prev" == "$container" ]] && WAS_RUNNING=true && break
390	        done
391
392	        if [[ "$WAS_RUNNING" == false ]]; then
393	            # Not in stop list — check current remote state
394	            REMOTE_STATUS=$(timeout 15 ssh -i "$SSH_KEY" \
395	                -o ConnectTimeout=10 root@"$REMOTE_SERVER" \
396	                "docker inspect -f '{{.State.Running}}' $container 2>/dev/null" 2>/dev/null)
397	            [[ "$REMOTE_STATUS" != "true" ]] && \
398	                log "$container not running on $REMOTE_SERVER_NAME — skipping remote restart" && \
399	                continue
400	        fi
401
402	        if [[ "$DRY_RUN" == true ]]; then
403	            warn "DRY RUN — would restart $container on $REMOTE_SERVER_NAME"
404	            continue
405	        fi
406
407	        timeout 15 ssh -i "$SSH_KEY" -o ConnectTimeout=10 root@"$REMOTE_SERVER" \
408	            "docker restart $container" >/dev/null 2>&1 && \
409	            log "$ICON_STARTED $container restarted on $REMOTE_SERVER_NAME ✅" || \
410	            warn "Failed to restart $container on $REMOTE_SERVER_NAME"
411	    done
412	fi
413
414	END=$(date +%s)
415	DURATION=$(( END - START ))
416
417	# ==============================================================================================
418	# ━━━ Bandwidth Logging ━━━
419	# ==============================================================================================
420	# Logs to bandwidth_monitor.sh — new format includes bytes transferred and warn flag.
421	# Only logs on actual runs (not dry-run) and only when bandwidth_monitor.sh exists.
422	BANDWIDTH_MONITOR="$SCRIPT_DIR/../Monitors/bandwidth_monitor.sh"
423
424	if [[ "$DRY_RUN" == false ]] && [[ -f "$BANDWIDTH_MONITOR" ]]; then
425	    STATUS="success"
426	    [[ "$RSYNC_SUCCESS" == false ]] && STATUS="failed"
427	    bash "$BANDWIDTH_MONITOR" --log-transfer \
428	        "$PROFILE_NAME" "$DURATION" "$STATUS" "$BYTES_TRANSFERRED"
429	    log "$ICON_BANDWIDTH Transfer logged to bandwidth monitor ($BYTES_TRANSFERRED bytes)"
430	fi
431
432	# ==============================================================================================
433	# ━━━ Summary ━━━
434	# ==============================================================================================
435	echo ""
436	echo "━━━━━ $ICON_SUMMARY RSYNC SUMMARY ━━━━━"
437	echo "$ICON_HOST    Identity:  $MY_ID ($LOCAL_SERVER_NAME)"
438	echo "$ICON_RUN     Directory: $DIRECTORY"
439	echo "$ICON_GEAR    Profile:   $PROFILE_NAME"
440	echo "$ICON_TIME    Duration:  $(format_duration $DURATION)"
441	[[ "$BYTES_TRANSFERRED" -gt 0 ]] && \
442	    echo "$ICON_BANDWIDTH Transferred: $(awk "BEGIN {printf \"%.2fGB\", $BYTES_TRANSFERRED / 1073741824}")"
443
444	if [[ "$DRY_RUN" == true ]]; then
445	    warn "DRY RUN — no changes made"
446	elif [[ "$RSYNC_SUCCESS" == true ]]; then
447	    echo "$ICON_DONE Status: $ICON_SUCCESS DONE"
448	else
449	    echo "$ICON_ERROR Status: FAILED after $RETRY_COUNT attempts"
450	    notify "Rsync FAILED — $DIRECTORY ($PROFILE_NAME) after $RETRY_COUNT attempts on $(hostname)" \
451	        "Rsync" "warning"
452	fi
453	echo "━━━━━━━━━━━━━━━━━━━━━━━"
454
455	# Flush tee and preserve log for UI — close both ends of the pipe so tee gets EOF
456	exec 1>&- 2>&-; wait
457	cp "$VV_LIVE_LOG" "$VV_LAST_LOG" 2>/dev/null
458	rm -f "$VV_LIVE_LOG"
459
460	[[ "$RSYNC_SUCCESS" == false ]] && [[ "$DRY_RUN" == false ]] && exit 1
461	exit 0

#!/bin/bash
# ==============================================================================================
# ============================= Rsync Core Script ==============================================
# ==============================================================================================
#
# PURPOSE
# ─────────────────────────────────────────────────────────────────────────────
# Core rsync engine for the two-server ecosystem. Called per share or per
# appdata profile by orchestrators (daily_sync_maintenance, weekly_sync_maintenance,
# critical_sync_maintenance) and directly for manual or scheduled dirty syncs.
#
# Profile is inferred from the directory basename (lowercased). Override with
# --profile=name for explicit selection. If no profile matches, global defaults
# from master.conf apply and no containers are stopped.
#
# After each sync, logs transfer data to bandwidth_monitor.sh for the weekly
# bandwidth report. Silent on success — only failures produce visible output.
#
# ==============================================================================================
# OPERATIONAL MODEL
# ==============================================================================================
#
# Profiles define per-share behavior:
#   PROFILE_CRITICAL_CONTAINER_NAMES  — containers stopped on both servers before sync
#   PROFILE_DELAYED_CONTAINERS        — containers with a delay before restart after sync
#   PROFILE_CONTAINER_DELAY           — seconds before delayed containers start
#   PROFILE_RSYNC_OPTS                — rsync flags (does not inherit DEFAULT_RSYNC_OPTS)
#   PROFILE_BW_LIMIT                  — bandwidth limit in KB/s
#   PROFILE_RETRY_COUNT               — retry attempts on failure
#   PROFILE_SLEEP                     — seconds between retry attempts
#   PROFILE_EXCLUDE_DIRS              — paths excluded from transfer
#   PROFILE_REMOTE_RESTART_CONTAINERS — containers restarted on remote after dirty sync
#                                       Was running → restart. Was stopped → leave stopped.
#
# Two-tier rsync enable/disable:
#   Tier 1: RSYNC_ENABLED=false → all rsync stops immediately (checked by this script)
#   Tier 2: per-orchestrator flag (DAILY_RSYNC_ENABLED etc.) → checked by caller
#
# Bandwidth logging: after each sync, logs profile/duration/status/bytes to
# bandwidth_monitor.sh --log-transfer. Bytes captured from rsync --stats via awk
# using version-stable field names.
#
# ==============================================================================================
# OPERATIONAL SAFEGUARDS
# ==============================================================================================
#
# Global Rsync Gate
#   check_rsync_enabled() — RSYNC_ENABLED=false exits cleanly before any operation.
#
# Partnership Blocklist
#   Refuses to sync if REMOTE_SERVER_NAME appears in the partnership blocklist.
#   Written at offboard — prevents stale access after a partnership ends.
#
# Version Parity
#   check_unraid_version_parity — refuses sync if servers on incompatible unRAID versions.
#
# Remote Health Pre-flights
#   check_connectivity() — Tailscale IP reachable before any SSH
#   check_remote_rootfs() — aborts if remote rootfs exceeds ROOTFS_WARN_PCT
#   check_remote_share() — aborts if target directory missing or empty on remote
#   check_remote_disks() — verifies all backing disks online on remote
#
# Drive Temperature Check
#   check_local_disk_temps() — runs before any transfer. Exit 1 = skip this profile,
#   exit 2 = abort all remaining syncs (CRITICAL temperature).
#
# Remote Docker Daemon Check
#   check_remote_docker_daemon — verified before any container stop/start operations.
#   If daemon unresponsive: container operations skipped, rsync proceeds without stopping.
#
# Per-Profile Concurrency Lock
#   acquire_rsync_lock() — per-profile lock prevents parallel runs of the same profile.
#   Global concurrent limit prevents too many simultaneous rsync processes.
#
# Notification Validated
#   platform_require_cmd confirms the notify script is present before use.
#
# ==============================================================================================
# CONFIGURATION
# ==============================================================================================
#
# master.conf
#
#   RSYNC_ENABLED
#       Global on/off toggle for all rsync operations. (default: true)
#
#   DEFAULT_RSYNC_OPTS
#       Base rsync flags for unproiled shares. Note: --delete is intentionally absent —
#       media shares spread files only, arr cleanup scripts own deletions. Profile-specific
#       opts set --delete explicitly where needed.
#
#   BW_LIMIT
#       Default bandwidth cap in KB/s when no PROFILE_BW_LIMIT is set. (default: 0 = unlimited)
#
#   RETRY_COUNT
#       Default retry attempts on rsync failure. (default: 3)
#
#   SLEEP
#       Default seconds between retry attempts. (default: 60)
#
#   ROOTFS_WARN_PCT
#       Abort threshold for remote rootfs percentage full. (default: 75)
#
#   PROFILES["profile_KEY"]
#       Profile definitions — one entry per PROFILE_* key per profile name.
#       See OPERATIONAL MODEL above for all supported keys.
#
#   BANDWIDTH_LOG / BANDWIDTH_WARN_GB
#       Shared with bandwidth_monitor.sh — set once, used by both.
#
# ==============================================================================================
# RUNTIME MODES
# ==============================================================================================
#
# rsync.sh /path/to/share
#     Sync the given path to the remote. Profile inferred from directory basename.
#
# rsync.sh /path/to/share --profile=name
#     Sync with explicit profile override — bypasses basename inference.
#
# rsync.sh /path/to/share --dry-run
#     Run all pre-flight checks and show what rsync would transfer. No transfer,
#     no container stops.
#
# rsync.sh /path/to/share --status
#     Show resolved profile, remote identity, and configuration. Then exit.
#
# rsync.sh /path/to/share --log
#     Verbose output throughout — every decision logged.
#
# ==============================================================================================

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

source "$SCRIPT_DIR/../load_config.sh"

# ── Separate positional directory arg from flags ───────────────────────────────────────────────
DIRECTORY=""
PROFILE_OVERRIDE=""
RAW_ARGS=()

for ARG in "$@"; do
    case "$ARG" in
        --profile=*) PROFILE_OVERRIDE="${ARG#--profile=}" ;;
        --*|*=*)     RAW_ARGS+=("$ARG") ;;
        *)           [[ -z "$DIRECTORY" ]] && DIRECTORY="$ARG" || RAW_ARGS+=("$ARG") ;;
    esac
done

parse_args "${RAW_ARGS[@]}"

[[ -z "$DIRECTORY" ]] && {
    error "No directory specified"
    error "Usage: rsync.sh <dir> [--dry-run] [--log] [--profile=name]"
    exit 1
}

# ==============================================================================================
# ━━━ Setup ━━━
# ==============================================================================================
if [[ "$EUID" -ne 0 ]]; then
    error "Must be run as root"
    exit 1
fi

acquire_lock

if ! command -v docker &>/dev/null; then
    error "Docker command not found"
    exit 1
fi

platform_require_cmd \
    "/usr/local/emhttp/plugins/dynamix/scripts/notify" \
    "" "" \
    "unRAID notify script" || warn "unRAID notify script not found — native notifications disabled"

detect_hosts

# Tier 1 global gate — Tier 2 (per-orchestrator) checked by caller
if ! check_rsync_enabled; then
    warn "RSYNC_ENABLED=false — exiting cleanly"
    exit 0
fi

# Blocklist gate — refuse to sync with a partner blocked after offboard
BLOCKLIST_FILE="${PARTNERSHIP_BLOCKLIST_FILE:-${STATE_DIR:-/boot/config}/partnership_blocklist.db}"
if [[ -f "$BLOCKLIST_FILE" ]] && grep -q "^${REMOTE_SERVER_NAME}|" "$BLOCKLIST_FILE" 2>/dev/null; then
    error "Rsync blocked — $REMOTE_SERVER_NAME is on the partnership blocklist"
    error "Re-onboard the partnership to restore access: partnership_manager.sh --onboard"
    exit 1
fi

resolve_remote_ip

# ── Profile inference ─────────────────────────────────────────────────────────────────────────
if [[ -n "$PROFILE_OVERRIDE" ]]; then
    PROFILE_NAME="$PROFILE_OVERRIDE"
    log "Profile override: $PROFILE_NAME"
else
    PROFILE_NAME=$(basename "$DIRECTORY" | tr '[:upper:]' '[:lower:]')
    log "Profile inferred: $PROFILE_NAME"
fi

# Acquire per-profile lock and check global concurrent limit
acquire_rsync_lock "$PROFILE_NAME"

# Tee all output to a live log file for the Varaverk UI
VV_LIVE_LOG="/tmp/unraid_locks/rsync_${PROFILE_NAME}.log"
VV_LAST_LOG="/tmp/unraid_locks/rsync_${PROFILE_NAME}.last.log"
: > "$VV_LIVE_LOG"
exec 1> >(tee -a "$VV_LIVE_LOG") 2>&1

# ── Load profile settings ─────────────────────────────────────────────────────────────────────
BW_LIMIT=${PROFILE_BW_LIMIT[$PROFILE_NAME]:-$BW_LIMIT}
RETRY_COUNT=${PROFILE_RETRY_COUNT[$PROFILE_NAME]:-$RETRY_COUNT}
SLEEP=${PROFILE_SLEEP[$PROFILE_NAME]:-$SLEEP}
CONTAINER_DELAY=${PROFILE_CONTAINER_DELAY[$PROFILE_NAME]:-$CONTAINER_DELAY}

read -r -a CRITICAL_CONTAINER_NAMES       <<< "${PROFILE_CRITICAL_CONTAINER_NAMES[$PROFILE_NAME]:-}"
read -r -a DELAYED_CONTAINERS             <<< "${PROFILE_DELAYED_CONTAINERS[$PROFILE_NAME]:-}"
read -r -a EXCLUDE_DIRS                   <<< "${PROFILE_EXCLUDE_DIRS[$PROFILE_NAME]:-}"
read -r -a REMOTE_RESTART_CONTAINERS      <<< "${PROFILE_REMOTE_RESTART_CONTAINERS[$PROFILE_NAME]:-}"

# Local containers use same names as remote (mirrored naming scheme)
LOCAL_CRITICAL_CONTAINER_NAMES=("${CRITICAL_CONTAINER_NAMES[@]}")

log "$ICON_GEAR Config: profile=${PROFILE_NAME} bw-limit=${BW_LIMIT}KB/s retry=${RETRY_COUNT} sleep=${SLEEP}s container-delay=${CONTAINER_DELAY}s"
log "$ICON_GEAR Containers: critical=${CRITICAL_CONTAINER_NAMES[*]:-none} delayed=${DELAYED_CONTAINERS[*]:-none} remote-restart=${REMOTE_RESTART_CONTAINERS[*]:-none}"

[[ "$SHOW_STATUS" == true ]] && show_status && exit 0
[[ "$DRY_RUN" == true ]] && warn "DRY RUN — no changes will be made"

# ==============================================================================================
# ━━━ Pre-flight Checks ━━━
# ==============================================================================================
echo ""
echo "━━━ $ICON_SHIELD Pre-flight ━━━"

# Disk temp — before touching remote or moving data
# Exit 1 = skip this profile | Exit 2 = abort all remaining profiles
check_local_disk_temps
TEMP_RESULT=$?
if [[ "$TEMP_RESULT" -eq 2 ]]; then
    error "Drive temps CRITICAL — aborting all remaining syncs"
    exit 2
elif [[ "$TEMP_RESULT" -eq 1 ]]; then
    warn "Drive temps high — skipping profile [$PROFILE_NAME]"
    exit 1
else
    log "Drive temps OK — $TEMP_CHECK_RESULT"
fi

# Version parity — refuse if servers on incompatible unRAID versions
check_unraid_version_parity || exit 1

check_connectivity
check_remote_rootfs
check_remote_share "$DIRECTORY"
check_remote_disks "$DIRECTORY"

# Remote Docker daemon — check before attempting container operations
if [[ ${#CRITICAL_CONTAINER_NAMES[@]} -gt 0 ]] || [[ ${#REMOTE_RESTART_CONTAINERS[@]} -gt 0 ]]; then
    check_remote_docker_daemon || {
        warn "Remote Docker daemon unresponsive — skipping container operations"
        warn "Proceeding with rsync only — containers will not be stopped or restarted"
        CRITICAL_CONTAINER_NAMES=()
        LOCAL_CRITICAL_CONTAINER_NAMES=()
        REMOTE_RESTART_CONTAINERS=()
    }
fi

# ==============================================================================================
# ━━━ Stop Containers ━━━
# ==============================================================================================
if [[ ${#CRITICAL_CONTAINER_NAMES[@]} -gt 0 || ${#LOCAL_CRITICAL_CONTAINER_NAMES[@]} -gt 0 ]]; then
    echo ""
    echo "━━━ $ICON_STOP $ICON_CONTAINERS Stop Containers ━━━"
    if [[ "$DRY_RUN" == true ]]; then
        for c in "${LOCAL_CRITICAL_CONTAINER_NAMES[@]:-}"; do
            [[ -n "$c" ]] && warn "DRY RUN — would stop local: $c"
        done
        for c in "${CRITICAL_CONTAINER_NAMES[@]:-}"; do
            [[ -n "$c" ]] && warn "DRY RUN — would stop remote: $c"
        done
    else
        # Local first — flush local databases before pushing
        stop_local_containers
        # Remote next — prevent writes while receiving
        stop_containers
    fi
fi

# ==============================================================================================
# ━━━ Transfer ━━━
# ==============================================================================================
echo ""
echo "━━━ $ICON_SYNC Transfer ━━━"
echo "$ICON_RUN  Source:   $DIRECTORY"
echo "$ICON_NET  Remote:   $REMOTE_SERVER:$DIRECTORY"
echo "$ICON_GEAR Profile:  $PROFILE_NAME"
echo "$ICON_HOST Identity: $MY_ID → $REMOTE_ID"
echo ""

get_rsync_opts

# Append profile excludes
for ex in "${EXCLUDE_DIRS[@]:-}"; do
    [[ -n "$ex" ]] && RSYNC_OPTS+=(--exclude="$ex")
done

# Add --stats to capture bytes transferred for bandwidth logging
RSYNC_OPTS+=(--stats)

[[ "$DRY_RUN" == true ]] && RSYNC_OPTS+=("--dry-run")

START=$(date +%s)
RSYNC_SUCCESS=false
BYTES_TRANSFERRED=0
ATTEMPT=0

for (( ATTEMPT=1; ATTEMPT<=RETRY_COUNT; ATTEMPT++ )); do
    log "$ICON_RETRY Attempt $ATTEMPT of $RETRY_COUNT..."
    echo "$ICON_SYNC  Rsync running — this may take a while..."

    RSYNC_OUTPUT=$(rsync "${RSYNC_OPTS[@]}" \
        -e "ssh -i $SSH_KEY -T -o Compression=no -o IPQoS=throughput" \
        "$DIRECTORY" "root@${REMOTE_SERVER}:$(dirname "$DIRECTORY")/" 2>&1)

    RSYNC_EXIT=$?

    if [[ "$RSYNC_EXIT" -eq 0 ]]; then
        # Parse bytes transferred from --stats output
        BYTES_TRANSFERRED=$(echo "$RSYNC_OUTPUT" | \
            awk '/Total transferred file size:/{gsub(/,/,"",$NF); gsub(/[^0-9]/,"",$NF); print $NF+0}')
        BYTES_TRANSFERRED="${BYTES_TRANSFERRED:-0}"

        log "$ICON_DONE Rsync complete — $BYTES_TRANSFERRED bytes transferred"
        RSYNC_SUCCESS=true
        break
    else
        warn "$ICON_RETRY Rsync failed (attempt $ATTEMPT/$RETRY_COUNT)"
        log "Exit code: $RSYNC_EXIT"
        if [[ "$ATTEMPT" -lt "$RETRY_COUNT" ]]; then
            log "Retrying in ${SLEEP}s..."
            sleep "$SLEEP"
        fi
    fi
done

# ==============================================================================================
# ━━━ Start Containers ━━━
# ==============================================================================================
if [[ ${#CRITICAL_CONTAINER_NAMES[@]} -gt 0 || ${#LOCAL_CRITICAL_CONTAINER_NAMES[@]} -gt 0 ]]; then
    echo ""
    echo "━━━ $ICON_START $ICON_CONTAINERS Start Containers ━━━"
    if [[ "$DRY_RUN" == true ]]; then
        for c in "${CRITICAL_CONTAINER_NAMES[@]:-}"; do
            [[ -n "$c" ]] && warn "DRY RUN — would start remote: $c"
        done
        for c in "${LOCAL_CRITICAL_CONTAINER_NAMES[@]:-}"; do
            [[ -n "$c" ]] && warn "DRY RUN — would start local: $c"
        done
    else
        # Remote first — can be coming up while local restarts
        start_containers
        # Local next
        start_local_containers
    fi
fi

# ==============================================================================================
# ━━━ Remote Restart (dirty sync profiles) ━━━
# ==============================================================================================
# For dirty sync profiles (critical-fallback, emby-fallback) — restart containers on remote
# that were running before sync so they pick up config changes from the dirty sync window.
# Was running → restart. Was stopped → leave stopped.
if [[ ${#REMOTE_RESTART_CONTAINERS[@]} -gt 0 ]]; then
    echo ""
    echo "━━━ $ICON_START $ICON_CONTAINERS Remote Restart (post dirty sync) ━━━"
    log "Restarting configured containers on $REMOTE_SERVER_NAME..."

    for container in "${REMOTE_RESTART_CONTAINERS[@]}"; do
        [[ -z "$container" ]] && continue

        # Check if container was running before sync (still tracked via RUNNING_CONTAINERS)
        WAS_RUNNING=false
        for prev in "${RUNNING_CONTAINERS[@]:-}"; do
            [[ "$prev" == "$container" ]] && WAS_RUNNING=true && break
        done

        if [[ "$WAS_RUNNING" == false ]]; then
            # Not in stop list — check current remote state
            REMOTE_STATUS=$(timeout 15 ssh -i "$SSH_KEY" \
                -o ConnectTimeout=10 root@"$REMOTE_SERVER" \
                "docker inspect -f '{{.State.Running}}' $container 2>/dev/null" 2>/dev/null)
            [[ "$REMOTE_STATUS" != "true" ]] && \
                log "$container not running on $REMOTE_SERVER_NAME — skipping remote restart" && \
                continue
        fi

        if [[ "$DRY_RUN" == true ]]; then
            warn "DRY RUN — would restart $container on $REMOTE_SERVER_NAME"
            continue
        fi

        timeout 15 ssh -i "$SSH_KEY" -o ConnectTimeout=10 root@"$REMOTE_SERVER" \
            "docker restart $container" >/dev/null 2>&1 && \
            log "$ICON_STARTED $container restarted on $REMOTE_SERVER_NAME ✅" || \
            warn "Failed to restart $container on $REMOTE_SERVER_NAME"
    done
fi

END=$(date +%s)
DURATION=$(( END - START ))

# ==============================================================================================
# ━━━ Bandwidth Logging ━━━
# ==============================================================================================
# Logs to bandwidth_monitor.sh — new format includes bytes transferred and warn flag.
# Only logs on actual runs (not dry-run) and only when bandwidth_monitor.sh exists.
BANDWIDTH_MONITOR="$SCRIPT_DIR/../Monitors/bandwidth_monitor.sh"

if [[ "$DRY_RUN" == false ]] && [[ -f "$BANDWIDTH_MONITOR" ]]; then
    STATUS="success"
    [[ "$RSYNC_SUCCESS" == false ]] && STATUS="failed"
    bash "$BANDWIDTH_MONITOR" --log-transfer \
        "$PROFILE_NAME" "$DURATION" "$STATUS" "$BYTES_TRANSFERRED"
    log "$ICON_BANDWIDTH Transfer logged to bandwidth monitor ($BYTES_TRANSFERRED bytes)"
fi

# ==============================================================================================
# ━━━ Summary ━━━
# ==============================================================================================
echo ""
echo "━━━━━ $ICON_SUMMARY RSYNC SUMMARY ━━━━━"
echo "$ICON_HOST    Identity:  $MY_ID ($LOCAL_SERVER_NAME)"
echo "$ICON_RUN     Directory: $DIRECTORY"
echo "$ICON_GEAR    Profile:   $PROFILE_NAME"
echo "$ICON_TIME    Duration:  $(format_duration $DURATION)"
[[ "$BYTES_TRANSFERRED" -gt 0 ]] && \
    echo "$ICON_BANDWIDTH Transferred: $(awk "BEGIN {printf \"%.2fGB\", $BYTES_TRANSFERRED / 1073741824}")"

if [[ "$DRY_RUN" == true ]]; then
    warn "DRY RUN — no changes made"
elif [[ "$RSYNC_SUCCESS" == true ]]; then
    echo "$ICON_DONE Status: $ICON_SUCCESS DONE"
else
    echo "$ICON_ERROR Status: FAILED after $RETRY_COUNT attempts"
    notify "Rsync FAILED — $DIRECTORY ($PROFILE_NAME) after $RETRY_COUNT attempts on $(hostname)" \
        "Rsync" "warning"
fi
echo "━━━━━━━━━━━━━━━━━━━━━━━"

# Flush tee and preserve log for UI — close both ends of the pipe so tee gets EOF
exec 1>&- 2>&-; wait
cp "$VV_LIVE_LOG" "$VV_LAST_LOG" 2>/dev/null
rm -f "$VV_LIVE_LOG"

[[ "$RSYNC_SUCCESS" == false ]] && [[ "$DRY_RUN" == false ]] && exit 1
exit 0


🔄  Shares (14):   ❌ Books — 0s   ❌ Intros — 0s   ❌ Kids_Movies — 0s   ❌ Kids_Tv_Shows — 0s   ❌ Movies — 0s   ❌ Music — 0s   ❌ Music_Videos — 0s   ❌ Nextcloud — 0s   ❌ stand-up_comedy — 0s   ❌ Sports…