The fixture used the wrong env var name (SOULSYNC_DB_PATH) when trying
to redirect ConfigManager at a tmp directory. ConfigManager actually
reads DATABASE_PATH (config/settings.py:49), so the test ConfigManager
loaded — and then saved — at the user's real database/music_library.db.
The retry stub in test_lock_errors_during_retries_log_at_debug_not_error
calls the real _save_to_database after its mocked failures, which then
clobbered the encrypted app_config row with the test fixture's stub
payload {"plex": {"base_url": "http://example.test"}}.
Three layers of fix so this can't happen again:
- Use the correct env var (DATABASE_PATH).
- Pin mgr.database_path / mgr.config_path on the instance after
construction, so the test fixture's tmp paths win even if
ConfigManager's resolution logic changes.
- Assert the resolved database_path is rooted under tmp_path before
returning the fixture, so the test refuses to run if it would touch
a non-tmp DB.
When users bind the same host music directory into both SoulSync
(e.g. /app/Transfer) and a media server like Plex (e.g.
/media/Music), both scans add a track row pointing at the same
physical file via different mount paths. The detector previously
flagged those as duplicate groups even though there's only one
file on disk.
New _is_same_physical_file helper filters pairs where:
- The trailing 3 path segments match (filename + album + artist
folder), so they're the same release on disk.
- The leading mount roots actually differ.
- Durations agree within 1s when both rows carry duration data.
Adds 10 regression tests covering the reported scenario plus
edge cases (Windows separators, case differences, missing
durations, sibling-album false-positive guard).
Pin the new save-retry contract so future changes can't silently
re-introduce the spam reported in #434:
- Happy-path saves emit zero ERROR logs.
- Transient locks during retries log at DEBUG, not ERROR.
- Six attempts run before giving up, with the documented backoff
schedule (0.2 + 0.5 + 1.0 + 2.0 + 4.0s).
- Genuine exhaustion logs a single ERROR and writes config.json.
- sqlite3.OperationalError("database is locked") routes to DEBUG;
any other OperationalError still logs ERROR.
- _connect_db() actually applies WAL + busy_timeout + synchronous=NORMAL.
Also moves `import time` from inside _save_config to the module
top so the tests can monkeypatch sleep cleanly.
User on Docker + HDDs saw "database is locked" errors on every
settings save. Two retries spaced 1 second apart isn't enough when
an enrichment worker is mid-commit on spinning-rust storage, and
each failed attempt logged at ERROR — multiplying the spam.
Three changes in config/settings.py:
- Centralized connection setup in a new _connect_db() helper that
always sets PRAGMA journal_mode=WAL + busy_timeout=30000 +
synchronous=NORMAL. NORMAL is the safe pairing with WAL and avoids
the per-commit fsyncs that make FULL brutal on HDDs, shrinking the
window the competing writer holds the lock.
- _save_to_database logs lock errors at DEBUG instead of ERROR. The
retry loop owns the user-visible message; otherwise every retry
spammed even when the next attempt succeeded.
- _save_config now retries 6 times with exponential backoff
(0.2 + 0.5 + 1.0 + 2.0 + 4.0s ≈ 7.7s of sleep, on top of the 30s
busy_timeout each attempt already runs internally) before logging
a single error and falling back to config.json.
Closes#434.
The bulk download_discography endpoint picked one metadata client
based on the configured primary source and called .get_album() on
every album with that single client. Albums whose IDs came from a
fallback/provider-specific source (e.g. Deezer-formatted IDs surfaced
through Hydrabase) failed with "Album not found" because the primary
client couldn't resolve them.
Bulk now uses the same source-aware resolver
(core.metadata.album_tracks.get_artist_album_tracks) the working
individual-album endpoint already uses, so the resolver's source-chain
walk finds each album under whichever provider actually has it. Also
adds explicit Discogs and Hydrabase support (the old if/elif chain
silently 500'd for those primaries).
Frontend (library.js + pages-extra.js) now sends a richer
`{ albums: [{id, name, artist_name, source}] }` payload so each album
can be resolved through its own source. The legacy `album_ids` payload
still works as a fallback path.
Closes#399.
Body byte-identical to the original. Spotify proxy via registry,
iTunes/Deezer client shims wrap registry helpers,
_resolve_library_file_path, _attempt_download_with_candidates, and
missing_download_executor are injected via init() right after
_init_wishlist_failed where all three deps are already defined.
web_server.py: 35239 → 35063 (-176 lines).
Body byte-identical to the original. Wishlist helpers come from
core.wishlist.* directly (aliased to the same names the body uses);
runtime state from core.runtime_state. automation_engine,
soulseek_client, and _sweep_empty_download_directories are injected
via init() right after _init_download_validation.
web_server.py: 35408 → 35239 (-169 lines).
Body byte-identical to the original. matching_engine and
soulseek_client are injected via init() right after _init_discover_hero
since both originals are constructed early in web_server.py boot
(L598/L610) and never rebound.
web_server.py: 35586 → 35408 (-178 lines).
Body byte-identical to the original. Spotify proxy via registry,
_get_active_discovery_source and get_current_profile_id redefined
as stateless shims, _get_metadata_fallback_client injected via init()
because it composes multiple registry helpers wired in web_server.py.
web_server.py: 35753 → 35586 (-167 lines).
Both function bodies (_discovery_score_candidates and
_search_spotify_for_tidal_track) are byte-identical to the originals.
The shared matching_engine instance is injected via init() right after
_init_connection_test; the spotify proxy + _get_metadata_fallback_source
shim follow the same pattern used elsewhere.
web_server.py: 36019 → 35753 (-266 lines).
Both function bodies byte-identical to the originals. The spotify
proxy resolves through core.metadata.registry; the tidal proxy is
backed by an injected getter so a Tidal re-auth that rebinds
web_server.tidal_client is visible. 13 state dicts and helpers are
injected via init() after _init_connection_test, when all deps
already exist.
web_server.py: 36260 → 36019 (-241 lines).
Body byte-identical to the original. Pure stdlib + requests, no
web_server-specific globals or runtime state — no init() needed.
web_server.py: 36500 → 36261 (-239 lines).
- Switch the download lifecycle over to the neutral wishlist track helper name
- Keep the old Spotify helper as a compatibility alias for older callers
- Store track_data as the primary failed-download wishlist payload key and add regression coverage
- Let the wishlist service accept both track_data and spotify_track_data
- Preserve the backward-compatible wrapper while avoiding the keyword argument crash
- Add a regression test for the alias path
- Replace Spotify-only labels in the wishlist and matching surface with metadata/provider-neutral wording
- Keep the existing matching behavior intact while removing the most visible Spotify-first text
- add neutral wishlist payload helpers while keeping legacy Spotify aliases
- route wishlist removal and classification through generic track data
- keep API and service compatibility for existing callers
Body byte-identical to the original. Five deps (soulseek_client,
qobuz_enrichment_worker, hydrabase_client, docker_resolve_url,
docker_resolve_path) are injected via init() right after the
register_runtime_clients block — that is the earliest point at which
hydrabase_client is guaranteed to exist.
web_server.py: 36833 → 36500 (-333 lines).
Body byte-identical to the original. The shared state dict, lock,
docker_resolve_path helper, and automation engine are injected via
init() at the lift point, where all four originals are already defined.
web_server.py: 37015 → 36833 (-182 lines).
Lifts _search_service and its _detect_provider helper. Both bodies are
byte-identical to the originals. The nine enrichment worker handles
(spotify/itunes/mb/lastfm/genius/tidal/qobuz/discogs/audiodb) are
injected via init() right after qobuz is constructed, which is the
last worker to come up — and well before Flask starts accepting
requests, so the route handlers never see unbound workers.
web_server.py: 37245 → 37015 (-230 lines).
Lifts _match_liked_artists_to_all_sources and
_backfill_liked_artist_images. Both bodies are byte-identical to the
originals. Uses the same _SpotifyClientProxy + _get_*_client shim
pattern as core/artists/map.py so the bodies resolve their original
names without modification.
web_server.py: 37501 → 37245 (-256 lines).
Class body byte-identical to original. Module-level IS_SHUTTING_DOWN
flag is mirrored from web_server's own flag in _shutdown_runtime_components
so the monitor loop still sees shutdown signals at the right moment.
Eight web_server-side helpers (_make_context_key, _on_download_completed,
_run_post_processing_worker, _download_track_worker,
_start_next_batch_of_downloads, _orphaned_download_keys,
missing_download_executor, soulseek_client) are injected via init() after
register_runtime_clients, when all symbols are defined and well before
Flask starts accepting requests.
web_server.py: 38220 → 37501 (-719 lines).
Lifts get_artist_map_data, get_artist_map_genre_list,
get_artist_map_genres, and get_artist_map_explore (plus the
_artmap_cache_* helpers and _artist_map_cache dict) to a new module.
Bodies are byte-identical to the originals. web_server.py keeps
thin route shells that delegate to the lifted functions.
A _SpotifyClientProxy resolves the global spotify_client lazily via
core.metadata.registry.get_spotify_client() so a Spotify re-auth that
rebinds the cached client stays visible to the lifted bodies.
web_server.py: 39124 → 38220 (-904 lines).
Class body byte-identical to original. The shared metadata_update_state
dict is bound at import time via init() so the class body can mutate
it without web_server.py rebinding.
web_server.py: 39754 → 39122 (-632 lines).
The Spotify enrichment worker was auto-starting unconditionally at boot,
hammering /v1/search to match every track in the library against the
Spotify catalog regardless of which metadata source the user had
actually chosen as their primary. Users on Deezer, iTunes, Discogs,
or Hydrabase saw multi-hour 429 bans (typically 14400s) on Spotify
even though they never wanted Spotify-driven enrichment in the first
place — the worker generated dead API traffic the user neither asked
for nor benefited from.
Compounded by Spotify's February 2026 API tightening:
- /v1/search max limit cut from 50 to 10 per request, default from
20 to 5 — every track now needs more pagination, more requests.
- Sustained-rate detection more aggressive — repeated calls over
hours trigger automated long-form bans even when each individual
30-second window is well under the rolling limit.
Result: a user on Deezer would see their Spotify connection get banned
for 4 hours after about 30 tracks of enrichment activity, with no
recourse other than manually pausing the worker each session.
Two-part fix:
1. Boot gate (web_server.py): only auto-start the worker when
`get_primary_source() == 'spotify'`. Otherwise initialize in the
paused state with an explanatory log line. The settings UI manual
unpause control remains functional for users who explicitly want
background Spotify enrichment regardless of primary source.
Boot logic:
- User manually paused (existing config) → stays paused (preserved).
- Primary = 'spotify' → starts running (preserved).
- Primary != 'spotify' → starts paused with log line.
2. Daily budget reduction (core/spotify_worker.py): drop from 3000 to
500 items per calendar day. The 3000 cap was set when /v1/search
returned 50 results per call; now that it caps at 10, each track
needs roughly 5x the API load to find a confident match. 500/day
keeps the worker productive without crossing Spotify's hidden
sustained-rate detection threshold.
The runtime side of the boot gate — auto-pausing when the user
switches primary source mid-session — is out of scope. The settings
UI already exposes the manual toggle, and primary-source switches are
infrequent enough that requiring a manual unpause after the fact is
acceptable.
Full suite: 1355 passing. Ruff clean.
Final lift in the web_server.py extraction effort. Pulls two route
handlers + one background worker out of `web_server.py` into new
focused packages:
- `core/streaming/prepare.py` — 258-line stream-prep worker that
downloads a track to the local Stream/ folder for the browser audio
player.
- `core/playlists/explorer.py` — 305-line route handler for
`POST /api/playlist-explorer/build-tree` that streams an NDJSON
discography tree from a mirrored playlist.
What `prepare_stream_task` does:
1. Reset stream state to 'loading' with the new track info.
2. Clear any prior file from Stream/ (only one stream lives there).
3. Spin up a fresh asyncio event loop and `soulseek_client.download()`.
4. Poll progress every 1.5s. Queue timeout 15s; overall 60s.
5. On succeeded + bytes-match: find the file with retry, move into
Stream/, signal slskd completion, mark state 'ready' with file_path.
6. On error/timeout/cancel: state goes to 'error' or 'stopped'.
7. Finally: tear down the event loop cleanly.
What `playlist_explorer_build_tree` does:
1. Validate request, load playlist + tracks from DB.
2. Pick active metadata source (Spotify if authed, else fallback).
3. Group tracks by artist using discovered matched_data when the
provider matches the active source.
4. Stream NDJSON: meta line → one artist line per group → complete line.
5. Per artist: cache check → resolve discography → tag releases with
`in_playlist` flag based on title-similarity match → filter by mode
(`albums` = only matches; `discographies` = full disco).
6. Mark playlist as explored on completion.
Strict 1:1 byte parity:
Both functions exposed their dependencies through proxy patterns
established in earlier lifts (PR4–PR8). For prepare_stream_task,
`stream_state` is a deps property; for the explorer, Flask `request` /
`jsonify` / `Response` are injected via deps so the lifted body keeps
its native syntax. Both lifts verified ZERO diff against the original
after `deps.X` → global X normalization.
258 lines orig = 258 lines lifted (prepare_stream_task).
305 lines orig = 305 lines lifted (explorer).
Bonus cleanup: web_server.py's module-level `import shutil` and
`import glob` were now unused (only `_prepare_stream_task` used them
at module scope; every other reference is via inline `import shutil`
in respective function bodies). Removed both module-level imports —
ruff caught the F811 redefinitions and confirmed they're truly
redundant.
Dependencies for `PrepareStreamDeps` (11 fields):
config_manager, soulseek_client, stream_lock, project_root,
docker_resolve_path, find_streaming_download_in_all_downloads,
find_downloaded_file, extract_filename, cleanup_empty_directories,
plus 2 stream_state property delegates.
Dependencies for `PlaylistExplorerDeps` (9 fields):
Flask request/Response/jsonify, spotify_client, get_database,
get_active_discovery_source, get_metadata_fallback_client,
get_metadata_fallback_source, get_metadata_cache.
Tests: 6 new under tests/streaming/test_prepare.py (state init,
Stream/ folder creation + clearing, download-init failure, completed
+ moved + ready state, partial-bytes incomplete-warning path) plus 9
new under tests/playlists/test_explorer.py (5 validation early-exit
paths, streaming response shape with meta/complete lines, mark-
explored side effect, discovered-artist grouping using matched_data,
provider mismatch falling back to raw artist name).
Full suite: 1355 passing (was 1340). Ruff clean.
End of the web_server.py extraction effort. Started at ~45,000 lines
across PR4–PR8 + this commit; finished around 35,000 lines with the
heavy worker + route logic now living in domain-cohesive packages
under core/. The remaining bulk in web_server.py is route handlers,
service initialization, and the deferred 1530-line
`_register_automation_handlers` (startup-only, marginal lift value).
Pulls the 284-line artist quality enhancement helper out of
`web_server.py` into a new `core/artists/` package. Flask route handler
split: route + request parsing stay in web_server.py, the body lifts to
a pure function returning `(payload_dict, http_status_code)`.
What `enhance_artist_quality` does:
1. Validate request: track_ids must be non-empty, artist must exist.
2. Build a `track_lookup` from `database.get_artist_full_detail` so each
selected track resolves with its album context.
3. Per track:
- Read current quality tier from the file extension.
- Build `matched_track_data` for the wishlist entry, in priority
order:
- Spotify direct lookup via stored `spotify_track_id` (preferred).
Uses raw API data when available; otherwise rebuilds the payload
and pulls album images via a follow-up `get_album` call.
- Spotify search fallback using matching_engine queries with
artist+title similarity scoring (album-type bonus for albums,
smaller bonus for EPs). Stops at first >= 0.9 confidence match.
- iTunes/fallback source search with the same scoring shape.
- Add to wishlist via `wishlist_service.add_spotify_track_to_wishlist`
with `source_type='enhance'` and a `source_context` carrying the
original file path, format tier, bitrate, original_tier, and
artist_name.
- Tally `enhanced_count` / `failed_count` / per-track failure reasons.
4. Return `{success, enhanced_count, failed_count, failed_tracks}` 200.
Dependencies injected via `ArtistQualityDeps` (7 fields) — spotify_client,
matching_engine, get_database, get_wishlist_service,
get_current_profile_id, get_quality_tier_from_extension,
get_metadata_fallback_client.
Diff vs original after `deps.X` → global X normalization is **1 line of
cosmetic drift** — the success return now uses an explicit `(payload, 200)`
tuple to keep all returns shape-consistent for the wrapper. Flask treats
`jsonify(x)` and `(jsonify(x), 200)` identically. 284 lines orig = 285
lines lifted, body otherwise byte-identical.
Tests: 10 new under tests/artists/test_quality.py covering input
validation (empty track_ids, artist not found), Spotify direct lookup
via raw_data, Spotify direct lookup with enhanced format requiring
album image rebuild, Spotify search fallback, iTunes/fallback source
match path, track-not-found and no-file-path failure modes, complete
no-match failure, and source_context payload assertions (enhance flag,
file path, format tier, bitrate, source_type).
Full suite: 1340 passing (was 1330). Ruff clean.
BoulderBadgeDad
Broque Thomas
Antti Kettunen