Patch bump for the post-2.5.6 fix cycle. Nine entries shipped since the
2.5.6 release moved into a fresh 2.5.7 WHATS_NEW block — original 2.5.6
release notes left intact.
Touched:
- web_server.py: `_SOULSYNC_BASE_VERSION` 2.5.6 -> 2.5.7
- webui/static/helper.js: new `'2.5.7'` block with date marker + the
nine shipped fixes; fallback default in `_getLatestWhatsNewVersion`
bumped to '2.5.7'
- .github/workflows/docker-publish.yml: workflow_dispatch description
+ default tag both bumped to 2.5.7
What's in 2.5.7 (all post-2.5.6 cycle work):
- MB manual search recall fix (strict -> bare-query)
- MB album-detail 404 fix (invalid cover-art-archive include)
- Fix popup MBID paste field (#647)
- MB added to Fix popup auto-search cascade (#655)
- Docker /app/Stream pre-baked for rootless Docker (#656)
- slskd unreachable log spam suppression (#649)
- MB 'Other' release-groups now visible in discography (#650)
- Quarantined-source dedup on auto-wishlist cycles (#652)
- Unknown Artist Fixer ImportError fix (#646)
The cancel-trigger diagnostic logging commit (a685f9ca) is also in
2.5.7 but isn't user-facing so no WHATS_NEW entry.
Diagnostic-only change for issue Technodude reported: Tidal sync-playlist
downloads getting mass-cancelled mid-flight with no clear cause in the
logs. App.log shows ~91 second gaps between Tidal download start and
cancel — matches the monitor's 90s queue-timeout exactly — but none of
the monitor's WARNING log lines fire, so the trigger is ambiguous
between five `_should_retry_task` paths, three web_server cancel paths,
and the API endpoints.
Added a single `[CancelTrigger:<label>]` INFO log line immediately
before every `download_orchestrator.cancel_download(...)` call so the
next log dump pins down which path is firing.
Labels (grep-able, prefix tells the file, suffix tells the trigger):
monitor.not_in_live_transfers_90s
monitor.errored_state_retry
monitor.queued_state_timeout
monitor.stuck_at_0pct_timeout
monitor.unknown_state_no_progress_timeout
candidates.worker_cancelled_during_download
web.orphan_cleanup
web.cancel_download_task
web.atomic_cancel_v2
api.manual_cancel_single
api.public_cancel
The monitor's `deferred_ops` tuple grew from 3 elements to 4 (added
trigger label as last element). The dispatch loop unpacks both legacy
and new shapes so the change is backward-compatible for any in-flight
ops mid-deploy.
Zero behavior change. 367 download tests still green. WHATS_NEW left
untouched — diagnostic only, not user-facing.
After ship: ask Technodude to re-run the same sync playlist scenario,
attach the new app.log, grep `[CancelTrigger:` lines for the trigger
context, then write the actual fix.
The Fix Track Match modal's auto-search was hardcoded to query only
Spotify -> Deezer -> iTunes, ignoring MusicBrainz entirely — even for
users with MB set as their primary metadata source. MB-niche recordings
(canonical entries with diacritics, fringe / non-mainstream tracks that
the commercial catalogues don't carry) had no chance.
Wiring:
- New `MusicBrainzSearchClient.search_tracks_with_artist(track, artist,
limit)` for surfaces that already have title + artist split. Uses MB's
bare-query mode (strict=False) — diacritic-folded, alias/sortname
indexed — same recall rationale as the earlier MBID-paste endpoint.
- New route `GET /api/musicbrainz/search_tracks` mirrors the existing
/api/{spotify,itunes,deezer}/search_tracks endpoints exactly: accepts
`track`+`artist` (or legacy `query`) + `limit`, returns
`{tracks: [{id, name, artists, album, duration_ms, image_url, source}]}`.
Applies the same `core.metadata.relevance.rerank_tracks` pass Deezer /
iTunes use, which is critical because MB's free-text scoring weighs
title-text matches heavily and would otherwise rank cover / tribute
recordings above the canonical version.
- `_search_tracks_text` gains a `min_score` parameter. The cascade path
passes 20 (vs the enhanced-search-tab default of 80) so MB recordings
whose title doesn't literally contain the artist name still enter the
candidate pool — without that, "Army of Me" + "Bjork" only surfaces
the HIRS Collective cover (score 100) and drops Björk's canonical
recording (score 28). The rerank pass then surfaces Björk by artist
match. Verified against real MB API: pre-fix returned only the cover;
post-fix top 5 are all Björk.
- Fix popup `allSources` array (wishlist-tools.js) gets MB appended.
The existing `activeIdx` reorder logic moves MB to the front when
it's the active primary; otherwise MB sits last (1 req/sec rate
limit makes it the slowest source).
7 new unit tests on the adapter: bare-query mode is used, missing
artist falls back to None (drops AND-clause), empty inputs short-circuit,
low-score candidates are kept for rerank to handle, default strict +
default min_score behaviour preserved for the existing search-tab path,
client errors are swallowed so the cascade falls through to the next
source.
Discogs intentionally absent — Discogs has no track-level search API
(see core/discogs_client.py:575 — returns []). Adding a Flask endpoint
that always returns empty would be a permanent no-op.
Power-user escape hatch on the Discovery Fix Track Match modal — when
fuzzy auto-search ranks the wrong recording among many same-title
versions (10 remasters, live cuts, alt sessions), paste the MusicBrainz
recording URL or bare UUID into the new field and resolve straight to
that record.
Layout:
- Shape adapter `get_recording_flat(mbid)` lives in
`core/musicbrainz_search.py` next to existing `get_track_details`.
Returns the flat Fix-popup track shape (artists as `string[]`,
album as string, single `image_url`) — distinct from the
Spotify-shaped nested dict `get_track_details` returns.
- New route `GET /api/musicbrainz/recording/<mbid>` is a thin wrapper:
validates MBID format with an anchored UUID regex, calls the adapter,
returns 400 / 404 / 200 with no inline shape massaging.
- Frontend `parseMusicBrainzMbid()` lives in `shared-helpers.js` —
pure URL/UUID parser, reusable from other surfaces (failed-MB cache,
manual match) without duplication.
- Fix modal HTML gets one new input row + button; existing search row
and result render pipeline are untouched. New `lookupDiscoveryFixByMbid()`
fetches the endpoint and feeds the single result through the existing
`renderDiscoveryFixResults` -> confirm-dialog -> match pipeline, so MB-
paste matches go through the exact same selection flow as auto-search
results.
- Enter-key bound on the MBID input via a separate handler ref so its
lifecycle matches the search-input handlers without conflating the
two submit targets.
7 unit tests cover the adapter: happy path, empty/None MBID, MB returns
None, recording-without-release (empty album), multi-artist credits,
includes-list contract, and client-error swallow.
Out of scope: the Fix popup's fuzzy cascade is still hardcoded to
spotify/deezer/itunes regardless of which primary source the user has
configured. Adding MB to that cascade (when MB is the active primary)
is a separate concern.
Two bugs surfacing on the Fix popup and enhanced-search MB tab:
1. Strict Lucene phrase queries (`recording:"X" AND artist:"Y"`) killed
recall on user-facing manual search — diacritics ("Bjork" vs canonical
"Björk"), bracketed suffixes like "(Live)", and any AND-clause
mismatch returned zero results. Added `strict: bool = True` param to
`search_release` / `search_recording`; when False, sends a bare query
joining title + artist so MB hits alias/sortname indexes with
diacritic folding. `/api/musicbrainz/search` (Fix popup) and
`core/library/service_search.py` (service tabs) now pass strict=False.
Enrichment workers stay on strict mode — precision matters there
because they auto-accept the top hit above a confidence threshold.
2. Every MB album click was silently 404-ing — `_render_release_as_album`
passed `cover-art-archive` as an MB `inc` param, but it's not a valid
include for the /release resource (MB rejects with 400). The CAA flags
come back on every release response by default, so dropping the bad
include preserves the image-scope picker logic intact.
Add MusicBrainz watchlist artist ID storage, badges, linked-provider editing, and per-artist preferred source support.
Backfill watchlist MusicBrainz matches from already-enriched library artists so existing MusicBrainz worker matches appear in watchlist cards and settings.
Extend bulk watchlist add, liked artist matching, artist map source picking, and service status labels to recognize MusicBrainz, with regression tests for watchlist ID persistence and backfill.
Register MusicBrainz as a first-class metadata source alongside Deezer, iTunes, Spotify, Discogs, and Hydrabase. Expose the shared client through metadata services, add the settings option, and expand the MusicBrainz search adapter with source-compatible artist, album, track, and detail methods.
Carry MusicBrainz IDs through similar-artist discovery, recommended artists, artist map serialization, and personalized playlist selection. Update DB migrations and lookup filters so similar_artist_musicbrainz_id is preserved on older schemas and used for source requirements and library exclusion.
Normalize MusicBrainz album adapter output for import context and add regression coverage for registry mapping, typed album conversion, and similar-artist filtering. Verified by user with 120 focused tests passing.
Move the existing-file missing-track import workflow out of web_server.py and into core/library/missing_track_import.py.
Keep the Flask route focused on request wiring and response formatting while the service handles staging copy, post-processing, album identity tag inheritance, DB upsert, and media-server sync.
Show actionable missing album tracks in the enhanced library from canonical metadata, with a practical Manage flow for Add to Library or I Have This.
Implement I Have This as a non-destructive copy/import path: copy the chosen existing file, run normal post-processing with the missing track context, insert the real library row, and inherit album identity tags from target siblings so Navidrome does not split albums.
Improve the modal with selectable search results, visible import progress, disabled controls during import, and missing-track row styling.
- _SOULSYNC_BASE_VERSION in web_server.py
- WHATS_NEW key + date in helper.js (strips unreleased flag from Amazon entries)
- fallback version string in helper.js
- Artist cards, hero section, and enhanced view now show Amazon Music badges
when amazon_id is populated (AMAZON_LOGO_URL constant, orange #FF9900 brand)
- Enhanced view artist and album match status rows include amazon_match_status
chip with click-to-rematch via openManualMatchModal
- getServiceUrl: added amazon (album/track ASIN → music.amazon.com) and fixed
missing discogs entries; serviceLabels adds tidal/qobuz/amazon
- Enhanced view enhanced-artist-id-badges includes amazon_id entry
- DB SELECTs for library artists list and artist detail now return amazon_id;
both response dicts include the field
- watchlist_artists migration adds amazon_artist_id column
- Watchlist config GET: amazon_artist_id in SELECT/WHERE/response (index 18)
- Watchlist artists list response includes amazon_artist_id
- link-provider endpoint: amazon added to valid_providers and col_map
- _populateLinkedProviderSection: amazonId param + Amazon Music source row
- Watchlist card source badges render Amazon pill (watchlist-source-amazon CSS)
- _openSourceSearch labels map includes amazon
- service_search: amazon_worker injected via init(); _search_service amazon branch
uses search_artists/albums/tracks, same {id,name,image,extra} return shape
- _SERVICE_ID_COLUMNS: amazon → amazon_id for artist/album/track
- _init_service_search call passes amazon_worker_obj
- amazon_client._fetch_album_metas: 5-minute TTL cache per ASIN — cached hits
skip _rate_limit() and HTTP call entirely; fixes ~10s artist detail load
- registry.py: removed amazon from METADATA_SOURCE_PRIORITY and
METADATA_SOURCE_LABELS — T2Tunes has no discography API, cannot serve as a
primary metadata source; Amazon remains a download source + ASIN enricher
- Settings metadata source dropdown and help text updated accordingly
Two bugs in the library artist detail page when Amazon is the source:
1. No album art: get_artist_albums returned Album dataclasses with
image_url=None — it collected ASINs but never called _fetch_album_metas.
Now fetches metas for up to 10 albums (same cap as search_albums),
populating image_url, release_date, and total_tracks on each Album.
2. No singles: Album.from_search_hit hardcodes album_type="album" and
T2Tunes exposes no release type in search results. Added inference:
total_tracks==1 → album_type="single", which routes them to the
singles bucket in the discography categorizer.
Also passes album_name through _strip_edition and artist through
_primary_artist in get_artist_albums (parity with search_albums).
3. amazon_id missing from artist_source_ids in get_artist_detail:
the discography lookup never received the stored Amazon slug so
it always fell back to name search. Added 'amazon': artist_info.
get('amazon_id') to the dict alongside spotify/deezer/itunes/etc.
_get_enrichment_status had a hardcoded workers_info list. Amazon was
registered in the generic enrichment blueprint but never added here,
so the rate-monitor speedometer overlay and status API omitted it.
Adds ('amazon_enrichment', 'Amazon Music', lambda: amazon_worker)
to workers_info — same pattern as Deezer, Discogs, Tidal, Qobuz.
Adds full parity with Deezer/Qobuz/Tidal/Discogs in every dashboard
UI layer — orb button, live tooltip, WebSocket push, rate speedometer.
- webui/index.html: Amazon enrichment orb button after Discogs
- webui/static/amazon.svg: local icon (a + smile, same pattern as
hydrabase.png — avoids external URL dependency)
- webui/static/style.css: Amazon button/spinner/tooltip CSS with
FF9900 brand color; added to mobile tooltip suppress list
- webui/static/worker-orbs.js: Amazon orb in WORKER_DEFS [255,153,0]
- webui/static/api-monitor.js: Amazon in rate gauge services list,
label, and color map
- webui/static/enrichment.js: updateAmazonEnrichmentStatusFromData,
toggleAmazonEnrichment, DOMContentLoaded init + 2s poll
- webui/static/core.js: socket.on enrichment:amazon-enrichment listener
- web_server.py: amazon-enrichment added to _emit_enrichment_status_loop
workers dict so WebSocket pushes fire every 2s
Background worker matching library artists/albums/tracks to Amazon ASINs
via T2Tunes search. Follows same 6-tier priority queue as Deezer/iTunes/
Spotify/Qobuz/Tidal workers. Backfills artist thumbnails from album cover
stand-ins (T2Tunes exposes no direct artist images).
- core/amazon_worker.py: new AmazonWorker class with full parity
- database/music_database.py: expand _add_amazon_columns to cover
amazon_id/amazon_match_status/amazon_last_attempted on artists,
albums, and tracks (was artists-only)
- web_server.py: import, init, register in enrichment panel, add to
scan pause/resume dicts and rate monitor key map
- helper.js: WHATS_NEW 2.5.3 entry for enrichment worker
AcoustID verification was quarantining every Amazon track because T2Tunes
embeds [Explicit] and [feat. X] in stream tag titles/artists, but AcoustID
returns bare titles — triggering version-mismatch rejection on every track.
- get_track_details: apply _strip_edition to name/album, _primary_artist to
artist; wire s.track_number / s.disc_number instead of hardcoded None
- get_album_tracks: apply _strip_edition to name, _primary_artist to artist
Also fix TypeError crash in album download paths when disc_number is None
(present in dict but explicitly None, so .get('disc_number', 1) returns None):
- master.py run_full_missing_tracks_process: or 1 guard on both max() and disc_num
- candidates.py track_info extraction: or 1 guard on both disc_number reads
- web_server.py enhanced + standard album download max() calls: or 1 guard
- All search_raw calls switched from single-type to types="track,album" — T2Tunes only
returns results when both types are requested together
- _fetch_album_metas: parallel fetch (up to 5 workers) of album cover art via
album_metadata(asin) — T2Tunes search results carry no image URLs
- search_tracks: populates image_url, release_date, total_tracks from album meta
- search_artists: strips feat. credits via _primary_artist() so "Artist feat. X" and
"Artist ft. Y" collapse to one "Artist" entry; uses album cover as artist image
stand-in (same approach as iTunes — T2Tunes has no artist images)
- search_albums: name-based dedup (display_name + artist key) instead of ASIN-based;
populates image_url, release_date, total_tracks from album meta (cap 10 ASIN fetches)
- _strip_edition(): strips [Explicit]/(Explicit) from track/album names — explicit is
the default version; Clean/Edited/Censored labels kept as-is so they stay distinct
- get_album(): applies _strip_edition to name and _primary_artist to artist so
MusicBrainz preflight matching doesn't fail on "[Explicit]" album names
- get_album_tracks(): populates track_number and disc_number from T2TunesStreamInfo
instead of hardcoding None — fixes track ordering in multi-track album downloads
- get_artist() / get_artist_albums(): _unslugify() converts slug artist IDs back to
search names; _primary_artist() in comparison handles feat-annotated results
- SOURCE_ONLY_ARTIST_SOURCES: added "amazon" so artist detail page doesn't 404
- build_source_only_artist_detail: added amazon_client param + dispatch branch
- web_server.py: resolve amazon_client in _build_source_only_artist_detail wrapper;
add source_override=="amazon" branch in get_spotify_album_tracks endpoint
- 77 tests covering all above paths; all pass
- Add 'amazon' to VALID_SOURCES (and transitively VALID_STREAM_SOURCES)
in core/search/orchestrator.py so the backend accepts it as a
requested source without returning 400
- Add resolve_client('amazon') case — mirrors musicbrainz pattern,
gets the cached AmazonClient from the metadata registry
- Add 'amazon' to _alternate_sources() so it appears as a tab when
another source is primary (always available, no credentials)
- Add SERVICE_CONFIG_REGISTRY entry 'amazon': {'always': True} so
/api/settings/config-status reports it as configured
- Add SOURCE_LABELS['amazon'] and SOURCE_ORDER entry in
shared-helpers.js so both enhanced search and global search show
the Amazon Music tab
- Add 'amazon' to _ALWAYS_CONFIGURED_SOURCES so the picker never
dims the tab (no credentials required)
- Add .enh-tab-amazon.active CSS (Amazon orange #FF9900)
- 3530 tests pass
`validation.py` had amazon absent from `_streaming_sources`, causing
Amazon TrackResult objects (bitrate=None, size=0) to fall through to
the Soulseek P2P code path and get rejected by
`filter_results_by_quality_preference`. Every album track was marked
not found.
Fix: add 'amazon' to every streaming-source guard tuple/set that was
previously missing it:
- core/downloads/validation.py — primary bug fix (quality-filter bypass)
- core/downloads/status.py — _STREAMING_SOURCE_NAMES frozenset
- core/downloads/task_worker.py — hybrid fallback client map
- core/imports/side_effects.py — || filename→stream-id extraction
- web_server.py — is_streaming_source, transfer list display,
candidate source label, _try_source_reuse, _store_batch_source
- tests/test_download_plugin_conformance.py — registry count + parametrize
Also updates the 2.5.3 What's New entry to drop the stale
"not yet wired" disclaimer.
Follows the exact same standard as Tidal, Qobuz, HiFi, and Deezer.
registry.py — import + register AmazonDownloadClient as 'amazon'.
amazon_download_client.py — read amazon_download.quality / allow_fallback
from config on init; pass quality as preferred_codec to AmazonClient;
_download_sync codec waterfall respects allow_fallback flag.
download_orchestrator.py — reload_settings() updates preferred_codec +
allow_fallback on the live client after a settings save. 'amazon' added
to _streaming_sources so search_and_download_best routes it correctly.
api_call_tracker.py — 'amazon' registered in RATE_LIMITS (120/min),
SERVICE_LABELS, and SERVICE_ORDER so API call monitoring shows Amazon.
web_server.py — 'amazon_download' added to the settings service loop.
'amazon' added to serverless_sources (no slskd probe needed). Streaming
file-finder extended to handle amazon username + ||asin||title encoding
(extension-less fuzzy match, same as Tidal/Qobuz/HiFi). New endpoint:
GET /api/amazon/test-connection → checks T2Tunes proxy status.
webui/index.html — amazon-download-settings-container: quality dropdown
(flac/opus/eac3), allow-fallback checkbox, test-connection button.
webui/static/settings.js — 'Amazon Music' added to HYBRID_SOURCES,
_hybridSourceEnabled, allSources mode list, loadSettings(), saveSettings()
payload, updateDownloadSourceUI() show/hide + auto-test. New
testAmazonConnection() function.
Reproduced on the personalized playlist pipeline: selecting Fresh Tape
(or any kind) and running the automation surfaced
"Working outside of application context" in the UI.
Root cause: `get_current_profile_id` reads Flask's `g.profile_id` and
only catches `AttributeError`. Outside a request — automation engine,
sync threads, watchlist scanner — `g` raises `RuntimeError` instead,
so the except misses and the handler dies.
Mirrored playlist pipeline never hit this because it hardcodes
profile_id=1 in its sync call. The personalized pipeline calls
`deps.get_current_profile_id()` from a background thread, which is
what tripped the bug. Fresh Tape's generator also resolves the
profile via the same function — same path, same crash.
Fix: broaden the except to `(AttributeError, RuntimeError)` in all
three copies of the helper (`web_server.py`, `core/artists/map.py`,
`core/discovery/hero.py`). All three now safely degrade to profile_id=1
(admin profile) when called outside a request context — matches the
existing intent that single-admin installs Just Work.
No test changes — the existing pipeline tests stub the helper, so
they never exercised the bug. The fix is in the layer above the
stubs.
The action was registered + the block declared, but the automation
builder's per-action config renderer didn't have a case for
`personalized_pipeline` so users only saw the bare card with the
generic delay-minutes input — no way to select which playlists to
sync. This commit adds the multi-select picker.
Backend:
- `core/personalized/api.list_kinds(manager=...)` now optionally
takes a manager and includes the resolved variant list per kind
(calls each spec's variant_resolver(deps) when present). Singleton
kinds get an empty `variants` list. Variant-bearing kinds
(time_machine / genre_playlist / daily_mix / seasonal_mix) get
their full enumerated set.
- `web_server.py` `/api/personalized/kinds` route now passes a built
manager so the variants list lands in the response.
Frontend:
- `webui/static/stats-automations.js` `_renderBlockConfigFields`
gains a `personalized_pipeline` branch that renders a scrollable
multi-select picker:
- Singletons (Hidden Gems, Discovery Shuffle, Popular Picks,
Fresh Tape, The Archives) = one checkbox row per kind
- Variant kinds = a section header + one checkbox row per variant
(e.g. Time Machine: 1960s/1970s/.../2020s; Seasonal: halloween/
christmas/valentines/summer/spring/autumn)
- Pre-checks rows that match the existing `kinds` config on edit
- New `_autoLoadPersonalizedKinds(slotKey)` fetches `/api/personalized/kinds`
(cached after first load), renders the picker DOM, and pre-checks
saved selections via `data-kind` / `data-variant` attributes on
the checkboxes.
- `_renderBuilderCanvas` calls the loader for any `cfg-*-kinds-picker`
it finds in the freshly-rendered slots.
- The save-time `_collectActionConfig` walks the picker's checked
inputs (matched by `data-kind` attribute) and emits
`{kinds: [{kind, variant?}, ...], refresh_first, skip_wishlist}`
in the same shape the handler expects.
Tests:
- `tests/automation/test_automation_blocks.py::_FIELD_TYPES` adds
'personalized_playlist_select' so the block-shape regression test
accepts the new field type. (Test was failing because it whitelists
every field type used across all blocks.)
- 189 automation + personalized API tests pass; full suite intact.
Follow-up to the personalized-playlists standardization PR. New
`personalized_pipeline` automation action syncs selected discover-
page playlists (Hidden Gems / Discovery Shuffle / Time Machine /
Genre / Daily Mix / Fresh Tape / The Archives / Seasonal Mix) to
the active media server + queues missing tracks for download.
Same pattern as the existing mirrored `playlist_pipeline` but two
phases instead of four — no REFRESH (no external source to re-pull)
and no DISCOVER (manager-backed snapshots are already metadata-
matched). Pipeline shape:
SNAPSHOT → SYNC → WISHLIST
Where SNAPSHOT either reads the persisted track list from
`PersonalizedPlaylistManager` (default) or refreshes it first when
`refresh_first=true` (cron use case: regenerate Hidden Gems nightly
and sync the fresh set).
Shared helper extraction:
PHASE 3 (SYNC loop) + PHASE 4 (WISHLIST tail) lifted out of mirrored
`playlist_pipeline` into `core/automation/handlers/_pipeline_shared.py`
as `run_sync_and_wishlist(deps, automation_id, playlists, sync_one_fn,
sync_id_for_fn, ...)`. Both pipelines call it. Mirrored injects
`auto_sync_playlist` as the per-playlist sync function; personalized
injects a thin wrapper that launches `_run_sync_task` directly with
a pre-built tracks_json. Same sync-state polling / progress emission
/ status counting / wishlist trigger logic — 0 duplication.
Files added:
- core/automation/handlers/_pipeline_shared.py
- core/automation/handlers/personalized_pipeline.py
- tests/automation/test_handlers_personalized_pipeline.py
Files changed:
- core/automation/handlers/playlist_pipeline.py: PHASE 3+4 replaced
with shared helper call (~100 lines deleted, 1 helper invocation
added; behavior identical).
- core/automation/deps.py: new `build_personalized_manager` field
(lazy builder so the pipeline gets a fresh PersonalizedPlaylistManager
per run).
- core/automation/handlers/__init__.py + registration.py: register
`personalized_pipeline` action with the shared `pipeline_running`
guard so it can't overlap mirrored.
- core/automation/blocks.py: new `personalized_pipeline` block
declaration with config_fields (kinds multi-select, refresh_first,
skip_wishlist).
- web_server.py: thread `_build_personalized_manager` into
AutomationDeps construction.
- All 5 automation test fixtures: `_build_deps` adds
`build_personalized_manager=lambda: None` stub.
- tests/automation/test_handler_registration.py:
EXPECTED_ACTION_NAMES + EXPECTED_GUARDED_ACTIONS gain
`personalized_pipeline`.
Trigger schema:
{
"_automation_id": "...",
"kinds": [
{"kind": "hidden_gems"},
{"kind": "time_machine", "variant": "1980s"},
{"kind": "seasonal_mix", "variant": "halloween"}
],
"refresh_first": false,
"skip_wishlist": false
}
Tests (14 new, 178 automation total):
- _track_to_sync_shape: basic shape, source ID fallback chain,
no-id returns empty string
- empty config / non-list kinds / empty kinds list all return
error + clear pipeline_running flag
- _build_payloads_for_kinds: skips invalid entries, skips kinds
with no tracks, refresh_first vs ensure dispatch, payload shape
+ sync_id format, manager exception swallowed continues
- _sync_personalized_playlist: launches background thread + returns
status='started'
- happy path: stubbed sync_states drives helper to completion, flag
cleaned up
Full suite: 3383 passed.
Note: the trigger UI block declares config_fields but the frontend
doesn't yet render the `personalized_playlist_select` multi-select
type — usable today via API; polished UI ships in a follow-up
frontend PR.
Wraps the manager + generator dispatch behind one HTTP surface so
the UI can drop the patchwork `/api/discover/personalized/*` calls
in favor of a single REST shape. Legacy endpoints stay alive for
backward compat during the UI migration window.
New endpoints:
- GET /api/personalized/kinds — list every registered kind + metadata
- GET /api/personalized/playlists — list every persisted playlist for the active profile
- GET /api/personalized/playlist/<kind> — fetch singleton + tracks
- GET /api/personalized/playlist/<kind>/<variant> — fetch variant + tracks
- POST /api/personalized/playlist/<kind>/refresh — regenerate singleton
- POST /api/personalized/playlist/<kind>/<variant>/refresh — regenerate variant
- PUT /api/personalized/playlist/<kind>/config — patch singleton config
- PUT /api/personalized/playlist/<kind>/<variant>/config — patch variant config
Per-call manager construction wires the deps each generator needs:
- database (MusicDatabase singleton)
- service (PersonalizedPlaylistsService for legacy generator calls)
- seasonal_service (SeasonalDiscoveryService for seasonal_mix)
- get_current_profile_id (active profile accessor)
- get_active_discovery_source (source dispatcher)
API handlers themselves live as pure functions in
`core/personalized/api.py` so they're testable without Flask. The
Flask layer in `web_server.py` is a thin parse-body / call-handler /
jsonify wrapper.
11 new boundary tests (122 personalized total):
- list_kinds enumerates registry, exposes default config + tags
- list_playlists returns empty list when none exist, serializes
PlaylistRecord shape correctly
- get_playlist_with_tracks auto-creates on first access, returns
persisted tracks, raises ValueError on unknown kind
- refresh_playlist runs generator and returns track snapshot,
forwards config_overrides to the generator
- update_config patches stored config
3365 tests pass total. Manager construction triggers generator
registration via `from core.personalized import generators` import
side-effect.
Cleans up the four remaining inline callbacks at the bottom of
`web_server._register_automation_handlers` so the function is now
purely deps-construction + register_all + a logger.info line.
Lifted:
- `_progress_init`, `_progress_finish`, `_record_automation_history`,
and `_on_library_scan_completed` -> core/automation/handlers/progress_callbacks.py
Each is a top-level function that takes deps as a parameter; the
engine sees thin lambdas through `register_progress_callbacks` /
`register_library_scan_completed_emitter` (called from `register_all`).
Two new deps fields:
- `init_automation_progress` (delegates into the live progress tracker)
- `record_progress_history` (delegates into _auto_progress.record_history)
12 new boundary tests in tests/automation/test_progress_callbacks.py
pin every shape:
- progress_init forwards to init_automation_progress
- progress_finish skips when handler manages its own progress
(prevents double-emit of finished status)
- progress_finish: completed -> finished/Complete/success;
error -> error/Error/error; msg falls through error -> reason ->
status -> 'done'
- record_history threads the live db into the recorder
- on_library_scan_completed: no engine = noop, server type taken
from web_scan_manager._current_server_type, defaults to 'unknown'
- register_library_scan_completed_emitter: no scan manager = noop,
registered callback emits the right event when invoked
3256 tests pass, no regression.
Final state of `_register_automation_handlers`:
- Was: 1530 lines, 21 nested closures + 4 progress callbacks
- Now: ~50 lines, builds AutomationDeps and calls register_all
web_server.py: 34,220 -> 34,187 lines (-33 net, -1,406 across the
whole branch).
Final commit of the automation-handler refactor. With this commit
every closure that used to live in
`web_server._register_automation_handlers` is now a top-level
function in `core/automation/handlers/`.
Handlers extracted in this commit:
- start_database_update + deep_scan_library
-> core/automation/handlers/database_update.py
Both share the db_update_state monitoring pattern (poll until
status flips, stall detection emits warning at 10 min, 2-hour
outer timeout). Lifted into a shared `_run_with_progress` helper
inside the module so the per-handler bodies stay tiny.
- run_duplicate_cleaner -> core/automation/handlers/duplicate_cleaner.py
- start_quality_scan -> core/automation/handlers/quality_scanner.py
- clear_quarantine, cleanup_wishlist, update_discovery_pool,
backup_database, refresh_beatport_cache
-> core/automation/handlers/maintenance.py
Grouped because each body is short (~20-50 lines) and they share
no state — splitting into per-handler files would just add import
noise.
- clean_search_history, clean_completed_downloads, full_cleanup
-> core/automation/handlers/download_cleanup.py
Grouped because all three reach the download orchestrator,
tasks_lock, and download_batches/download_tasks accessors. The
full_cleanup multi-step orchestration shares phase-detection
logic with clean_completed_downloads.
- run_script -> core/automation/handlers/run_script.py
- search_and_download -> core/automation/handlers/search_and_download.py
`AutomationDeps` grew with the new dependency surface:
- get_db_update_state + db_update_lock + db_update_executor +
run_db_update_task + run_deep_scan_task
- get_duplicate_cleaner_state + duplicate_cleaner_lock +
duplicate_cleaner_executor + run_duplicate_cleaner
- get_quality_scanner_state + quality_scanner_lock +
quality_scanner_executor + run_quality_scanner
- download_orchestrator + run_async + tasks_lock +
get_download_batches + get_download_tasks +
sweep_empty_download_directories + get_staging_path
- docker_resolve_path + get_current_profile_id +
get_watchlist_scanner + get_app + get_beatport_data_cache
- set_db_update_automation_id (writes the legacy global so the live
DB-update progress callbacks still living in web_server.py keep
emitting against the active automation card)
`web_server._register_automation_handlers` is now ~50 lines: build
deps once, call register_all. The 667-line block of remaining
closure definitions and engine register calls is gone.
The final orphan was the `_db_update_automation_id` module global —
the DB-update progress callbacks at line ~14080 still read it
directly, so the extracted database_update handler propagates the
automation id through `deps.set_db_update_automation_id` (a closure
in web_server that writes the global). When the legacy callbacks
get extracted in a future PR the setter goes away.
Tests:
- tests/automation/test_handlers_maintenance.py adds 21 boundary
tests covering every newly-extracted handler shape: guard
short-circuits (already-running returns skipped), deps wiring
(set_db_update_automation_id called with the right id),
exception swallow contract, status returns, path-traversal
blocked in run_script, source-mode skip in clean_search_history,
active-batch skip in clean_completed_downloads, etc.
- 3244 tests pass (was 3223 — 21 new), no regression.
web_server.py: 35,593 -> 34,220 lines (-1,373 net across 3 commits).
Issue #1 from the extraction punch list is now COMPLETE.
Continues the lift from `web_server._register_automation_handlers`.
This commit extracts the four playlist-lifecycle closures:
- `refresh_mirrored` -> core/automation/handlers/refresh_mirrored.py
- `sync_playlist` -> core/automation/handlers/sync_playlist.py
- `discover_playlist` -> core/automation/handlers/discover_playlist.py
- `playlist_pipeline` -> core/automation/handlers/playlist_pipeline.py
The pipeline composes refresh + sync + discover, so all four ship
together. The pipeline imports the other three handler modules
directly (cross-handler call) instead of going through the engine,
preserving the "single trigger from the user's perspective" UX.
`AutomationDeps` grew to cover the new dependency surface:
- run_playlist_discovery_worker, run_sync_task, load_sync_status_file
(pre-existing background-task entry points)
- get_deezer_client, parse_youtube_playlist (per-source clients)
- get_sync_states (live mutable accessor for the sync UI's state dict)
`web_server._register_automation_handlers` now wires those plus the
existing infrastructure into a single `AutomationDeps` and calls
`register_all`. The 669-line block of closure definitions and engine
register calls (lines 959-1627 pre-edit) is gone -- the file shed
743 lines net on this commit.
`tests/automation/test_handlers_playlist.py` adds 17 new boundary
tests:
- discover_playlist: no_id error, specific_id starts worker, all=True
enumerates, no playlists in db
- refresh_mirrored: error path, source filter (file/beatport excluded),
Spotify happy path with auto-discovered marker, per-playlist
exception captured into errors counter
- sync_playlist: no_id, not_found, no_tracks, no-discovered-tracks
skip, discovered-track happy path, unchanged-since-last-sync skip
- playlist_pipeline: no_playlist clears running flag, no-refreshable
clears running flag, exception clears running flag
3223 tests pass. web_server.py: 35,593 -> 34,850 lines (743 removed).
Begins the lift of `web_server._register_automation_handlers` (1530
lines, 20 nested closures) into `core/automation/handlers/`. Each
extracted handler is a top-level function that accepts
`(config, deps)` instead of reaching for module-level globals --
makes them unit-testable in isolation.
Infrastructure:
- `core/automation/deps.py`: `AutomationDeps` (dependency-injection
bundle of clients + callables) and `AutomationState` (mutable flags
shared across handler invocations, with thread-safe accessors).
- `core/automation/handlers/__init__.py` + `registration.py`: one-stop
`register_all(deps)` that wires every extracted handler to the
engine.
First batch of handlers extracted:
- `process_wishlist` -> `core/automation/handlers/process_wishlist.py`
- `scan_watchlist` -> `core/automation/handlers/scan_watchlist.py`
- `scan_library` -> `core/automation/handlers/scan_library.py`
`web_server._register_automation_handlers` now builds the deps once
and calls `register_all(deps)` for the extracted batch. Remaining
17 closures still live below; subsequent commits in this branch
finish the lift.
14 boundary tests in `tests/automation/test_handlers_simple.py` pin
every shape: success path, exception swallow contract, fresh-vs-stale
state detection (scan_watchlist's id() trick), guard short-circuits,
state cleanup on exceptions, AutomationState concurrent-safe accessors.
All 101 automation tests pass; no regression.
The first token-leak fix scrubbed the artwork URL fixer's own log
calls. This catches three more sites that ALSO leaked tokens, plus
one upstream gap that let URL-encoded tokens slip through the
redactor.
Three sites in `web_server.py` (artist endpoint at line 8765-8773):
- "Artist image before fix: '...'" -- logged the raw image_url with
the auth token in plain form.
- "Artist image after fix: '...'" -- logged the URL-encoded form
after it had been wrapped in the image proxy
(`/api/image-proxy?url=<percent-encoded-token>`).
- "Final artist data being sent: {...}" -- dumped the entire
artist_info dict on every render, including the image_url field.
All three were dev-time debug noise. Removed entirely. The "No
artist image URL found" warning at line 8770 stays (no URL, just
the artist name).
One site in `core/discovery/sync.py:402`:
- "[PLAYLIST IMAGE] image_url=..." -- logged the playlist poster URL
during sync. Same auth-token leak risk for Plex / Jellyfin
playlists. Changed to log only `has_image=True/False`.
Upstream gap in `_redact_url_secrets`:
- The original regex only matched plain query params (`?key=value`).
When an auth-bearing URL gets wrapped inside another URL's query
string (our `/api/image-proxy?url=<encoded>` flow) the auth params
end up percent-encoded -- `%3FX-Plex-Token%3D...` -- and slipped
through.
- New second pattern catches the URL-encoded form. Both passes run
on every redact call; idempotent.
Verified manually:
/api/image-proxy?url=...%3FX-Plex-Token%3DABC...
-> /api/image-proxy?url=...%3FX-Plex-Token%3D***REDACTED***
6 artwork tests pass.
Adds an opt-in alternative metadata source for reorganize. The
existing API path (query Spotify / iTunes / Deezer / Discogs /
Hydrabase for the canonical tracklist) stays the default and is
unchanged. The new tag mode reads each file's embedded tags as the
source of truth instead -- useful for well-enriched libraries where
API drift can produce inconsistent renames, and avoids API calls
entirely.
- New pure helper `core/library/reorganize_tag_source.py` adapts the
output of `read_embedded_tags` (the same mutagen path the audit-
trail modal uses) to the `api_album` / `api_track` shapes that
`_build_post_process_context` already consumes. Handles ID3-style
"5/12" track + disc shapes, multi-value Artists tags, year
normalization across 5 date formats, releasetype canonical tokens,
multi-artist string splits across 9 separators.
- `plan_album_reorganize` accepts `metadata_source: 'api' | 'tags'`
(default 'api') and `resolve_file_path_fn`. Tag mode branches into
a new `_plan_from_tags` that reads each track's file and produces
per-item `api_album` + `api_track` instead of a shared one.
- `_run_post_process_for_track` accepts a per-item `api_album`
override so each file's own album metadata flows through post-
process (not a single shared dict).
- `total_discs` in tag mode honors the `totaldiscs` tag and the
trailing `/N` of an ID3 `discnumber = "1/2"`. Partial-album
reorganize still routes into the correct `Disc N/` subfolder when
the tag knows the total even if not all discs are present locally.
- Bare `discnumber = "1"` no longer poisons `total_discs` -- it
carries no total signal.
- `reorganize_album` surfaces a tag-mode-specific error when no
files are readable, instead of the API-mode "run enrichment first"
message which would mislead in tag mode.
- `QueueItem.metadata_source` field, `enqueue` / `enqueue_many`
pass-through, runner injects `item.metadata_source` into
`reorganize_album`.
- `web_server.py` endpoints accept `mode` body param. Falls back to
the `library.reorganize_metadata_source` config setting, then to
'api'. Strict allowlist (api / tags) -- anything else falls back.
- Frontend: per-album modal + reorganize-all modal both grow a new
"Metadata Mode" dropdown above the source picker. Tag mode hides
the source picker (irrelevant). Choice persisted in localStorage.
Both preview + execute fetches send `mode` in body.
Tests:
- 49 boundary tests on the pure helper pin every shape: ID3 "5/12",
multi-artist split, year normalization, releasetype validation,
total_discs precedence, defensive paths.
- 6 planner-level integration tests pin the wiring: tag-mode with
good tags, partial-disc with totaldiscs tag, file missing,
some-match-some-fail, defensive resolve_file_path_fn=None,
API-mode regression guard.
- All 3171 tests pass; 52 existing reorganize tests unchanged.
Discord report (netti93). The download flow runs `enhance_file_metadata`
(clears all tags) then `generate_lrc_file` (writes .lrc sidecar AND
embeds USLT). The retag flow only ran the first half — `enhance_file_metadata`
cleared USLT and there was no follow-up to restore it.
Two coordinated fixes (no new setting per kettui scope discipline —
user described it as "might even be an idea," consistency was the
load-bearing ask).
Fix 1 — retag calls generate_lrc_file after enhance
`core/library/retag.py:execute_retag` now invokes
`deps.generate_lrc_file` right after the `enhance_file_metadata`
call, mirroring the download pipeline. New `generate_lrc_file`
field on `RetagDeps`, defaults to None for backward compat with
any test caller that builds RetagDeps without it. Web_server's
`_build_retag_deps()` factory wires in the real
`core.metadata.lyrics.generate_lrc_file`.
Placement matters — runs BEFORE `safe_move_file` so the helper
sees the audio file at its current path with its existing sidecar
(which retag hasn't moved yet). After the embed, the audio file
gets moved with USLT now present; the sidecar move step that
follows is unaffected.
Fix 2 — create_lrc_file re-embeds from existing sidecar
`core/lyrics_client.py:create_lrc_file` used to early-return True
when an .lrc / .txt sidecar already existed (skipping the LRClib
fetch). For the retag case the sidecar is already there, so the
shortcut hit and USLT was never re-written. Now the helper reads
the existing sidecar and calls `_embed_lyrics` with its content
before returning. Empty / unreadable sidecars short-circuit
silently — defensive, no crash. Download flow unaffected because
no sidecar exists at fetch time.
7 boundary tests pin: existing .lrc triggers re-embed, existing
.txt triggers re-embed, empty sidecar skips embed, unreadable
sidecar swallows error, no sidecar falls through to LRClib (download
path regression guard), RetagDeps.generate_lrc_file field accepted,
field optional for backward compat.
Full suite: 3120 passed.
Closes#585. When a Spotify source track had a versioned suffix not
present in the local file ("Iron Man - 2012 - Remaster" vs "Iron Man"),
the auto-matcher missed the pair. User could click Find & Add to pick
the right local file — that worked, file got added to the Plex
playlist — but the source track stayed in Missing while the added
file appeared in Extra, because the matcher kept no record of the
user-confirmed pairing. On the next sync the source track re-tried
to download.
Fix: every Find & Add selection now writes a (spotify_track_id →
server_track_id) override into sync_match_cache at confidence=1.0.
The matching algorithm runs an override pass BEFORE the existing
exact and fuzzy passes, so any user-confirmed pair short-circuits
straight to "matched" without going through title normalization.
Covers every mismatch class — dash-suffix remasters, covers /
karaoke, alt masters, cross-language titles, typo'd local files.
- core/sync/match_overrides.py (new) — pure helpers
resolve_match_overrides + record_manual_match. 18 boundary tests
pin: cache hits, cache misses falling through to normal matching,
stale-cache (server track removed) handled gracefully, str/int
id coercion, partial cache hits, defensive against non-dict
inputs and DB exceptions.
- web_server.py — get_server_playlist_tracks runs the override
pre-pass before exact/fuzzy matching. server_playlist_add_track
accepts source_track_id + source_title + source_artist and
persists the override after every successful add (Plex / Jellyfin
/ Navidrome). source_track_id added to source_tracks payload so
the frontend has it.
- webui/static/pages-extra.js — _serverSelectTrack sends
source_track_id + source_title + source_artist when adding a
track from a mirrored playlist context.
- Sync match cache schema unchanged — already had UNIQUE
(spotify_track_id, server_source) which fits the override
semantics perfectly. Manual overrides distinguished from
auto-discovered matches by confidence=1.0.
Full suite: 3010 passed.
Closes#584. Quarantined files used to sit in ss_quarantine/ with a
thin sidecar — no UI, no recovery, no way to see what got dropped.
This adds the management surface the user needs without going to the
filesystem.
UI: new "Quarantine" button on the downloads page header opens a
modal with every quarantined file (filename, expected track/artist,
reason, when, size). Three actions per row:
- Approve (one-click): restores the file, re-runs the post-process
pipeline with ONLY the failing check skipped, lands in the library
with full tags + lyrics + scan
- Recover (legacy fallback): moves to Staging for thin-sidecar
entries that lack the embedded context Approve needs
- Delete: permanent removal of file + sidecar
Per-check bypass: context['_skip_quarantine_check'] = 'integrity' /
'acoustid' / 'bit_depth'. Skips ONLY the named check — other quality
gates stay live. No blanket bypass-all flag.
Sidecar expansion: move_to_quarantine now persists the full
json-serializable context via serialize_quarantine_context (drops
non-JSON-safe values, walks nested dicts/lists/sets, str-coerces
unknown objects) plus the trigger name. Existing thin sidecars are
detected and routed to Recover instead of Approve.
Pure helpers in core/imports/quarantine.py: list_quarantine_entries
/ delete_quarantine_entry / approve_quarantine_entry /
recover_to_staging / serialize_quarantine_context. 27 tests pin
every shape: orphan files / orphan sidecars / corrupt sidecars /
collision-safe filename restoration / full-context vs thin-sidecar
dispatch / json round-trip safety.
Four new endpoints in web_server.py — thin glue around the helpers:
GET /api/quarantine/list, DELETE /api/quarantine/<id>,
POST /api/quarantine/<id>/approve, POST /api/quarantine/<id>/recover.
Download modal status differentiates "🛡️ Quarantined" from
"❌ Failed" so recoverable files are visible at a glance — checked
against the error_message text, no schema change needed.
Pipeline changes are three minimal per-check conditionals at the
existing quarantine sites in core/imports/pipeline.py. Each
move_to_quarantine call now passes its trigger name so the sidecar
records which check fired.
Full suite: 2992 passed.
Move the remaining manual import endpoint logic out of web_server.py and into core.imports.routes behind ImportRouteRuntime. The Flask endpoints now stay as thin compatibility wrappers for album/track search, album match/process, single-file import processing, and batched singles processing.
Keep legacy test patch points intact by re-exporting build_album_import_match_payload from web_server and routing singles_process through an injected process_single_import_file callable. This preserves existing route-level monkeypatch behavior while keeping the extracted helper testable.
Add focused helper coverage for Hydrabase enqueueing, search limit clamping, album match payload forwarding, album import side effects, single-file worker outcomes, malformed manual matches, and singles aggregation/injected-worker behavior.
Verification: py_compile and git diff --check passed locally; bundled-Python smoke covered the extracted helpers. Claude reran the project tests and reported all tests passing.
Move import staging files/groups/hints/suggestions controller logic out of web_server.py and into core.imports.routes behind an ImportRouteRuntime dependency object. Keep the existing Flask routes as thin compatibility wrappers so the UI endpoint surface stays unchanged.
Add focused tests for staging file filtering, album grouping, hint generation, cached suggestions, empty missing staging paths, and error payloads from failed path/metadata reads.
Verification: py_compile passed for web_server.py, core/imports/routes.py, and tests/imports/test_import_routes.py. A bundled-Python smoke pass covered the extracted helper behavior; pytest was not available in this Windows shell because the bundled Python lacks pytest and the repo venv is WSL/Linux-only here.
Discord report: prolific artists (Bach, Beatles complete box,
deep dance/electronic catalogues) only showed ~50 entries in the
"Download Discography" modal.
`MetadataLookupOptions(limit=50, max_pages=0)` was hardcoded at
three call sites. Spotify's `max_pages=0` already paginates
through everything (per-page is clamped to 10 internally), so
Spotify-primary users were unaffected. But Deezer / iTunes /
Discogs / Hydrabase all honor the outer `limit` as a hard cap,
so non-Spotify users were silently clipped.
Bump `limit` to 200 at all three call sites — matches iTunes's
and Discogs's own internal caps and covers near-everyone's full
catalogue. Spotify behavior unchanged.
- web_server.py:9221 — discography endpoint (modal)
- web_server.py:8700 — artist-detail discography view
- core/artist_source_detail.py:129 — source-specific artist detail
- Replace the shell convenience script with a cross-platform Python launcher.
- Keep dev.sh as a Unix compatibility wrapper.
- Let the direct backend bind with host and port overrides.
- Update the root and webui README guidance for the new launcher.
- Preserve the backend startup behavior used by the old dev flow.
- Drop unused _resolve_webui_initial_* helpers from web_server.py.
- Remove template-side initial_nav_page and initial_client_page conditionals.
- Keep Vite asset injection and runtime page activation in the client.
Remove the Flask route-to-page helpers and stop passing initial active-page flags into the shell template.
The web UI now renders static page and nav markup, while the client-side shell remains responsible for establishing active page state after load. This keeps the hybrid Flask + Vite asset setup intact while reducing duplicated route/page ownership logic in the backend template layer.
Also added a previously missing /stream path to the spa exclusions
- File-based routing with tanstack router
- Persist top-level navigation state in url, even for most legacy pages
- Striving for an intuitive and simple folder structure where
route-related code is colocated, but the amount of files is still
kept to a minimum
- Replace native fetch with `ky`
- Familiar api, but more polished
Broque Thomas
BoulderBadgeDad
Antti Kettunen