mirror of https://github.com/Nezreka/SoulSync.git
dev
main
fix/usenet-album-poll-sab-handoff
fix/quarantine-source-dedup
release/2.5.3
fix/disable-beatport-features
johnbaumb-discover-redesign
1.0
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
2.0
2.1
2.2
2.3
2.4.0
2.4.1
2.4.2
2.5.0
2.5.1
2.5.2
2.5.3
2.5.4
2.5.5
2.5.6
2.5.7
2.5.9
2.6.0
2.6.1
2.6.2
2.6.3
2.6.4
2.6.5
2.6.6
2.6.7
2.6.8
v0.65
${ noResults }
2882 Commits (f9f74ac51157166e01a2513993eaea2add079dec)
| Author | SHA1 | Message | Date |
|---|---|---|---|
Broque Thomas
|
f9f74ac511 |
Lift auto-import matching to testable helper + pin contracts
Cin-pass on the #524 + multi-disc fixes. Pre-merge polish. Lifts: `core/imports/album_matching.py` `AutoImportWorker._match_tracks` was a 100+-line method buried in a 1400-line class. Testing it required monkey-patching `_read_file_tags` + mocking the metadata client just to exercise the matching algorithm. Per Cin's "lift logic out of monolithic classes" pattern (same shape as the album-info builders / discography / quality scanner lifts), moved the dedup + scoring into `core/imports/album_matching.py` as pure functions over already-fetched data. Helper exposes: - Constants for every match weight (TITLE_WEIGHT, ARTIST_WEIGHT, POSITION_WEIGHT, NEAR_POSITION_WEIGHT, CROSS_DISC_POSITION_WEIGHT, ALBUM_WEIGHT, MATCH_THRESHOLD). Magic numbers killed. - `dedupe_files_by_position(audio_files, file_tags, *, quality_rank)` — position-keyed quality dedup. - `score_file_against_track(file_path, file_tags, track, *, target_album, similarity)` — pure per-(file, track) scorer. - `match_files_to_tracks(audio_files, file_tags, tracks, *, target_album, similarity, quality_rank)` — full matching with greedy best-per-track + first-come-first-serve over deduped files. Worker shrinks from 100 lines of inline algorithm to 8 lines that fetch tags + delegate to the helper. Tests added (26 new across 3 files): `tests/imports/test_album_matching_helper.py` (19 tests): - Constants pin: weights sum to 1.0, threshold above position-only - `dedupe_files_by_position`: quality wins, cross-disc preserved, tag-less files passed through, first-wins on equal quality - `score_file_against_track`: perfect-agreement = 1.0, position needs both disc+track, near-position only same-disc, missing artist tags handled, disc field aliases (Spotify/Deezer/iTunes), filename fallback when title tag missing - `match_files_to_tracks`: happy path, file used at-most-once, below-threshold left unmatched - Edge case Cin would flag: tag-less file with strong filename title matches multi-disc album track via title alone (perfect-name scenario works); tag-less file with weak filename title against multi-disc API correctly stays unmatched (the behavior delta from the disc-aware fix — pinned so future readers see it's intentional) `tests/test_import_album_match_endpoint.py` (3 tests): - Backend warning fires when source missing from match POST - No warning fires on the legit path (catches noisy-warning regression) - Endpoint actually forwards source/name/artist to the payload builder (catches "logging the right warning but doing the wrong lookup" regression) `tests/test_import_page_album_lookup_pattern.py` (4 tests): - Source-text guard for the import-page #524 fix in stats-automations.js. Until the file is modularized enough for a behavioral JS test (under the existing tests/static/*.mjs pattern), regex-based assertions pin: the `_albumLookup` field exists, the click handler reads from it, both card renderers populate it before emitting onclick, and the cache stores `source` per entry. Caveat documented in the test module docstring. Verification: - All 26 new tests pass. - Existing multi-disc tests (test_auto_import_multi_disc_matching.py) still pass after the lift — proves the helper is behavior-equivalent to the inline implementation it replaced. - Full suite: 2293 passed, 1 flaky-timing failure (test_library_reorganize_orchestrator.py::test_watchdog_warns_about_stuck_workers — passes in isolation, fails only in full-suite runs, pre-existing, unrelated to this PR). - Ruff clean. Notes for the reviewer: - The frontend stats-automations.js JS test is structural-only. Behavioral JS testing for that file requires modularizing the ~7k-line monolith first — out of scope for this fix. - The cross-disc 5% consolation bonus is a small behavior change for users with weak/missing tag info on multi-disc albums. Pinned explicitly in `test_tagless_file_with_weak_title_unmatched_in_multidisc` so the trade-off is visible: correct multi-disc matching wins over optimistic position-only matching that produced wrong-disc files. |
1 month ago |
|
Broque Thomas
|
c03edc3cb4 |
Auto-import: respect disc_number in dedup + match scoring
Caught while live-testing the #524 fix with kendrick lamar mr morale & the big steppers (3 discs). User dropped discs 1+2 loose in staging root + disc 3 in its own folder, every file perfectly tagged with disc_number/track_number/title — only 9 tracks ended up in the library, the rest got integrity-rejected and quarantined. Two related bugs in `AutoImportWorker._match_tracks`: 1. **Quality dedup keyed on track_number alone.** The dedup loop kept `seen_track_nums[track_number] = file` and dropped any later file with the same number, treating it as a quality duplicate. On a multi-disc release where every disc has tracks 1..N, that collapses the album to one disc's worth of files BEFORE the matcher runs. User's 18 loose disc-1+disc-2 files reduced to 9 before any title/disc info was even consulted. 2. **Match scoring ignored disc_number.** The 30% track-number bonus fired whenever `ft[track_number] == track_num` regardless of disc. File with tag (disc=2, track=6, "Auntie Diaries", 281s) got the full bonus matching API track (disc=1, track=6, "Rich Interlude", 103s) — wrong file → wrong destination → integrity check correctly rejected and quarantined the file. Same for tracks 7, 8, 9. Fix: - Dedup keys on `(disc_number, track_number)` tuples — multi-disc files with parallel numbering all survive. - Match scoring's 30% bonus only when BOTH disc AND track agree. Cross-disc same-track-number collisions get a small 5% consolation bonus so title similarity has to carry the match (covers cases where tag disc info is missing or wrong). - API track disc_number read from `disc_number` (Spotify) / `disk_number` (Deezer) / `discNumber` (iTunes) defaulting to 1. 4 new pinning tests in `tests/imports/test_auto_import_multi_disc_matching.py`: - 18-file 2-disc regression case (dedup preserves all) - (disc=2, track=6) file matches API (disc=2, track=6) track, not the disc-1 same-numbered track - Single-disc albums still match normally (no regression) - Quality dedup within a single (disc, track) position still picks higher-quality format (.flac over .mp3) Verification: - 2268 full pytest suite passes (+4 new), 1 skipped, 0 failed - Ruff clean Same branch as the #524 fix because both surfaced from the same import session — easier reviewer context if they ship together. |
1 month ago |
|
Broque Thomas
|
f58f202d32 |
Fix manual album import losing source — issue #524
radoslav-orlov reported every imported album landing in the soulsync
standalone library as "Unknown Artist" + the raw 10-digit album id
as the title + 0 tracks. Audit traced it to the click handler in the
import page dropping the source-of-the-album_id on its way to the
backend match endpoint.
Root cause:
`importPageSelectAlbum(albumId)` (the onclick on every suggestion /
search-result card) only passed the album_id string. The full search
response carried `source`, `name`, and `artist` per row — the
backend's `get_artist_album_tracks` needs source so it can route the
lookup to the metadata source the id actually came from. Without it,
the source chain tries each source's `get_album(id)` against an id
shaped for a different source — a Deezer numeric id against
Spotify's id format returns 404, against iTunes's collectionId range
returns 404, etc. — and falls through to the failure-fallback dict
in `get_artist_album_tracks`:
{
'success': False,
'album': {'name': album_name or album_id, 'total_tracks': 0,
'release_date': '', ...}, # no artist field at all
'tracks': [],
}
That broken album dict then flowed through `build_album_import_context`
→ post-processing pipeline → `record_soulsync_library_entry`, writing
"Unknown Artist" + album_id-as-title + 0 tracks rows into the
soulsync standalone library tables.
Why hybrid users hit it most: a Spotify-primary user searching for an
album → search returns the Spotify result PLUS Deezer fallbacks
(via `_search_albums_for_source`'s priority chain). Clicking a Deezer
fallback row then sent only the Deezer id to /album/match without
flagging that source — Spotify-first chain failed against the Deezer
id and the broken fallback got written.
Fix:
Frontend (`webui/static/stats-automations.js`):
- New `importPageState._albumLookup: { albumId: { id, name, artist,
source } }` populated by both card renderers (`_renderSuggestionCard`
+ the search-results render block) before they emit the onclick.
- `importPageSelectAlbum` reads source / name / artist from that
cache and includes them in the match POST body, so the backend
routes to the correct provider's `get_album` on the very first try.
- `_escAttr` applied to album_id in the onclick (defensive — ids
shouldn't contain quotes but `_escAttr` was already being used on
every other field interpolated into onclick attributes).
Backend (`web_server.py:import_album_match`):
- Defensive log warning when source is missing from the request body.
Catches any future regression where another caller (curl /
third-party / new UI flow) drops source again — it'll show up as
a visible warning in app.log instead of silently corrupting the
library.
Verification:
- Full pytest suite: 2264 passed, 1 skipped, 0 failed
- Ruff clean
- JS syntax clean
- Manual repro requires a real user flow (search albums on the
import page → click one → import) which isn't covered by the
existing unit tests; reviewer should verify against issue #524's
steps before merge.
|
1 month ago |
BoulderBadgeDad
|
2da1e8b2d9
|
Merge pull request #532 from Nezreka/fix/docker-image-ffmpeg-bloat
Fix/docker image ffmpeg bloat |
1 month ago |
|
Broque Thomas
|
48aefbacdd |
Drop redundant `import sys` inside _auto_download_disabled
Ruff F811 — `sys` is already imported at module top (line 13). The local `import sys` inside `_auto_download_disabled` shadowed it needlessly. Caught by CI ruff check on the dev-nightly workflow. |
1 month ago |
|
Broque Thomas
|
950857ba40 |
ffmpeg gate also covers is_available — fixes the actual leak path
Previous commit split _check_ffmpeg into a side-effect-free
_locate_ffmpeg + the original auto-download _check_ffmpeg, and moved
__init__ to call _locate_ffmpeg. That alone wasn't enough — caught
the gap during a deeper audit:
is_configured() → is_available() → _check_ffmpeg() (with download)
The orchestrator registry, download engine, and the orchestrator's
own configured_clients() all probe is_configured() polymorphically at
boot. So when tests import web_server, the registry probes
youtube.is_configured() → is_available() → _check_ffmpeg() →
DOWNLOAD. My __init__ change didn't help because the registry boot
fires the same code path right after.
Real fix: gate the download branch inside _check_ffmpeg itself.
Returns False (and logs a warning) when running under pytest or when
SOULSYNC_NO_FFMPEG_DOWNLOAD=1. End users on a fresh install still get
auto-download on first real YouTube use (gate is off in production).
Container is unaffected (system ffmpeg via apt is found on PATH, the
download branch never runs).
Three detection paths in _auto_download_disabled():
- SOULSYNC_NO_FFMPEG_DOWNLOAD env var (explicit opt-out for CI /
build steps that want to disable outside pytest)
- PYTEST_CURRENT_TEST env var (set by pytest per-test — covers
in-test-body call path)
- 'pytest' in sys.modules (covers calls fired during pytest collection
/ import phase, BEFORE the per-test env var is set — which is
exactly when registry.py probes is_configured() at web_server
import time)
Verified by inspecting tools/ after a full suite run — empty (was
~388 MB after a single test_tidal_auth_instructions.py run before
the gate). Container behavior unchanged: shutil.which('ffmpeg')
returns /usr/bin/ffmpeg from the apt-installed package, so the
download branch is never reached anyway.
5 new pinning tests:
- pytest-in-sys.modules detection works
- PYTEST_CURRENT_TEST env detection works
- SOULSYNC_NO_FFMPEG_DOWNLOAD env detection works
- _check_ffmpeg returns False (no urlretrieve, no tools/ dir created)
when gate is on and ffmpeg is missing — pinned by trapping
urlretrieve to AssertionError so a regression blows up loud
- _locate_ffmpeg never triggers download or creates tools/ —
pinned by trapping both urlretrieve AND Path.mkdir on tools-prefixed
paths
2264 passed (+5), 1 skipped, 0 failed.
|
1 month ago |
|
Broque Thomas
|
70e1750948 |
Stop docker image bloat from auto-downloaded ffmpeg
kettui reported the dev image roughly doubled in size after a recent nightly build. codex investigation traced it back to: 1. nightly workflow runs `python -m pytest` before docker build 2. one of the new tests imports web_server (test_tidal_auth_instructions.py) 3. importing web_server constructs YouTubeClient 4. YouTubeClient.__init__ called _check_ffmpeg() — which auto-downloads a ~388 MB ffmpeg/ffprobe bundle into ./tools/ when system ffmpeg isn't on PATH (CI runner doesn't have it) 5. .dockerignore didn't exclude tools/ffmpeg or tools/ffprobe 6. docker `COPY . .` shipped the binaries 7. the immediately-following `chown -R /app` rewrote every file into a new layer — so the 388 MB payload got counted twice in image size three fixes: 1. .dockerignore — block the auto-downloaded binaries even if they leak into the workspace (tools/ffmpeg, tools/ffprobe, .exe variants, .zip and .tar.xz download archives). Defense-in-depth so a future regression in the test/import path can't bloat the image again. 2. youtube_client — split _check_ffmpeg into a side-effect-free _locate_ffmpeg (pure existence check) and the original auto- download _check_ffmpeg. __init__ now calls _locate_ffmpeg + logs a warning when missing instead of triggering download. is_available() and the actual download dispatch paths still call _check_ffmpeg — so end users still get auto-download on first YouTube use, but `import web_server` doesn't drag a 388 MB binary into the workspace. 3. Dockerfile — replaced `COPY . .` + `chown -R /app` with `COPY --chown=soulsync:soulsync . .` + a scoped chown on just the runtime mount-point dirs. eliminates the layer that duplicated the entire /app tree just to flip ownership bits, so even legit workspace content isn't double-counted in the image. Combined effect: image size returns to baseline + future ffmpeg leaks can't bloat it. Inside the container nothing changes — the Dockerfile already installs system ffmpeg via apt, so YouTube downloads find it on PATH on first use and the auto-download path never fires. 2259 passed, 1 skipped, 0 failed. |
1 month ago |
|
BoulderBadgeDad
|
661f00cb35
|
Merge pull request #531 from Nezreka/feat/candidates-modal-manual-search
Feat/candidates modal manual search |
1 month ago |
|
Broque Thomas
|
e20994e1c7 |
Manual picks: stream results, don't auto-retry, fix stuck-at-0%
Three follow-on fixes to the manual-search candidates modal once people started actually using it: 1. NDJSON streaming. Manual search waited for every source to return before showing anything. Now streams one event per source as each completes — header line, source_results per source, done terminator. Frontend appends rows incrementally via response.body.getReader(). 2. Manual picks no longer auto-retry on failure. New _user_manual_pick flag set on the task in /download-candidate. Both monitor retry paths (not-in-live-transfers stuck + Errored state) bail on the flag. Surfaces the failure to the user instead of silently picking a different candidate via fresh search. 3. Non-Soulseek manual picks (youtube/tidal/qobuz/hifi/deezer/ soundcloud/lidarr) no longer stuck at "downloading 0%" forever. The live_transfers IF branch now marks manual-pick tasks failed directly when the engine reports Errored, instead of deferring to the monitor (which bails on manual picks). Engine fallback in else branch covers the rare race where the orchestrator's pre-populated transfer lookup is missing the entry. Plus a deadlock fix discovered along the way: the new failure path synchronously called on_download_completed while holding tasks_lock, which itself re-acquires the same Lock — non-reentrant threading.Lock self-deadlocked the polling thread. While wedged, every other endpoint that needed the lock (including /candidates → other failed rows couldn't open modals) hung waiting. Moved completion callbacks onto a daemon thread so the lock releases first. Plus failed/not_found/cancelled rows are now ALWAYS clickable (not just when the auto-search cached candidates) — the modal carries the manual search bar, which is the user's recourse for empty results. Plus manual download worker now runs on a dedicated thread instead of competing with the batch's 3-worker missing_download_executor pool — saturated batches no longer queue manual picks indefinitely. All scoped to manual picks via the _user_manual_pick flag — auto attempt flow byte-identical to before. Engine fallback gated on the flag too so auto attempts in the else branch keep the original do-nothing behavior (safety valve handles the stuck-forever case). Also dropped _handle_failed_download from web_server.py — defined but had no callers (dead code). 17 new unit tests pin the gate behavior: - engine fallback: Errored/Cancelled/Succeeded/InProgress transitions, manual-pick gate, terminal-state skip, soulseek skip, missing download_id skip, engine returning None, orchestrator exception - monitor: manual-pick skips not-in-live-transfers retry + Errored retry - IF-branch end-to-end: Errored marks failed, "Completed, Errored" hits failure branch, auto attempts defer to monitor Manual-search endpoint tests rewritten for NDJSON: 11 cases (validation, single-source dispatch, parallel "all" dispatch, one-event-per-source streaming shape, unconfigured-source skip + reject, header metadata, per-source exception isolation). Full suite 2259 passed, 1 skipped. |
1 month ago |
|
Broque Thomas
|
996575fab3 |
Add manual search to the failed-track candidates modal
When an auto-download fails or returns "not found" with leftover
candidates, the user can already click the status cell to open a
modal showing those candidates and pick a different one. This adds
a manual search bar to that modal — type any query, hit search,
get a fresh round of results without having to bail out and start
over from the main search page.
Solves the case where the auto-query was bad (featured artist not
in title, parentheticals like "(Remastered 2019)" tripping the
matcher, slight artist-name variants, transliteration) but the
file genuinely exists on the source.
Frontend (downloads.js)
- Added a manual-search section above the existing auto-candidates
table inside the candidates modal.
- Source picker is smart per download mode:
- Single-source mode (soulseek-only / youtube-only / etc) shows
a "Searching X" label, no dropdown.
- Hybrid mode shows a dropdown with "All sources" default + every
configured source. Picking "All" runs parallel searches across
them and tags each result row with its source badge.
- Only configured sources show up; unconfigured are hidden.
- Validation: button disabled until query length >= 2, "Type at
least 2 characters" hint until threshold crosses.
- Loading state on search button while the request is in flight.
- Manual results render in a separate table above the existing
auto-candidates table, using the same row template (file /
quality / size / duration / user / ⬇ button) so the renderer
helper is shared.
- Click ⬇ reuses the existing `downloadCandidate(taskId, candidate,
trackName)` flow — same retry path, same AcoustID verification
when the file lands, no shortcut around the safety net.
- Re-running the search with a different query replaces the
previous manual results.
Backend (web_server.py)
- Extended `GET /api/downloads/task/<id>/candidates` response with:
- `download_mode` (e.g. 'hybrid', 'soulseek')
- `available_sources` (list of configured source IDs + labels)
- `source` field on each candidate (purely additive — frontend
auto-renderer ignores it on legacy code paths, manual-search
renderer uses it for the badge)
- Added `POST /api/downloads/task/<id>/manual-search`:
- Body: `{ query, source: 'all' | <source_id> }`
- Validates query length (>=2 trimmed) → 400
- Validates source against the configured-sources gate → 400
(rejects unconfigured sources even when explicitly named)
- For 'all': parallel `ThreadPoolExecutor` dispatch across every
configured download source, merged results
- For specific source: just that source
- Returns same shape as `/candidates` so the frontend renderer
is reused
- New module-level helpers: `_STREAMING_SOURCE_NAMES`,
`_infer_candidate_source`, `_serialize_candidate`,
`_list_available_download_sources`. The existing `/candidates`
endpoint also goes through `_serialize_candidate` so the source
badge is consistent across both flows.
Behavior preserved
- Existing modal layout / candidates table / ⬇ button are
byte-identical when the user doesn't use manual search.
- `downloadCandidate()` JS function untouched.
- `/candidates` and `/download-candidate` endpoints
backwards-compatible — only NEW fields added, nothing changed
or removed.
Tests
`tests/test_manual_search_endpoint.py` — 10 tests:
- `test_manual_search_validates_query_length`
- `test_manual_search_validates_source` (whitelist gate)
- `test_manual_search_handles_task_not_found` (404)
- `test_manual_search_dispatches_to_configured_source_only`
- `test_manual_search_all_dispatches_parallel`
- `test_manual_search_skips_unconfigured_sources`
- `test_manual_search_rejects_unconfigured_source_explicitly`
- `test_manual_search_returns_same_shape_as_candidates`
- `test_manual_search_single_source_mode_lists_source` (verifies
`available_sources` reflects the active mode)
- `test_manual_search_isolates_per_source_exceptions` (one source
throwing doesn't kill the merged result)
2242/2242 full suite green (was 2232 + 10 new). Ruff clean.
JS parses clean.
|
1 month ago |
|
BoulderBadgeDad
|
4277648734
|
Merge pull request #526 from Nezreka/release/2.4.3
Bump version to 2.4.3 + make sidebar version dynamic |
1 month ago |
|
Broque Thomas
|
d556ec0fa7 |
Bump version to 2.4.3 + make sidebar version dynamic
- `_SOULSYNC_BASE_VERSION` 2.4.2 → 2.4.3
- helper.js — flip 2.4.3 WHATS_NEW header to "May 8, 2026 — 2.4.3
release"; bump fallback default from 2.4.2 → 2.4.3
- docker-publish.yml — manual-trigger default tag 2.4.2 → 2.4.3
Drive-by — make sidebar version + version-modal subtitle dynamic.
The sidebar version button (`v2.4.1`) and version-modal subtitle
(`Version 2.4.1 — Latest Changes`) were hardcoded text in the HTML.
2.4.2 shipped without these getting bumped — silent drift, easy to
miss at every release.
Added a Flask context_processor that injects `soulsync_version` and
`soulsync_base_version` into every template, then templated the two
hardcoded values:
v{{ soulsync_base_version }}
Version {{ soulsync_base_version }} — Latest Changes
Now bumping `_SOULSYNC_BASE_VERSION` updates the UI everywhere it's
rendered. No more "I forgot to bump the sidebar" at release.
2232/2232 full suite green. Ruff clean. JS parses clean.
|
1 month ago |
|
BoulderBadgeDad
|
d7ab37e3b9
|
Merge pull request #525 from Nezreka/fix/discover-track-selection-correction
Fix/discover track selection correction |
1 month ago |
|
Broque Thomas
|
d75ae48981 |
Discover: sharpen track selection (diversity, source-aware popularity, library dedup, SQL genre)
Four selection-quality fixes on the SoulSync-made discover playlists.
None change public method signatures; all are tightenings on what's
already there.
(1) Diversity for Hidden Gems + Discovery Shuffle
Both used to be `RANDOM() LIMIT N` with no diversity. Could return
50 tracks from one artist or 20 from one album if the discovery
pool happened to be skewed. Both now over-fetch 3x and run the
existing `_apply_diversity_filter`:
- Hidden Gems: max 2 per album, 3 per artist
- Discovery Shuffle: max 2 per album, 2 per artist (tighter — shuffle
should feel maximally varied)
(2) Source-aware popularity thresholds
`popularity >= 60` for "Popular Picks" and `popularity < 40` for
"Hidden Gems" was Spotify-shaped (0-100 scale). Deezer writes its
`rank` value into that column (often six-digit integers); iTunes
writes nothing meaningful. For Deezer-primary users:
- Popular Picks pulled essentially everything (rank >= 60 = all)
- Hidden Gems pulled essentially nothing (rank < 40 = none)
New `_get_popularity_thresholds(source)` helper returns per-source
values:
- Spotify: (60, 40) — the existing 0-100 scale
- Deezer: (500_000, 100_000) — ballpark from real rank values
- iTunes / unknown: (None, None) — skip the popularity filter
entirely, fall back to random + diversity
`get_popular_picks` and `get_hidden_gems` now consult the helper.
When threshold is None they skip the popularity SQL filter. Diversity
+ ID gate still apply.
(3) Push genre keyword filter into SQL
`get_genre_playlist` used to fetch `limit=1_000_000` rows into Python
then run a substring keyword filter on `artist_genres`. Bad on big
discovery pools.
Now the keyword OR chain is generated as SQL placeholders:
AND (artist_genres LIKE ? OR artist_genres LIKE ? OR ...)
Each placeholder gets `f'%{keyword.lower()}%'` via `extra_params`.
`fetch_limit` drops back to `limit * 10`. `_genre_matches` Python
helper deleted (only intra-file caller; verified via grep).
Parent-genre expansion via `GENRE_MAPPING` preserved — keywords list
feeds the LIKE chain unchanged.
(4) Filter out tracks already in library
Discovery pool can include tracks the user already owns. Hidden Gems
/ Shuffle / Popular Picks shouldn't surface those.
`_select_discovery_tracks` gained `exclude_owned: bool = True`
parameter. When True, adds a correlated NOT EXISTS subquery against
the `tracks` table covering all 3 source IDs:
AND NOT EXISTS (
SELECT 1 FROM tracks t WHERE
(t.spotify_track_id IS NOT NULL AND t.spotify_track_id = discovery_pool.spotify_track_id)
OR (t.itunes_track_id IS NOT NULL AND t.itunes_track_id = discovery_pool.itunes_track_id)
OR (t.deezer_id IS NOT NULL AND t.deezer_id = discovery_pool.deezer_track_id)
)
Note column-name asymmetry: tracks.deezer_id vs
discovery_pool.deezer_track_id. Inline comment marks the trap. All
5 public discovery methods automatically benefit (default True).
Seasonal Playlist doesn't go through the helper so it's unaffected
(curated content, dedup is wrong intent there).
Tests
12 new tests in `tests/test_personalized_playlists_id_gate.py` (27
total in the file):
- Hidden Gems + Discovery Shuffle apply diversity (cap proven by
inserting 10 same-artist + same-album rows and asserting return
count ≤ per-album cap)
- Popularity thresholds: Spotify (60, 40), Deezer larger scale,
iTunes None / None
- Popular Picks skips threshold filter when None
- Genre playlist pushes filter to SQL (parent + child genre expansion)
- Owned-track exclusion: filtered when match, kept when no match,
opt-out flag works
- Deezer column-name asymmetry pinned (regression footgun)
Test fixture re-added the minimal `tracks` table (4 columns: id,
spotify_track_id, itunes_track_id, deezer_id) — only what the new
NOT EXISTS subquery needs to join. Plus `insert_library_track`
helper.
Verification
- 27/27 in this test file pass (15 prior + 12 new)
- 2232/2232 full suite green
- ruff clean
LOC delta:
- core/personalized_playlists.py: 1030 → 1101 (+71)
- tests/test_personalized_playlists_id_gate.py: 352 → 616 (+264)
|
1 month ago |
|
Broque Thomas
|
d123581a39 |
Fix: ID gate missed Deezer-track-id-only rows
The original gate baked into `_select_discovery_tracks` only checked
Spotify + iTunes:
AND (spotify_track_id IS NOT NULL OR itunes_track_id IS NOT NULL)
For Deezer-primary users, discovery_pool rows have populated
`deezer_track_id` but NULL Spotify + NULL iTunes IDs. The gate
filtered every row out — Time Machine, Genre Browser, Hidden Gems,
Discovery Shuffle, Popular Picks all rendered "no tracks found" for
every tab on every Deezer-primary install.
Extended the gate to include `deezer_track_id` and added that column
to the standard SELECT column tuple. `_build_track_dict` already
exposed `deezer_track_id` in its output shape, so frontend rendering
needed no changes.
Regression pinned via new test
`test_discovery_helper_accepts_deezer_only_id_rows` — inserts a row
with NULL Spotify + NULL iTunes but a populated `deezer_track_id`
and asserts it survives the gate.
2220/2220 full suite green.
|
1 month ago |
|
Broque Thomas
|
959562f6b0 |
Delete Recently Added / Top Tracks / Forgotten Favorites / Familiar Favorites
Owner decision: not worth shipping. The four library-driven personalized sections were stubbed returning [] for ages because their schema prereqs didn't exist; the prior commit re-enabled them by routing through a new `_select_library_tracks` helper. Owner reviewed and chose to delete the sections entirely instead. Removed everywhere: - `core/personalized_playlists.py` — `get_recently_added`, `get_top_tracks`, `get_forgotten_favorites`, `get_familiar_favorites` + the `_select_library_tracks` helper (no other callers; verified via grep). - `web_server.py` — 4 route handlers (`/api/discover/personalized/recently-added`, `top-tracks`, `forgotten-favorites`, `familiar-favorites`). - `webui/index.html` — 4 `<div class="discover-section">` blocks (`#personalized-recently-added`, `#personalized-top-tracks`, `#personalized-forgotten-favorites`, `#personalized-familiar-favorites`). - `webui/static/discover.js` — 4 load functions (`loadPersonalizedRecentlyAdded`, `loadPersonalizedTopTracks`, `loadPersonalizedForgottenFavorites`, `loadFamiliarFavorites`), plus their entries in `loadDiscoverPage`'s Promise.all, plus 4 module-level state vars + 6 dead branches across `openDownloadModalForDiscoverPlaylist` / `startDiscoverPlaylistSync` and the sync-progress / rehydrate dispatchers. - `webui/static/helper.js` — 4 tooltip / docs entries. - `webui/static/sync-spotify.js` — 1 stale rehydrate dispatcher branch (`discover_familiar_favorites`) caught during the global grep pass. - `tests/test_personalized_playlists_id_gate.py` — 3 library-method tests + the test infrastructure that supported them (`tracks` schema, `insert_library_track` helper). Documentation header updated to reflect the deletion. Net: -527 / +2 lines across 7 files. What stays: - Daily Mixes (also in personalized package, intentionally paused — separate decision). - Popular Picks + Hidden Gems + Discovery Shuffle (alive, not affected by this deletion). - All 14 tests in the personalized-playlists test file still pass. - The PersonalizedPlaylistsService lift from the prior commit (`_select_discovery_tracks` etc) — those are still in active use by the surviving discovery_pool methods. DISCOVER_TRACK_SELECTION_REVIEW.md at repo root contains historical references to the four deleted endpoints. Treated as historical context (same policy as WHATS_NEW), left alone. 2219/2219 full suite green (was 2222 - 3 deleted tests = 2219). JS parses clean, ruff clean. |
1 month ago |
|
Broque Thomas
|
44dd7f980f |
Discover: unify Decade + Genre tabbed browsers
Both tabbed-browser sections — Time Machine ("Decade") and Browse by
Genre — re-implemented the same lifecycle by hand: fetch tabs list,
render the tab strip, attach click handlers, fetch content per tab,
render track list with sync + download action buttons + sync-status
block, handle empty/error/loading states. ~314 lines of identical
boilerplate split across two browsers.
Lifted into one shared `createTabbedBrowserSection(config)` helper.
Each browser is now a thin wrapper:
```js
const ctrl = createTabbedBrowserSection({
id: 'decade-browser',
tabsContainerEl: '#decade-tabs',
contentContainerEl: '#decade-content',
fetchTabs: async () => { ... },
renderTabButton: (tab, isActive) => `<button>...</button>`,
fetchTabContent: async (tab) => { ... },
renderTabContent: (tracks, tab) => `...`,
onTabContentRendered: (tab, contentEl) => { ... },
emptyMessage / errorMessage,
});
```
Migrated:
- `loadDecadeBrowserTabs` 85 → 3 lines
- `loadDecadeTracks` 67 → 3 lines
- `loadGenreBrowserTabs` 92 → 3 lines
- `loadGenreTracks` 70 → 3 lines
Helper: ~125 lines + ~100 lines of per-browser config blocks +
~25 lines of shared `_renderTabbedTrackList` (the two browsers had
byte-identical track-row markup so it lifted cleanly).
Public function names preserved — the four migrated functions stay
on the same signature so existing callers (`loadDiscoverPage`,
refresh buttons, inline handlers) don't change.
Side effects preserved — `decadeTracksCache[year]`, `activeDecade`,
`genreTracksCache[name]`, `activeGenre`, `availableGenres` still
mutated at the same lifecycle moments. The decade-specific
`startDecadeSync(decade)` and genre-specific `startGenreSync(name)`
sync-button handlers stay where they are; they're click handlers
attached to rendered content, not part of the tab lifecycle.
What didn't fit (intentionally left alone):
- `_renderCompactTrackRow` (the existing shared track-row helper) is
NOT used by the tabbed browsers — they had their own template
with a `track_data_json` fallback chain `_renderCompactTrackRow`
doesn't do. Unifying these two would change behavior for
non-tabbed sections, so the tabbed-browser variant lives as
`_renderTabbedTrackList`. Future cleanup could merge them by
giving `_renderCompactTrackRow` an opt-in fallback flag.
- `switchDecadeTab` / `switchGenreTab` still know about cache shape
so they can skip refetch on already-loaded tabs. Keeping that
in the per-browser switch is fine — it's a click handler, not
lifecycle.
Net: 8546 → 8578 LOC on `discover.js` (+32). Helper boilerplate
offsets the line count, but the win is single-source-of-truth, not
raw line reduction.
`node --check` clean. 2222/2222 full suite green.
|
1 month ago |
|
Broque Thomas
|
0701bcc213 |
PersonalizedPlaylistsService: bake in ID-validity gate, lift selectors
User-facing bug found in the discover-page audit: multiple sections (hidden gems, discovery shuffle, popular picks, decade browser, genre browser) had no `WHERE (spotify_track_id IS NOT NULL OR itunes_track_id IS NOT NULL ...)` gate. Tracks with no source IDs in the discovery pool got displayed, the user clicked download, the download silently failed because there was nothing to look up. Lift + gate `PersonalizedPlaylistsService` had 5 selection methods that all shared the same shape — connect to DB, run a SELECT against `discovery_pool` with different WHERE clauses, optionally apply diversity, return list of track dicts. ~366 lines of business logic, ~55% of which was repeated boilerplate. Three new private helpers consolidate everything: - `_select_discovery_tracks(*, source, extra_where, extra_params, order_by, fetch_limit, extra_columns)` — shared SELECT against `discovery_pool`. The mandatory ID gate is hard-coded into the WHERE clause: no opt-out flag, every method inherits it for free. Plus the source filter and the blacklist filter — same shape every selector needs. - `_apply_diversity_filter(tracks, *, max_per_album, max_per_artist, limit)` — per-album / per-artist cap loop, returns trimmed list. Lifted from the inline duplicates in decade / genre / popular_picks. - `_compute_adaptive_diversity_limits(tracks, *, relaxed=False)` — step-function tiers based on unique-artist count. `relaxed=True` gives the slightly looser limits the genre playlist used vs the decade playlist. Re-enable 4 library methods `get_recently_added`, `get_top_tracks`, `get_forgotten_favorites`, `get_familiar_favorites` were all stubs (`return []`) because they predated the schema columns they need. Schema now has them: `tracks.created_at`, `tracks.play_count`, `tracks.last_played`, and the source ID columns added in earlier work. New `_select_library_tracks(*, where_clause, params, order_by, limit)` helper mirrors the discovery selector but targets the `tracks` table joined against `albums` + `artists`. Mandatory ID gate lives in the helper too: every library method automatically rejects rows where spotify_track_id, itunes_track_id, deezer_id, musicbrainz_recording_id, AND audiodb_id are all NULL. Selection rules: - `get_recently_added` — ORDER BY created_at DESC - `get_top_tracks` — WHERE play_count > 0 ORDER BY play_count DESC - `get_forgotten_favorites` — WHERE play_count > 5 AND last_played < (now - 90 days) ORDER BY play_count DESC - `get_familiar_favorites` — WHERE play_count BETWEEN 3 AND 15 Tests `tests/test_personalized_playlists_id_gate.py` — 17 tests pinning: - `_select_discovery_tracks` filters NULL-id rows, honors source + blacklist + extra_where - `_apply_diversity_filter` caps per-album + per-artist + stops at limit - `_compute_adaptive_diversity_limits` returns the right tier for unique-artist count + relaxed flag - All 5 discovery methods (decade, popular_picks, hidden_gems, discovery_shuffle, genre is exercised via the helper) reject NULL-id rows - All 4 library methods reject NULL-id rows + honor their play-count rules Behavior preserved Same diversity tiers, same over-fetch multipliers (10x for decade / genre, 3x for popular_picks), same `popularity DESC, RANDOM()` ordering, same `popularity >= 60` / `< 40` thresholds, same blacklist filter. Public method signatures unchanged — `web_server.py` needs zero edits. Net file: 1089 → ~1170 LOC (helpers + docstrings), but actual business logic across the 9 methods went from ~418 lines down to ~195 (-53%). 2222/2222 full suite green (was 2205 + 17 new). Ruff clean. |
1 month ago |
|
BoulderBadgeDad
|
1f2b8f8ccd
|
Merge pull request #522 from Nezreka/feat/discover-section-controller
Feat/discover section controller |
1 month ago |
|
Broque Thomas
|
c557d9196e |
Discover controller — Cin pre-review polish
Three changes tightening the controller before opening the PR. DROP MAGIC `extractItems` DEFAULTS Controller used to auto-pull `data.items` / `data.albums` / `data.artists` / `data.tracks` / `data.results` when no extractor was supplied. Removed the fallback chain — every section now MUST provide an explicit `extractItems(data) => array`. Validated at register-time so misuse fails immediately, not silently on first load against an endpoint that happened to return two arrays. Cin standard: explicit > implicit. Magic key-grabbing could pick the wrong one in edge cases (e.g. an endpoint returning both `data.albums` and `data.results` would have grabbed albums when the section actually wanted results). All 10 existing controller call sites already passed explicit extractors, so no migration churn — this is purely tightening the contract for future sections. REPLACE `renderItems` NULL-RETURN CONVENTION WITH `manualDom: true` Your Albums and similar sections that delegate to existing renderers that target a CHILD element of `contentEl` used to signal "leave the container alone" by returning null/undefined from `renderItems`. That convention is easy to confuse with an accidental missing-return error. Replaced with an explicit `manualDom: true` config flag. Renderer is still called for its side-effects, controller just skips the innerHTML swap. Clearer intent at the call site. Updated `loadYourAlbums` to use the new flag. PIN THE CONTROLLER CONTRACT WITH JS TESTS Added `tests/static/test_discover_section_controller.mjs` — 32 tests covering the controller's lifecycle contract: - Config validation (every required field, mutual exclusivity of fetchUrl/data, type checks on contentEl) - Happy-path fetch → parse → render - Empty state (default empty render, hideWhenEmpty + sectionEl, success=false treated as empty, custom isSuccess override) - Stale state (fires when isStale returns true, wins over empty, custom renderStale override) - Error state (HTTP non-ok, fetch throws, showErrorToast fires window.showToast, default off doesn't fire) - No-fetch `data:` mode (value + function form, doesn't call fetch) - manualDom mode (skips innerHTML swap, still calls renderer) - Callable `fetchUrl` (resolved at load time, refresh re-resolves) - Load coalescing (concurrent loads share one fetch) - Refresh bypasses coalescing (re-fires fetch every call) - Hook error containment (throwing renderer/onSuccess hooks don't crash the controller) Runs via Node's stable built-in `--test` runner — no package.json, no jest/vitest dependency, no compile step. Just `node --test`. Pytest wrapper at `tests/test_discover_section_controller_js.py` shells out to node and asserts clean exit, so the JS tests fail the regular pytest sweep if the controller contract drifts. Skipped gracefully when node isn't available or is < 22. Closes the "controller is a contract, pin it at the test boundary" gap that Cin would have flagged on review. VERIFICATION - 2205/2205 full pytest suite green (was 2204 + 1 new wrapper) - 32/32 `node --test` pass on the controller test file directly - ruff clean - node --check clean on all touched JS files |
1 month ago |
|
Broque Thomas
|
dc2323cde6 |
Discover cleanup: controller extensions, toast errors, migrate skipped sections
Follow-up to the controller migration commits. Closes out the extension list the per-section migrations surfaced as needed. CONTROLLER EXTENSIONS - Callable `fetchUrl: () => string` — resolves the seasonal-playlist recreate-on-key-change hack from the prior commit. - No-fetch `data:` mode — value or `() => value`. Lets render-only sections like Seasonal Albums use the controller without inventing a fake endpoint. Mutually exclusive with `fetchUrl`; validated up front so misuse fails at register-time. - `beforeLoad(ctx)` hook — runs before the spinner shows. Lets dynamically-inserted sections like Because You Listen To ensure their `contentEl` exists before the visibility check. - `onSuccess(data, ctx)` hook — runs after the success gate but before isEmpty / isStale. Cleaner home for sibling header / subtitle / button updates than folding them into renderItems. - `isStale(items, data)` + `onStale(ctx)` + `renderStale(items, data)` + `staleMessage` — third render state for "data is empty BUT upstream is still discovering". Stale wins over empty when both apply. Default stale UI is the same spinner block used elsewhere. - `showErrorToast: true` config — opens a global `showToast(...)` in addition to the in-section error block. Default off; sections that have no recovery action shouldn't shout at the user. - `renderItems` returning null/undefined now leaves contentEl untouched. Lets a renderer do its own DOM manipulation (e.g. delegating to an existing grid-render fn that targets a child element) without fighting the controller's innerHTML swap. MIGRATED THE 2 SKIPPED SECTIONS - `loadYourAlbums` — uses `isStale`/`onStale`/`renderStale` for the stale-fetch state, `onSuccess` for the subtitle/filters/download side-effects, `hideWhenEmpty` + `sectionEl` for the truly-empty case, `renderItems` returning null since it delegates to the existing `_renderYourAlbumsGrid` + `_renderYourAlbumsPagination`. - `loadSeasonalAlbums` — uses no-fetch `data:` mode because the parent `loadSeasonalContent` already fetched the season payload. `beforeLoad` updates the sibling title/subtitle text. ERROR TOASTS ON ALL MIGRATED SECTIONS Every migrated section now has `showErrorToast: true`. Section load failures surface a global toast instead of silently spinning forever or swallowing into console.debug. Same pattern JohnBaumb #369 asked for at the Python layer, applied at the UI layer. SHARED SYNC-STATUS BLOCK Lifted the duplicated decade-tab + genre-tab sync-status HTML (✓ completed | ⏳ pending | ✗ failed | percentage) into a single `_renderSyncStatusBlock(idPrefix)` helper. Two call sites now share one implementation. ListenBrainz playlists keep their own block because the semantics differ — matching progress (total / matched / failed) vs download progress. DEAD-SECTION AUDIT — NONE DEAD Audited the 13 supposedly-dead hidden sections from DISCOVER_REVIEW.md. All 13 are alive: gated on user data (discovery pool, library content, metadata cache) and self-surface when their data exists via `style.display = 'block'` on the success path. The review's grep missed the toggle. No deletions made. DAILY MIXES ORPHAN CALL Removed the orphaned `loadPersonalizedDailyMixes()` call from `blockDiscoveryArtist` — Daily Mixes is intentionally paused (its load call in `loadDiscoverPage` is commented out) so refreshing it from the post-block hook was a no-op. 2204/2204 full suite green. JS parses clean (`node --check`). |
1 month ago |
|
Broque Thomas
|
4ee78bb973 |
Migrate 7 more discover sections to the shared controller
Follow-up to the foundation commit. Drops the hand-rolled try/catch + spinner injection + empty-state HTML + error-swallow in seven sections by routing them through `createDiscoverSectionController`. Each section keeps its existing public function name + signature so callers, refresh buttons, and dashboard wiring don't notice the swap. Migrated: - `loadDiscoverReleaseRadar` (Fresh Tape) - `loadDiscoverWeekly` (The Archives) - `loadDecadeBrowser` (Time Machine intro carousel) - `loadGenreBrowser` (Browse by Genre intro carousel) - `loadSeasonalPlaylist` (Seasonal Mix) - `loadYourArtists` - `loadBecauseYouListenTo` Skipped (don't fit the controller's single-fetch / single-render-target shape): - `loadYourAlbums` — paginated grid + filters, updates four separate UI elements (subtitle, filter chips, download button, grid). - `loadSeasonalAlbums` — receives pre-fetched data from `loadSeasonalContent`; no fetch URL to satisfy. Hidden / dead sections (~13 of them — `loadPersonalized*`, `loadDiscoveryShuffle`, `loadFamiliarFavorites`, `loadCache*`) untouched in this pass. Separate audit commit will surface or kill them. Two side-effects worth noting: - `loadDecadeBrowser` and `loadGenreBrowser` migrated for completeness, but neither appears wired into `loadDiscoverPage` or any inline handler. May be dead code — flagged for the audit pass. - `loadSeasonalPlaylist` needs a per-load fetch URL (varies by `currentSeasonKey`); worked around by recreating the controller when the key changes. Cleaner option: extend the controller to accept a `fetchUrl: () => string` callable form. Tracked in the follow-up extension list below. Controller extension candidates surfaced for follow-up: - Callable `fetchUrl` (resolves the seasonal playlist recreate-on-key-change hack) - Explicit `isStale` / `onStale` hook (so Your Artists doesn't fold stale handling into renderItems) - `beforeLoad` / `ensureContentEl` hook (so Because You Listen To can let the controller own the dynamic container creation) - No-fetch `data:` mode (so render-only sections like Seasonal Albums can use the controller too) - `onSuccess(data)` hook (cleaner home for header / subtitle side-effects vs folding them into renderItems) Net: -76 lines in `discover.js` even after adding the per-section render helpers. 2204/2204 full suite green. JS parses clean. |
1 month ago |
|
Broque Thomas
|
07a71f0432 |
Discover section controller foundation + migrate Recent Releases
Every section on the discover page (Recent Releases, Your Artists,
Your Albums, Seasonal Albums, Seasonal Mix, Fresh Tape, The Archives,
Build Playlist, Time Machine, Browse by Genre, ListenBrainz Playlists,
Because You Listen To, plus ~13 hidden sections) currently
re-implements the same lifecycle by hand:
1. show a loading spinner in the carousel container
2. fetch the section's endpoint
3. parse the response, decide if the data is empty
4. either render the items, show an empty-state, or show an error
5. wire post-render handlers (download buttons, hover behavior, etc)
6. maybe expose refresh()
~30 sections worth of duplicated boilerplate, all subtly drifting.
Different empty-state messages. Different error handling (some
`console.debug`, some silently swallowed, some leave the spinner
spinning forever). Different sync-status icons (✓/⏳/✗ vs ♪/✓/✗).
No consistent error toast.
Lifted the lifecycle into a shared `createDiscoverSectionController`
in `webui/static/discover-section-controller.js`. Renderers stay
per-section because section data shapes legitimately differ — album
cards vs artist circles vs playlist tiles vs track rows. The
controller is the wrapper, not a forced visual abstraction.
Foundation contract:
createDiscoverSectionController({
id: 'recent-releases', // for diagnostic logging
contentEl: '#carousel', // selector or Element
fetchUrl: '/api/discover/...',
extractItems: (data) => [...], // pull list from response
renderItems: (items, data, ctx) => '<html>',
onRendered: (ctx) => { ... }, // optional post-render hook
loadingMessage / emptyMessage / errorMessage: copy
sectionEl + hideWhenEmpty: optional whole-section visibility
isSuccess / isEmpty: optional gate overrides
})
Returns `{ load, refresh, destroy, getState }`. Validates config up
front so misuse fails at register-time, not silently on load. Coalesces
concurrent loads (same in-flight promise returned) so a double-click
or repeated trigger doesn't double-fetch. `refresh()` bypasses the
coalesce so the refresh button always re-fires. Errors are logged
(console.debug by default, console.error when verboseErrors=true).
Renderer hook errors are caught + logged so a buggy render callback
can't tear down the controller — keeps the page resilient.
Migrated `Recent Releases` as the proof — simplest album-card shape,
no source-gating, no refresh button. Verified the contract covers it
end-to-end. The legacy `loadDiscoverRecentReleases` entry-point stays
public so existing callers don't change; internally it lazy-builds
the controller and triggers `load()`.
NOT in this commit:
- Other section migrations (one section per follow-up commit, keeps
reviews small + lets us sequence the work)
- Registry-driven section list (so the dead-section audit becomes
registry deletions instead of section-by-section removal)
- Global error toast wrapper
- Per-section "requires X primary source" gate
- Sync-status icon renderer unification
Once every section is on the controller, the discover-page cleanup
work (kill the 13 dead sections, standardize sync-status icons, add
error toasts) becomes single-line registry-level edits instead of
30 separate section-by-section rewrites.
2204/2204 full suite green. JS parses clean (`node --check`). Manual
smoke deferred until follow-up commits — Recent Releases unchanged
on the wire (same endpoint, same payload shape, same render output).
|
1 month ago |
|
BoulderBadgeDad
|
6637c29964
|
Merge pull request #520 from Nezreka/release/2.4.2
Bump version to 2.4.2 |
1 month ago |
|
Broque Thomas
|
6aafcaae93 |
Bump version to 2.4.2
- `web_server.py` — `_SOULSYNC_BASE_VERSION` 2.4.1 → 2.4.2
- `webui/static/helper.js` — flip the 2.4.2 WHATS_NEW header from
"Unreleased — 2.4.2 dev cycle" to "May 7, 2026 — 2.4.2 release"
so the per-version block stops being filtered out by
`_getLatestWhatsNewVersion`. Also bumps the safety-net default
inside that helper from 2.4.1 → 2.4.2.
- `.github/workflows/docker-publish.yml` — manual-trigger default
tag bumped to match.
Drive-by fix: escaped a stray single quote in the `Internal: Download
Engine` 2.4.2 entry that broke `node --check` on the file
(`orchestrator.client('soulseek')` inside a single-quoted desc string
silently terminated the string mid-entry). Pre-existing, unrelated to
the bump but caught while validating JS parse for the release.
VERSION_MODAL_SECTIONS not rotated in this commit — separate
editorial pass.
|
1 month ago |
|
BoulderBadgeDad
|
f2fb66340a
|
Merge pull request #519 from Nezreka/feat/artist-top-tracks-download
Add download buttons + bulk action to artist top-tracks sidebar |
1 month ago |
|
Broque Thomas
|
1a2da016e4 |
Add download buttons + bulk action to artist top-tracks sidebar
Closes #513 (s66jones). The artist detail page already showed a "Popular on Last.fm" sidebar — list of an artist's top tracks by playcount, with a play button per row but no download action. Issue #513 wanted a way to grab those tracks the same way zotify let users grab "top X songs" without pulling the full discography. Pulls from the configured primary metadata source (Spotify `artist_top_tracks`, Deezer `/artist/{id}/top`) when available, falls back to the existing Last.fm display-only mode for sources that don't expose popularity ranking (iTunes / Discogs / MusicBrainz). Source label in the section title shifts to match. Each row gets a hover-revealed download button that wishlists the single track via the existing /api/add-album-to-wishlist endpoint (preserves the track's real album metadata, so the wishlist worker later places the file in its proper album folder). A "Download All" footer button opens the standard download modal in PLAYLIST context, not album context — the virtual playlist_id is `top_tracks_<source>_<artistId>` which doesn't match any of the album-prefix checks in `startMissingTracksProcess` (downloads.js). That keeps `is_album_download=false`, so the master worker doesn't inject a wrapper context as `_explicit_album_context`. Each track downloads using its own real album metadata, files land in proper per-album folders on disk (not a fake "Top Tracks" folder). Backend additions: - `SpotifyClient.get_artist_top_tracks(artist_id, country, limit)` — wraps `spotipy.artist_top_tracks`, returns up to 10 tracks for the market (Spotify's API cap). UI-side limit trim only. - `DeezerClient.get_artist_top_tracks(artist_id, limit)` — wraps `/artist/{id}/top?limit=N`, converts Deezer's raw shape to the same Spotify-compatible dict layout (id, name, artists, album with album_type / total_tracks / images, duration_ms, track_number, disc_number) so downstream code doesn't branch on source. - `GET /api/artist/<id>/top-tracks` — dispatches to whichever client matches the primary source. Resolves per-source artist IDs from the DB row first (matching what /discography already does) so a Spotify ID in the URL still works when Deezer is primary, and vice versa. Returns `{success, source, tracks, resolved_artist_id}` on hit; `{success: False, reason: 'unsupported_source' | 'spotify_not_authenticated' | 'deezer_unavailable' | 'no_tracks_found'}` on miss so the frontend can decide whether to fall through to Last.fm. Frontend: - `_loadArtistTopTracks` tries the metadata source first, falls through to the legacy `/api/artist/0/lastfm-top-tracks` call if the source can't deliver. Section title and per-row UI shift based on which source answered. - New per-row `.hero-top-track-download` button (hover-revealed). - New `.hero-top-tracks-download-all` footer button — only visible when metadata-source mode rendered the list (Last.fm fallback hides it since rows have no track IDs to download). Tests: 10 new tests pin the client methods — - Spotify: returns track list, honors UI limit cap, returns empty when unauthed / artist_id missing / API throws. - Deezer: shape conversion to Spotify-compatible dict, empty when no data / artist_id missing, limit clamping at upper bound, default fallback when limit=0, malformed entries skipped. The Flask endpoint dispatcher itself isn't covered by the new test file because importing web_server at test-collection time spins up worker threads that race with caplog-using tests elsewhere in the suite (specifically test_library_reorganize_orchestrator). Endpoint verified manually; the underlying client methods (the load-bearing logic) are covered. 2204/2204 full suite green (was 2194 + 10 new). |
1 month ago |
|
Broque Thomas
|
dd48dc8c6e |
Update style.css
|
1 month ago |
|
BoulderBadgeDad
|
5eff659220
|
Merge pull request #518 from Nezreka/fix/acoustid-version-mismatch
Reject AcoustID matches whose version disagrees with the expected track |
1 month ago |
|
Broque Thomas
|
01c528fd5f |
Reject AcoustID matches whose version disagrees with the expected track
Discord report (corruption [BWC]): downloads coming through as the
instrumental cut when a vocal track was requested. The verification
step's `_normalize` function strips parentheticals and version-suffix
tags ("(Instrumental)", "- Live", etc) so legitimate name variations
don't false-fail the title-similarity check. That also means "In My
Feelings" and "In My Feelings (Instrumental)" both normalize to "in
my feelings", title similarity is 1.0, and the wrong cut passes
verification.
Detect the version label on each side BEFORE normalization runs. If
the expected and matched recordings disagree on version (one is
original, the other is instrumental / live / acoustic / remix /
etc), return FAIL — the fingerprint identified a real song, just
not the version the caller asked for.
Reuses `MusicMatchingEngine.detect_version_type` so the same regex
patterns the pre-download Soulseek matcher applies also drive
post-download verification. No duplicated tables.
Also gates the secondary fallback scan, so a wrong-version variant
sitting in the same fingerprint cluster can't win the loop after
the best match has already been version-rejected.
6 tests pin the behavior:
- instrumental returned for vocal request → FAIL
- vocal returned for instrumental request → FAIL
- live vs acoustic → FAIL
- matching versions on both sides → PASS
- original-to-original happy path → PASS (regression guard)
- secondary scan skips wrong-version recordings → not PASS
2194/2194 full suite green (was 2188 + 6 new).
|
1 month ago |
|
BoulderBadgeDad
|
caef3dc9f1
|
Merge pull request #517 from Nezreka/fix/primary-source-non-admin-profiles
Fix non-admin profiles defaulting to Spotify on search picker |
1 month ago |
|
Broque Thomas
|
caa1c198e5 |
Fix non-admin profiles defaulting to Spotify on search picker
Closes #515 (jaruca). Search-picker controller in shared-helpers.js resolved the user's configured primary metadata source by fetching `/api/settings`. That endpoint is `@admin_only` (it returns full config including credentials), so non-admin profiles got a 403 and the controller silently fell back to the hardcoded `'spotify'` default — admin's chosen source (deezer / itunes / discogs / etc) was ignored on every non-admin profile, forcing manual reselection each session. Switched to `/status`, which is public and already exposes the resolved `metadata_source` for the dashboard. Same value the picker needs — different endpoint that doesn't gate non-admins. Admins see no behavior change. Non-admins now see admin's configured primary source as the default active icon. Refs #515 |
1 month ago |
|
BoulderBadgeDad
|
627d32cebd
|
Merge pull request #516 from Nezreka/fix/silent-exception-swallowing
Fix/silent exception swallowing |
1 month ago |
|
Broque Thomas
|
9602d1827c |
Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites
Catches the silent excepts the awk-based earlier sweeps missed:
- Bare `except:` followed by `pass` (also swallows KeyboardInterrupt
and SystemExit — actively wrong). Upgraded to `except Exception as
e: logger.debug("...: %s", e)`. ~14 sites across connection_detect,
soulseek_client, listenbrainz_manager, watchlist_scanner,
youtube_client, navidrome_client, jellyfin_client, web_server.
- `except Exception:` + pass that the awk pattern missed (e.g.
multi-line or unusual whitespace). ~31 sites across automation_engine,
database_update_worker, music_database, spotify_client, web_server,
others.
- 14 legitimate cleanup sites left silent with explicit `# noqa: S110`
+ comment explaining why (atexit handlers, finally-block conn.close
calls). Logging during shutdown can itself crash because file handles
get torn down before the handler fires.
Also enables `S110` rule in pyproject.toml so this pattern fails CI
going forward — drift fails at PR review instead of at runtime against
a wedged worker thread. Tests path keeps S110 ignored (test fixtures
legitimately use try-except-pass for cleanup).
Adds a WHATS_NEW entry to helper.js summarizing the full #369 sweep.
Verified: `python -m ruff check .` → All checks passed.
Verified: `python -m pytest tests/` → 2188 passed.
Closes #369
|
1 month ago |
|
Broque Thomas
|
aa54bed818 |
Surface silent exceptions across remaining modules — ~70 sites
Final sweep. Covers: - Downloads: candidates / lifecycle / master / monitor / wishlist_failed - Metadata: source / registry / cache / common / artwork (+ plex_client) - Imports: pipeline / resolution / file_ops / paths / guards - Library: path_resolver / retag / duplicate_cleaner - Stats / playlists / wishlist / discovery / automation / enrichment - Misc: hydrabase_client, soulsync_client, tag_writer, debug_info, api_call_tracker, album_consistency, beatport_unified_scraper, reorganize_runner, seasonal_discovery, lidarr_download_client, services/sync_service.py, automation_engine, automation/progress Two `_e` renames in imports/file_ops.py (outer scope binding `e`). A few finally-block sites in metadata/album_mbid_cache.py, library/track_identity.py, listening_stats_worker.py, watchlist/ auto_scan.py left silent — same reason as the rest of the sweep (logger calls during cleanup paths can themselves raise). Refs #369 |
1 month ago |
|
Broque Thomas
|
e95452b465 |
Surface silent exceptions in workers + repair jobs — ~30 sites
Across all background workers (Spotify/Tidal/Deezer/Qobuz/iTunes/
Discogs/Genius/AudioDB/MusicBrainz/Last.fm/SoulID + the metadata-update
worker) and the repair-job scanners. All converted to
`logger.debug("...: %s", e)`.
Two `_e` renames in genius_worker and soulid_worker where outer scope
was already binding `e`. Two finally-block sites in repair_jobs/
library_reorganize.py left silent (conn.close on shutdown path).
Refs #369
|
1 month ago |
|
Broque Thomas
|
8219771304 |
Add module logger + surface silent exceptions in 7 logger-less files — 12 sites
These files had silent `except Exception: pass` blocks but no module logger. Added `import logging` + `logger = logging.getLogger(__name__)` at the top of each, then replaced the silent excepts with `logger.debug(...)`. - core/replaygain.py — 4 sites (id3 txxx + vorbis + mp4 atom reads) - core/wishlist/presence.py — 3 sites (wishlist row parsing + queries) - core/runtime_state.py — 1 site (activity toast emit) - core/automation/signals.py — 1 site (collect known signals) - core/download_engine/rate_limit.py — 1 site (plugin rate_limit_policy) - api/system.py — 1 site (hydrabase status probe) - api/search.py — 1 site (hydrabase search) Refs #369 |
1 month ago |
|
Broque Thomas
|
8dc9f79f97 |
Surface silent exceptions in watchlist + discovery + reorganize — 18 sites
- watchlist_scanner.py: 6 sites
- discovery/playlist.py: 5 sites
- discovery/sync.py: 4 sites
- watchlist/auto_scan.py: 1 site (1 left silent — finally-block scanner cleanup)
- library_reorganize.py: 2 sites (4 left silent — all in finally blocks:
conn.close, staging rmtree, sidecar delete, cleanup_empty_dir)
All non-finally sites converted to `logger.debug("...: %s", e)`.
Finally-block sites kept silent because logger calls during cleanup
(after exception was already raised) can themselves raise.
Refs #369
|
1 month ago |
|
Broque Thomas
|
de348981a5 |
Surface silent exceptions in import pipeline — 11 sites
- imports/side_effects.py: 8 sites (post-import cleanup paths,
thumbnail+lyrics pulls, Plex refresh)
- auto_import_worker.py: 3 sites (queue/dedup helpers)
All converted to `logger.debug("...: %s", e)`.
Refs #369
|
1 month ago |
|
Broque Thomas
|
aa9429d733 |
Surface silent exceptions in core/artists — 23 sites
- map.py: 15 sites (cache lookups + per-track DB inserts in artist→source
mapping)
- liked_match.py: 8 sites (Spotify liked-songs match heuristics)
All converted to `logger.debug("...: %s", e)`. No control-flow changes.
Refs #369
|
1 month ago |
|
Broque Thomas
|
cc7a3f76ac |
Surface silent exceptions in metadata clients — 37 sites
- spotify_client.py: 15 sites (mostly `publish_spotify_status` + cache
parse fallbacks)
- deezer_client.py: 10 sites (cache get/store + dataclass parsing)
- itunes_client.py: 4 sites (cache parsing + lookup fallback)
- discogs_client.py: 3 sites (cache parsing + token-from-config)
- tidal_client.py: 2 sites (used `_e` to avoid shadowing `e` from a
sibling `except` clause in the same function — defensive)
- deezer_download_client.py: 3 sites
All converted to `logger.debug("...: %s", e)` with lazy formatter.
No control-flow changes.
Refs #369
|
1 month ago |
|
Broque Thomas
|
e4e6b6bd5a |
Surface silent exceptions in repair_worker — 16 sites
Mostly progress-callback try/except (caller-provided fns we don't control) and best-effort file-move / DB-cleanup paths in the auto-fixers. All converted to `logger.debug(...)`. No control-flow changes. Refs #369 |
1 month ago |
|
Broque Thomas
|
bfef2c7579 |
Surface silent exceptions in music_database.py — 18 sites
Mostly schema-migration ALTER TABLE fallbacks (column-already-exists
is the silent expected case) plus a few cache-purge/notify-migration
spots. Same pattern as the web_server sweep: `except Exception as e:
logger.debug("...: %s", e)`.
Refs #369
|
1 month ago |
|
Broque Thomas
|
b0c58a0f91 |
Surface silent exceptions in web_server.py — 81 sites
Replaces `except Exception: pass` blocks with `except Exception as e: logger.debug(...)` so failures are inspectable in the log instead of disappearing silently. Per JohnBaumb's request in #369. - Pattern is consistent: `logger.debug("<context>: %s", e)` with lazy formatter and 2-6 word context describing the operation. - 2 atexit handlers (lines 2977, 2983) intentionally left silent — the log file handles can be torn down before atexit fires, and a separate `_atexit_silence_shutdown_logger_errors` already exists for this exact reason. - No behavior changes; control flow is unchanged. Test suite green (2188 passed). Refs #369 |
1 month ago |
|
BoulderBadgeDad
|
763d160691
|
Merge pull request #514 from Nezreka/fix/repair-job-card-pending-count
Repair job card badge — show pending count, not last-scan count |
1 month ago |
|
Broque Thomas
|
4c11375930 |
Repair job card badge — show pending count, not last-scan count
Discord report: Duplicate Detector card said "372 findings" and Cover
Art Filler said "60 findings", but clicking the Findings tab's Pending
filter showed 0. User read it as "findings aren't being created" —
looked like a detector bug.
Actual cause: the badge sourced ``last_run.findings_created``
(historical "found in last scan") without considering current state.
After the user (or bulk-fix automation) resolved or dismissed those
findings, they no longer appeared on the Pending tab — but the badge
kept showing the last-scan number in red urgent styling.
Backend was correct end-to-end: detectors create pending rows,
bulk-fix moves them to resolved, Findings tab filters by status.
Only the badge display lied about current state.
Fix:
- ``RepairWorker._get_pending_count_by_job()`` — single SQL aggregation
returning ``{job_id: pending_count}`` for every job with pending
findings. O(1) lookup per job instead of N round trips.
- ``get_all_job_info()`` calls it once per request and adds
``pending_findings_count`` to each job's API response.
- ``enrichment.js`` job card now branches on the count:
- ``> 0`` → red ``"X pending"`` badge (urgent, action needed)
- ``= 0`` AND last scan found something → muted grey ``"X found in
last scan"`` (historical context, no action needed)
- New CSS class ``.repair-flow-badge.findings-historical`` for the
muted slate color so the two states are visually distinct.
User-visible result with the screenshotted state (372 dup / 60 cover-
art findings, all resolved):
- Before: red "372 findings" / "60 findings" — implied 432 things to
do, but Findings tab showed 0 pending
- After: grey "372 found in last scan" / "60 found in last scan" —
the badge text tells the user the count is historical, no surprise
when Pending is empty
Tests: 3 new tests in ``tests/test_create_finding_dedup_counter.py``
pin the per-job pending count helper:
- returns ``{job_id: count}`` based on status='pending' rows only;
resolved + dismissed rows excluded
- empty dict when no pending findings exist
- gracefully returns ``{}`` on DB error (badge falls back to
historical count via the existing JS ``or 0`` safety)
2188/2188 full suite green. Pure UI/state-display fix — no detector
logic, no backend behavior change.
|
1 month ago |
|
BoulderBadgeDad
|
7bd15438da
|
Merge pull request #506 from JohnBaumb/dependencies-pin
Pin all dependencies to exact resolved versions |
1 month ago |
|
BoulderBadgeDad
|
7ceeee2715
|
Merge pull request #496 from dlynas/feat/fast-entrypoint-permissions
fix: skip recursive chown on startup when UIDs are already correct |
1 month ago |
|
BoulderBadgeDad
|
8e389fc187
|
Merge pull request #512 from Nezreka/fix/slskd-http-timeout-prevents-worker-deadlock
Bound slskd HTTP timeout — fixes worker thread deadlock |
1 month ago |
|
Broque Thomas
|
5c69b853b4 |
Bound slskd HTTP timeout — fixes worker thread deadlock
GitHub issue #499 (@bafoed). Big initial sync of Spotify playlists worked for 2-3 hours then downloads silently stopped: - 3 active tasks stuck in "Searching" state, replaced every ~10 min by different ones - slskd UI showed no actual searches happening - Debug log: orphaned-task count grew over time, no jobs executed - Container restart was the only fix (bought another 2-3 hours) - Not a rate limit (rates showed 0/min) Root cause: ``core/soulseek_client.py`` constructed ``aiohttp.ClientSession()`` with no timeout at four sites. When slskd hung on a request (overloaded, transient network blip, internal stall), the HTTP call blocked indefinitely — and the worker thread blocked with it. The download executor only has ``ThreadPoolExecutor(max_workers=3)``, so once 3 worker threads were wedged on hung calls, no further downloads could start. Batch-level "stuck detection" (10-minute timer in ``check_batch_completion_v2``) was correctly marking tasks ``not_found`` and trying to start replacements, but the executor pool was exhausted — replacements queued forever inside the executor with no thread to run them. Symptom: tasks rotating every ~10 min at the batch level while the underlying executor stayed wedged. Fix: bounded ``aiohttp.ClientTimeout`` (total 120s, connect 15s, sock_read 60s) on every slskd ``ClientSession`` construction. Module- level constant ``_SLSKD_DEFAULT_TIMEOUT`` so the four sites stay in lockstep — future sites get the same protection by reusing the constant. Why these timeouts are safe: - Every slskd API call is metadata-level (search submission, status polls, download enqueue, transfer state queries). None stream files — slskd handles file transfer via its own peer-to-peer infrastructure entirely outside our HTTP requests. - Legitimate metadata calls finish in seconds. 120s ceiling is ~50× the normal latency. Timeout handling: - ``asyncio.TimeoutError`` caught explicitly BEFORE the generic ``except Exception`` — surfaces "slskd timed out" specifically in logs (debuggable instead of buried as "Error making API request"). - Returns None to the caller (same code path as a 5xx response or any other failure). No new error path; callers already handle None as "request failed". - Worker thread unblocks immediately → executor pool stays healthy → downloads keep flowing. Sites updated: - ``_make_request`` (general /api/v0/ helper, line 152) — used for every slskd API operation - ``_make_direct_request`` (non-/api/v0/ helper, line 235) - ``_explore_api_endpoints`` Swagger fetch (line 1566) — diagnostic - ``_explore_api_endpoints`` per-endpoint probe (line 1617) — diagnostic Tests: 3 new tests in ``tests/downloads/test_soulseek_pinning.py`` pin: - ``_SLSKD_DEFAULT_TIMEOUT`` is bounded (total set, ≤300s ceiling, connect ≤60s) — guards against future regressions that drop or unbound the timeout - ``_make_request`` returns None on ``asyncio.TimeoutError`` rather than raising — pins the caller contract - ``_make_direct_request`` returns None on ``asyncio.TimeoutError`` 2185/2185 full suite green. Closes #499. |
1 month ago |