SoulSync

Commit Graph

Author	SHA1	Message	Date
Broque Thomas	59992d42a8	Deezer search: free-text fallback when advanced query returns 0 Defensive followup to the relevance fix. Deezer's advanced search syntax (`artist:"X"`) is documented as substring match, but in practice it's brittle on artist name variants ("Foreigner [US]", "The Foreigner") and on tracks indexed under non-canonical title spellings. When the advanced query returns nothing, we'd previously land at "No matches" — a regression vs. pre-fix behaviour where free-text would have returned a less-relevant but non-empty set. Fix: when the advanced query returns 0 results AND the caller used field-scoped kwargs, fall back to a free-text join of the same kwargs and re-query. Caller-side rerank still tightens whatever the fallback returns, so the worst-case post-fix behaviour is the pre-fix behaviour — never strictly worse. Pulled the cache + parse + store dance into a private helper (`_search_tracks_with_query`) so the orchestration can call it twice (advanced → fallback) without code duplication. Single API call when the advanced query has results — no wasted requests. Diagnostic logger.debug fires when the fallback triggers so we can see in production whether it's happening (and to which queries). # Tests added (4) - `test_falls_back_to_free_text_when_advanced_empty` — advanced query returns 0, free-text returns hits; client returns the free-text hits + both API calls fire. - `test_no_fallback_when_advanced_query_has_results` — single hit on advanced query → no second API call. - `test_no_fallback_when_legacy_free_text_call` — legacy callers already exhausted the only path; empty result is final. - `test_no_fallback_when_query_unchanged` — empty kwargs path doesn't trigger the fallback branch (used_advanced=False). # Existing tests updated The 4 prior `TestSearchTracksQueryWiring` + `TestSearchTracksCacheKey` tests were stubbing `_api_get` to return empty `{'data': []}` and asserting `assert_called_once`. With the new fallback, those stubs trigger a second API call and the assertions break — even though the FIRST call construction is what the tests cared about. Updated the stubs to return one fake hit so the fallback doesn't fire, and switched to `call_args_list[0]` for first-call inspection. # Verification - 18/18 deezer query tests pass (14 prior + 4 new) - 2445 full suite passes (+4 from prior commit) - Ruff clean	3 weeks ago
Broque Thomas	1cc37081a6	Fix Deezer search relevance — issue #534 # Background User reported (#534) that the import-modal "Search for Match" dialog returned irrelevant results when Deezer was the metadata source. Searching `Dirty White Boy` + `Foreigner` returned 5+ karaoke / "originally performed by" / "in the style of" / "re-recorded" / tribute-band results ranked above the actual Foreigner studio cut from Head Games. User had to scroll past the junk every time, or fall back to iTunes search which is much slower. # Root cause — two layers 1. Endpoint joined `track + artist` into free-text query. `/api/deezer/search_tracks` was passing `q=Dirty White Boy Foreigner` to Deezer's `/search/track` API. Deezer fuzzy-matches that string across title / lyrics / artist / album / contributors and orders by global popularity — anything that appears across many compilations outranks the canonical recording. 2. No local rerank. None of the search-modal endpoints applied any post-filtering. Deezer's API order shipped straight to the user. # Fix — same architectural shape Cin would build ## Layer 1: field-scoped query at the client boundary `core/deezer_client.py::search_tracks()` now accepts optional `track`, `artist`, `album` kwargs. When provided, builds Deezer's advanced search syntax: `q=track:"X" artist:"Y" album:"Z"`. Massive relevance improvement because each term matches the right field instead of fuzzy-matching everywhere. Backward compat preserved: legacy free-text `query=` callers still work unchanged. Field-scoped path takes precedence when both are provided. Empty input fast-fails without an API call. Embedded double-quotes stripped (Deezer's syntax has no escape mechanism). ## Layer 2: provider-neutral relevance reranker New `core/metadata/relevance.py` module — pure-function rerank over the canonical `Track` dataclass. Composable scoring: - Cover/karaoke patterns (multiplier 0.05, effectively buries): matches "karaoke", "originally performed by", "in the style of", "made famous by", "tribute", "vocal version", "backing track", "cover version", "re-recorded", "cover by", etc. across title, album, AND artist fields. Catches the screenshot's exact junk: artist credits like "Pop Music Workshop" / "The Karaoke Channel" / "Foreigner Tribute Band". - Variant tags (multiplier 0.4): live / acoustic / demo / instrumental / remix / radio edit / club mix etc. — softer penalty since the user MAY want them. Skipped entirely when the expected_title contains the same tag (so searching "Track (Live)" still ranks Live versions first). - Exact artist boost (multiplier 1.5): primary artist exactly matches expected_artist after normalisation. Single strongest signal for "this is the canonical recording". - Title + artist similarity via SequenceMatcher (parentheticals + punctuation stripped before comparison). - Album-type weighting: album=1.0 > single/ep=0.85 > compilation=0.7. Compilations are more likely tribute / karaoke repackages. Each component is a standalone function so tests pin them individually without standing up the full pipeline. ## Wired at three search-modal endpoints - `/api/deezer/search_tracks` — uses both layers (field-scoped query + rerank). - `/api/itunes/search_tracks` — uses rerank only (iTunes API has no advanced-syntax search, but karaoke / cover variants still leak through and need the local penalty). - `/api/spotify/search_tracks` — already builds field-scoped `track:X artist:Y` query; rerank added as the consistency safety net so all three sources behave the same from the user's perspective. Other Deezer call sites (matching engine, watchlist scanner, auto-import single-track ID) deliberately not touched in this PR — they have their own elaborate scoring pipelines tuned to their specific contexts and aren't surfacing the user-reported issue. Per Cin: "don't refactor beyond what the task requires." # Tests 71 new tests across 3 files: - `tests/metadata/test_relevance.py` (50 tests) — every scoring component pinned individually + the issue #534 screenshot reproduced as a regression test (real Foreigner cut wins after rerank, karaoke variants drop to bottom). - `tests/metadata/test_deezer_search_query.py` (14 tests) — advanced-syntax query construction, field-scoped wiring at the client boundary, free-text path unchanged, kwargs win when ambiguous, limit clamping, cache key consistency. - `tests/imports/test_search_match_endpoints.py` (7 tests) — end-to-end through Flask test client: Deezer endpoint passes kwargs not joined query; karaoke buried at bottom for all three sources; legacy query param still works without rerank. # Verification - 2441 full suite passes (+71 from baseline 2370) - 0 failures (the prior watchdog flake fix held) - Ruff clean across all changed files - JS parses clean (`node -c webui/static/helper.js`) # Architectural standards followed - Logic at the right boundary. Query construction lives in the client (every caller benefits from one change). Rerank lives in a neutral module (`core/metadata/relevance.py`) over the canonical `Track` dataclass — works for any source, not Deezer- specific. - Explicit > implicit. Every scoring rule has its own named function. Pattern tables are module-level constants tests can introspect. - Scope discipline. Audited every Deezer search call site; fixed the user-reported one + the consistent siblings. Did NOT speculatively normalise every Deezer call across the codebase. - Backward compat. Free-text `query=` callers untouched. Kwargs added to existing client method signature with safe defaults. - Tests pin contract at correct boundary. Pure-function rerank tests don't mock anything; client-query tests stub at `_api_get`; endpoint tests run through the real Flask app.	3 weeks ago

Author

SHA1

Message

Date

Broque Thomas

59992d42a8

Deezer search: free-text fallback when advanced query returns 0

Defensive followup to the relevance fix. Deezer's advanced search
syntax (`artist:"X"`) is documented as substring match, but in
practice it's brittle on artist name variants ("Foreigner [US]",
"The Foreigner") and on tracks indexed under non-canonical title
spellings. When the advanced query returns nothing, we'd previously
land at "No matches" — a regression vs. pre-fix behaviour where
free-text would have returned a less-relevant but non-empty set.

Fix: when the advanced query returns 0 results AND the caller used
field-scoped kwargs, fall back to a free-text join of the same
kwargs and re-query. Caller-side rerank still tightens whatever the
fallback returns, so the worst-case post-fix behaviour is the
pre-fix behaviour — never strictly worse.

Pulled the cache + parse + store dance into a private helper
(`_search_tracks_with_query`) so the orchestration can call it
twice (advanced → fallback) without code duplication. Single API
call when the advanced query has results — no wasted requests.

Diagnostic logger.debug fires when the fallback triggers so we can
see in production whether it's happening (and to which queries).

# Tests added (4)

- `test_falls_back_to_free_text_when_advanced_empty` — advanced
  query returns 0, free-text returns hits; client returns the
  free-text hits + both API calls fire.
- `test_no_fallback_when_advanced_query_has_results` — single hit
  on advanced query → no second API call.
- `test_no_fallback_when_legacy_free_text_call` — legacy callers
  already exhausted the only path; empty result is final.
- `test_no_fallback_when_query_unchanged` — empty kwargs path
  doesn't trigger the fallback branch (used_advanced=False).

# Existing tests updated

The 4 prior `TestSearchTracksQueryWiring` + `TestSearchTracksCacheKey`
tests were stubbing `_api_get` to return empty `{'data': []}` and
asserting `assert_called_once`. With the new fallback, those stubs
trigger a second API call and the assertions break — even though
the FIRST call construction is what the tests cared about. Updated
the stubs to return one fake hit so the fallback doesn't fire, and
switched to `call_args_list[0]` for first-call inspection.

# Verification

- 18/18 deezer query tests pass (14 prior + 4 new)
- 2445 full suite passes (+4 from prior commit)
- Ruff clean

Broque Thomas

1cc37081a6

Fix Deezer search relevance — issue #534

# Background

User reported (#534) that the import-modal "Search for Match" dialog
returned irrelevant results when Deezer was the metadata source.
Searching `Dirty White Boy` + `Foreigner` returned 5+ karaoke /
"originally performed by" / "in the style of" / "re-recorded" /
tribute-band results ranked above the actual Foreigner studio cut
from Head Games. User had to scroll past the junk every time, or
fall back to iTunes search which is much slower.

# Root cause — two layers

1. **Endpoint joined `track + artist` into free-text query.**
   `/api/deezer/search_tracks` was passing `q=Dirty White Boy Foreigner`
   to Deezer's `/search/track` API. Deezer fuzzy-matches that
   string across title / lyrics / artist / album / contributors and
   orders by global popularity — anything that appears across many
   compilations outranks the canonical recording.

2. **No local rerank.** None of the search-modal endpoints applied
   any post-filtering. Deezer's API order shipped straight to the
   user.

# Fix — same architectural shape Cin would build

## Layer 1: field-scoped query at the client boundary

`core/deezer_client.py::search_tracks()` now accepts optional
`track`, `artist`, `album` kwargs. When provided, builds Deezer's
advanced search syntax: `q=track:"X" artist:"Y" album:"Z"`. Massive
relevance improvement because each term matches the right field
instead of fuzzy-matching everywhere.

Backward compat preserved: legacy free-text `query=` callers still
work unchanged. Field-scoped path takes precedence when both are
provided. Empty input fast-fails without an API call. Embedded
double-quotes stripped (Deezer's syntax has no escape mechanism).

## Layer 2: provider-neutral relevance reranker

New `core/metadata/relevance.py` module — pure-function rerank over
the canonical `Track` dataclass. Composable scoring:

- **Cover/karaoke patterns** (multiplier 0.05, effectively buries):
  matches "karaoke", "originally performed by", "in the style of",
  "made famous by", "tribute", "vocal version", "backing track",
  "cover version", "re-recorded", "cover by", etc. across title,
  album, AND artist fields. Catches the screenshot's exact junk:
  artist credits like "Pop Music Workshop" / "The Karaoke Channel"
  / "Foreigner Tribute Band".
- **Variant tags** (multiplier 0.4): live / acoustic / demo /
  instrumental / remix / radio edit / club mix etc. — softer
  penalty since the user MAY want them. Skipped entirely when the
  expected_title contains the same tag (so searching
  "Track (Live)" still ranks Live versions first).
- **Exact artist boost** (multiplier 1.5): primary artist exactly
  matches expected_artist after normalisation. Single strongest
  signal for "this is the canonical recording".
- **Title + artist similarity** via SequenceMatcher (parentheticals
  + punctuation stripped before comparison).
- **Album-type weighting**: album=1.0 > single/ep=0.85 > compilation=0.7.
  Compilations are more likely tribute / karaoke repackages.

Each component is a standalone function so tests pin them
individually without standing up the full pipeline.

## Wired at three search-modal endpoints

- `/api/deezer/search_tracks` — uses both layers (field-scoped
  query + rerank).
- `/api/itunes/search_tracks` — uses rerank only (iTunes API has
  no advanced-syntax search, but karaoke / cover variants still
  leak through and need the local penalty).
- `/api/spotify/search_tracks` — already builds field-scoped
  `track:X artist:Y` query; rerank added as the consistency safety
  net so all three sources behave the same from the user's
  perspective.

Other Deezer call sites (matching engine, watchlist scanner,
auto-import single-track ID) deliberately not touched in this PR
— they have their own elaborate scoring pipelines tuned to their
specific contexts and aren't surfacing the user-reported issue.
Per Cin: "don't refactor beyond what the task requires."

# Tests

71 new tests across 3 files:

- `tests/metadata/test_relevance.py` (50 tests) — every scoring
  component pinned individually + the issue #534 screenshot
  reproduced as a regression test (real Foreigner cut wins after
  rerank, karaoke variants drop to bottom).
- `tests/metadata/test_deezer_search_query.py` (14 tests) —
  advanced-syntax query construction, field-scoped wiring at the
  client boundary, free-text path unchanged, kwargs win when
  ambiguous, limit clamping, cache key consistency.
- `tests/imports/test_search_match_endpoints.py` (7 tests) —
  end-to-end through Flask test client: Deezer endpoint passes
  kwargs not joined query; karaoke buried at bottom for all three
  sources; legacy query param still works without rerank.

# Verification

- 2441 full suite passes (+71 from baseline 2370)
- 0 failures (the prior watchdog flake fix held)
- Ruff clean across all changed files
- JS parses clean (`node -c webui/static/helper.js`)

# Architectural standards followed

- **Logic at the right boundary.** Query construction lives in the
  client (every caller benefits from one change). Rerank lives in
  a neutral module (`core/metadata/relevance.py`) over the
  canonical `Track` dataclass — works for any source, not Deezer-
  specific.
- **Explicit > implicit.** Every scoring rule has its own named
  function. Pattern tables are module-level constants tests can
  introspect.
- **Scope discipline.** Audited every Deezer search call site;
  fixed the user-reported one + the consistent siblings. Did NOT
  speculatively normalise every Deezer call across the codebase.
- **Backward compat.** Free-text `query=` callers untouched. Kwargs
  added to existing client method signature with safe defaults.
- **Tests pin contract at correct boundary.** Pure-function rerank
  tests don't mock anything; client-query tests stub at `_api_get`;
  endpoint tests run through the real Flask app.

2 Commits (048e4e85d5ceb6bbb9f6a6d2b2efbc259cc331ff)