3 Commits (dev)

Author	SHA1	Message	Date
Broque Thomas	877d0e7d81	Personalized pipeline: auto-refresh stale snapshots after watchlist scan ... Snapshots now track when their source data changes. Watchlist scan emits stale flags on the playlists whose underlying pool just got refreshed; the next pipeline run sees the flag and regenerates the snapshot before syncing, so the server playlist never lags the source. Schema: - new `is_stale INTEGER NOT NULL DEFAULT 0` column on `personalized_playlists`, plus an idempotent ADD COLUMN migration in `ensure_personalized_schema` for installs created before this PR. - `PlaylistRecord.is_stale: bool = False` exposed on the dataclass so callers can branch on freshness without re-querying. Manager: - new `mark_kinds_stale(kinds, profile_id=None)` flips the flag in bulk for a list of kinds (used by upstream data refreshers). - `_persist_snapshot` clears `is_stale = 0` on successful refresh. - SELECT statements + `_row_to_record` updated to read the column (with tuple-form length guard for safety). Pipeline: - `_build_payloads_for_kinds` now branches: refresh_first=True OR `existing.is_stale` -> refresh_playlist, else read existing snapshot. So the auto-refresh kicks in without needing the user to toggle the refresh-each-run option. Watchlist scanner emits stale flags at three sites: - after `update_discovery_pool_timestamp` -> marks pool-fed kinds stale: hidden_gems, discovery_shuffle, popular_picks, time_machine, genre_playlist, daily_mix. - after release_radar `save_curated_playlist` -> marks `fresh_tape`. - after discovery_weekly `save_curated_playlist` -> marks `archives`. All three calls go through a module-level `_mark_personalized_kinds_stale` helper that builds a PersonalizedPlaylistManager with `deps=None` (only DB access is needed for the flag update — no generator dispatch). Each call is wrapped in try/except so a flag failure can never abort the scan itself. Tests: - new `TestStaleFlag` class in `test_personalized_manager.py` (6 tests): default-false, single-kind flip, multi-kind, profile scoping, refresh-clears, empty-list noop. - two new pipeline tests pin the auto-refresh dispatch: `test_stale_snapshot_auto_refreshes_even_without_refresh_first` and `test_non_stale_snapshot_skips_refresh`. - existing stub-manager `SimpleNamespace` returns gained `is_stale=False` so the new attribute read doesn't AttributeError. Full suite: 3391 pass. User-facing WHATS_NEW entry added under 2.5.2 (above the prior pipeline auto-sync entry) describing the auto-refresh behavior.	1 week ago
Broque Thomas	cc0828e9ff	Personalized playlists (4/N): staleness post-filter (exclude_recent_days) ... Adds the first quality feature on top of the manager: when `config.exclude_recent_days > 0`, the manager drops any track from the generator's output whose primary id was served by this kind for this profile in the last N days. Lives at the manager layer, not in each generator, so: - generators stay focused on selection logic - staleness behavior stays uniform across every kind - enabling/disabling per playlist is just a config patch Implementation: - New `PersonalizedPlaylistManager._apply_quality_filters` runs after generator returns, before `_persist_snapshot`. - Reads recent ids via existing `recent_track_ids` accessor. - Tracks without a primary id pass through unchanged (nothing to dedupe on -- happens for sourceless tracks during edge cases). - Returns a new list (never mutates input). Default `exclude_recent_days = 0` preserves pre-overhaul behavior. Per-playlist override via `PUT /api/personalized/playlist/<kind>/config` with `{"exclude_recent_days": N}`. Recommended values: - Discovery Shuffle: 1-3 days (high churn desired) - Hidden Gems: 7-14 days (avoid same gems weekly) - Time Machine / Genre: 30+ days (slow rotation OK, stable view preferred) 4 new boundary tests: - Zero days = no filter (default behavior preserved) - Positive days drops tracks served in window - Filter preserves new tracks alongside dropped ones - Tracks without primary id pass through unchanged 3369 tests pass total. Note: listening-history cross-ref + seeded shuffle are deferred to a future PR. Each requires deeper integration -- listening history needs a play-events table the discovery pool can query against; seeded shuffle needs the legacy generators to accept a seed param without breaking their existing diversity / popularity logic.	1 week ago
Broque Thomas	79224ed294	Personalized playlists (1/N): unified storage + manager foundation ... Begins the standardization of the personalized-playlist subsystem. Pre-existing state was a patchwork: Group A (Fresh Tape / Archives / Seasonal Mix) lived in `discovery_curated_playlists` and `curated_seasonal_playlists` with inconsistent shapes; Group B (Hidden Gems / Discovery Shuffle / Time Machine / Popular Picks / Genre / Daily Mixes) was computed on-demand by `PersonalizedPlaylistsService` with no persistence -- every call reran the generator with `ORDER BY RANDOM()` so results rotated. Post-overhaul (this PR) every personalized playlist lands in one unified storage layer with stable identity, persistent track lists, explicit refresh, and per-playlist user-tweakable config. Foundation in this commit (no behavior change yet): - `database/personalized_schema.py`: 3 tables created idempotently at app startup (wired into `MusicDatabase._initialize_database`). - `personalized_playlists`: one row per (profile, kind, variant) with config_json, track_count, last_generated_at, last_synced_at, last_generation_source, last_generation_error. Variant '' (empty string) for singletons; non-empty for time_machine / seasonal_mix / genre_playlist / daily_mix. - `personalized_playlist_tracks`: current snapshot per playlist. Atomically replaced on refresh. - `personalized_track_history`: append-only log powering the `exclude_recent_days` config knob. - `core/personalized/types.py`: `Track`, `PlaylistConfig`, `PlaylistRecord` dataclasses. `PlaylistConfig.merged()` for partial-update PATCH semantics; `Track.from_dict()` accepts the legacy generator output shape unchanged. - `core/personalized/specs.py`: `PlaylistKindSpec` (kind, name_template, default_config, generator, variant_resolver) and a module-level registry. Generators register at import time; manager dispatches by kind. - `core/personalized/manager.py`: `PersonalizedPlaylistManager` -- the only thing that touches the new tables. Owns: - ensure_playlist (auto-create row from kind defaults) - get_playlist / list_playlists - refresh_playlist (atomic snapshot replace; generator exception preserves previous good snapshot + records error on row) - get_playlist_tracks - update_config (deep-merge with stored config, including extra dict) - recent_track_ids (staleness lookup for generators) 35 boundary tests in `tests/test_personalized_manager.py` pin every shape: config round-trip / merge semantics / extra deep-merge / defaults; Track.from_dict tolerance + primary_id fallback chain; registry dedup / display_name with+without variant; manager ensure_playlist auto-create + idempotency, variant separation, required-variant enforcement, unknown-kind error; refresh persists + replaces atomically + survives generator exception with previous snapshot intact + records source from first track + round-trips nested track_data_json; update_config patch semantics; list_playlists profile scoping; staleness history scoped to (profile, kind, days). 3304 tests pass total. Generators ship in subsequent commits on this branch -- each kind migrated one at a time with its own per-kind boundary tests.	1 week ago