Foundation for the isolated video download settings. The Downloads tab is almost
entirely music-specific, so on the video side the music download sections are hidden
(video-side.css) and a data-video-only 'Video Download Folders' section takes their
place — an input (download) and output (transfer/library) folder, stored SEPARATELY
from the music soulseek.* paths in video.db's video_settings KV table.
- api/video/downloads.py: GET/POST /api/video/downloads/config (download_path,
transfer_path), registered in the video blueprint. Imports nothing from music.
- video-settings.js: loadDownloads/saveDownloads wired into onPageShown + the
video save-button chain.
- The shared 'Indexers & Downloaders' tab is left untouched (identical for both).
2 tests (round-trip + the isolation guard). Quality profile, video hybrid, and the
shared slskd block are the next phases.
AniList defaults off (anime opt-in), so it showed 'Not configured' — implying a
missing API key, when it's keyless and just toggled off in Settings > Community Data
(No Key). Workers now report needs_key in get_stats (True for the key-gated
fanart/opensubtitles/trakt + all matchers; False for the keyless toggles). The
manager rail/pill + dashboard-header tooltip show 'Disabled' / 'Off — enable in
Settings' for a disabled keyless worker, and keep 'Not configured' only for ones
that genuinely need a key.
(The AniList on/off toggle already exists in the collapsed 'Community Data (No Key)'
settings frame — this just makes the status honest about what's needed.)
They were registered in the manager modal (WORKERS) + status poll (SERVICES) but the
dashboard HEADER renders its own per-worker .video-enrich-container buttons (with the
floating orb animation) keyed by data-video-enrich, and the orb engine has its own
WORKER_DEFS list — neither had the new services, so trakt/tvmaze/anilist/dearrow/
wikidata were absent from the header.
Added a header button + tooltip block for each (matching the fanart/opensubtitles
pattern, accent colors consistent with the manager orbs) and the matching WORKER_DEFS
entries in video-worker-orbs.js. Status was already wired via the SERVICES list, so
they now animate + report in the header identically to the existing workers.
Fifth/final service. Keyless, movies + shows, on by default → toggle in the
'Community Data (No Key)' frame.
- WikidataWorker: two-step lookup — find the entity by IMDb id (haswbstatement P345)
then read its official website (P856); stores wikidata_url. Registered in
build_backfill_workers.
- DB: wikidata_url/status/attempted on movies + shows + _BACKFILL/_BACKFILL_COLS;
both detail payloads return wikidata_url.
- config GET/POST wikidata_enabled; manager orb (green 🔗) + status poll; detail page
'Official Site' link badge alongside IMDb/TMDB/TVDB.
- 3 new tests (incl. the two-step fetch) + fixed-set/config assertions.
34 backfill tests green, ruff clean. All five new services (Trakt, TVmaze, AniList,
DeArrow, Wikidata) now have the full worker/DB/connections/orb/manager/detail parity.
Fourth service. Rides the YouTube-video enrich path (like RYD/SponsorBlock), keyless,
on by default → toggle in the YouTube-Extras frame.
- DeArrowWorker: fetches the branding API and stores the first non-original crowd
title for a cached YouTube video. apply_youtube_dearrow + youtube_video_dearrow_title
DB methods; dearrow_status added to the youtube_enrich next/breakdown whitelists.
- Schema: dearrow_title/status/attempted on youtube_video_stats (video_schema.sql +
_COLUMN_MIGRATIONS for existing DBs).
- config GET/POST dearrow_enabled; manager orb (blue 🏹, video-kind) + status poll;
YT video-detail panel shows a 'DeArrow' alt-title (payload via youtube.py + CSS).
- 4 new tests + fixed-set/config assertions.
30 backfill tests green. (My change is ruff-clean; the 12 pre-existing S110s in
api/video/youtube.py are unrelated tech debt on this branch, left untouched.)
Third service. Keyless GraphQL (new _http_post_json helper) → enable toggle in the
'Community Data (No Key)' frame, OFF by default (anime-niche + title-search match).
- AniListWorker: TV-only, searches AniList by title and stores the anime averageScore
(0-100), with a conservative normalized-title guard so a fuzzy anime search can't
attach a score to a non-anime show. anilist_enabled toggle (default off).
- DB: anilist_score/status/attempted on shows + _BACKFILL/_BACKFILL_COLS (keyed on
title); show_detail returns anilist_score.
- config GET/POST anilist_enabled (default 0); manager orb (blue 🎌) + status poll;
detail page 'AniList 85%' chip (+ CSS).
- 4 new tests (incl. the title-mismatch rejection) + fixed-set/config assertions.
27 backfill tests green, ruff clean. (Note: the youtube_status_route test still
flaps on the sandbox WSL WAL disk-I/O — environmental, unrelated.)
Second service. Keyless (no API key) → an enable toggle in a new 'Community Data
(No Key)' connections frame, mirroring the YouTube-Extras pattern.
- TVmazeWorker: TV-only (no movie DB), looks a show up by imdb/thetvdb id and
gap-fills the TVmaze community rating. On by default; tvmaze_enabled toggle.
- DB: tvmaze_rating/status/attempted on shows + _BACKFILL/_BACKFILL_COLS (show only);
show_detail returns tvmaze_rating.
- config GET/POST tvmaze_enabled; manager orb (teal 📺, show-kind) + status poll;
detail page TVmaze rating chip (+ CSS).
- 4 new tests + the 3 fixed-set/config-exact assertions updated.
Note: one UNRELATED test (youtube_status_route) flaps on a WSL WAL 'disk I/O error'
in this sandbox (pass/fail/fail across reruns of identical code) — environmental, not
this change; the TVmaze + config tests pass consistently.
First of the new enrichment services, wired to the SAME standard as the rest:
- TraktWorker backfill (core/video/enrichment/backfill.py): looks a title up by its
IMDb id (?extended=full) and gap-fills the community rating + vote count. Registered
in build_backfill_workers.
- DB: trakt_rating/trakt_votes/trakt_status/trakt_attempted columns + _BACKFILL /
_BACKFILL_COLS registration; detail payloads return trakt_rating/votes.
- Connections tab: Trakt service frame (Client ID field + Test button), wired into
video-settings.js load/save/bindings and the /enrichment/config GET+POST.
- Worker-manager modal + orb: WORKERS registry entry (red ★) + status-poll SERVICES
list, so it gets the same orb animation + per-service matched/pending/error card.
- Detail page: a 'Trakt 8.2' rating chip alongside IMDb/RT/Metacritic (+ CSS).
5 new tests + 2 fixed-set assertions updated; 249 video tests green, ruff clean.
The OpenSubtitles backfill worker already collects which subtitle languages exist
for each title (movies.subtitle_langs / shows.subtitle_langs), but nothing showed
it. Now the detail payload returns it (parsed to a list) and the hero renders a
'Subtitles: English · Spanish · …' CC-tinted chip row under the genres — so you can
tell subs exist before grabbing the file. Hidden entirely when there's no data.
(The clearlogo hero was already implemented, so this targets the one genuinely
invisible enrichment field.)
Also bound the per-iteration loop var in the ratings-breakdown counter (pre-existing
ruff B023). 154 video tests green, ruff clean.
The video dashboard's Memory Usage / System Uptime cards were stuck at their
markup defaults — flatten() only fed library + download figures from
/api/video/dashboard, so data-video-stat='memory'/'uptime' never updated.
Now it pulls those two from the SAME /api/system/stats the music dashboard uses
(one machine → identical figures), with an immediate fill on page-shown and a 10s
poll for live parity with music's push loop. Reached over HTTP (not music's
socket), so the video isolation contract holds; the poll cheaply no-ops whenever
the dashboard isn't the visible page.
- Dashboard header now shows fanart.tv / OpenSubtitles / YouTube Votes / SponsorBlock
buttons alongside TMDB/TVDB/OMDb/YouTube (same chip + spinner + tooltip + click
pause/resume + worker-orb animation).
- Socket status loop now iterates ALL engine workers (matchers + backfill) instead
of a hardcoded 3, so new buttons get live 2s status with no extra wiring.
- video-enrichment.js SERVICES + video-worker-orbs.js WORKER_DEFS extended.
header-actions already flex-wraps, so 8 buttons reflow cleanly.
- YouTube video cards show 👍/👎 like/dislike counts (from Return YouTube Dislike)
next to views, threaded through ytEpisodeOf.
- New GET /youtube/video/<id>/segments exposes stored SponsorBlock segments so the
player can offer skips. Update config-shape test for the new keys/toggles.
- Manage Workers modal: register fanart.tv / OpenSubtitles / YouTube Votes (RYD)
/ SponsorBlock in the worker rail (cards + animations + pause/resume come free
via the shared .em-* design); add the 'video' entity kind.
- Settings: fanart.tv + OpenSubtitles API-key fields (with Test buttons) and a
no-key 'YouTube Extras' toggle frame (RYD + SponsorBlock on/off).
- API /enrichment/config now reads/writes the two keys + the two toggles; a key
change rebuilds the engine so the worker turns on immediately.
Adds a VideoBackfillWorker base that enriches already-identified items BY id
(vs the matcher workers), with the exact same lifecycle + get_stats() shape so
the engine registry, /api/video/enrichment routes, and Manage-Workers modal
drive them identically.
Workers:
- fanart.tv (free key): gap-fills logo/clearart/banner/backdrop/poster art
- OpenSubtitles (free key): records subtitle-language availability per title
- Return YouTube Dislike (no key): like/dislike estimates on cached videos
- SponsorBlock (no key): crowd segments per video
DB: youtube_video_stats + youtube_video_segments tables; fanart/subs columns;
backfill_next/mark/breakdown + youtube_enrich_* helpers; likes/dislikes merged
into get_channel_videos. Seam tests in tests/test_video_backfill.py.
Correcting the cookies theory — ytdl-sub is just CLI yt-dlp (no browser). The real
differentiators are a CURRENT yt-dlp + not fighting YouTube's client identity. We
were pinning a static user_agent in the yt-dlp opts, which yt-dlp recommends against
(it trips YouTube's heuristics and truncates large-playlist pagination). Dropped it
so yt-dlp manages its own (current) client — the lever that lets a fresh ytdl-sub
page further than us. (_UA is still used for the InnerTube requests.) Softened the
partial-playlist note to just 'Showing N of TOTAL videos.'
Correcting my earlier 'YouTube caps at ~200' claim — that's the ANONYMOUS ceiling.
YouTube throttles unauthenticated playlist extraction to ~100 (yt-dlp) / ~200
(InnerTube); ytdl-sub gets everything because it runs with browser cookies. Our
code already wires cookiesfrombrowser via youtube.cookies_browser (same as the
music client) — it's just unset. The detail endpoint now resolves with a high
limit so a cookie-authenticated yt-dlp pages the WHOLE playlist, and uses whichever
source (yt-dlp vs InnerTube) returned more, with the true header count. The partial
note now tells you to add cookies in Settings to load the full list.
A big playlist read '97' because yt-dlp flat caps playlist extraction at ~100 and
its playlist_count is often unset, so we showed the FETCHED count. Now the detail
endpoint overlays the InnerTube playlist browse (browseId VL<id>): curator-ordered
videos up to ~200 (vs yt-dlp's ~100) AND the real count from the header's
numVideosText (e.g. '512 videos'). The billboard shows the true total; when the
list is partial it says 'Showing the first N of TOTAL videos (YouTube limits
playlist browsing).' Live-validated (Veritasium uploads: total 512, 200 loaded).
103 tests green.
Honest cap: YouTube's browse only exposes ~200 of a playlist (page 2 returns no
continuation), so very large playlists show the right COUNT but list ~200.
Clicking a video thumbnail now plays it inline — the thumbnails are play buttons
(data-vd-yt-play) that reuse the existing trailer overlay (youtube.com/embed/<id>
?autoplay=1). Applies to channel + playlist video cards and the playlist-section
mini-cards (which previously just linked out to YouTube). The rest of the row
still expands to details.
On a playlist page the hero actions were hardcoded for channels: 'Open on YouTube'
linked to /channel/<PL id> (opened as a channel, failed), and the Watchlist button
called the channel-follow handler. renderActions is now kind-aware — playlists link
to /playlist?list=<id> and the button follows/unfollows the PLAYLIST (new
toggleYtPlaylistFollowHero via a yt-pl-follow action).
The channel card's 'N videos' was actually the count of WISHED videos (0 until you
wish some) — so every channel read '0 videos'. It now shows the REMEMBERED catalog
size (from youtube_channel_videos, which fills in as a channel is enriched/opened),
falling back to 'Channel' when nothing's cached yet. Wished count is kept as a
separate wished_count field. Playlists likewise show their video count (cached when
the playlist is followed-with-videos or opened) instead of a static 'Playlist'.
list_watchlist_channels/playlists return the remembered count; the playlist detail +
follow endpoints cache the list so the count is known. 138 tests green.
The per-video wishlist toggle now reuses .library-artist-watchlist-btn (icon +
text, accent gradient) via a shared ytWishBtn() helper — compact in the dense
episode rows, icon-only in the tight playlist-section cards. 'Wishlist' ↔ 'In
Wishlist' with a green .watching state. Same behavior + data-vd-yt-wish hook;
just consistent chrome.
The per-video + Wish derived its channel from data._channel, which only exists on
CHANNEL pages — on a playlist detail page it's data._playlist, so the add posted
no youtube_id and the endpoint 400'd (button reverted, nothing happened). Now it
falls back to the playlist's owner channel (channel_id/title, else the playlist
itself) so wishing a video works on playlist pages too.
Channel playlists (the collapsible sections on a channel page) now each carry the
standard watchlist button — Add to Watchlist / In Watchlist — so you can follow a
channel's playlist without pasting its link. The /youtube/playlists/<id> endpoint
flags each playlist with its follow state to hydrate the button; the button click
is handled before the row's expand toggle (stopPropagation), and adds via
followPlaylist — the same plumbing the search chip + watchlist use. Followed ones
then appear in the watchlist Channels tab alongside channels.
The pasted-playlist chip used the YouTube 'Follow' toggle; swapped it for the
app-standard watchlist button (library-artist-watchlist-btn — same look/wording
used across video + music): ＋ Add to Watchlist ↔ ✓ In Watchlist. The search
toggle handler updates the icon/text spans + 'watching' class accordingly. No
global handler keys off that class, so reusing it is purely cosmetic.
- search: paste a playlist link → it resolves to a 'YouTube playlist' chip
(cover, owner, count) with Add-to-watchlist; clicking the chip opens it.
(isPlaylistRef + playlistCard + followPlaylist/unfollowPlaylist helpers.)
- routing: /video-detail/youtube/playlist/<PL…> deep-links like a channel
(string id); the open-detail event handles kind='playlist'.
- detail view: a FLAT list in the curator's order (no year-seasons, no season
nav / view toggle, no catalog streaming — it's a partial set). Billboard shows
'Playlist · N videos · owner'. Reuses the show-detail shell + the new
duration/views cards.
- watchlist: followed playlists sit beside channels in the Channels tab (rounded
square + a list badge), open on click, ✕ to unfollow.
Live-validated (Lex Fridman playlist resolves + renders in order). The 2 shell
test 'failures' are the pre-existing window.-assertion (no new globals added).
A playlist is now a first-class watchlist type alongside channels:
- core: parse_playlist_id (URL/bare id; rejects mixes RD… + personal lists) +
resolve_playlist → {playlist_id, title, channel_title, video_count, thumbnail,
videos} in the CURATOR's order (partial set, not by year).
- db: add/remove/list/watch_state for kind='playlist' video_watchlist rows
(mirrors channels; same surrogate scheme, separate kind so they don't collide).
- api: /youtube/resolve now detects playlist links; /youtube/playlist/follow +
/unfollow; the existing /youtube/playlist/<id> is upgraded to also return the
playlist meta + following (serves the channel-page expansion AND the new detail
view from one route); /youtube/channels also returns followed playlists.
Validated live (Lex Fridman playlist resolves title+owner+videos). 190 tests green.
Frontend (search detect + result card + detail view + watchlist display) next.
The rail 'season poster' is a video thumbnail shown in a 2:3 portrait slot
(object-fit: cover), but InnerTube hands us hqdefault (480x360, often sqp-shrunk)
— so it got crop-zoomed and looked soft. Now the poster pulls maxresdefault
(1280x720; ~300KB vs ~23KB) via the image proxy, with a fallback chain
(maxres → original thumbnail → hide) so videos without a maxres thumb still
render. Only the big rail poster is upgraded; the small episode stills already
downscale crisply.
The search/sort change hardcoded pillsHTML() for the youtube branch, so the
view toggle did nothing and the rail view's season posters vanished. The youtube
branch now honours seasonView (rail / timeline / tabs / list) for the year nav,
with the new search+sort controls stacked above it; flat mode (search / most-
viewed / longest) still hides the per-year nav. Default view (rail) shows the
year posters again.
The standalone channel page had sort/filter/search; the merge into show-detail
dropped them. Now that the whole catalog is client-side, they're instant:
- A controls row above the year pills: a debounced title search + a sort select
(Newest / Oldest / Most viewed / Longest).
- Newest/Oldest keep the year-season grouping (reordered); search + Most-viewed +
Longest collapse into one flat sorted 'results' list. Empty search → a clear
'No videos match …' state.
- Refactored ytToShow's grouping into reusable helpers (ytGroupByYear / ytEpisodeOf)
+ ytRegroup(), which the streaming loop now uses too — so newly-streamed videos
fold into whatever view/sort is active, and the master list stays intact.
Search focus is preserved across the regroup (same hack the music manager uses).
TV episode rows show runtime; channel video cards showed nothing per video. The
InnerTube lockup already carries both — now we capture duration ('12:34') + an
approximate view count (parsed from '2.6M views') in innertube_parse_video_items,
remember them on youtube_channel_videos (new duration/view_count cols + migration),
and render a duration badge bottom-right of the thumb + 'N views' in the card meta.
The background stream backfills them onto the recent (yt-dlp) videos too, so the
whole catalog gets them. Live-validated (MrBeast: 32:08 / 50M, 30/30). 65 tests green.
Each /videos batch was 3 InnerTube pages + 0.4s sleeps (>1s → tripped the
slow-request WARNING) and carried the ~2KB continuation token in the URL, so
every batch dumped a giant warning line — a dozen per channel open. Now it's a
POST (token in the body, never the URL) fetching ONE page per call with no
server sleep, so each request is fast (<1s, no warning) and the frontend's 120ms
pacing keeps it polite. 17 youtube API tests green.
Channel pages re-fetched everything every open (re-stream + ~3s yt-dlp metadata).
Now we remember what we learn about a channel and serve it cache-first:
- schema: youtube_channel_videos (list) + youtube_channel_meta (avatar/subs/tags/
banner); dates stay in youtube_video_dates, merged on read. DB: cache_/get_
channel_videos + cache_/get_channel_meta (upserts COALESCE so refreshes never
drop fields).
- enricher: _enrich now REMEMBERS a channel — caches the full InnerTube catalog
(list, via new innertube_channel_catalog) + metadata + dates, not just dates.
Since it already sweeps followed channels, watchlisted channels get pre-warmed
in the background → opening them is instant.
- channel endpoint: cache-first. A remembered channel renders from cache with NO
network (returns from_cache:true); a miss resolves live (yt-dlp) and remembers
it. The /videos batch endpoint caches every page it streams.
- frontend: cache hit renders the full catalog instantly, then re-streams to
refresh QUIETLY (new uploads/date fixes); only the first (miss) load shows the
'loading full history' banner.
Live-validated (MrBeast catalog: 90/3 pages, all titled+dated). 190 tests green.
The channel page was capped at the recent 90 (yt-dlp flat, hard min(90,limit)),
so prolific channels (Ninja Kidz etc.) showed only ~90 of hundreds. Now it loads
everything via YouTube's InnerTube API, paginated by continuation token — each
page fetched once (O(n), light on rate limits), unlike yt-dlp offset batches
which re-scan from the start every time.
- core: innertube_parse_video_items (keeps title+thumbnail, not just dates) +
innertube_channel_videos_page(channel_id, continuation) → {videos, continuation}.
- api: GET /youtube/channel/<id>/videos?continuation=<tok> returns ~a batch of
videos (approx dates, refined from the date cache) + the next token.
- frontend: ytLoadAllVideos streams batches after the fast initial 90 render,
folding each into the year-seasons live — backfills dates on the videos already
shown AND appends older ones, re-rendering per batch (episode grid only when the
viewed season changed). Replaces the old date-only re-poll (this dates AND
expands). Safety ceiling 2000; a 'Loading full history… N so far' indicator.
Validated live (MrBeast: 30/page, clean continuation, all dated). 101 tests green.
Only the detail page produced a real URL before; top-level video pages (search,
library, discover, calendar, watchlist, wishlist, …) pushed nothing and even
cleared the URL to '/'. Now each page deep-links to '/' + pageId (e.g.
/video-search), mirroring music's '/<page>' and the existing /video-detail/ scheme:
- navigate() pushes/replaces { videoPage } history state; parsePagePath/buildPagePath
added alongside parseDetailPath/buildDetailPath.
- popstate restores a page URL (and hands the side back to music when Back crosses
out of /video-*); boot restores a page deep link, re-asserting the URL against
music's boot clobber (same tactic the detail boot already uses).
- applySide() is now chrome-only; switchSide()/boot drive navigation explicitly.
- Nav anchors carry the real href=/video-<page> (was '#') and the click handler
lets ⌘/Ctrl/middle-click open a new tab — full parity with the music nav + cards.
Server already serves /video-* via the SPA catch-all; music side untouched
(isolated IIFE). Reload / Back / Forward / new-tab now work for every page.
- The hover snap was jarring because collapsed panels floored at min-width:10%
(so panels jumped ~10%↔80%) on a fast-start easeOutCirc curve. Raised the floor
to 25% (3 panels now settle 50/25/25 and hovering just glides the 50% across)
and switched to a gentler ease (0.66s cubic-bezier(0.45,0,0.2,1)).
- Lengthened the dim-overlay + sub/actions fades (0.4→0.55s, 0.3→0.45s) so they
move with the expansion instead of finishing early. Mobile stack unaffected
(it overrides min-width:0).
The video modal reused music's shell but had drifted into a parallel mini-design:
it emitted its own markup (.em-ph-main / .em-pause-btn / .em-item / .em-pg /
.vem-logo / .vem-icon) styled separately in video-side.css, instead of the
music classes that already exist in style.css. So the panel header, list rows,
rail icons and pager all looked 'redesigned'.
Now the modal emits music's real markup and inherits its global .em-* styling:
- Panel header -> .em-hero (glow + .em-icon--lg + .em-pill + .em-ph-sub)
- Unmatched list -> .em-row (status stripe, art+glyph, 'tried Nd ago', ghost Retry)
- Rail icon -> .em-icon chip (with --i stagger); pager -> .em-pager + .em-btn
- Pause -> .em-btn/.em-btn--go; coverage cards gain the First/Done/N-left badges
- Open/close entrance+exit animation (.em-in / .em-closing), dropping .em-in
after it plays so the 3s rail re-render doesn't replay the stagger
- Panel sets --accent-rgb so music's accent-keyed rules recolor per worker
Removed the now-dead bespoke CSS. Episodes stay a coverage card (per the backend:
episode art cascades from a show match; /priority only accepts movie/show), so
the top bar is correctly Movies/Shows/Auto. Music side untouched (shared classes
are global; video overrides were all #vem-overlay-scoped or now deleted).
Channels enriched before InnerTube existed (partial coverage, e.g. CaseyNeistat
36, Good Mythical MORE 25) were locked behind the 24h coverage gate and would
never reach the full ~240-date catalog on their own. Tag each enrichment row
with the source ('innertube'|'fallback'); legacy rows (method NULL after the
additive migration) bypass the gate ONCE so they re-enrich and upgrade, then
settle under the normal window. Non-destructive: existing follows, wished videos
(24), and cached dates (1176) are untouched — only coverage refreshes on next
open/sweep. Migration is additive (method TEXT); regression test added.
Two reports:
- 'Following · 0 videos added' toast: following from the channel page sent a
stub with no videos, so 0 were wished. Watchlisting a channel shouldn't bulk-
wish its whole catalog anyway (that conflates watchlist/track with wishlist/
acquire, unlike TV shows). follow() now sends only the channel fields and the
toast reads 'Added to watchlist'. Videos are wished explicitly via + Wish.
- '90 videos' in every channel header: that was just our fetch cap (limit=90),
not the real total — YouTube no longer exposes a trustworthy channel video
count (videoCountText reports a shelf count: 142 for Veritasium, 71 for MrBeast).
Dropped it; the accurate subscriber count already conveys the channel's scale.
The date re-poll only re-rendered if the season COUNT increased — but for a
daily channel the 90 recent videos all land in the current year, so it went from
[2026(3), Earlier(87)] (2 seasons) to [2026(90)] (1 season): count dropped, so
the years (which WERE cached, per the logs) never showed and the spinner hung.
Now it re-renders whenever the 'Earlier videos' bucket shrinks too. Also extended
the poll window (20/45/80/120/160s) for channels queued behind others.
- The InnerTube enrichment takes ~30-45s; while videos are still undated the page
now shows a spinner 'Fetching upload dates from YouTube… your year-seasons will
fill in shortly' (reuses the episode-syncing indicator), which clears once the
years populate via the re-poll (or after the poll window).
- The Watchlist button on a channel was calling loadChannel() on follow — a full
re-fetch + scroll-to-top that read as a page refresh. Now it just flips the
button in place (data.following + renderActions), like unfollow already did.
Bug from the logs: opening MrBeast showed only '2026: 3, Earlier: 87' because
the InnerTube date enrichment only fired for FOLLOWED channels — MrBeast was
opened, not followed, so it never ran and the page had only RSS dates (and RSS
includes Shorts that don't match the Videos tab → ~3 real matches). Now opening
any channel page enqueues enrichment, so its full catalog gets dated in the
background and the years populate via the existing re-poll. (24h gate still
prevents re-sweeping; the enqueue still no-ops under tests.)
A Beatport-style custom parser, but pointed at YouTube's own InnerTube browse API
— no key, no Java, no third-party proxies. Reads the channel 'Videos' tab's
lockupViewModel items (contentId + relative 'N ago' text → approximate date,
which is fine for year-seasons), paginating via continuation tokens.
Validated LIVE before wiring in: GMM/MrBeast/Veritasium each return 240 dated
videos in ~7s with correct year spreads (e.g. Veritasium 2017→2026), ~8 requests
per channel (light on rate limits). End-to-end enricher run cached 240 dates.
Enricher order is now: InnerTube (primary) → configured proxy (opt-in, only if
empty) → yt-dlp per-video (the fallback / basic method, for undated gaps + exact
dates). Pure parsing is fully unit-tested (relative-date conversion, lockup
filtering, continuation-token selection, pagination, guards). 182 video tests
green; additive — if InnerTube ever breaks, it returns {} and falls back, so
nothing breaks.
Live-tested the proxy against real channels — it does not work: Invidious public
APIs are 401/403 (auth-walled/disabled), and the one reachable Piped instance
returns EMPTY video lists for every channel (MrBeast/LTT/Veritasium all 0). Also
fixed a real bug: _harvest bailed on an empty first page instead of following the
nextpage token.
So the proxy is now OPT-IN via a youtube_proxy_instances setting (empty by
default → skipped entirely, no wasted dead calls). The default path is RSS +
the parallelized yt-dlp fallback, which actually works (your log: 9/36/25 dates
cached). Power users can point the setting at a live instance if they find one.
Verified the three engine workers (tmdb/tvdb/omdb) log per-item INFO under
video_enrichment.worker. Matched the youtube enricher to that exactly:
- INFO is now purely per-item ('Dated <ch> '<video>' -> <date>' / 'No date
for …') + one terse per-channel summary ('Dated N/M videos for <ch>'),
mirroring 'Matched … -> TMDB ID' / 'Synced full episode list…'.
- The 'YouTube dates:'-prefixed phase/diagnostic lines (enriching / proxy
returned / failures) demoted to DEBUG.
- Logger renamed video.youtube_enrichment -> video_enrichment.youtube (same
namespace as video_enrichment.worker).
- Per-item lines as it dates each video — 'Dated <channel> '<video>' -> <date>'
/ 'No date for …', mirroring the workers' 'Matched <kind> '<title>' -> TMDB
ID: …'. (Phase lines kept for context: enriching / proxy returned / done.)
Your log showed 'proxy returned 0' (public proxies dead) → the slow per-video
path. Date those recent uploads in a 3-worker thread pool instead of serially,
so a channel finishes in ~30s rather than 1-2 min; cached as each completes.
Addresses 'says running but I see nothing':
- Real logs: 'enriching <ch>…', 'proxy returned N', 'done — N proxy + N
per-video (X/Y dated)' so you can see exactly what it's doing.
- Proxy timeout 8s→4s so dead instances (most public ones) fail fast instead of
hanging ~40s before the yt-dlp fallback.
- Channel page now re-fetches a few times (25/60/110s) after load while videos
are still undated, re-rendering only when NEW year-seasons appear — so dates
the background enricher fills in pop in without a manual reload.
The date enricher was the odd one out — the dashboard polled
/api/video/enrichment/youtube/status every 3s (flooding the access log) instead
of using the shared WebSocket the other three workers push on. Now the existing
_emit_video_enrichment_status_loop also emits 'enrichment:youtube' (same stats
shape), and video-enrichment.js binds it on the socket alongside tmdb/tvdb/omdb.
Removed the bespoke polling loop. One-time prime fetch on load stays (instant
initial state); everything after is socket-driven. Consistent with the others.
The enrichment daemon was starting during API tests (it uses the default DB +
network), touching a stray real DB and causing sqlite disk-IO flakiness. enqueue()
now no-ops when PYTEST_CURRENT_TEST is set; _enrich() is still tested directly
with a tmp DB. 176 video tests green.
BoulderBadgeDad