Cin observed that database/api_call_history.json was occasionally
landing on disk truncated mid-write — `_load()` would log
`History file is not valid JSON, starting fresh` and 24h of metrics
would be lost.
Root cause: `save()` opened the file in 'w' mode (which truncates to
0 bytes immediately) and then streamed JSON via `json.dump`. Any
SIGINT/SIGTERM/crash between truncate and final write left the file
half-formed — exactly Cin's symptom of the JSON cutting off mid-array.
Switch to the standard atomic pattern: write to a sibling .tmp file,
flush + fsync, then `os.replace` (atomic on every platform we run on).
Failed writes also clean up the leftover .tmp file. The canonical
file is now either the previous good copy or the new good copy —
never a partial one.
API Call Tracker:
- Save/load 24h minute-bucketed history + events to database/api_call_history.json
- Persists across server restarts via atexit + signal handler hooks
- New record_event() for rate limit bans (called from _set_global_rate_limit)
- New get_debug_summary() for Copy Debug Info — 24h totals, peak cpm with
timestamp, per-endpoint breakdown, and last 20 rate limit events
- Fixed race condition: events iteration now inside lock during save
Spotify Rate Limit Mitigation:
- Enrichment worker: max_pages=5 on get_artist_albums (was unlimited — artist
with 217 albums caused 22 paginated API calls, now capped at 5)
- Enrichment worker: inter_item_sleep raised from 0.5s to 1.5s
Spotify Re-Auth Fix:
- Both OAuth callbacks (port 8008 + 8888) now clear rate limit ban AND
post-ban cooldown after successful re-auth — Spotify usable immediately
instead of stuck on Deezer fallback for 5 minutes
- Auth cache invalidated on both global client and enrichment worker client
- New Discogs section on Settings → Connections with personal token input
- Discogs added as fallback metadata source option alongside iTunes/Deezer
- Token saved to discogs.token config key
- Discogs added to API rate monitor gauges (60/min with auth)
- Help text links to discogs.com/settings/developers for token generation
- New core/api_call_tracker.py — centralized tracker with rolling 60s
timestamps (speedometer) and 24h minute-bucketed history (charts)
- Instrument all 9 service client rate_limited decorators to record
actual API calls with per-endpoint tracking for Spotify
- 1-second WebSocket push loop for real-time gauge updates
- Modern radial arc gauges with service brand colors, glowing active
arc, endpoint dot, 0/max scale labels, smooth CSS transitions
- Click any gauge to open detail modal with 24h call history chart
(Canvas 2D, HiDPI, gradient fill, grid lines, danger zone band)
- Spotify modal shows per-endpoint history lines with color legend
and live per-endpoint breakdown bars
- Rate limited state indicator — blinking red badge with countdown
timer appears on gauge card when Spotify ban is active
- REST endpoint GET /api/rate-monitor/history/<service> for chart data
- Responsive grid layout (5 cols desktop, 3 tablet, 2 phone)
Broque Thomas
Antti Kettunen