aherman
/
SoulSync
mirror of https://github.com/Nezreka/SoulSync.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

readme update

Browse Source
pull/130/head
Broque Thomas 4 days ago
parent 3cf8461560
commit 45500892aa
1 changed files with 134 additions and 71 deletions
Show Stats Download Patch File Download Diff File

205
README.md
Unescape View File

@ -16,12 +16,13 @@
SoulSync bridges streaming services to your media server with automated discovery: SoulSync bridges streaming services to your media server with automated discovery:
1. **Monitors artists** → Automatically detects new releases 1. **Monitors artists** → Automatically detects new releases from your watchlist
2. **Generates playlists** → Release Radar, Discovery Weekly, Seasonal, Decade/Genre mixes 2. **Generates playlists** → Release Radar, Discovery Weekly, Seasonal, Decade/Genre mixes
3. **Downloads missing tracks** → From Soulseek, Beatport charts, playlists 3. **Downloads missing tracks** → From Soulseek, YouTube, Beatport charts, Spotify/Tidal/YouTube playlists
4. **Enriches metadata** → LRC lyrics, album art, proper tags 4. **Verifies downloads** → AcoustID fingerprinting confirms you got the right track
5. **Organizes files** → Custom templates for clean folder structures 5. **Enriches metadata** → LRC lyrics, album art, MusicBrainz IDs, proper tags
6. **Syncs media server** → Plex, Jellyfin, or Navidrome stay updated 6. **Organizes files** → Custom templates for clean folder structures
7. **Syncs media server** → Plex, Jellyfin, or Navidrome stay updated automatically
--- ---
@ -40,61 +41,94 @@ SoulSync bridges streaming services to your media server with automated discover
- Built from 1000+ track discovery pool - Built from 1000+ track discovery pool
- Refreshes every 24 hours - Refreshes every 24 hours
**Seasonal Playlists** - Halloween, Christmas, Valentine's, Summer, Spring, Autumn (auto-generated) **Seasonal Playlists** - Halloween, Christmas, Valentine's, Summer, Spring, Autumn (auto-generated based on time of year)
**Personalized Playlists** (12+ types) **Personalized Playlists** (12+ types)
- Recently Added, Top Tracks, Forgotten Favorites - Recently Added, Top Tracks, Forgotten Favorites
- Decade Playlists (1960s-2020s), Genre Playlists (15 categories) - Decade Playlists (1960s-2020s), Genre Playlists (15 categories)
- Daily Mixes, Hidden Gems, Popular Picks, Custom Builder - Daily Mixes, Hidden Gems, Popular Picks, Discovery Shuffle, Familiar Favorites
- Custom Playlist Builder
**ListenBrainz** - Import recommendation and community playlists **ListenBrainz** - Import recommendation and community playlists
**Beatport** - Electronic music charts by genre (House, Techno, Trance, etc.) **Beatport** - Full electronic music integration with genre browser (39+ genres)
- Top 100, Hype Charts, DJ Charts, Staff Picks - Top 100, Hype Charts, DJ Charts, Staff Picks, New Releases
- Browse by genre with dedicated discovery UI
### Multi-Source Downloads ### Multi-Source Downloads
**Sources**: Soulseek (FLAC priority), YouTube (Audio), Beatport charts, Spotify/Tidal playlists **Download Sources**: Soulseek (FLAC priority), YouTube (audio extraction), or Hybrid mode (tries primary source, falls back automatically)
**Features**
- Quality profiles: Audiophile, Balanced, Mobile
- Automatic format fallback (FLAC → MP3)
- Duplicate prevention against library
- Batch processing with retry logic
- Synchronized lyrics (LRC) for every track
### Metadata & Reliability
**Dual-Source System**
- **Primary**: Spotify (Preferred for richer data and discovery features)
- **Backup**: iTunes (No authentication required)
- **Redundancy**: System automatically manages both sources. If Spotify is authorized, it is prioritized. If Spotify is unavailable, rate-limited, or unauthorized, SoulSync **seamlessly switches to iTunes** for metadata, cover art, and artist tracking.
- **Fail-Safe**: Even with Spotify authorized, iTunes metadata is maintained as a redundant layer to ensure zero downtime.
**Playlist Sources**: Spotify, Tidal, YouTube, Beatport charts, ListenBrainz
### Advanced Matching **Features**
- Quality profiles with presets: Audiophile, Balanced, Space Saver
- Unicode/accent handling (KoЯn, Björk, A$AP Rocky) - Per-format configuration (FLAC, MP3, OGG, AAC, WMA) with min/max size and priority
- Fuzzy matching with confidence scoring - Automatic format fallback (FLAC → MP3 → next best)
- Album variation detection (Deluxe, Remastered, etc.) - Duplicate prevention against existing library
- Multi-strategy: exact → normalized → fallback - Batch processing with concurrent workers and automatic retry logic
- Source reuse — when downloading an album, reuses the same uploader for consistency
- Synchronized lyrics (LRC) via LRClib for every track
### Audio Verification
**AcoustID Fingerprinting** (optional) - Verifies downloaded files actually match the expected track
- Generates audio fingerprints and checks against AcoustID database
- Compares title and artist using fuzzy matching with configurable thresholds
- Fail-open design: verification errors never block downloads, only confident mismatches are rejected
- Mismatched files are flagged and can be re-queued via the wishlist
### Dual-Source Metadata
- **Primary**: Spotify — richer data, discovery features, playlist sync
- **Backup**: iTunes/Apple Music — no authentication required, works out of the box
- **Seamless switching**: If Spotify is unavailable, rate-limited, or unauthorized, SoulSync automatically falls back to iTunes for metadata, cover art, and artist info
- **MusicBrainz enrichment** — background worker matches artists, albums, and tracks to MusicBrainz IDs with 90-day caching
### Advanced Matching Engine
- Version-aware matching: strictly rejects remixes when you want the original (and vice versa)
- Unicode and accent handling (KoЯn, Bjork, A$AP Rocky)
- Fuzzy matching with weighted confidence scoring (title, artist, duration)
- Album variation detection (Deluxe, Remastered, Taylor's Version, etc.)
- Multi-query search strategy: generates 4-6 query variations per track
- Short title protection: prevents "Love" from matching "Loveless"
- Source-specific weighting: Soulseek prioritizes artist match, YouTube prioritizes title match
### Automation ### Automation
**Watchlist** - Monitor unlimited artists, auto-discover similar artists via music-map.com **Watchlist** - Monitor unlimited artists with per-artist configuration
- Choose which release types to track: Albums, EPs, Singles
- Content filters: Live, Remixes, Acoustic, Compilations
- Auto-discover similar artists via music-map.com
- Periodic scanning for new releases
**Wishlist** - Failed downloads retry every 30 minutes automatically **Wishlist** - Failed downloads are automatically queued for retry
- Auto-processing on a configurable timer
- Organized by category (albums vs singles)
- Cleanup tool removes tracks you've since acquired
**Background Tasks** - Database sync, discovery pool updates, seasonal content **Background Tasks** - Database sync, discovery pool updates, seasonal content, watchlist scanning
### Library Management ### Library Management
- **Quality Scanner** - Find low-bitrate files to replace - **Dashboard** with service status, system stats, and activity feed
- **Duplicate Cleaner** - Identify redundant tracks - **Database Updater** - Incremental or full sync from your media server (smart early-stopping for speed)
- **Completion Tracking** - Album progress percentages - **Metadata Updater** - Refresh artist photos, genres, and album art from Spotify/iTunes
- **Enhanced Search** - Unified search across Spotify, library, Soulseek - **Quality Scanner** - Find low-bitrate files that could be replaced with higher quality
- **Duplicate Cleaner** - Identify and remove redundant tracks, reclaim disk space
- **Completion Tracking** - See album/EP/single progress percentages per artist
- **Enhanced Search** - Unified search across Spotify, your library, and Soulseek simultaneously
- **Import System** - Drop existing audio files into a staging folder, match them to albums, and process with full metadata
- **Template Organization** - `$albumartist/$album/$track - $title` (fully customizable) - **Template Organization** - `$albumartist/$album/$track - $title` (fully customizable)
### Built-in Streaming Player
- Stream any track directly from Soulseek before downloading
- HTML5 audio player in the sidebar with play/pause, seeking, volume control
- Format detection with browser compatibility checking
- Retry logic with exponential backoff for reliability
--- ---
## Installation ## Installation
@ -107,7 +141,16 @@ docker-compose up -d
# Access at http://localhost:8008 # Access at http://localhost:8008
``` ```
### Python ### Unraid
SoulSync is available as an Unraid template. Install from Community Applications or manually add the template from:
```
https://raw.githubusercontent.com/Nezreka/SoulSync/main/templates/soulsync.xml
```
PUID/PGID are exposed in the template — set them to match your Unraid permissions (default: 99/100 for nobody/users).
### Python (No Docker)
```bash ```bash
git clone https://github.com/Nezreka/SoulSync git clone https://github.com/Nezreka/SoulSync
@ -124,7 +167,7 @@ python web_server.py
### Prerequisites ### Prerequisites
- **slskd** running and accessible ([Download](https://github.com/slskd/slskd/releases)) - **slskd** running and accessible ([Download](https://github.com/slskd/slskd/releases))
- **Spotify API** credentials ([Dashboard](https://developer.spotify.com/dashboard)) - **Spotify API** credentials ([Dashboard](https://developer.spotify.com/dashboard)) — optional but recommended
- **Media Server** (optional): Plex, Jellyfin, or Navidrome - **Media Server** (optional): Plex, Jellyfin, or Navidrome
### Step 1: Set Up slskd ### Step 1: Set Up slskd
@ -188,10 +231,12 @@ Open SoulSync at `http://localhost:8008` (or your server's IP) and go to Setting
**Paths:** **Paths:**
- **Download Path**: This should be the container path where slskd's downloads are mapped. If you followed the volume mapping above, that's `/app/downloads` - **Download Path**: This should be the container path where slskd's downloads are mapped. If you followed the volume mapping above, that's `/app/downloads`
- **Transfer Path**: This is where SoulSync puts your organized/renamed music. The default is `/app/Transfer` - just make sure you have a volume mapping for it - **Transfer Path**: This is where SoulSync puts your organized/renamed music. The default is `/app/Transfer` - just make sure you have a volume mapping for it so the files persist and your media server can see them
- **Staging Path**: Optional — for importing existing audio files. Map a host folder to `/app/Staging` if you want to use the import feature
**Media Server** (optional): **Media Server** (optional):
- **Plex/Jellyfin/Navidrome URL**: Use your machine's actual IP address, not `localhost`. For example `http://192.168.1.100:32400` for Plex. Same reason as slskd - `localhost` inside Docker means the container, not your host machine. - **Plex/Jellyfin/Navidrome URL**: Use your machine's actual IP address, not `localhost`. For example `http://192.168.1.100:32400` for Plex. Same reason as slskd - `localhost` inside Docker means the container, not your host machine.
- After connecting, run a **Database Update** from the Dashboard to populate your library
### Step 4: Docker Path Mapping ### Step 4: Docker Path Mapping
@ -206,6 +251,7 @@ This is where most issues come from. The key concept: SoulSync runs inside a Doc
| Database | `/app/data` | Use a named volume (see below) | | Database | `/app/data` | Use a named volume (see below) |
| slskd Downloads | `/app/downloads` | Same folder slskd downloads to | | slskd Downloads | `/app/downloads` | Same folder slskd downloads to |
| Music Library/Transfer | `/app/Transfer` | Where you want organized music to go | | Music Library/Transfer | `/app/Transfer` | Where you want organized music to go |
| Import Staging | `/app/Staging` | Optional — folder for importing existing files |
**Example docker-compose volumes for Linux:** **Example docker-compose volumes for Linux:**
```yaml ```yaml
@ -215,6 +261,7 @@ volumes:
- soulsync_database:/app/data - soulsync_database:/app/data
- /path/to/slskd/downloads:/app/downloads - /path/to/slskd/downloads:/app/downloads
- /path/to/music/library:/app/Transfer - /path/to/music/library:/app/Transfer
- ./staging:/app/Staging
``` ```
**Example for Unraid:** **Example for Unraid:**
@ -225,6 +272,7 @@ volumes:
- soulsync_database:/app/data - soulsync_database:/app/data
- /mnt/user/Music/Downloads:/app/downloads - /mnt/user/Music/Downloads:/app/downloads
- /mnt/user/Music/Library:/app/Transfer - /mnt/user/Music/Library:/app/Transfer
- /mnt/user/appdata/soulsync/staging:/app/Staging
``` ```
**Important:** The database should use a named volume (`soulsync_database:/app/data`), not a direct host path mount. Mounting a host folder to `/app/database` will overwrite Python module files the app needs to run. **Important:** The database should use a named volume (`soulsync_database:/app/data`), not a direct host path mount. Mounting a host folder to `/app/database` will overwrite Python module files the app needs to run.
@ -233,7 +281,11 @@ The paths you enter in SoulSync's settings page should always be the **container
### Unraid Users ### Unraid Users
If you're using the Unraid template from Community Applications, double check that the container path for your music share is set to match what SoulSync actually uses. The download folder should map to `/app/downloads` and your library/transfer folder should map to `/app/Transfer`. Then use those same container paths in SoulSync's settings page. If you're using the Unraid template from Community Applications:
- PUID/PGID are shown during setup — set them to match your permissions (default 99/100)
- The Music Share mount gives SoulSync access to your downloads and library
- Make sure the Transfer Dir path is **writable** — read-only mounts will cause post-processing failures
- After installing, run a **Database Update** from the Dashboard to populate your library
--- ---
@ -241,13 +293,13 @@ If you're using the Unraid template from Community Applications, double check th
**Perfect for:** **Perfect for:**
- Self-hosters with Plex/Jellyfin/Navidrome - Self-hosters with Plex/Jellyfin/Navidrome
- Music enthusiasts with 500+ album collections - Music enthusiasts building and maintaining large collections
- Electronic music fans (Beatport integration) - Electronic music fans (Beatport integration)
- Former Spotify users wanting local discovery - Former Spotify users wanting local discovery that actually works
**Not ideal for:** **Not ideal for:**
- Casual users wanting simple sync - Casual users wanting simple one-click sync
- Slow/metered internet connections - Slow or metered internet connections
- Users uncomfortable with APIs or Docker - Users uncomfortable with APIs or Docker
--- ---
@ -256,36 +308,41 @@ If you're using the Unraid template from Community Applications, double check th
| Feature | SoulSync | Lidarr | Headphones | Beets | | Feature | SoulSync | Lidarr | Headphones | Beets |
|---------|----------|--------|------------|-------| |---------|----------|--------|------------|-------|
| Custom Discovery Algorithm | ✓ | ✗ | ✗ | ✗ | | Custom Discovery Playlists (12+) | ✓ | ✗ | ✗ | ✗ |
| Personalized Playlists (12+) | ✓ | ✗ | ✗ | ✗ | | Seasonal/Personalized Playlists | ✓ | ✗ | ✗ | ✗ |
| Beatport Integration | ✓ | ✗ | ✗ | ✗ | | Beatport Integration | ✓ | ✗ | ✗ | ✗ |
| ListenBrainz Playlists | ✓ | ✗ | ✗ | ✗ | | ListenBrainz Playlists | ✓ | ✗ | ✗ | ✗ |
| Multi-Source (Spotify/Tidal/YouTube) | ✓ | ✓ | ✗ | ✗ | | Multi-Source Playlists (Spotify/Tidal/YouTube) | ✓ | ✗ | ✗ | ✗ |
| Watchlist Monitoring | ✓ (100+) | ✓ | ✓ | ✗ | | Soulseek Downloads | ✓ | ✗ | ✗ | ✗ |
| YouTube Downloads | ✓ | ✗ | ✗ | ✗ |
| Audio Fingerprint Verification | ✓ | ✗ | ✗ | ✓ |
| Watchlist Monitoring | ✓ | ✓ | ✓ | ✗ |
| LRC Lyrics | ✓ | ✗ | ✗ | Plugin | | LRC Lyrics | ✓ | ✗ | ✗ | Plugin |
| Advanced Matching | ✓ | ✗ | ✗ | ✓ | | Version-Aware Matching | ✓ | ✗ | ✗ | ✗ |
| Quality Scanner + Duplicate Cleaner | ✓ | ✗ | ✗ | ✓ | | Quality Scanner + Duplicate Cleaner | ✓ | ✗ | ✗ | ✓ |
| Template-Based Organization | ✓ | ✗ | ✗ | ✓ | | Template-Based Organization | ✓ | ✗ | ✗ | ✓ |
| Seasonal Playlists | ✓ | ✗ | ✗ | ✗ | | Built-in Streaming Player | ✓ | ✗ | ✗ | ✗ |
| Import Existing Files | ✓ | ✓ | ✗ | ✓ |
**SoulSync is the only tool combining intelligent discovery with multi-source automation and library management.** | Web UI | ✓ | ✓ | ✓ | ✗ |
--- ---
## Architecture ## Architecture
**Scale**: 83,000+ lines Python, 120+ API endpoints, handles 10,000+ album libraries **Scale**: ~100,000 lines across Python backend and JavaScript frontend, 60+ API endpoints, handles 10,000+ album libraries
**Integrations**: Spotify, Tidal, YouTube, Plex, Jellyfin, Navidrome, Slskd, ListenBrainz, LRClib, music-map.com, Beatport **Integrations**: Spotify, iTunes/Apple Music, Tidal, YouTube, Soulseek (slskd), Beatport, ListenBrainz, MusicBrainz, AcoustID, LRClib, music-map.com, Plex, Jellyfin, Navidrome
**Stack**: Python 3.8+, Flask, SQLite, PyQt6 (desktop GUI in maintenance mode) **Stack**: Python 3.11, Flask, SQLite (WAL mode), vanilla JavaScript SPA
**Core Components**: **Core Components**:
- Matching engine with Unicode/fuzzy logic - **Matching Engine** — version-aware fuzzy matching with multi-strategy query generation and source-specific confidence weighting
- Discovery system with custom algorithms - **Download Orchestrator** — routes between Soulseek and YouTube with hybrid fallback, batch processing with concurrent workers, automatic retry on failure/timeout
- Download pipeline with quality profiles - **Discovery System** — custom algorithms for personalized playlists, seasonal content, and similar artist exploration
- Metadata enhancement (lyrics, art, tags) - **Metadata Pipeline** — dual-source (Spotify/iTunes) with MusicBrainz enrichment, AcoustID verification, LRC lyrics, album art embedding via mutagen
- Template-based file organization - **Database Update Worker** — incremental sync from media servers with smart early-stopping (Jellyfin fast-path: ~2 API calls vs thousands)
- **Web Scan Manager** — debounced media server scanning with completion callbacks
- **Template-based File Organization** — configurable folder structures with automatic fallback
--- ---
@ -311,11 +368,18 @@ Transfer/Artist/Artist - Album/01 - Track.flac
**Enable Debug Logging**: Settings → Log Level → DEBUG → Check `logs/app.log` **Enable Debug Logging**: Settings → Log Level → DEBUG → Check `logs/app.log`
**Common Issues** **Common Issues**
- **Files not organizing**: Verify transfer path, check template syntax, use "Reset to Defaults"
- **Docker paths**: Ensure drives mounted in docker-compose.yml, use `/host/mnt/X/` prefix | Problem | Solution |
- **OAuth from remote**: Manually edit callback URL to server IP (Spotify requires 127.0.0.1) |---------|----------|
- **Wishlist stuck**: Auto-retry runs every 30 mins, check logs for failures | Library not showing up | Run a Database Update from Dashboard (full refresh for first time) |
- **Multi-library**: Select correct library in settings dropdown | Files not organizing | Verify transfer path is writable, check template syntax, try "Reset to Defaults" |
| Docker path issues | Ensure host paths are mapped in docker-compose.yml, use container paths in SoulSync settings |
| slskd not connecting | Use `host.docker.internal` or your machine's LAN IP, not `localhost` |
| OAuth from remote machine | Manually change `127.0.0.1` to your server's IP in the callback URL |
| Wishlist not processing | Auto-retry runs on a timer, check logs for persistent failures |
| Wrong track downloaded | Enable AcoustID verification in settings for fingerprint-based checking |
| Post-processing file not found | Check that download and transfer paths are correctly mapped and writable |
| Unraid permission errors | Set PUID/PGID to match your user (typically 99/100) |
--- ---
@ -330,10 +394,9 @@ Transfer/Artist/Artist - Album/01 - Track.flac
- Mobile-responsive improvements - Mobile-responsive improvements
### Under Consideration ### Under Consideration
- MusicBrainz ID integration - Additional streaming sources (Deezer, Apple Music direct)
- Additional streaming sources (Deezer, Apple Music)
- Playlist collaboration between instances - Playlist collaboration between instances
- Machine learning for matching - Machine learning for matching improvement
--- ---
@ -345,7 +408,7 @@ MIT License - See [LICENSE](LICENSE) file for details
## Acknowledgments ## Acknowledgments
**Services**: slskd, music-map.com, LRClib.net, Spotify, Tidal, Plex, Jellyfin, Navidrome **Services**: slskd, music-map.com, LRClib.net, Spotify, iTunes, Tidal, YouTube, Plex, Jellyfin, Navidrome, MusicBrainz, AcoustID, Beatport
**Community**: Contributors, testers, and users providing feedback **Community**: Contributors, testers, and users providing feedback

Write Preview
Loading…
Cancel
Save