proxysql

Commit Graph

Author	SHA1	Message	Date
Rene Cannao	ffdb334dc3	Add WHERE filters to prevent empty input errors and fix SQL syntax Changes: - Filter Posts by PostTypeId IN (1,2) (Questions and Answers) - Filter by minimum text length > 30 characters (Title + Body) - Update get_total_posts to count only eligible posts for accurate progress - Fix SQL syntax error in process_batch WHERE clause - Update documentation with filter details Rationale: - Empty or very short text causes embedding generation failures - PostTypeId 1,2 are most relevant content (Questions and Answers) - Ensures consistent counting between total, remaining, and processed	2 months ago
Rene Cannao	36a59f3f56	Add Posts embeddings processing script with exponential backoff New script: scripts/process_posts_embeddings.py Features: - Connects to SQLite3 server via MySQL connector with configurable credentials - Configures API client using API_KEY environment variable (fails if not set) - Processes unembedded Posts rows in configurable batch sizes (default: 10) - Uses LEFT JOIN WHERE IS NULL pattern to track unprocessed rows - Implements exponential backoff for retry failures with 5-minute maximum cap - Shows progress: remaining rows, processed count, percentage complete - Fails if Posts_embeddings table doesn't exist (no automatic creation) - Handles concurrent processing race conditions with small delays Script prerequisites: 1. Posts table must exist (copied from MySQL) 2. Posts_embeddings virtual table must exist: CREATE VIRTUAL TABLE Posts_embeddings USING vec0(embedding float[768]); Backoff behavior: - Default retry delay: 5 seconds - Exponential increase: 5s, 10s, 20s, 40s, ... up to 300s maximum - Resets on any successful operation (even if 0 rows processed)	2 months ago
Rene Cannao	95a95cb479	Add script to copy StackExchange Posts table from MySQL to SQLite3 server This Python script uses mysql.connector to copy the entire Posts table (248,905 rows) from MySQL to the ProxySQL SQLite3 server. The script handles schema conversion, proper escaping, and provides progress reporting. Tested with full copy taking 30 seconds at ~8,300 rows/sec.	2 months ago
Javier Jaramago Fernández	a443e3ded0	Improve RESTAPI example scripts 'kill_idle_backend_conns.py' and doc	4 years ago
Javier Jaramago Fernández	691c19a0b1	Fix 'change_host_status' README example for RESTAPI example scripts	4 years ago
Javier Jaramago Fernández	d6a753d0f6	Update 'scripts' folder with several RESTAPI example scripts	4 years ago
Valentin Rakush	66400a35c7	Implementation of the GET and POST methods	6 years ago
Valentin Rakush	f0687f3688	Implement restapi functionality - restapi_routes table in admin module - calling python script (json validation, error handling) - scripts: metrics, export_users - tests adjusted	6 years ago

8 Commits (ffdb334dc30929103603f1bbdb02cdeed8f73373)