10 Commits (bdd2ef70e6cad18d087b4d52ef63b72703f08f8b)

Author	SHA1	Message	Date
Rene Cannao	221831afc1	Add usage examples to script documentation and help output ... Changes: 1. Add comprehensive usage examples to script docstring with: - Remote API example (requires API_KEY environment variable) - Local Ollama example (uses --local-ollama flag) - Both examples show all common command line options 2. Add epilog to argparse help with concise examples: - Shows minimal command line for both modes - Points to docstring for full examples - Uses RawDescriptionHelpFormatter for proper formatting Users now have multiple ways to access usage information: - Read script header with `head -50 scripts/process_posts_embeddings.py` - Run `python3 scripts/process_posts_embeddings.py --help` - Both show appropriate examples for remote API and local Ollama modes	5 months ago
Rene Cannao	4aba7137b4	Add --local-ollama option for local Ollama server support ... New option: --local-ollama - Uses Ollama format with localhost:11434 API endpoint - Model: nomic-embed-text-v1.5 (without hf: prefix) - No API_KEY environment variable required - Overrides api-format, api-url, and api-model flags Changes: 1. Add --local-ollama boolean flag to parse_args() 2. Modify check_env() to skip API_KEY check when local-ollama is set 3. Update configure_client() to generate Ollama-specific SQL without 'key' parameter 4. Update main() to display correct configuration based on mode 5. Update documentation with local Ollama usage Behavior: - Without --local-ollama: Requires API_KEY, uses remote API with configurable format/url/model - With --local-ollama: No API_KEY needed, uses fixed local Ollama configuration	5 months ago
Rene Cannao	ffdb334dc3	Add WHERE filters to prevent empty input errors and fix SQL syntax ... Changes: - Filter Posts by PostTypeId IN (1,2) (Questions and Answers) - Filter by minimum text length > 30 characters (Title + Body) - Update get_total_posts to count only eligible posts for accurate progress - Fix SQL syntax error in process_batch WHERE clause - Update documentation with filter details Rationale: - Empty or very short text causes embedding generation failures - PostTypeId 1,2 are most relevant content (Questions and Answers) - Ensures consistent counting between total, remaining, and processed	5 months ago
Rene Cannao	36a59f3f56	Add Posts embeddings processing script with exponential backoff ... New script: scripts/process_posts_embeddings.py Features: - Connects to SQLite3 server via MySQL connector with configurable credentials - Configures API client using API_KEY environment variable (fails if not set) - Processes unembedded Posts rows in configurable batch sizes (default: 10) - Uses LEFT JOIN WHERE IS NULL pattern to track unprocessed rows - Implements exponential backoff for retry failures with 5-minute maximum cap - Shows progress: remaining rows, processed count, percentage complete - Fails if Posts_embeddings table doesn't exist (no automatic creation) - Handles concurrent processing race conditions with small delays Script prerequisites: 1. Posts table must exist (copied from MySQL) 2. Posts_embeddings virtual table must exist: CREATE VIRTUAL TABLE Posts_embeddings USING vec0(embedding float[768]); Backoff behavior: - Default retry delay: 5 seconds - Exponential increase: 5s, 10s, 20s, 40s, ... up to 300s maximum - Resets on any successful operation (even if 0 rows processed)	5 months ago
Rene Cannao	95a95cb479	Add script to copy StackExchange Posts table from MySQL to SQLite3 server ... This Python script uses mysql.connector to copy the entire Posts table (248,905 rows) from MySQL to the ProxySQL SQLite3 server. The script handles schema conversion, proper escaping, and provides progress reporting. Tested with full copy taking 30 seconds at ~8,300 rows/sec.	5 months ago
Javier Jaramago Fernández	a443e3ded0	Improve RESTAPI example scripts 'kill_idle_backend_conns.py' and doc	4 years ago
Javier Jaramago Fernández	691c19a0b1	Fix 'change_host_status' README example for RESTAPI example scripts	4 years ago
Javier Jaramago Fernández	d6a753d0f6	Update 'scripts' folder with several RESTAPI example scripts	4 years ago
Valentin Rakush	66400a35c7	Implementation of the GET and POST methods	6 years ago
Valentin Rakush	f0687f3688	Implement restapi functionality ... - restapi_routes table in admin module - calling python script (json validation, error handling) - scripts: metrics, export_users - tests adjusted	6 years ago