proxysql

Commit Graph

Author	SHA1	Message	Date
Rene Cannao	62cbd6c71e	Fix issues identified in AI code review - Fix combined_search: Build SQL query dynamically based on search_term - Only include MATCH relevance when search_term is provided - Fix parameter binding (fulltext_params vs params) - Fix search_term/query parameter mismatch in run_demo - combined_search mode now correctly receives query from kwargs - Add environment variable support for database credentials - nlp_search_demo.py: DB_HOST, DB_PORT, DB_USER, DB_PASSWORD, DB_NAME - stackexchange_posts.py: SOURCE_DB_* and TARGET_DB_* variables - Fix README CreationDate SQL example to use JSON_EXTRACT - Add zero division checks in get_table_stats and similarity_search_preparation	3 months ago
Rene Cannao	d37d291488	Implement comprehensive StackExchange posts processing with search capabilities - Unified script that creates target table with search-optimized schema - Adds full-text search indexes and text processing for better search results - Implements duplicate detection and prevention - Fixed MySQL functional index compatibility issue (removed problematic Tags index) - Added support for --limit 0 to process all posts - Added --warning-large-batches flag for batch size > 1000 - Improved text cleaning and normalization for search optimization - Enhanced progress tracking and batch processing efficiency Removes separate scripts (populate_search_columns.py, nlp_search_demo.py) and consolidates all functionality into unified solution.	4 months ago
Rene Cannao	d94dc036e8	Add StackExchange posts processing script with JSON storage - Retrieves parent posts (PostTypeId=1) and their replies (PostTypeId=2) from MySQL - Combines posts and tags into structured JSON format - Supports batch processing with memory-efficient chunking - Includes duplicate checking to avoid reprocessing - Creates target table automatically with JSON storage and BLOB for embeddings - Handles large datasets with offset-based pagination - Optimized to skip duplicate processing work - Includes verification and progress tracking Files: - scripts/stackexchange_posts.py: Main processing script	4 months ago

Author

SHA1

Message

Date

Rene Cannao

62cbd6c71e

Fix issues identified in AI code review

- Fix combined_search: Build SQL query dynamically based on search_term
  - Only include MATCH relevance when search_term is provided
  - Fix parameter binding (fulltext_params vs params)
- Fix search_term/query parameter mismatch in run_demo
  - combined_search mode now correctly receives query from kwargs
- Add environment variable support for database credentials
  - nlp_search_demo.py: DB_HOST, DB_PORT, DB_USER, DB_PASSWORD, DB_NAME
  - stackexchange_posts.py: SOURCE_DB_* and TARGET_DB_* variables
- Fix README CreationDate SQL example to use JSON_EXTRACT
- Add zero division checks in get_table_stats and similarity_search_preparation

Rene Cannao

d37d291488

Implement comprehensive StackExchange posts processing with search capabilities

- Unified script that creates target table with search-optimized schema
- Adds full-text search indexes and text processing for better search results
- Implements duplicate detection and prevention
- Fixed MySQL functional index compatibility issue (removed problematic Tags index)
- Added support for --limit 0 to process all posts
- Added --warning-large-batches flag for batch size > 1000
- Improved text cleaning and normalization for search optimization
- Enhanced progress tracking and batch processing efficiency

Removes separate scripts (populate_search_columns.py, nlp_search_demo.py)
and consolidates all functionality into unified solution.

Rene Cannao

d94dc036e8

Add StackExchange posts processing script with JSON storage

- Retrieves parent posts (PostTypeId=1) and their replies (PostTypeId=2) from MySQL
- Combines posts and tags into structured JSON format
- Supports batch processing with memory-efficient chunking
- Includes duplicate checking to avoid reprocessing
- Creates target table automatically with JSON storage and BLOB for embeddings
- Handles large datasets with offset-based pagination
- Optimized to skip duplicate processing work
- Includes verification and progress tracking

Files:
- scripts/stackexchange_posts.py: Main processing script

3 Commits (2d840faa7f80e90bbb722bf41ad6edcfc4f007fe)