proxysql

Commit Graph

Author	SHA1	Message	Date
Rene Cannao	637b2a669c	feat: Implement NL2SQL vector cache and complete Anomaly threat pattern management NL2SQL_Converter improvements: - Implement get_query_embedding() using GenAI module - Implement check_vector_cache() with KNN search via sqlite-vec - Implement store_in_vector_cache() with embedding storage - All stub methods now fully functional Anomaly_Detector improvements: - Implement add_threat_pattern() with embedding generation - Stores patterns in both main table and virtual vec table - Returns pattern ID on success, -1 on error Documentation: - Add comprehensive VECTOR_FEATURES documentation - README.md (471 lines): User guide and quick start - API.md (736 lines): Complete API reference - ARCHITECTURE.md (358 lines): System architecture - TESTING.md (767 lines): Testing guide and procedures This completes the vector features implementation, enabling: - Semantic similarity caching for NL2SQL queries - Embedding-based threat pattern detection - Full CRUD operations for threat patterns	1 month ago
Rene Cannao	782f6cb66b	feat: Implement threat pattern management and improve statistics Improve Anomaly_Detector with full threat pattern CRUD operations: Changes to lib/Anomaly_Detector.cpp: - Implement list_threat_patterns(): * Returns JSON array of all threat patterns * Shows pattern_name, pattern_type, query_example, severity, created_at * Ordered by severity DESC (highest risk first) - Implement remove_threat_pattern(): * Deletes from both anomaly_patterns and anomaly_patterns_vec tables * Proper error handling with error messages * Returns true on success, false on failure - Improve get_statistics(): * Add threat_patterns_count to statistics * Add threat_patterns_by_type breakdown * Shows patterns grouped by type (sql_injection, dos, etc.) - Add count_by_pattern_type query for categorization Features: - Full CRUD operations for threat patterns - JSON-formatted output for API integration - Statistics include both counts and categorization - Proper cleanup of both main and virtual tables	1 month ago
Rene Cannao	f226c0e687	feat: Implement embedding-based threat similarity for Anomaly Detection Implemented embedding-based threat pattern detection using GenAI and sqlite-vec: Changes to lib/Anomaly_Detector.cpp: - Add GenAI_Thread.h include and GloGATH extern - Implement get_query_embedding(): * Calls GloGATH->embed_documents() via llama-server * Normalizes query before embedding for better quality * Returns std::vector<float> with embedding - Implement check_embedding_similarity(): * Generates embedding for query if not provided * Performs sqlite-vec KNN search against anomaly_patterns table * Uses cosine distance (vec_distance_cosine) for similarity * Calculates risk score based on severity and distance * Returns AnomalyResult with pattern details and blocking decision - Implement add_threat_pattern(): * Generates embedding for threat pattern example * Stores pattern with embedding in anomaly_patterns table * Updates anomaly_patterns_vec virtual table for KNN search * Returns pattern ID on success Features: - Semantic similarity detection against known threat patterns - Configurable similarity threshold (ai_anomaly_similarity_threshold) - Risk scoring based on pattern severity (1-10) and similarity - Automatic threat pattern management with vector indexing	1 month ago
Rene Cannao	52a70b0b09	feat: Implement AI-based Anomaly Detection for ProxySQL Phase 3: Anomaly Detection Implementation This commit implements a comprehensive multi-stage anomaly detection system for real-time SQL query security analysis. Core Detection Methods: 1. SQL Injection Pattern Detection (lib/Anomaly_Detector.cpp) - Regex-based detection of 11 SQL injection patterns - Suspicious keyword detection (11 patterns) - Covers: tautologies, union-based, comment-based, stacked queries 2. Query Normalization (lib/Anomaly_Detector.cpp:normalize_query) - Converts to lowercase - Removes SQL comments - Replaces string/numeric literals with placeholders - Normalizes whitespace 3. Rate Limiting (lib/Anomaly_Detector.cpp:check_rate_limiting) - Per user/host query rate tracking - Configurable time windows (3600s default) - Auto-block on threshold exceeded - Prevents DoS and brute force attacks 4. Statistical Anomaly Detection (lib/Anomaly_Detector.cpp:check_statistical_anomaly) - Z-score based outlier detection - Abnormal execution time detection (>5s) - Large result set detection (>10000 rows) - Behavioral profiling per user 5. Embedding-based Similarity (lib/Anomaly_Detector.cpp:check_embedding_similarity) - Placeholder for vector similarity search - Framework for sqlite-vec integration - Detects novel attack variations Query Flow Integration: - Added `detect_ai_anomaly()` to MySQL_Session (line 3626) - Integrated after libinjection SQLi detection (line 5150) - Blocks queries when risk threshold exceeded (default: 0.70) - Sends error response with anomaly details Status Variables Added: - `ai_detected_anomalies`: Total anomalies detected - `ai_blocked_queries`: Total queries blocked - Available via: `SELECT * FROM stats_mysql_global` Configuration (defaults): - `enabled`: true - `risk_threshold`: 70 (0-100) - `similarity_threshold`: 85 (0-100) - `rate_limit`: 100 queries/hour - `auto_block`: true - `log_only`: false Detection Pipeline: ``` Query → SQLi Check → AI Anomaly Check → [Block if needed] → Execute (libinjection) (Multi-stage) ``` Files Modified: - include/MySQL_Session.h: Added detect_ai_anomaly() declaration - include/MySQL_Thread.h: Added AI status variables - lib/Anomaly_Detector.cpp: Full implementation (700+ lines) - lib/MySQL_Session.cpp: Integration and query flow - lib/MySQL_Thread.cpp: Status variable definitions Next Steps: - Add unit tests for each detection method - Add integration tests with sample attacks - Add user and developer documentation Related: Phase 1-2 (NL2SQL foundation and testing) Related: Phase 4 (Vector storage for embeddings)	1 month ago
Rene Cannao	af68f347d4	fix: Add missing verbosity level to proxy_debug call in Anomaly_Detector The proxy_debug macro requires a verbosity level as the second parameter. Fixed the call in Anomaly_Detector::analyze() to include the level.	1 month ago
Rene Cannao	d9346fe64d	feat: Add AI features manager foundation - Add AI_Features_Manager coordinator class - Add AI_Vector_Storage interface (stub) - Add Anomaly_Detector class (stub for Phase 3) - Update includes and main initialization	1 month ago

6 Commits (ee74384c79cd64fb714c1b609f2e14d2a49bf85e)