proxysql

Commit Graph

Author	SHA1	Message	Date
René Cannaò	ea16ebb8ad	Merge branch 'v3.0-5473' into v3.0-5499 Signed-off-by: René Cannaò <rene@proxysql.com>	1 month ago
René Cannaò	d872a392a6	Merge branch 'v3.0-5473' into v3.0-5498 Signed-off-by: René Cannaò <rene@proxysql.com>	1 month ago
René Cannaò	8d7459e560	Merge branch 'v3.0-5473' into v3.0-5497 Signed-off-by: René Cannaò <rene@proxysql.com>	1 month ago
René Cannaò	9bccd2d1fa	Merge branch 'v3.0-5473' into v3.0-5495 Signed-off-by: René Cannaò <rene@proxysql.com>	1 month ago
René Cannaò	cffd85f0ff	Merge pull request #5511 from sysown/v3.0-5494 Phase 3.6: Backend variable sync decisions + unit tests	1 month ago
René Cannaò	c40eed9405	Merge branch 'v3.0-5473' into v3.0-5492 Signed-off-by: René Cannaò <rene@proxysql.com>	1 month ago
René Cannaò	f07cd83d16	Extract MySQL protocol utilities for FFTO unit testing (#5499 ) New: include/MySQLProtocolUtils.h, lib/MySQLProtocolUtils.cpp Functions: - mysql_read_lenenc_int(): reads MySQL length-encoded integers from raw buffers (mirrors read_lenenc_int in MySQLFFTO.cpp) - mysql_build_packet(): constructs MySQL wire-format packets (3-byte length + seq_id + payload) for crafting test data	1 month ago
René Cannaò	e82bcb391b	Extract PgSQL error classification logic (Phase 3.10, #5498 ) New: include/PgSQLErrorClassifier.h, lib/PgSQLErrorClassifier.cpp Functions: - classify_pgsql_error(): classifies by SQLSTATE class — connection (08), transaction rollback (40), resources (53) = retryable; operator intervention (57), system error (58) = fatal; all others (syntax 42, constraints 23, data 22) = report to client - pgsql_can_retry_error(): checks retry conditions (retries left, not in transaction — PgSQL transactions are atomic)	1 month ago
René Cannaò	78400743ff	Extract PgSQL monitor health decisions (Phase 3.9, #5497 ) New: include/PgSQLMonitorDecision.h, lib/PgSQLMonitorDecision.cpp Functions: - pgsql_should_shun_on_ping_failure(): threshold-based shunning (simpler than MySQL — always aggressive with kill_all) - pgsql_should_offline_for_readonly(): read-only server in writer hostgroup should go OFFLINE_SOFT Unshun recovery is already covered by MonitorHealthDecision.h can_unshun_server() (same logic for both protocols).	1 month ago
René Cannaò	ead2331fb9	Extract backend variable sync decisions (Phase 3.6, #5494 ) New: include/BackendSyncDecision.h, lib/BackendSyncDecision.cpp determine_backend_sync_actions() returns bitmask of needed sync: - SYNC_USER: username mismatch → CHANGE USER required - SYNC_SCHEMA: schema mismatch → USE <db> required (skipped if SYNC_USER set, since CHANGE USER handles schema) - SYNC_AUTOCOMMIT: autocommit state mismatch Covers both MySQL (7 verify functions) and PgSQL (2 verify functions) at the decision level — the core comparisons are the same.	1 month ago
René Cannaò	5c227a3bc9	Extract MySQL error classification logic (Phase 3.7, #5495 ) New: include/MySQLErrorClassifier.h, lib/MySQLErrorClassifier.cpp Functions: - classify_mysql_error(): classifies error codes as retryable (1047 WSREP, 1053 shutdown) or fatal, checking retry conditions - can_retry_on_new_connection(): checks if offline server retry is possible given connection state (reusable, no txn, no transfer)	1 month ago
René Cannaò	5148989a72	Extract PgSQL CommandComplete tag parser for unit testing (#5499 ) New files: - include/PgSQLCommandComplete.h: PgSQLCommandResult struct + parse_pgsql_command_complete() declaration - lib/PgSQLCommandComplete.cpp: implementation mirroring extract_pg_rows_affected() from PgSQLFFTO.cpp Parses PostgreSQL CommandComplete tags (INSERT/UPDATE/DELETE/SELECT/ FETCH/MOVE/COPY/MERGE) to extract row counts. Pure function with no FFTO or session dependencies.	1 month ago
René Cannaò	7b06515d7f	Merge branch 'v3.0-5473' into v3.0-5496 Signed-off-by: René Cannaò <rene@proxysql.com>	1 month ago
René Cannaò	3f0281651e	Extract transaction state tracking logic (Phase 3.8, #5496 ) New files: - include/TransactionState.h: 2 pure function declarations - lib/TransactionState.cpp: implementations Functions: - update_transaction_persistent_hostgroup(): mirrors session logic for locking/unlocking HG on transaction start/end - is_transaction_timed_out(): checks if transaction exceeded max time Logic identical for MySQL and PgSQL (shared via Base_Session).	1 month ago
René Cannaò	aaa02bb400	Extract hostgroup routing decision logic (Phase 3.5, #5493 ) New files: - include/HostgroupRouting.h: HostgroupRoutingDecision struct + resolve_hostgroup_routing() declaration - lib/HostgroupRouting.cpp: implementation mirroring the routing block in get_pkts_from_client() (~lines 5340-5377) Logic covers: QP destination override, transaction affinity, hostgroup lock acquisition/enforcement, error on HG mismatch. Identical for both MySQL and PgSQL protocols.	1 month ago
René Cannaò	dba1294585	Extract server selection algorithm into pure functions (Phase 3.4, #5492 ) New files: - include/ServerSelection.h: ServerCandidate struct (lightweight, no MySrvC dependency) + is_candidate_eligible() + select_server_from_candidates() - lib/ServerSelection.cpp: implementations ServerCandidate decouples selection logic from connection pool: contains only weight, status, connections, latency, and replication lag — no MySQL_Connection pointers or pool objects. Selection algorithm: 1. Filter candidates by eligibility (ONLINE, below max_connections, within latency/lag thresholds) 2. Weighted random selection using seeded LCG (deterministic for testing, avoids polluting global srand() state) Uses its own ServerSelectionStatus enum to avoid pulling in proxysql_structs.h and its entire dependency chain.	1 month ago
René Cannaò	59654e48a3	Extract monitor health decision logic into pure functions (Phase 3.3, #5491 ) New files: - include/MonitorHealthDecision.h: declarations for 4 pure functions - lib/MonitorHealthDecision.cpp: implementations Extracted functions: - should_shun_on_connect_errors(): mirrors MySrvC::connect_error() threshold logic (min(shun_on_failures, connect_retries+1)) - can_unshun_server(): mirrors MyHGC recovery loop with time check, connect_timeout_max cap, min 1s floor, and kill_all drain check - should_shun_on_replication_lag(): mirrors MySQL_HostGroups_Manager lag check with consecutive count threshold - can_recover_from_replication_lag(): mirrors lag recovery check All functions are pure (no global state, no I/O) and added to libproxysql.a via lib/Makefile.	1 month ago
René Cannaò	3f1b0a1510	Add static linking for zstd compression library - Add zstd v1.5.7 as a static dependency in deps/ - Add zstd build rules to deps/Makefile - Add ZSTD_PATH, ZSTD_IDIR, ZSTD_LDIR to makefiles_paths.mk - Add zstd include path to lib/Makefile and src/Makefile - Link zstd statically in LIBPROXYSQLAR for all platforms - Remove -lzstd from dynamic linking flags (Linux and macOS) - Replace clickhouse-cpp bundled zstd with our static zstd This eliminates the runtime dependency on system/Homebrew zstd and ensures consistent behavior across platforms. Related: #5462	1 month ago
René Cannaò	88be1449d3	Fix compilation issues on macOS - Add Homebrew include paths for zstd headers on macOS - Fix pthread_t cast on macOS where it's a pointer type On macOS, pthread_t is defined as _opaque_pthread_t * (a pointer), unlike Linux where it's an integral type. Use reinterpret_cast<uintptr_t> for proper conversion. Related: #5462	1 month ago
Rene Cannao	c2f678a9de	feat: implement release tiers and feature guards for v3.0, v3.1, and v4.0 This commit introduces a structured release tier system controlled by environment variables, allowing three distinct ProxySQL versions to be built from the same codebase while maintaining a clear upgrade path. Key Changes: - Tier 1 (v3.0.x Stable): Default build including bug fixes and core enhancements. - Tier 2 (v3.1.x Innovative): Enabled via PROXYSQL31=1. Adds FFTO and TSDB. - Tier 3 (v4.0.x AI/MCP): Enabled via PROXYSQLGENAI=1. Adds the AI stack. Implementation Details: - Makefile: Implemented dynamic versioning logic that increments the minor version for v3.1 and the major version for v4.0. Added tier documentation. - Feature Guards: Introduced PROXYSQLFFTO and PROXYSQLTSDB guards across ProxySQL_Admin, ProxySQL_Statistics, REST API, and HTTP server. - FFTO: Guarded Fast Forward Traffic Observer in MySQL and PgSQL sessions, including thread-local variables and configuration parameters. - TSDB: Guarded the entire Time Series Database subsystem, including SQLite table definitions, background loops, and REST API endpoints. - Build Fixes: * Fixed 'proxy_debug_func' linkage by adding extern "C" and moving the definition outside of DEBUG guards to ensure non-debug builds link correctly. * Corrected misplaced/nested DEBUG and PROXYSQLFFTO guards. * Verified build stability for all tiers (unset, PROXYSQL31=1, PROXYSQLGENAI=1).	2 months ago
Rene Cannao	acef17a8c1	Merge branch 'v3.0-ff_inspect' into v3.0-unified-large-pr # Conflicts: # test/tap/tests/Makefile	2 months ago
Rene Cannao	e2ebc0c449	Merge branch 'v4.0-mcp-stats2' into v3.0-unified-large-pr	2 months ago
Rene Cannao	5d786520b0	Merge remote-tracking branch 'origin/v3.0' into merge/v3.0-into-v3.0-5243 # Conflicts: # include/PgSQL_Logger.hpp # lib/Makefile # lib/PgSQL_Logger.cpp # lib/ProxySQL_Admin.cpp # test/tap/groups/groups.json	2 months ago
René Cannaò	9c6945fae8	Merge branch 'v3.0' into v3.0-tsdb-feature Signed-off-by: René Cannaò <rene@proxysql.com>	2 months ago
René Cannaò	4a6809b60f	Merge branch 'v3.0' into v3.0-tsdb-feature Signed-off-by: René Cannaò <rene@proxysql.com>	2 months ago
Rene Cannao	20d6282a60	build(clean): ignore and remove generated MySQL parser artifacts Context: - Bison/Flex-generated parser files in `lib/` can be created locally during parser work. - These files are generated artifacts and should not be tracked. Changes: - Add generated parser artifacts to `.gitignore`: - `lib/MySQL_Lexer.yy.c` - `lib/MySQL_Parser.output` - `lib/MySQL_Parser.tab.c` - `lib/MySQL_Parser.tab.h` - Extend `lib/Makefile` `clean` target to remove the same generated files. Why: - Prevent accidental inclusion of regenerated parser outputs in feature commits. - Keep worktrees clean and make `make clean` deterministic for parser-related changes.	2 months ago
Rene Cannao	3a3be5a559	Implement Fast Forward Traffic Observer (FFTO) for MySQL and PostgreSQL - Added TrafficObserver base interface - Implemented MySQLFFTO and PgSQLFFTO classes for protocol-aware observation - Integrated FFTO hooks into MySQL and PostgreSQL session handlers in FAST_FORWARD state - Added configuration variables: mysql-ffto_enabled, mysql-ffto_max_buffer_size, pgsql-ffto_enabled, pgsql-ffto_max_buffer_size - Implemented query digest extraction and reporting parity with internal stats - Added session-level bypass when packet size exceeds buffer threshold - Included architectural documentation in doc/ffto_design.md	2 months ago
René Cannaò	568fe1524e	Merge pull request #5364 from mevishalr/feature/query-logging-improvements Optimize query logging performance (#5243)	2 months ago
René Cannaò	762afaa44f	Merge branch 'v3.0' into v4.0-mcp-stats Signed-off-by: René Cannaò <rene@proxysql.com>	2 months ago
Rene Cannao	b8c6b4f573	tsdb: address code review comments and improve stability - Refactored variable flushing to use prepared statements for safety. - Replaced brittle command parsing with robust string searching. - Extracted TSDB dashboard HTML/JS to separate source file for maintainability. - Added NULL guards for GloProxyStats in Admin and REST API. - Implemented batched metric insertions using transactions for performance. - Refactored backend probing to use non-blocking connect() with timeout. - Fixed potential integer overflows in timer calculations. - Improved error handling and fixed memory leaks in statistics reporting. - Updated TAP tests to verify UI and REST API functionality.	2 months ago
Rene Cannao	9685cdaa4b	mcp discovery: enforce target-scoped run model and add protocol-aware static harvesting (mysql+pgsql) Refactor the MCP query/discovery stack to remove prototype-era MySQL-only assumptions and make discovery/catalog semantics explicitly target-scoped. This commit is intentionally not backward compatible with legacy single-target catalog metadata. Key implementation details: - Discovery_Schema: - runs table now stores target_id, protocol, and server_version - resolve_run_id() now requires target_id and resolves schemas within target scope - create_run() now records target/protocol/server version - added legacy schema detection + destructive catalog table rebuild when old runs layout is found - fixed resultset lifetime/escaping issues in target-aware run resolution - New PostgreSQL static harvester: - added PgSQL_Static_Harvester class and build wiring - harvests schemas/objects/columns/indexes/fks/view definitions into Discovery_Schema - maps pg metadata to object_id-based insert APIs used by Discovery_Schema - Query_Tool_Handler: - constructor simplified to catalog_path only (no mcp-mysql_* runtime ctor deps) - discovery.run_static now requires target_id and dispatches protocol-aware harvester at runtime - catalog., agent.run_start, and llm. tool contracts now require target_id when resolving run_id - all run_id resolution call sites switched to resolve_run_id(target_id, ...) - list_schemas catalog query now scoped by target_id via runs join - digest logging path now resolves run_id in target context - ProxySQL_MCP_Server: - updated Query_Tool_Handler construction to new signature - Docs updated: - added explicit target_id scoping guidance for discovery/catalog/agent/llm workflows - clarified protocol-aware routing and noted legacy examples still present in large script docs Build validation: - Recompiled changed objects with PROXYSQLGENAI=1: - Query_Tool_Handler.oo - Discovery_Schema.oo - PgSQL_Static_Harvester.oo - ProxySQL_MCP_Server.oo	2 months ago
vramesha	15a4ed66e9	Optimize query logging performance (#5243 ) Detailed Changes: - Introduced LogBuffer and LogBufferThreadContext to implement per-thread buffering, reducing lock contention. - Replaced localtime() with localtime_r() for improved performance and thread safety during timestamp generation. - Implemented configurable sampling for event logs to reduce I/O overhead. - Added global configuration variables to control flush size, flush interval, and sampling rate for event and audit logs.	3 months ago
Wazir Ahmed	ae92ae265f	Merge branch 'v3.0' into v4.0-mcp-stats	3 months ago
Wazir Ahmed	643b322f29	MCP: Add stats endpoint and tools Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	3 months ago
Rene Cannao	b965fc6df4	Fix PROXYSQLGENAI build - resolve circular includes and missing headers This commit fixes compilation errors when building with PROXYSQLGENAI=1: Header file fixes: - Add missing #endif /* PROXYSQLGENAI */ before header guards in multiple headers - Remove #include cpp.h from GenAI headers to avoid circular dependencies Source file fixes: - Add #include proxysql.h to GenAI .cpp files that were missing it - Add #include Static_Harvester.h to Query_Tool_Handler.cpp for forward decl Build system fixes: - Remove vec.o from libproxysql.a (it's linked separately in src/Makefile) - Prevents duplicate symbol errors during linking Test files: - Rename MCP test files with .sh suffix to prevent make clean deletion Both build modes now work: - make build_lib_debug (without GenAI) - PROXYSQLGENAI=1 make build_lib_debug (with GenAI)	3 months ago
Rene Cannao	48bc7dd7bf	Merge v4.0 GenAI features into v3.0 with conditional compilation This commit merges the experimental v4.0 GenAI/MCP features into the stable v3.0 branch using conditional compilation. All v4.0 features are disabled by default and only enabled when PROXYSQLGENAI=1 is set at compile time. Changes: Build System: - Modified main Makefile to pass PROXYSQLGENAI flag to sub-makefiles - Modified deps/Makefile to conditionally build sqlite-vec and sqlite-rembed - Modified lib/Makefile to add PSQLGA flag and include GenAI object files - Modified src/Makefile to add PSQLGA flag and conditional linking Headers (wrapped with #ifdef PROXYSQLGENAI): - All 20 new GenAI header files in include/ - Modified cpp.h, proxysql_glovars.hpp, proxysql_admin.h - Modified ProxySQL_Admin_Tables_Definitions.h for GenAI/MCP tables Source Files: - All 22 new GenAI source files in lib/ wrapped with #ifdef PROXYSQLGENAI - Modified src/main.cpp for conditional global variables and init/shutdown - Modified Admin_Handler.cpp for conditional command handlers - Modified Admin_Bootstrap.cpp for conditional table registration - Modified Admin_FlushVariables.cpp for conditional variable flushing - Modified ProxySQL_Admin.cpp for conditional admin methods - Modified ProxySQL_Admin_Stats.cpp for conditional MCP stats functions - Modified proxy_sqlite3_symbols.cpp to always compile (needed by core) - Modified MySQL_Session.cpp for conditional GenAI function calls Test Files: - Renamed test_mcp_query_rules-t to test_mcp_query_rules-t.sh - Renamed test_mcp_rag_metrics-t to test_mcp_rag_metrics-t.sh - Modified anomaly_detection-t.cpp for conditional test execution Usage: # Build without GenAI (v3.0 mode - default) make clean && make build_deps -j$(nproc) && make build_lib -j$(nproc) && make build_src -j$(nproc) # Build with GenAI (v4.0 mode) make clean && PROXYSQLGENAI=1 make build_deps -j$(nproc) && PROXYSQLGENAI=1 make build_lib -j$(nproc) && PROXYSQLGENAI=1 make build_src -j$(nproc)	3 months ago
Wazir Ahmed	22c4e94d53	AI: Fix sqlite-vec extension loading Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	3 months ago
Rene Cannao	b4f521c634	Merge v3.1-MCP2 into v3.1-vec Conflict resolution summary: Features preserved from v3.1-vec: - mcp_use_ssl variable (HTTP/HTTPS mode support) - RAG tool handler support - FTS (Full Text Search) functionality in MySQL_Tool_Handler Changes accepted from v3.1-MCP2: - Schema isolation for catalog (removed mcp_catalog_path/fts_path variables) - Schema parameter added to catalog functions - Comprehensive query rewriting improvements in Query_Tool_Handler - Format string fixes in vector_db_performance-t.cpp Key decisions: - catalog_path/fts_path: Removed per v3.1-MCP2 commit `35b0b224` (intentional) - MySQL_Catalog.cpp: Accepted v3.1-MCP2 version (schema isolation improvements) - Query_Tool_Handler.cpp: Accepted v3.1-MCP2 version (significant improvements) - Makefile: Kept v3.1-vec's EXCLUDE_TRACKING_VARIABLES (correct spelling) Build notes: - MySQL_Tool_Handler now requires schema parameter for catalog operations - Catalog path is hardcoded to datadir/mcp_catalog.db - FTS functionality preserved but may need updates for schema isolation	3 months ago
Rene Cannao	8dc4246bdc	Introduce canonical proxy_sqlite3 symbol TU; update lib Makefile; remove MAIN_PROXY_SQLITE3 from main.cpp	3 months ago
Rahim Kanji	0d56918744	Add full-text search (FTS) tools to MCP query server Implement BM25-ranked full-text search capability for MySQL/MariaDB tables using SQLite-based external FTS index. Changes: - Add MySQL_FTS class for managing SQLite FTS indexes - Add FTS tools: fts_index_table, fts_search, fts_reindex, fts_delete_index, fts_list_indexes, fts_rebuild_all - Add thread-safe FTS lifecycle management with fts_lock mutex - Add reset_fts_path() for runtime FTS database path configuration - Add comprehensive FTS test scripts (test_mcp_fts.sh, test_mcp_fts_detailed.sh)	3 months ago
Rene Cannao	1dc5eb6581	fix: Fix RAG implementation compilation issues - Use public GenAI_Thread embed_documents() method instead of private LLM_Bridge get_text_embedding() - Fix signedness comparison warning in validate_query_length() - Fix JSON ternary operator type mismatch - Remove unused variables to eliminate warnings - Add GloGATH extern declaration	3 months ago
Rene Cannao	6f23d5bcd0	feat: Implement two-phase schema discovery architecture Phase 1 (Static/Deterministic): - Add Discovery_Schema: SQLite catalog with deterministic and LLM tables - Add Static_Harvester: MySQL INFORMATION_SCHEMA metadata extraction - Harvest schemas, objects, columns, indexes, foreign keys, view definitions - Compute derived hints: is_time, is_id_like, has_pk, has_fks, has_time - Build quick profiles and FTS5 indexes Phase 2 (LLM Agent): - Add 19 new MCP tools for two-phase discovery - discovery.run_static: Trigger ProxySQL's static harvest - Catalog tools: init, search, get_object, list_objects, get_relationships - Agent tools: run_start, run_finish, event_append - LLM tools: summary_upsert, relationship_upsert, domain_upsert, etc. Files: - include/Discovery_Schema.h, lib/Discovery_Schema.cpp - include/Static_Harvester.h, lib/Static_Harvester.cpp - include/Query_Tool_Handler.h, lib/Query_Tool_Handler.cpp (updated) - lib/Makefile (updated) - scripts/mcp/DiscoveryAgent/ClaudeCode_Headless/prompts/ - scripts/mcp/DiscoveryAgent/ClaudeCode_Headless/two_phase_discovery.py	3 months ago
Rene Cannao	a3f0bade4e	feat: Convert NL2SQL to generic LLM bridge - Rename NL2SQL_Converter to LLM_Bridge for generic prompt processing - Update MySQL protocol handler from /* NL2SQL: / to / LLM: / - Remove SQL-specific fields (sql_query, confidence, tables_used) - Add GENAI_OP_LLM operation type to GenAI module - Rename all genai_nl2sql_ variables to genai_llm_* - Update AI_Features_Manager to use LLM_Bridge - Deprecate ai_nl2sql_convert MCP tool with error message - LLM bridge now handles any prompt type via MySQL protocol This enables generic LLM access (summarization, code generation, translation, analysis) while preserving infrastructure for future NL2SQL implementation via Web UI + external agents.	3 months ago
Rene Cannao	3f44229e28	feat: Add MCP AI Tool Handler for NL2SQL with test script Phase 5: MCP Tool Implementation for NL2SQL This commit implements the AI Tool Handler for the MCP (Model Context Protocol) server, exposing NL2SQL functionality as an MCP tool. New Files: - include/AI_Tool_Handler.h: Header for AI_Tool_Handler class - Provides ai_nl2sql_convert tool via MCP protocol - Wraps NL2SQL_Converter and Anomaly_Detector - Inherits from MCP_Tool_Handler base class - lib/AI_Tool_Handler.cpp: Implementation - Implements ai_nl2sql_convert tool execution - Accepts parameters: natural_language (required), schema, context_tables, max_latency_ms, allow_cache - Returns JSON response with sql_query, confidence, explanation, cached, cache_id - scripts/mcp/test_nl2sql_tools.sh: Test script for NL2SQL MCP tool - Tests ai_nl2sql_convert via JSON-RPC over HTTPS - 10 test cases covering SELECT, WHERE, JOIN, aggregation, etc. - Includes error handling test for empty queries - Supports --verbose, --quiet options Modified Files: - include/MCP_Thread.h: Add AI_Tool_Handler forward declaration and pointer - lib/Makefile: Add AI_Tool_Handler.oo to _OBJ_CXX list - lib/ProxySQL_MCP_Server.cpp: Initialize and register AI tool handler - Creates AI_Tool_Handler with GloAI components - Registers /mcp/ai endpoint - Adds cleanup in destructor MCP Tool Details: - Endpoint: /mcp/ai - Tool: ai_nl2sql_convert - Parameters: - natural_language (string, required): Natural language query - schema (string, optional): Database schema name - context_tables (string, optional): Comma-separated table list - max_latency_ms (integer, optional): Max acceptable latency - allow_cache (boolean, optional): Check semantic cache (default: true) Testing: Run the test script with: ./scripts/mcp/test_nl2sql_tools.sh [--verbose] [--quiet] See scripts/mcp/test_nl2sql_tools.sh --help for usage. Related: Phase 1-4 (Documentation, Unit Tests, Integration Tests, E2E Tests) Related: Phase 6-8 (User Docs, Developer Docs, Test Docs)	3 months ago
Rene Cannao	147a059781	feat: Add NL2SQL converter with hybrid LLM support - Add NL2SQL_Converter with prompt building and model selection - Add LLM clients for Ollama, OpenAI, Anthropic APIs - Update Makefile for new source files	3 months ago
René Cannaò	313f637cf0	Merge branch 'v3.1-vec' into v3.1-MCP1 Signed-off-by: René Cannaò <rene.cannao@gmail.com>	4 months ago
Rene Cannao	c86a048d9c	Implement MCP multi-endpoint architecture with dedicated tool handlers This commit implements Option 1 (Multiple Tool Handlers) for the MCP module, where each of the 5 endpoints has its own dedicated tool handler with specific tools. ## Architecture Changes - Created MCP_Tool_Handler base class interface for all tool handlers - Each endpoint now has its own dedicated tool handler: - /mcp/config → Config_Tool_Handler (configuration management) - /mcp/query → Query_Tool_Handler (database exploration) - /mcp/admin → Admin_Tool_Handler (administrative operations) - /mcp/cache → Cache_Tool_Handler (cache management) - /mcp/observe → Observe_Tool_Handler (monitoring & metrics) ## New Files Base Interface: - include/MCP_Tool_Handler.h - Base class for all tool handlers Tool Handlers: - include/Config_Tool_Handler.h, lib/Config_Tool_Handler.cpp - include/Query_Tool_Handler.h, lib/Query_Tool_Handler.cpp - include/Admin_Tool_Handler.h, lib/Admin_Tool_Handler.cpp - include/Cache_Tool_Handler.h, lib/Cache_Tool_Handler.cpp - include/Observe_Tool_Handler.h, lib/Observe_Tool_Handler.cpp Documentation: - doc/MCP/Architecture.md - Comprehensive architecture documentation ## Modified Files - include/MCP_Thread.h, lib/MCP_Thread.cpp - Added 5 tool handler pointers - include/MCP_Endpoint.h, lib/MCP_Endpoint.cpp - Use tool_handler base class - lib/ProxySQL_MCP_Server.cpp - Create and pass handlers to endpoints - lib/Makefile - Added new source files ## Implementation Status - Config_Tool_Handler: Functional (get_config, set_config, list_variables, get_status) - Query_Tool_Handler: Functional (wraps MySQL_Tool_Handler, all 18 tools) - Admin_Tool_Handler: Stub implementations (TODO: implement) - Cache_Tool_Handler: Stub implementations (TODO: implement) - Observe_Tool_Handler: Stub implementations (TODO: implement) See GitHub Issue #8 for detailed TODO list. Co-authored-by: Claude <claude@anthropic.com>	4 months ago
Rene Cannao	221ff23991	Add MySQL exploration MCP tools with SQLite catalog Implemented MCP (Model Context Protocol) server providing tools for LLM-based MySQL database exploration: - MySQL_Catalog: SQLite-based catalog for LLM external memory with upsert, get, search, list, merge, delete operations and FTS support - MySQL_Tool_Handler: 17+ database exploration tools with guardrails: * Inventory: list_schemas, list_tables * Structure: describe_table, get_constraints, describe_view * Profiling: table_profile, column_profile * Sampling: sample_rows (max 20), sample_distinct (max 50) * Query: run_sql_readonly (max 200 rows, 2s timeout, SELECT-only) * Relationship: suggest_joins, find_reference_candidates * Catalog: catalog_upsert, catalog_get, catalog_search, catalog_list, catalog_merge, catalog_delete - MCP Module Integration: * Added 6 new configuration variables for MySQL tool handler (mysql_hosts, mysql_ports, mysql_user, mysql_password, mysql_schema, catalog_path) * Added MySQL_Tool_Handler pointer to MCP_Threads_Handler * Implemented tool routing in MCP endpoint for tools/list, tools/describe, and tools/call methods - TAP Tests: Updated to expect 14 MCP variables (was 8) Files: - include/MySQL_Catalog.h, lib/MySQL_Catalog.cpp - include/MySQL_Tool_Handler.h, lib/MySQL_Tool_Handler.cpp - include/MCP_Thread.h, lib/MCP_Thread.cpp - include/MCP_Endpoint.h, lib/MCP_Endpoint.cpp - lib/Makefile, test/tap/tests/mcp_module-t.cpp	4 months ago
Rene Cannao	87fff9e046	Add MCP (Model Context Protocol) module skeleton Add new MCP module supporting multiple MCP server endpoints over HTTPS with JSON-RPC 2.0 protocol skeleton. Each endpoint (/mcp/config, /mcp/observe, /mcp/query, /mcp/admin, /mcp/cache) is a distinct MCP server with its own authentication configuration. Features: - HTTPS server using existing ProxySQL TLS certificates - JSON-RPC 2.0 skeleton implementation (actual protocol TBD) - 5 MCP endpoints with per-endpoint auth configuration - LOAD/SAVE MCP VARIABLES admin commands - Configuration file support (mcp_variables section) Implementation follows GenAI module pattern: - MCP_Threads_Handler: Main module handler with variable management - ProxySQL_MCP_Server: HTTPS server wrapper using libhttpserver - MCP_JSONRPC_Resource: Base endpoint class with JSON-RPC skeleton	4 months ago
Rene Cannao	c476f56f97	Add initial GenAI module placeholder Implement a new GenAI module for ProxySQL with basic infrastructure: - GenAI_Threads_Handler class for managing GenAI module configuration - Support for genai- prefixed variables in global_variables table - Dummy variables: genai-var1 (string) and genai-var2 (integer) - Config file support via genai_variables section - Flush functions for runtime_to_database and database_to_runtime - Module lifecycle: initialization at startup, graceful shutdown - LOAD/SAVE GENAI VARIABLES admin command infrastructure Core functionality verified: - Config file loading works - Variables persist in global_variables table - Disk save/load via SQL works - Module initializes and shuts down properly Related files: - include/GenAI_Thread.h: New GenAI thread handler class - lib/GenAI_Thread.cpp: Implementation with dummy variables - lib/Admin_Handler.cpp: Added GENAI command vectors and handlers - lib/Admin_FlushVariables.cpp: Added genai flush functions - lib/ProxySQL_Admin.cpp: Added init_genai_variables() and load_save_disk_commands entry - include/proxysql_admin.h: Added function declarations - lib/Makefile: Added GenAI_Thread.oo to build - src/main.cpp: Added module initialization and cleanup - src/proxysql.cfg: Added genai_variables configuration section	4 months ago

1 2 3 4 5

204 Commits (f2db55f0c8aac088a1c40d51bc0630f4ef27eb86)