proxysql/doc/ssl_keylog/ssl_keylog_user_guide.md

# SSL/TLS Key Logging - User Guide

## What is SSL/TLS Key Logging?

SSL/TLS key logging is a debugging feature that allows ProxySQL to write TLS encryption secrets to a file. These secrets can be used by network analysis tools like **Wireshark** to decrypt and inspect TLS traffic.

### Why Would You Use This?

This feature is primarily useful for:

- **Debugging TLS connection issues** between clients and ProxySQL
- **Analyzing encrypted traffic** without modifying application code
- **Troubleshooting TLS handshake problems**
- **Performance analysis** of TLS connections
- **Security auditing** of TLS configurations

### Important Security Warning

> **WARNING:** The key log file contains cryptographic secrets that can decrypt ALL TLS traffic. Anyone with access to this file can decrypt your encrypted communications.
>
> **Only enable this feature for debugging purposes.** Disable it in production environments.

---

## Variable Names: Important Distinction

ProxySQL variables belong to **modules**. When referencing a variable from the SQL interface, you must prefix it with the module name.

### SQL Interface (Runtime)

From the ProxySQL admin interface, use the **module prefix**:

```sql
-- Correct: uses admin- prefix for admin module variables
SET admin-ssl_keylog_file = '/var/log/proxysql/sslkeys.txt';

-- Also correct
SET admin-ssl_keylog_file = 'sslkeys.txt';

-- Disable key logging
SET admin-ssl_keylog_file = '';

-- Apply to runtime
LOAD ADMIN VARIABLES TO RUNTIME;
```

### Configuration File

In the configuration file (e.g., `/etc/proxysql.cnf`), variables are grouped by module section:

```ini
# Configuration file format
admin_variables=
{
    admin_credentials="admin:admin"
    mysql_ifaces="0.0.0.0:6032"
    
    # NO prefix needed in config file - already in admin section
    ssl_keylog_file='/var/log/proxysql/sslkeys.txt'
}

mysql_variables=
{
    threads=4
    max_connections=2048
    # ... other mysql variables
}
```

**Key Points:**
- In **SQL commands**: Use `SET admin-ssl_keylog_file = '...'` (with prefix)
- In **config files**: Use `ssl_keylog_file='...'` (no prefix, inside `admin_variables` section)

---

## How to Enable SSL Key Logging

### Method 1: Using SQL Commands (Runtime)

Connect to the ProxySQL admin interface (default port 6032):

```bash
mysql -h 127.0.0.1 -P 6032 -u admin -padmin
```

Then set the variable:

```sql
-- Enable key logging with absolute path
SET admin-ssl_keylog_file = '/var/log/proxysql/sslkeys.txt';

-- Apply to runtime immediately
LOAD ADMIN VARIABLES TO RUNTIME;

-- Verify it's set
SELECT * FROM global_variables WHERE variable_name = 'admin-ssl_keylog_file';
```

### Method 2: Using Configuration File

Edit your ProxySQL configuration file (typically `/etc/proxysql.cnf`):

```ini
admin_variables=
{
    admin_credentials="admin:admin"
    mysql_ifaces="0.0.0.0:6032"
    
    # Add this line
    ssl_keylog_file='/var/log/proxysql/sslkeys.txt'
}
```

Then restart ProxySQL:

```bash
sudo systemctl restart proxysql
# or
sudo service proxysql restart
```

---

## Path Resolution

The `ssl_keylog_file` variable accepts two types of paths:

| Path Type | Format | Example | Resolved To |
|-----------|--------|---------|-------------|
| **Absolute** | Starts with `/` | `/var/log/proxysql/keys.txt` | `/var/log/proxysql/keys.txt` |
| **Relative** | No leading `/` | `sslkeys.txt` | `$DATADIR/sslkeys.txt` |

**Example:**

```sql
-- If ProxySQL data directory is /var/lib/proxysql
SET admin-ssl_keylog_file = 'debug/sslkeys.txt';
-- Resolves to: /var/lib/proxysql/debug/sslkeys.txt
```

---

## Verifying Key Logging

After enabling key logging and generating TLS traffic, verify the key log file:

```bash
# Check if file exists
ls -la /var/log/proxysql/sslkeys.txt

# View contents (should contain secrets!)
cat /var/log/proxysql/sslkeys.txt
```

The file should contain lines like:

```
CLIENT_RANDOM 3a4b5c6d7e8f0123456789abcdef... 48_byte_secret_here...
```

---

## Disabling SSL Key Logging

### Using SQL Commands

```sql
-- Set to empty string to disable
SET admin-ssl_keylog_file = '';

-- Apply to runtime
LOAD ADMIN VARIABLES TO RUNTIME;
```

### Using Configuration File

Remove or comment out the `ssl_keylog_file` line in your config file and restart ProxySQL.

---

## Log Rotation

ProxySQL supports rotating the SSL key log file using the `PROXYSQL FLUSH LOGS` command:

```sql
PROXYSQL FLUSH LOGS;
```

This command:
1. Closes the current key log file
2. Reopens the file for appending

**Note:** The file is reopened in append mode, so existing contents will be preserved. If you want to start with a fresh file, rename/move the old file manually before running `FLUSH LOGS`.

### Manual Log Rotation Example

```bash
# 1. Rename the current key log file
mv /var/log/proxysql/sslkeys.txt /var/log/proxysql/sslkeys.txt.old

# 2. Tell ProxySQL to create a new file
mysql -h 127.0.0.1 -P 6032 -u admin -padmin -e "PROXYSQL FLUSH LOGS;"

# 3. Secure the old file
chmod 600 /var/log/proxysql/sslkeys.txt.old
```

---

## Analyzing TLS Traffic with Key Logs

In production environments, you typically don't run Wireshark directly on the server. Instead, you:

1. Capture traffic to a pcap file using `tcpdump`
2. Copy both the pcap file and key log file to an analysis system
3. Analyze offline using Wireshark (GUI) or tshark (command-line)

### Production Capture Workflow

#### Step 1: Capture Traffic with tcpdump

On the ProxySQL server, capture network traffic to a pcap file:

```bash
# Capture on the interface ProxySQL is listening on (e.g., eth0)
# Replace 6033 with your ProxySQL MySQL port
sudo tcpdump -i eth0 -w /tmp/proxysql_debug.pcap port 6033

# Or capture traffic between specific hosts
sudo tcpdump -i eth0 -w /tmp/proxysql_debug.pcap host client_ip and host proxysql_ip

# Run for a specific duration
sudo timeout 60 tcpdump -i eth0 -w /tmp/proxysql_debug.pcap port 6033
```

**Notes:**
- Use `-i any` to capture on all interfaces if unsure
- The `-w` flag writes to pcap format (binary)
- Capture size is limited by disk space - monitor with `df -h`

#### Step 2: Collect Files for Analysis

Copy both the pcap file and the key log file to your analysis system:

```bash
# On the ProxySQL server
scp /tmp/proxysql_debug.pcap user@analysis-system:/path/to/analysis/
scp /var/log/proxysql/sslkeys.txt user@analysis-system:/path/to/analysis/

# Or archive them together
tar czf proxysql_debug.tar.gz /tmp/proxysql_debug.pcap /var/log/proxysql/sslkeys.txt
```

**Security:** Use secure copy (scp/sftp) and ensure the key log file is transmitted securely, as it contains cryptographic secrets.

#### Step 3: Analyze with Wireshark (GUI)

On your analysis system with Wireshark installed:

1. **Configure TLS key log:**
   - Open Wireshark
   - Go to **Edit → Preferences → Protocols → TLS** (or **SSL** in older versions)
   - Set **"(Pre)-Master-Secret log filename"** to the key log file path

2. **Open the pcap file:**
   - File → Open → Select `proxysql_debug.pcap`
   - Wireshark will decrypt TLS traffic using the key log file

3. **Filter decrypted traffic:**
   ```
   # Show only MySQL packets
   mysql
   
   # Show TLS handshake
   tls.handshake.type == 1
   
   # Show decrypted application data
   tls.app_data
   ```

4. **View decrypted content:**
   - Right-click on a TLS packet → **Follow → TCP Stream**
   - Or right-click → **Follow → TLS Stream** (Wireshark 3.0+)

#### Step 4: Analyze with tshark (Command-Line)

**tshark** is Wireshark's command-line counterpart - useful for servers or headless analysis.

```bash
# Read pcap with TLS decryption using key log file
tshark -r /tmp/proxysql_debug.pcap \
  -o tls.keylog_file:/path/to/sslkeys.txt \
  -Y "tls" \
  -V

# Show only MySQL packets
tshark -r /tmp/proxysql_debug.pcap \
  -o tls.keylog_file:/path/to/sslkeys.txt \
  -Y "mysql"

# Export decrypted TLS payloads to JSON
tshark -r /tmp/proxysql_debug.pcap \
  -o tls.keylog_file:/path/to/sslkeys.txt \
  -T json \
  -Y "tls.app_data" \
  > decrypted.json

# Show summary of decrypted connections
tshark -r /tmp/proxysql_debug.pcap \
  -o tls.keylog_file:/path/to/sslkeys.txt \
  -q -z tls,tree
```

**Common tshark filters for ProxySQL debugging:**

```bash
# Show TLS handshake details
tshark -r proxysql_debug.pcap -o tls.keylog_file:sslkeys.txt -Y "tls.handshake"

# Show all TLS app data (decrypted MySQL queries/responses)
tshark -r proxysql_debug.pcap -o tls.keylog_file:sslkeys.txt -Y "tls.app_data" -V

# Convert to readable text format
tshark -r proxysql_debug.pcap -o tls.keylog_file:sslkeys.txt -T fields \
  -e frame.time -e ip.src -e ip.dst -e tcp.srcport -e tcp.dstport \
  -e tls.app_data.data

# Statistics: TLS sessions by cipher suite
tshark -r proxysql_debug.pcap -o tls.keylog_file:sslkeys.txt -q -z tls,ctext
```

### Alternative: Live Capture with tshark

If you need to monitor traffic in real-time (not recommended for production debugging):

```bash
# Live capture with TLS decryption
sudo tshark -i eth0 -f "port 6033" \
  -o tls.keylog_file:/var/log/proxysql/sslkeys.txt \
  -Y "tls.app_data" \
  -V
```

**Note:** This still requires running on the ProxySQL server. For production, prefer the tcpdump → offline analysis workflow.

---

## Configuration File Reference

### Sample Configuration with Key Logging

```ini
# /etc/proxysql.cnf

datadir="/var/lib/proxysql"

admin_variables=
{
    admin_credentials="admin:admin"
    mysql_ifaces="0.0.0.0:6032"
    
    # Enable SSL key logging for debugging
    ssl_keylog_file='/var/log/proxysql/sslkeys.txt'
}

mysql_variables=
{
    threads=4
    max_connections=2048
    interfaces="0.0.0.0:6033"
    default_schema="information_schema"
    # ... other mysql variables
}
```

---

## Troubleshooting

### Variable Not Found Error

**Problem:** `ERROR 1045 (28000): Unknown variable 'admin-ssl_keylog_file'`

**Solution:** 
- Make sure you're connected to the admin interface (port 6032, not 6033)
- Check that you're using the correct prefix: `admin-ssl_keylog_file`

### File Not Created

**Problem:** The key log file is not being created.

**Solutions:**
1. Check that the directory exists and is writable:
   ```bash
   ls -la /var/log/proxysql
   ```
2. Check ProxySQL error logs for permission errors
3. Verify the variable is set:
   ```sql
   SELECT * FROM global_variables WHERE variable_name = 'admin-ssl_keylog_file';
   ```

### No Secrets in File

**Problem:** File exists but is empty or has no secrets.

**Solutions:**
1. Verify TLS is actually being used:
   ```sql
   -- Check if connections are using TLS
   SELECT * FROM stats_mysql_connection_pool;
   ```
2. Make sure clients are connecting with SSL/TLS
3. Check that `admin-ssl_keylog_file` is loaded into runtime:
   ```sql
   LOAD ADMIN VARIABLES TO RUNTIME;
   ```

### tcpdump Permission Denied

**Problem:** `tcpdump: snaplen: ioctl: Permission denied`

**Solution:** Run tcpdump with sudo:
```bash
sudo tcpdump -i eth0 -w /tmp/capture.pcap port 6033
```

---

## Best Practices

### Security

1. **Never enable in production** unless actively debugging
2. **Set restrictive file permissions:**
   ```bash
   chmod 600 /var/log/proxysql/sslkeys.txt
   chown proxysql:proxysql /var/log/proxysql/sslkeys.txt
   ```
3. **Securely delete old key log files:**
   ```bash
   shred -u /var/log/proxysql/sslkeys.txt.old
   ```
4. **Monitor file size** - key log files can grow quickly

### Operational

1. **Use absolute paths** to avoid confusion
2. **Document when key logging is enabled** for audit purposes
3. **Rotate regularly** during long debugging sessions
4. **Disable immediately after** debugging is complete
5. **Use tcpdump for production captures** - don't run Wireshark on production servers

---

## Quick Reference

| Context | Variable Name | Example |
|---------|---------------|---------|
| **SQL commands** | `admin-ssl_keylog_file` | `SET admin-ssl_keylog_file = '/path/file.txt';` |
| **Config file** | `ssl_keylog_file` | `ssl_keylog_file='/path/file.txt'` (in `admin_variables` section) |

| Command | Description |
|---------|-------------|
| `SET admin-ssl_keylog_file = '/path/to/file.txt';` | Enable key logging |
| `SET admin-ssl_keylog_file = '';` | Disable key logging |
| `LOAD ADMIN VARIABLES TO RUNTIME;` | Apply changes |
| `PROXYSQL FLUSH LOGS;` | Rotate key log file |
| `SELECT * FROM global_variables WHERE variable_name = 'admin-ssl_keylog_file';` | Check current setting |

| Tool | Use Case |
|------|----------|
| `tcpdump` | Capture traffic to pcap file (production) |
| `tshark` | Analyze pcap files with key log (command-line) |
| `Wireshark` | Analyze pcap files with key log (GUI) |

---

## Additional Resources

- **Developer Documentation:** See `ssl_keylog_developer_guide.md` for implementation details
- **NSS Key Log Format:** https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format
- **Wireshark TLS Decryption:** https://wiki.wireshark.org/TLS
- **tshark Manual:** `man tshark` or https://www.wireshark.org/docs/man-pages/tshark.html
- **ProxySQL Configuration:** https://github.com/sysown/proxysql/wiki