mirror of https://github.com/sysown/proxysql
Rene Cannao
4 months ago
parent
faa64a570d
commit
365b0c6d80
@ -0,0 +1,144 @@
|
||||
# ProxySQL 3.0.4 Release Notes
|
||||
|
||||
This release of ProxySQL 3.0.4 includes significant improvements to PostgreSQL support, MySQL protocol handling, monitoring accuracy, and security.
|
||||
|
||||
Release commit: `faa64a570d19fe35af43494db0babdee3e3cdc89`
|
||||
|
||||
## New Features:
|
||||
|
||||
### PostgreSQL Improvements:
|
||||
|
||||
- **PostgreSQL-Specific Tokenizer for Query Digest Generation** (285fb1b4, #5254)
|
||||
- Adds PostgreSQL dialect support: dollar-quoted strings (`$$...$$`), identifier quoting, and dialect-specific comment rules
|
||||
- Improved tokenizer for PostgreSQL with nested comments support (f5079037)
|
||||
- Adds dedicated handling for double-quoted PostgreSQL identifiers and type cast syntax (`value::type`) (e70fcbf0)
|
||||
- Processes array literals (`ARRAY[...]` and `{...}`) and prefixed literals (`E''`, `U&''`, `x''`, `b''`)
|
||||
- **SSL Support for Backend Connections in PostgreSQL Monitor** (7205f424, #5237)
|
||||
- PostgreSQL monitor now supports client certificates for SSL/TLS connections to backend servers
|
||||
- Adds new metrics `PgSQL_Monitor_ssl_connections_OK` and `PgSQL_Monitor_non_ssl_connections_OK` for connection status visibility (fae283cf)
|
||||
|
||||
### MySQL Protocol Enhancements:
|
||||
|
||||
- **Special Handling for Unexpected `COM_PING` Packets** (d0e88599, #5257)
|
||||
- Implements workaround for unexpected `COM_PING` packets received during query processing
|
||||
- Queues `COM_PING` packets as counters and sends corresponding OK packets after query completion
|
||||
- Improves logging with client address, session status, and data stream status (6fea828e)
|
||||
- **Handling of `SELECT @@version` and `SELECT VERSION()` Without Backend** (df287b20, #4889)
|
||||
- ProxySQL now responds directly to `SELECT @@version` and `SELECT VERSION()` queries without backend connection
|
||||
- Returns hardcoded server version (currently 8.0.0), reducing unnecessary backend load
|
||||
- **Client-Side `wait_timeout` Support** (abe16e66, #4901)
|
||||
- Adds support for client-specified `wait_timeout` values, previously ignored by ProxySQL
|
||||
- Terminates client connections (not backend) when client timeout is reached
|
||||
- Clamps client timeout to not exceed global `mysql-wait_timeout`
|
||||
- **Fast Forward Grace Close Feature to Prevent Data Loss** (44aa606c, #5203)
|
||||
- Implements graceful connection closure for fast-forward sessions
|
||||
- Ensures all pending data is flushed to client before closing connection
|
||||
- Prevents data loss during connection termination in replication scenarios
|
||||
|
||||
### Monitoring & Diagnostics:
|
||||
|
||||
- **TCP Keepalive Warnings When Disabled** (cf454b8e, #5228)
|
||||
- Adds warning messages when TCP keepalive is disabled on connections
|
||||
- Helps administrators identify potential connection stability issues
|
||||
- Includes detailed connection information for troubleshooting
|
||||
|
||||
|
||||
## Bug Fixes:
|
||||
|
||||
### MySQL:
|
||||
|
||||
- **Make `cur_cmd_cmnt` Thread-Safe** (91e20648, #5259)
|
||||
- Fixes thread-safety issue where `cur_cmd_cmnt` (current command comment) was shared across threads
|
||||
- **Fix `cache_empty_result=0` Not Caching Non-Empty Resultsets** (2987242d, #5250)
|
||||
- Corrects behavior of `cache_empty_result` field in query rules
|
||||
- When set to "0", now correctly caches non-empty resultsets while skipping empty resultsets
|
||||
- Previously prevented caching of any resultsets regardless of row count
|
||||
- **Incorrect Affected Rows Reporting for DDL Queries** (05960b5d, #5232)
|
||||
- Fixes issue #4855 where `mysql_affected_rows()` in ProxySQL Admin interface incorrectly reported affected rows for DDL queries
|
||||
- DDL queries now correctly return 0 affected rows instead of previous DML's count
|
||||
|
||||
### Monitoring:
|
||||
|
||||
- **Fix Artificially High Ping Latency in MySQL Backend Monitoring** (24e02e95, #5199)
|
||||
- Addresses artificially high ping latency measurements
|
||||
- Introduces batching for task dispatching (batches of 30) with `poll()` calls between batches
|
||||
- Reduces monitor ping `poll()` timeout from 100ms to 10ms for more responsive monitoring
|
||||
|
||||
### Security & Configuration:
|
||||
|
||||
- **Fix SQL Injection Vulnerability in `Read_Global_Variables_from_configfile()`** (0b2bc1bf, #5247)
|
||||
- Replaces `sprintf`-based SQL query construction with prepared statements using bound parameters
|
||||
- Fixes automatic prefix stripping for `mysql_variables`, `pgsql_variables`, and `admin_variables` config parsing (b4683569)
|
||||
- Handles cases where users mistakenly include module prefixes (e.g., "mysql-") in variable names
|
||||
|
||||
|
||||
## Improvements:
|
||||
|
||||
### Performance Optimizations:
|
||||
|
||||
- **Improve Fast Forward Replication `CLIENT_DEPRECATE_EOF` Validation** (5485bb02, #5240)
|
||||
- Enhances validation logic for fast forward replication with `CLIENT_DEPRECATE_EOF` capability flag
|
||||
- Ensures proper handling of replication packets and improves compatibility with MySQL 8.0+ clients
|
||||
- **Refactored Prepared-Statement Cache Design (Lock-Free Hot Path)** (c0f99c0e, #5225)
|
||||
- Implements lock-free hot path for prepared statement cache operations
|
||||
- Reduces contention and improves performance for high-concurrency workloads
|
||||
- Optimizes transaction command parsing to avoid unnecessary tokenization (e744c2bb)
|
||||
- Replaces `std::string` with `char[]` to avoid heap allocation in hot paths (7a3a5c71)
|
||||
- **GTID: Refactor Reconnect Logic & Prevent `events_count` Reset** (50c60284, #5226)
|
||||
- Refactors GTID reconnect logic to prevent `events_count` from being reset during reconnection attempts
|
||||
- Preserves replication state and reduces unnecessary replication restarts
|
||||
|
||||
|
||||
## Documentation:
|
||||
|
||||
- **Comprehensive Documentation Additions** (cf8cbfd8, #5258)
|
||||
- Query rules capture groups and backreferences documentation (6966b79d)
|
||||
- "Closing killed client connection" warnings documentation (de214cb5)
|
||||
- `coredump_filters` feature documentation addressing issue #5213
|
||||
- `vacuum_stats()` and `stats_pgsql_stat_activity` Doxygen documentation with bug fix (efe0d4fe)
|
||||
- **Permanent Fast-Forward Sessions and `check_data_flow()` Documentation** (ec1247f2, #5245)
|
||||
- Documents behavior of permanent fast-forward sessions
|
||||
- Documents `MySQL_Data_Stream::check_data_flow()` method
|
||||
- Explains when bidirectional data checks are skipped for specific session types (4044a407)
|
||||
- **Claude Code Agent Definitions and Architecture Documentation** (291e5f02, #5115)
|
||||
- Adds architecture documentation for Claude Code agent integration
|
||||
- Includes automation framework and internal documentation
|
||||
- **Comprehensive Doxygen Documentation for GTID Refactoring** (9a55e974, #5229)
|
||||
- Adds detailed Doxygen documentation for GTID-related code changes
|
||||
- **TAP Test Writing Guide and GitHub Automation Improvements** (49899601, #5215)
|
||||
- Provides comprehensive guide for writing TAP tests for ProxySQL
|
||||
- Improves GitHub automation workflows
|
||||
|
||||
|
||||
## Testing:
|
||||
|
||||
- **Add Comments to `SELECT @@version` Queries to Bypass ProxySQL Interception** (66119b74, #5251)
|
||||
- Adds `/* set_testing */` comments to `SELECT @@version` queries in test files
|
||||
- Prevents ProxySQL interception, allowing queries to reach backend MySQL servers for testing
|
||||
- **Add Fast Forward Replication Deprecate EOF Test and Update Test Infrastructure** (9df7407f, #5241)
|
||||
- Adds comprehensive test coverage for fast forward replication `CLIENT_DEPRECATE_EOF` validation
|
||||
- Updates test infrastructure to better handle replication scenarios
|
||||
- **Add Delay to Let ProxySQL Process `mysql_stmt_close()`** (fa74de3c, #5198)
|
||||
- Adds appropriate delays in tests to allow ProxySQL time to process `mysql_stmt_close()` operations
|
||||
- Prevents race conditions in prepared statement tests
|
||||
- **Regroup Tests to Balance Group Test Time** (458ff778, #5207)
|
||||
- Reorganizes test groups to balance execution time across test runners
|
||||
- Improves CI pipeline efficiency and reduces overall test runtime
|
||||
|
||||
|
||||
## Build/Packaging:
|
||||
|
||||
- **Add OpenSUSE 16 Packaging Support** (bce71a95, #5230)
|
||||
- Adds packaging support for OpenSUSE 16
|
||||
- Expands supported distribution matrix for ProxySQL installations
|
||||
|
||||
|
||||
## Other Changes:
|
||||
|
||||
- **Bump Version to 3.0.4 at Beginning of Development Cycle** (ba664785, #5200)
|
||||
- Updates version number to 3.0.4 at start of development cycle for clarity and version tracking
|
||||
|
||||
|
||||
## Hashes
|
||||
|
||||
The release commit is: `faa64a570d19fe35af43494db0babdee3e3cdc89`
|
||||
@ -0,0 +1,90 @@
|
||||
# ProxySQL Release Notes Workflow
|
||||
|
||||
This document describes the complete workflow for generating release notes like ProxySQL v3.0.3.
|
||||
|
||||
## Quick Start (For ProxySQL 3.0.4)
|
||||
|
||||
```bash
|
||||
# 1. Run the orchestration script
|
||||
python scripts/release-tools/orchestrate_release.py \
|
||||
--from-tag v3.0.3 \
|
||||
--to-tag v3.0 \
|
||||
--output-dir release-data \
|
||||
--verbose
|
||||
|
||||
# 2. Provide the generated files to LLM with this prompt:
|
||||
cat release-data/llm-prompt-v3.0.md
|
||||
```
|
||||
|
||||
## Complete Procedure
|
||||
|
||||
### Step 1: Prepare Environment
|
||||
```bash
|
||||
git fetch --tags
|
||||
git checkout v3.0 # Ensure you're on the release branch
|
||||
```
|
||||
|
||||
### Step 2: Run Orchestration Script
|
||||
```bash
|
||||
python scripts/release-tools/orchestrate_release.py \
|
||||
--from-tag PREVIOUS_TAG \
|
||||
--to-tag CURRENT_BRANCH_OR_NEW_TAG \
|
||||
--output-dir release-data
|
||||
```
|
||||
|
||||
This generates:
|
||||
- `release-data/pr-data-*.json` - PR details from GitHub
|
||||
- `release-data/pr-summary-*.md` - PR summary
|
||||
- `release-data/structured-notes-*.md` - Commit-level data
|
||||
- `release-data/llm-prompt-*.md` - Complete LLM prompt
|
||||
- `release-data/workflow-summary.md` - Instructions
|
||||
|
||||
### Step 3: Provide Files to LLM
|
||||
|
||||
Give the LLM:
|
||||
1. All files in `release-data/` directory
|
||||
2. The prompt: `release-data/llm-prompt-*.md`
|
||||
|
||||
### Step 4: LLM Generates Release Notes
|
||||
|
||||
The LLM should create:
|
||||
- `ProxySQL-X.X.X-Release-Notes.md` - Main release notes
|
||||
- `CHANGELOG-X.X.X-detailed.md` - Detailed changelog
|
||||
- `CHANGELOG-X.X.X-commits.md` - Complete commit list (optional)
|
||||
|
||||
## Key Requirements for Release Notes
|
||||
|
||||
1. **Descriptive Content**: Explain what each feature/fix does and why it matters
|
||||
2. **Logical Grouping**: Organize under categories (PostgreSQL, MySQL, Monitoring, etc.)
|
||||
3. **Backtick Formatting**: Use `backticks` around all technical terms
|
||||
4. **Commit References**: Include commit hashes and PR numbers
|
||||
5. **No WIP/skip-ci Tags**: Make all entries production-ready
|
||||
6. **Follow v3.0.3 Format**: Structure like previous release notes
|
||||
|
||||
## Example Output
|
||||
|
||||
See `ProxySQL-3.0.4-Release-Notes.md` in the root directory for a complete example of descriptive release notes with backtick formatting.
|
||||
|
||||
## Tools Directory
|
||||
|
||||
All scripts are in `scripts/release-tools/`:
|
||||
|
||||
| Script | Purpose |
|
||||
|--------|---------|
|
||||
| `orchestrate_release.py` | Main orchestration script |
|
||||
| `collect_pr_data.py` | Fetch PR details from GitHub |
|
||||
| `generate_structured_notes.py` | Create commit-level data |
|
||||
| `categorize_commits.py` | Categorize commits by type |
|
||||
| `generate_release_notes.py` | Basic release notes (without LLM) |
|
||||
| `generate_changelog.py` | Basic changelog generation |
|
||||
|
||||
See `scripts/release-tools/README.md` for detailed documentation.
|
||||
|
||||
## For ProxySQL 3.0.4
|
||||
|
||||
The release notes for 3.0.4 have already been generated:
|
||||
- `ProxySQL-3.0.4-Release-Notes.md` - Final release notes
|
||||
- Backup: `ProxySQL-3.0.4-Release-Notes-backup.md` - Original version
|
||||
- Example: `ProxySQL-3.0.4-Release-Notes-Descriptive.md` - Descriptive version
|
||||
|
||||
These notes follow all requirements: descriptive content, logical grouping, backtick formatting, and proper references.
|
||||
@ -0,0 +1,207 @@
|
||||
# ProxySQL Release Tools
|
||||
|
||||
This directory contains Python scripts to help generate release notes and changelogs for ProxySQL releases.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.6+
|
||||
- Git command line tools
|
||||
- GitHub CLI (`gh`) installed and authenticated (for scripts that fetch PR details)
|
||||
|
||||
## Scripts Overview
|
||||
|
||||
### 1. `categorize_commits.py`
|
||||
|
||||
Categorizes git commits based on keywords in commit messages.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
python categorize_commits.py --from-tag v3.0.3 --to-tag v3.0
|
||||
python categorize_commits.py --input-file /tmp/commits.txt --output-format text
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--from-tag`, `--to-tag`: Git tags/branches to compare
|
||||
- `--input-file`: Read commits from a file (format: git log --pretty=format:'%H|%s|%b')
|
||||
- `--output-format`: `markdown` (default) or `text`
|
||||
- `--verbose`: Show detailed output
|
||||
|
||||
### 2. `generate_changelog.py`
|
||||
|
||||
Generates a changelog from git merge commits (pull requests). Uses second parent of merge commits to get PR-specific commits.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
python generate_changelog.py --from-tag v3.0.3 --to-tag v3.0 --output changelog.md
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--from-tag`, `--to-tag`: Git tags/branches to compare
|
||||
- `--output`, `-o`: Output file (default: changelog.md)
|
||||
- `--verbose`: Show progress
|
||||
|
||||
### 3. `generate_release_notes.py**
|
||||
|
||||
Generates formatted release notes using GitHub API (via `gh` CLI). Provides automatic categorization based on PR labels and titles, with optional manual mapping.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
python generate_release_notes.py --from-tag v3.0.3 --to-tag v3.0 --output release-notes.md
|
||||
python generate_release_notes.py --from-tag v3.0.3 --to-tag v3.0 --config category_mapping.json --verbose
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--from-tag`, `--to-tag`: Git tags/branches to compare
|
||||
- `--output`, `-o`: Output file for release notes (default: release-notes.md)
|
||||
- `--changelog`, `-c`: Output file for detailed changelog
|
||||
- `--config`: JSON file with manual category mapping (see example)
|
||||
- `--verbose`: Show detailed progress
|
||||
|
||||
### 4. `fetch_prs.py` (legacy)
|
||||
|
||||
Legacy script that was used for ProxySQL 3.0.4. Consider using `generate_release_notes.py` instead.
|
||||
|
||||
### 5. `gen_release_notes.py` (legacy)
|
||||
|
||||
Legacy script with hardcoded mapping for 3.0.4.
|
||||
|
||||
## Category Mapping
|
||||
|
||||
For `generate_release_notes.py`, you can provide a JSON file with manual categorization overrides. The format can be:
|
||||
|
||||
```json
|
||||
{
|
||||
"5259": ["Bug Fixes", "MySQL"],
|
||||
"5257": ["New Features", "MySQL"],
|
||||
"5258": "Documentation"
|
||||
}
|
||||
```
|
||||
|
||||
Each key is a PR number. The value can be:
|
||||
- A string: category name (e.g., `"Documentation"`)
|
||||
- An array: `[category, subcategory]` (e.g., `["New Features", "PostgreSQL"]`)
|
||||
|
||||
See `category_mapping.example.json` for a complete example.
|
||||
|
||||
## Complete Workflow with LLM Integration
|
||||
|
||||
For high-quality release notes like ProxySQL v3.0.3, use the orchestrated workflow:
|
||||
|
||||
### Option 1: Automated Orchestration (Recommended)
|
||||
|
||||
```bash
|
||||
# Run the complete workflow
|
||||
python orchestrate_release.py --from-tag v3.0.3 --to-tag v3.0.4 --output-dir release-data --verbose
|
||||
|
||||
# This generates:
|
||||
# - release-data/pr-data-3.0.4.json # PR details from GitHub
|
||||
# - release-data/pr-summary-3.0.4.md # PR summary
|
||||
# - release-data/structured-notes-3.0.4.md # Commit-level data
|
||||
# - release-data/llm-prompt-3.0.4.md # Complete prompt for LLM
|
||||
# - release-data/workflow-summary.md # Instructions summary
|
||||
```
|
||||
|
||||
### Option 2: Manual Steps
|
||||
|
||||
#### Step 1: Prepare the environment
|
||||
|
||||
Ensure you're on the correct branch and have all tags fetched:
|
||||
```bash
|
||||
git fetch --tags
|
||||
git checkout v3.0 # or your release branch
|
||||
```
|
||||
|
||||
#### Step 2: Collect PR data for LLM analysis
|
||||
|
||||
```bash
|
||||
python collect_pr_data.py --from-tag v3.0.3 --to-tag v3.0.4 --output pr-data.json --verbose
|
||||
```
|
||||
|
||||
#### Step 3: Generate structured analysis files
|
||||
|
||||
```bash
|
||||
python generate_structured_notes.py --input pr-data.json --output structured-notes.md --verbose
|
||||
python categorize_commits.py --from-tag v3.0.3 --to-tag v3.0.4 --output-format markdown > commit-categories.md
|
||||
```
|
||||
|
||||
#### Step 4: Provide data to LLM with this prompt:
|
||||
|
||||
```markdown
|
||||
Generate ProxySQL 3.0.4 release notes and changelogs using the provided data files.
|
||||
|
||||
Available files:
|
||||
1. pr-data.json - All PR details from GitHub
|
||||
2. structured-notes.md - Commit-level organized data
|
||||
3. commit-categories.md - Commits categorized by type
|
||||
|
||||
Requirements:
|
||||
1. Write DESCRIPTIVE release notes (not just PR titles)
|
||||
2. Group changes logically (PostgreSQL, MySQL, Monitoring, Bug Fixes, etc.)
|
||||
3. Use `backticks` around all technical terms
|
||||
4. Include commit hashes and PR numbers
|
||||
5. Remove any WIP/skip-ci tags
|
||||
6. Follow the format of ProxySQL v3.0.3 release notes
|
||||
|
||||
Output files:
|
||||
- ProxySQL-3.0.4-Release-Notes.md
|
||||
- CHANGELOG-3.0.4-detailed.md
|
||||
- CHANGELOG-3.0.4-commits.md (optional)
|
||||
```
|
||||
|
||||
### Option 3: Quick Generation (Without LLM)
|
||||
|
||||
For basic changelogs without descriptive analysis:
|
||||
```bash
|
||||
python generate_release_notes.py --from-tag v3.0.3 --to-tag v3.0.4 --output release-notes.md --changelog detailed-changelog.md --verbose
|
||||
python generate_changelog.py --from-tag v3.0.3 --to-tag v3.0.4 --output changelog.md
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
See the `examples/` directory for output generated for ProxySQL 3.0.4:
|
||||
- `ProxySQL-3.0.4-Release-Notes.md`: Final release notes
|
||||
- `CHANGELOG-3.0.4-detailed.md`: Detailed changelog with PR summaries
|
||||
- `CHANGELOG-3.0.4-commits.md`: Complete list of commits
|
||||
|
||||
## Generating Descriptive Release Notes
|
||||
|
||||
ProxySQL release notes (see v3.0.3 example) are descriptive, not just collections of PR titles. They:
|
||||
1. Group related changes under feature categories
|
||||
2. Describe what each feature/fix does and why it matters
|
||||
3. Reference PR numbers and commit hashes
|
||||
4. Use backticks around technical terms
|
||||
|
||||
The scripts in this directory help collect data, but the LLM should:
|
||||
1. Analyze PR descriptions and commit messages
|
||||
2. Write descriptive paragraphs explaining changes
|
||||
3. Group related changes logically
|
||||
4. Apply backtick formatting to technical terms
|
||||
|
||||
### Backtick Formatting for Technical Terms
|
||||
|
||||
ProxySQL release notes use backticks (`) around technical terms such as:
|
||||
- Function names: `Read_Global_Variables_from_configfile()`
|
||||
- Variable names: `wait_timeout`, `cur_cmd_cmnt`
|
||||
- SQL queries: `SELECT @@version`, `SELECT VERSION()`
|
||||
- Protocol commands: `COM_PING`, `CLIENT_DEPRECATE_EOF`
|
||||
- Configuration options: `cache_empty_result=0`
|
||||
- Metrics: `PgSQL_Monitor_ssl_connections_OK`
|
||||
|
||||
The scripts do not automatically apply backtick formatting. When generating final release notes with the LLM, ensure you:
|
||||
1. Manually add backticks around technical terms
|
||||
2. Use the LLM's understanding of the codebase to identify what needs formatting
|
||||
3. Review the final output for consistency
|
||||
|
||||
## Tips
|
||||
|
||||
1. **GitHub CLI Authentication**: Ensure `gh auth login` has been run and you have permissions to access the repository.
|
||||
|
||||
2. **Tag Names**: Use exact tag names (e.g., `v3.0.3`) or branch names (e.g., `HEAD` for current branch).
|
||||
|
||||
3. **Manual Review**: Always review the generated notes. Automatic categorization is not perfect.
|
||||
|
||||
4. **Customization**: Feel free to modify the categorization keywords in the scripts to match your project's conventions.
|
||||
|
||||
## License
|
||||
|
||||
These tools are part of the ProxySQL project and follow the same licensing terms.
|
||||
@ -0,0 +1,162 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Categorize git commits based on keywords.
|
||||
|
||||
This script reads git commit messages from a git log range or from a file
|
||||
and categorizes them based on keyword matching.
|
||||
|
||||
Usage:
|
||||
python categorize_commits.py --from-tag v3.0.3 --to-tag v3.0
|
||||
python categorize_commits.py --input-file /tmp/commits.txt
|
||||
"""
|
||||
|
||||
import sys
|
||||
import re
|
||||
import subprocess
|
||||
import argparse
|
||||
|
||||
# Categories mapping keywords
|
||||
CATEGORIES = {
|
||||
'Bug Fix': ['fix', 'bug', 'issue', 'crash', 'vulnerability', 'error', 'wrong', 'incorrect', 'failure', 'broken'],
|
||||
'New Feature': ['add', 'new', 'support', 'implement', 'feature', 'introduce', 'enable'],
|
||||
'Improvement': ['improve', 'optimize', 'enhance', 'speed', 'performance', 'better', 'reduce', 'faster', 'efficient'],
|
||||
'Documentation': ['doc', 'documentation', 'comment', 'doxygen', 'readme'],
|
||||
'Testing': ['test', 'tap', 'regression', 'validation'],
|
||||
'Build/Packaging': ['build', 'package', 'makefile', 'cmake', 'docker', 'opensuse', 'deb', 'rpm'],
|
||||
'Refactoring': ['refactor', 'cleanup', 'restructure', 'reorganize', 'rename'],
|
||||
'Security': ['security', 'injection', 'vulnerability', 'secure', 'sanitize'],
|
||||
'Monitoring': ['monitor', 'metric', 'log', 'warning', 'alert'],
|
||||
'PostgreSQL': ['postgresql', 'pgsql', 'pg'],
|
||||
'MySQL': ['mysql'],
|
||||
}
|
||||
|
||||
|
||||
def categorize_commit(message):
|
||||
"""Categorize a commit message based on keyword matching."""
|
||||
msg_lower = message.lower()
|
||||
scores = {}
|
||||
for cat, keywords in CATEGORIES.items():
|
||||
score = 0
|
||||
for kw in keywords:
|
||||
if re.search(r'\b' + re.escape(kw) + r'\b', msg_lower):
|
||||
score += 1
|
||||
if score:
|
||||
scores[cat] = score
|
||||
if scores:
|
||||
# return max score category
|
||||
return max(scores.items(), key=lambda x: x[1])[0]
|
||||
return 'Other'
|
||||
|
||||
|
||||
def get_git_log(from_tag, to_tag):
|
||||
"""Get git log between two tags/branches in a parsable format."""
|
||||
cmd = f"git log {from_tag}..{to_tag} --no-merges --pretty=format:'%H|%s|%b'"
|
||||
try:
|
||||
output = subprocess.check_output(cmd, shell=True, text=True).strip()
|
||||
return output.split('\n') if output else []
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"Error running git log: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def read_commits_from_file(filename):
|
||||
"""Read commits from a file with the same format as git log output."""
|
||||
with open(filename, 'r') as f:
|
||||
lines = f.readlines()
|
||||
return lines
|
||||
|
||||
|
||||
def parse_commits(lines):
|
||||
"""Parse commit lines in format 'hash|subject|body'."""
|
||||
commits = []
|
||||
current = []
|
||||
for line in lines:
|
||||
line = line.rstrip('\n')
|
||||
if line and '|' in line and line.count('|') >= 2:
|
||||
if current:
|
||||
commits.append(''.join(current))
|
||||
current = []
|
||||
current.append(line + '\n')
|
||||
if current:
|
||||
commits.append(''.join(current))
|
||||
|
||||
parsed = []
|
||||
for commit in commits:
|
||||
parts = commit.split('|', 2)
|
||||
if len(parts) < 3:
|
||||
continue
|
||||
hash_, subject, body = parts[0], parts[1], parts[2]
|
||||
parsed.append((hash_, subject, body))
|
||||
return parsed
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Categorize git commits based on keywords.',
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
%(prog)s --from-tag v3.0.3 --to-tag v3.0
|
||||
%(prog)s --input-file /tmp/commits.txt
|
||||
%(prog)s --from-tag v3.0.3 --to-tag v3.0 --output-format markdown
|
||||
"""
|
||||
)
|
||||
parser.add_argument('--from-tag', help='Starting tag/branch (e.g., v3.0.3)')
|
||||
parser.add_argument('--to-tag', help='Ending tag/branch (e.g., v3.0)')
|
||||
parser.add_argument('--input-file', help='Input file with git log output')
|
||||
parser.add_argument('--output-format', choices=['text', 'markdown'], default='markdown',
|
||||
help='Output format (default: markdown)')
|
||||
parser.add_argument('--verbose', '-v', action='store_true', help='Verbose output')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if not (args.from_tag and args.to_tag) and not args.input_file:
|
||||
parser.error('Either --from-tag and --to-tag must be specified, or --input-file')
|
||||
|
||||
if args.from_tag and args.to_tag:
|
||||
lines = get_git_log(args.from_tag, args.to_tag)
|
||||
else:
|
||||
lines = read_commits_from_file(args.input_file)
|
||||
|
||||
commits = parse_commits(lines)
|
||||
|
||||
categorized = {}
|
||||
for hash_, subject, body in commits:
|
||||
full_msg = subject + ' ' + body
|
||||
cat = categorize_commit(full_msg)
|
||||
categorized.setdefault(cat, []).append((hash_, subject, body))
|
||||
|
||||
# Output
|
||||
if args.output_format == 'markdown':
|
||||
for cat in sorted(categorized.keys()):
|
||||
print(f'\n## {cat}\n')
|
||||
for hash_, subject, body in categorized[cat]:
|
||||
print(f'- {hash_[:8]} {subject}')
|
||||
if body.strip():
|
||||
for line in body.strip().split('\n'):
|
||||
if line.strip():
|
||||
print(f' {line.strip()}')
|
||||
print()
|
||||
|
||||
print('\n---\n')
|
||||
for cat in sorted(categorized.keys()):
|
||||
print(f'{cat}: {len(categorized[cat])}')
|
||||
else:
|
||||
# plain text output
|
||||
for cat in sorted(categorized.keys()):
|
||||
print(f'\n=== {cat} ===')
|
||||
for hash_, subject, body in categorized[cat]:
|
||||
print(f' {hash_[:8]} {subject}')
|
||||
if body.strip() and args.verbose:
|
||||
for line in body.strip().split('\n'):
|
||||
if line.strip():
|
||||
print(f' {line.strip()}')
|
||||
print()
|
||||
|
||||
print('\nSummary:')
|
||||
for cat in sorted(categorized.keys()):
|
||||
print(f' {cat}: {len(categorized[cat])}')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@ -0,0 +1,28 @@
|
||||
{
|
||||
"5259": ["Bug Fixes", "MySQL"],
|
||||
"5257": ["New Features", "MySQL"],
|
||||
"5258": "Documentation",
|
||||
"5254": ["New Features", "PostgreSQL"],
|
||||
"5237": ["New Features", "PostgreSQL"],
|
||||
"5250": ["Bug Fixes", "MySQL"],
|
||||
"5251": "Testing",
|
||||
"4889": ["New Features", "MySQL"],
|
||||
"5247": ["Bug Fixes", "Configuration"],
|
||||
"5245": "Documentation",
|
||||
"4901": ["New Features", "MySQL"],
|
||||
"5199": ["Bug Fixes", "Monitoring"],
|
||||
"5241": "Testing",
|
||||
"5232": ["Bug Fixes", "MySQL"],
|
||||
"5240": ["Improvements", "Performance"],
|
||||
"5225": ["Improvements", "Performance"],
|
||||
"5230": "Build/Packaging",
|
||||
"5115": "Documentation",
|
||||
"5229": "Documentation",
|
||||
"5215": "Documentation",
|
||||
"5203": ["New Features", "MySQL"],
|
||||
"5207": "Testing",
|
||||
"5200": "Other",
|
||||
"5198": "Testing",
|
||||
"5228": ["New Features", "Monitoring"],
|
||||
"5226": ["Improvements", "Performance"]
|
||||
}
|
||||
@ -0,0 +1,128 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Collect PR data for release notes generation.
|
||||
|
||||
This script fetches all PRs between two git tags and saves their details
|
||||
to a JSON file for analysis and release notes generation.
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
import re
|
||||
import json
|
||||
import argparse
|
||||
import sys
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
def run(cmd):
|
||||
"""Run shell command and return output."""
|
||||
return subprocess.check_output(cmd, shell=True, text=True).strip()
|
||||
|
||||
|
||||
def get_merge_commits(from_tag, to_tag):
|
||||
"""Get merge commits between two tags."""
|
||||
merge_log = run(f"git log {from_tag}..{to_tag} --merges --pretty=format:'%H|%s'")
|
||||
lines = merge_log.split('\n')
|
||||
prs = []
|
||||
for line in lines:
|
||||
if 'Merge pull request' in line:
|
||||
hash_, subject = line.split('|', 1)
|
||||
match = re.search(r'#(\d+)', subject)
|
||||
if match:
|
||||
pr_num = match.group(1)
|
||||
prs.append((pr_num, hash_, subject))
|
||||
return prs
|
||||
|
||||
|
||||
def fetch_pr_details(pr_numbers, verbose=False):
|
||||
"""Fetch PR details using GitHub CLI."""
|
||||
pr_details = []
|
||||
for pr_num in pr_numbers:
|
||||
if verbose:
|
||||
print(f"Fetching PR #{pr_num}...")
|
||||
try:
|
||||
data = run(f'gh pr view {pr_num} --json title,body,number,url,labels,state,createdAt,mergedAt,author')
|
||||
pr = json.loads(data)
|
||||
# Also get commits in PR if possible
|
||||
try:
|
||||
commits_data = run(f'gh pr view {pr_num} --json commits')
|
||||
commits_json = json.loads(commits_data)
|
||||
pr['commits'] = commits_json.get('commits', [])
|
||||
except subprocess.CalledProcessError:
|
||||
pr['commits'] = []
|
||||
pr_details.append(pr)
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"Failed to fetch PR {pr_num}: {e}", file=sys.stderr)
|
||||
continue
|
||||
return pr_details
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Collect PR data for release notes generation.',
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
%(prog)s --from-tag v3.0.3 --to-tag v3.0 --output pr-data.json
|
||||
%(prog)s --from-tag v3.0.3 --to-tag HEAD --verbose
|
||||
"""
|
||||
)
|
||||
parser.add_argument('--from-tag', required=True, help='Starting tag/branch')
|
||||
parser.add_argument('--to-tag', required=True, help='Ending tag/branch')
|
||||
parser.add_argument('--output', '-o', default='pr-data.json', help='Output JSON file')
|
||||
parser.add_argument('--verbose', '-v', action='store_true', help='Verbose output')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.verbose:
|
||||
print(f"Collecting PR data from {args.from_tag} to {args.to_tag}")
|
||||
|
||||
# Get merge commits
|
||||
prs = get_merge_commits(args.from_tag, args.to_tag)
|
||||
pr_numbers = [pr_num for pr_num, _, _ in prs]
|
||||
|
||||
if args.verbose:
|
||||
print(f"Found {len(pr_numbers)} PR merges")
|
||||
|
||||
# Fetch PR details
|
||||
pr_details = fetch_pr_details(pr_numbers, args.verbose)
|
||||
|
||||
# Add git hash to each PR
|
||||
for pr in pr_details:
|
||||
for pr_num, hash_, subject in prs:
|
||||
if str(pr['number']) == pr_num:
|
||||
pr['merge_hash'] = hash_
|
||||
pr['merge_subject'] = subject
|
||||
break
|
||||
|
||||
# Save to JSON
|
||||
with open(args.output, 'w') as f:
|
||||
json.dump(pr_details, f, indent=2, default=str)
|
||||
|
||||
if args.verbose:
|
||||
print(f"PR data saved to {args.output}")
|
||||
print(f"Collected {len(pr_details)} PRs")
|
||||
|
||||
# Also generate a summary markdown for quick review
|
||||
summary_file = args.output.replace('.json', '-summary.md')
|
||||
with open(summary_file, 'w') as f:
|
||||
f.write(f'# PR Summary: {args.from_tag} to {args.to_tag}\n\n')
|
||||
f.write(f'Total PRs: {len(pr_details)}\n\n')
|
||||
for pr in pr_details:
|
||||
f.write(f'## PR #{pr["number"]}: {pr["title"]}\n')
|
||||
f.write(f'- URL: {pr["url"]}\n')
|
||||
f.write(f'- Author: {pr["author"]["login"]}\n')
|
||||
f.write(f'- Labels: {", ".join([l["name"] for l in pr.get("labels", [])])}\n')
|
||||
f.write(f'- Merge hash: {pr.get("merge_hash", "N/A")[:8]}\n')
|
||||
if pr.get('body'):
|
||||
# Take first 200 chars of body
|
||||
body_preview = pr['body'][:200].replace('\n', ' ')
|
||||
f.write(f'- Preview: {body_preview}...\n')
|
||||
f.write('\n')
|
||||
|
||||
if args.verbose:
|
||||
print(f"Summary saved to {summary_file}")
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@ -0,0 +1,70 @@
|
||||
# ProxySQL 3.0.4 Release Notes
|
||||
|
||||
This release of ProxySQL 3.0.4 includes new features, bug fixes, and improvements across PostgreSQL, MySQL, monitoring, and configuration management.
|
||||
|
||||
Release commit: `faa64a570d19fe35af43494db0babdee3e3cdc89`
|
||||
|
||||
## New Features:
|
||||
|
||||
### PostgreSQL:
|
||||
- Add PostgreSQL-Specific Tokenizer for Query Digest Generation (0f7ff1f3, #5254)
|
||||
- Add SSL support for backend connections in PGSQL monitor (d1b003a0, #5237)
|
||||
|
||||
### MySQL:
|
||||
- Add special handling for unexpected COM_PING packets (e73ba2b6, #5257)
|
||||
- Added handling of SELECT @@version and SELECT VERSION() without backend (0d55ab5e, #4889)
|
||||
- [WIP] Setting client side wait_timeout (2b900b34, #4901)
|
||||
- Implement Fast Forward Grace Close Feature to Prevent Data Loss (83300374, #5203)
|
||||
|
||||
### Monitoring:
|
||||
- Add TCP keepalive warnings when disabled (issue #5212) (0c3582f1, #5228)
|
||||
|
||||
|
||||
## Bug Fixes:
|
||||
|
||||
### MySQL:
|
||||
- Fix: Make cur_cmd_cmnt thread-safe (91e20648, #5259)
|
||||
- Fix cache_empty_result=0 not caching non-empty resultsets (issue #5248) (2987242d, #5250)
|
||||
- Fix issue 4855: Incorrect affected rows reporting for DDL queries (d188715a, #5232)
|
||||
|
||||
### Monitoring:
|
||||
- Fix artificially high ping latency in MySQL backend monitoring (24e02e95, #5199)
|
||||
|
||||
### Security:
|
||||
- Fix SQL injection vulnerability in Read_Global_Variables_from_configfile (automatic prefix stripping) (0b2bc1bf, #5247)
|
||||
|
||||
|
||||
## Improvements:
|
||||
|
||||
### Performance:
|
||||
- Improve fast forward replication CLIENT_DEPRECATE_EOF validation (closes #5062) (5485bb02, #5240)
|
||||
- Refactored Prepared-Statement Cache Design (Lock-Free Hot Path) - Part 2 (9c0e14a5, #5225)
|
||||
- gtid: Refactor reconnect logic & prevent `events_count` reset (20975923, #5226)
|
||||
|
||||
|
||||
## Documentation:
|
||||
- Documentation additions and bug fix for vacuum_stats() (cf8cbfd8, #5258)
|
||||
- Permanent FF sessions + check_data_flow docs (ec1247f2, #5245)
|
||||
- [skip-ci] V3.0.agentics - Add Claude Code Agent Definitions and Architecture Documentation (291e5f02, #5115)
|
||||
- docs: Add comprehensive Doxygen documentation for GTID refactoring (9a55e974, #5229)
|
||||
- Add TAP test writing guide and GitHub automation improvements (49899601, #5215)
|
||||
|
||||
|
||||
## Testing:
|
||||
- Add comments to select @@version queries to bypass ProxySQL interception (66119b74, #5251)
|
||||
- Add fast forward replication deprecate EOF test and update test infrastructure (closes #5062) (9df7407f, #5241)
|
||||
- Add delay to let ProxySQL process mysql_stmt_close() (fa74de3c, #5198)
|
||||
- regroup tests to balance group test time (41b0e96c, #5207)
|
||||
|
||||
|
||||
## Build/Packaging:
|
||||
- add opensuse16 packaging (bce71a95, #5230)
|
||||
|
||||
|
||||
## Other Changes:
|
||||
- bump version to 3.0.4 at the beginning of the development cycle (ba664785, #5200)
|
||||
|
||||
|
||||
## Hashes
|
||||
|
||||
The release commit is: `faa64a570d19fe35af43494db0babdee3e3cdc89`
|
||||
@ -0,0 +1,62 @@
|
||||
# ProxySQL 3.0.4 Release Notes
|
||||
|
||||
This release of ProxySQL 3.0.4 includes new features, bug fixes, and improvements across PostgreSQL, MySQL, monitoring, and configuration management.
|
||||
|
||||
Release commit: faa64a57 (faa64a57)
|
||||
|
||||
## New Features:
|
||||
### PostgreSQL:
|
||||
- Add PostgreSQL-Specific Tokenizer for Query Digest Generation (0f7ff1f3, #5254)
|
||||
- Add SSL support for backend connections in PGSQL monitor (d1b003a0, #5237)
|
||||
|
||||
### MySQL:
|
||||
- Add special handling for unexpected COM_PING packets (e73ba2b6, #5257)
|
||||
- Added handling of SELECT @@version and SELECT VERSION() without backend (0d55ab5e, #4889)
|
||||
- [WIP] Setting client side wait_timeout (2b900b34, #4901)
|
||||
- Implement Fast Forward Grace Close Feature to Prevent Data Loss (83300374, #5203)
|
||||
|
||||
### Monitoring:
|
||||
- Add TCP keepalive warnings when disabled (issue #5212) (0c3582f1, #5228)
|
||||
|
||||
## Bug Fixes:
|
||||
### MySQL:
|
||||
- Fix: Make cur_cmd_cmnt thread-safe (91e20648, #5259)
|
||||
- Fix cache_empty_result=0 not caching non-empty resultsets (issue #5248) (2987242d, #5250)
|
||||
- Fix issue 4855: Incorrect affected rows reporting for DDL queries (d188715a, #5232)
|
||||
|
||||
### Monitoring:
|
||||
- Fix artificially high ping latency in MySQL backend monitoring (24e02e95, #5199)
|
||||
|
||||
### Configuration:
|
||||
- Fix: Automatic prefix stripping for mysql_variables, pgsql_variables, and admin_variables config parsing (0b2bc1bf, #5247)
|
||||
|
||||
## Improvements:
|
||||
### Performance:
|
||||
- Improve fast forward replication CLIENT_DEPRECATE_EOF validation (closes #5062) (5485bb02, #5240)
|
||||
- Refactored Prepared-Statement Cache Design (Lock-Free Hot Path) - Part 2 (9c0e14a5, #5225)
|
||||
- gtid: Refactor reconnect logic & prevent `events_count` reset (20975923, #5226)
|
||||
|
||||
## Documentation:
|
||||
- Documentation additions and bug fix for vacuum_stats() (cf8cbfd8, #5258)
|
||||
- Permanent FF sessions + check_data_flow docs (ec1247f2, #5245)
|
||||
- [skip-ci] V3.0.agentics - Add Claude Code Agent Definitions and Architecture Documentation (291e5f02, #5115)
|
||||
- docs: Add comprehensive Doxygen documentation for GTID refactoring (9a55e974, #5229)
|
||||
- Add TAP test writing guide and GitHub automation improvements (49899601, #5215)
|
||||
|
||||
## Testing:
|
||||
- Add comments to select @@version queries to bypass ProxySQL interception (66119b74, #5251)
|
||||
- Add fast forward replication deprecate EOF test and update test infrastructure (closes #5062) (9df7407f, #5241)
|
||||
- Add delay to let ProxySQL process mysql_stmt_close() (fa74de3c, #5198)
|
||||
- regroup tests to balance group test time (41b0e96c, #5207)
|
||||
|
||||
## Build/Packaging:
|
||||
- add opensuse16 packaging (bce71a95, #5230)
|
||||
|
||||
## Other Changes:
|
||||
- bump version to 3.0.4 at the beginning of the development cycle (ba664785, #5200)
|
||||
|
||||
|
||||
## Hashes
|
||||
|
||||
The release commit is: `faa64a570d19fe35af43494db0babdee3e3cdc89`
|
||||
|
||||
@ -0,0 +1,38 @@
|
||||
# ProxySQL 3.0.4 Release Notes
|
||||
|
||||
ProxySQL 3.0.4 includes numerous bug fixes, improvements, and new features.
|
||||
|
||||
## New Features
|
||||
- Add special handling for unexpected COM_PING packets [#5257](https://github.com/sysown/proxysql/pull/5257)
|
||||
- Add PostgreSQL-Specific Tokenizer for Query Digest Generation [#5254](https://github.com/sysown/proxysql/pull/5254)
|
||||
- Add SSL support for backend connections in PGSQL monitor [#5237](https://github.com/sysown/proxysql/pull/5237)
|
||||
- Add comments to select @@version queries to bypass ProxySQL interception [#5251](https://github.com/sysown/proxysql/pull/5251)
|
||||
- Added handling of SELECT @@version and SELECT VERSION() without backend [#4889](https://github.com/sysown/proxysql/pull/4889)
|
||||
- Add fast forward replication deprecate EOF test and update test infrastructure (closes #5062) [#5241](https://github.com/sysown/proxysql/pull/5241)
|
||||
- add opensuse16 packaging [#5230](https://github.com/sysown/proxysql/pull/5230)
|
||||
- [skip-ci] V3.0.agentics - Add Claude Code Agent Definitions and Architecture Documentation [#5115](https://github.com/sysown/proxysql/pull/5115)
|
||||
- docs: Add comprehensive Doxygen documentation for GTID refactoring [#5229](https://github.com/sysown/proxysql/pull/5229)
|
||||
- Add TAP test writing guide and GitHub automation improvements [#5215](https://github.com/sysown/proxysql/pull/5215)
|
||||
- Implement Fast Forward Grace Close Feature to Prevent Data Loss [#5203](https://github.com/sysown/proxysql/pull/5203)
|
||||
|
||||
## Bug Fixes
|
||||
- Fix: Make cur_cmd_cmnt thread-safe [#5259](https://github.com/sysown/proxysql/pull/5259)
|
||||
- Documentation additions and bug fix for vacuum_stats() [#5258](https://github.com/sysown/proxysql/pull/5258)
|
||||
- Fix cache_empty_result=0 not caching non-empty resultsets (issue #5248) [#5250](https://github.com/sysown/proxysql/pull/5250)
|
||||
- Fix: Automatic prefix stripping for mysql_variables, pgsql_variables, and admin_variables config parsing [#5247](https://github.com/sysown/proxysql/pull/5247)
|
||||
- Fix artificially high ping latency in MySQL backend monitoring [#5199](https://github.com/sysown/proxysql/pull/5199)
|
||||
- Fix issue 4855: Incorrect affected rows reporting for DDL queries [#5232](https://github.com/sysown/proxysql/pull/5232)
|
||||
- Add TCP keepalive warnings when disabled (issue #5212) [#5228](https://github.com/sysown/proxysql/pull/5228)
|
||||
|
||||
## Improvements
|
||||
- Improve fast forward replication CLIENT_DEPRECATE_EOF validation (closes #5062) [#5240](https://github.com/sysown/proxysql/pull/5240)
|
||||
- Refactored Prepared-Statement Cache Design (Lock-Free Hot Path) - Part 2 [#5225](https://github.com/sysown/proxysql/pull/5225)
|
||||
- gtid: Refactor reconnect logic & prevent `events_count` reset [#5226](https://github.com/sysown/proxysql/pull/5226)
|
||||
|
||||
## Documentation
|
||||
- Permanent FF sessions + check_data_flow docs [#5245](https://github.com/sysown/proxysql/pull/5245)
|
||||
|
||||
## Testing
|
||||
- Add delay to let ProxySQL process mysql_stmt_close() [#5198](https://github.com/sysown/proxysql/pull/5198)
|
||||
- regroup tests to balance group test time [#5207](https://github.com/sysown/proxysql/pull/5207)
|
||||
|
||||
@ -0,0 +1,182 @@
|
||||
#!/usr/bin/env python3
|
||||
import subprocess
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
|
||||
def run(cmd):
|
||||
return subprocess.check_output(cmd, shell=True, text=True).strip()
|
||||
|
||||
# Get merge commits since v3.0.3
|
||||
merge_log = run("git log v3.0.3..v3.0 --merges --pretty=format:'%H %s'")
|
||||
lines = merge_log.split('\n')
|
||||
pr_numbers = []
|
||||
for line in lines:
|
||||
if 'Merge pull request' in line:
|
||||
match = re.search(r'#(\d+)', line)
|
||||
if match:
|
||||
pr_numbers.append(match.group(1))
|
||||
print(f'Found {len(pr_numbers)} PRs')
|
||||
|
||||
# Fetch PR details using gh
|
||||
prs = []
|
||||
for num in pr_numbers:
|
||||
try:
|
||||
data = run(f'gh pr view {num} --json title,body,number,url,labels')
|
||||
pr = json.loads(data)
|
||||
prs.append(pr)
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f'Failed to fetch PR {num}: {e}')
|
||||
continue
|
||||
|
||||
# Categorize based on labels and title
|
||||
categories = {
|
||||
'Bug Fixes': [],
|
||||
'New Features': [],
|
||||
'Improvements': [],
|
||||
'Documentation': [],
|
||||
'Testing': [],
|
||||
'Build/Packaging': [],
|
||||
'Refactoring': [],
|
||||
'Security': [],
|
||||
'Monitoring': [],
|
||||
'PostgreSQL': [],
|
||||
'MySQL': [],
|
||||
'Other': [],
|
||||
}
|
||||
|
||||
def categorize(pr):
|
||||
title = pr['title'].lower()
|
||||
labels = [l['name'].lower() for l in pr.get('labels', [])]
|
||||
# label hints
|
||||
for label in labels:
|
||||
if 'bug' in label:
|
||||
return 'Bug Fixes'
|
||||
if 'feature' in label:
|
||||
return 'New Features'
|
||||
if 'documentation' in label:
|
||||
return 'Documentation'
|
||||
if 'test' in label:
|
||||
return 'Testing'
|
||||
if 'security' in label:
|
||||
return 'Security'
|
||||
if 'refactor' in label:
|
||||
return 'Refactoring'
|
||||
if 'improvement' in label:
|
||||
return 'Improvements'
|
||||
# title keywords
|
||||
if any(word in title for word in ['fix', 'bug', 'issue', 'crash', 'vulnerability', 'error']):
|
||||
return 'Bug Fixes'
|
||||
if any(word in title for word in ['add', 'new', 'support', 'implement', 'feature', 'introduce', 'enable']):
|
||||
return 'New Features'
|
||||
if any(word in title for word in ['improve', 'optimize', 'enhance', 'performance', 'better']):
|
||||
return 'Improvements'
|
||||
if any(word in title for word in ['doc', 'documentation', 'doxygen']):
|
||||
return 'Documentation'
|
||||
if any(word in title for word in ['test', 'tap', 'regression']):
|
||||
return 'Testing'
|
||||
if any(word in title for word in ['build', 'package', 'opensuse', 'docker']):
|
||||
return 'Build/Packaging'
|
||||
if any(word in title for word in ['refactor', 'cleanup', 'restructure']):
|
||||
return 'Refactoring'
|
||||
if any(word in title for word in ['security', 'injection', 'vulnerability']):
|
||||
return 'Security'
|
||||
if any(word in title for word in ['monitor', 'metric', 'log']):
|
||||
return 'Monitoring'
|
||||
if any(word in title for word in ['postgresql', 'pgsql', 'pg']):
|
||||
return 'PostgreSQL'
|
||||
if any(word in title for word in ['mysql']):
|
||||
return 'MySQL'
|
||||
return 'Other'
|
||||
|
||||
for pr in prs:
|
||||
cat = categorize(pr)
|
||||
categories[cat].append(pr)
|
||||
|
||||
# Generate markdown
|
||||
output = []
|
||||
output.append('# ProxySQL 3.0.4 Detailed Changelog\n')
|
||||
output.append('\n')
|
||||
output.append('This changelog lists all pull requests merged since ProxySQL 3.0.3.\n')
|
||||
output.append('\n')
|
||||
|
||||
for cat in sorted(categories.keys()):
|
||||
entries = categories[cat]
|
||||
if not entries:
|
||||
continue
|
||||
output.append(f'## {cat}\n')
|
||||
for pr in entries:
|
||||
output.append(f'- **PR #{pr["number"]}**: [{pr["title"]}]({pr["url"]})\n')
|
||||
if pr.get('body'):
|
||||
# take first non-empty line as summary
|
||||
lines = pr['body'].split('\n')
|
||||
summary = ''
|
||||
for line in lines:
|
||||
if line.strip():
|
||||
summary = line.strip()
|
||||
break
|
||||
if summary:
|
||||
output.append(f' - {summary}\n')
|
||||
output.append('\n')
|
||||
|
||||
with open('CHANGELOG-3.0.4-detailed.md', 'w') as f:
|
||||
f.writelines(output)
|
||||
|
||||
print('Generated CHANGELOG-3.0.4-detailed.md')
|
||||
|
||||
# Also generate a concise version for release notes
|
||||
release_notes = []
|
||||
release_notes.append('# ProxySQL 3.0.4 Release Notes\n')
|
||||
release_notes.append('\n')
|
||||
release_notes.append('ProxySQL 3.0.4 includes numerous bug fixes, improvements, and new features.\n')
|
||||
release_notes.append('\n')
|
||||
|
||||
# New Features section
|
||||
new_features = categories['New Features'] + categories['PostgreSQL'] + categories['MySQL']
|
||||
if new_features:
|
||||
release_notes.append('## New Features\n')
|
||||
for pr in new_features:
|
||||
release_notes.append(f'- {pr["title"]} [#{pr["number"]}]({pr["url"]})\n')
|
||||
release_notes.append('\n')
|
||||
|
||||
# Bug Fixes
|
||||
bug_fixes = categories['Bug Fixes'] + categories['Security']
|
||||
if bug_fixes:
|
||||
release_notes.append('## Bug Fixes\n')
|
||||
for pr in bug_fixes:
|
||||
release_notes.append(f'- {pr["title"]} [#{pr["number"]}]({pr["url"]})\n')
|
||||
release_notes.append('\n')
|
||||
|
||||
# Improvements
|
||||
improvements = categories['Improvements'] + categories['Refactoring'] + categories['Monitoring']
|
||||
if improvements:
|
||||
release_notes.append('## Improvements\n')
|
||||
for pr in improvements:
|
||||
release_notes.append(f'- {pr["title"]} [#{pr["number"]}]({pr["url"]})\n')
|
||||
release_notes.append('\n')
|
||||
|
||||
# Documentation
|
||||
if categories['Documentation']:
|
||||
release_notes.append('## Documentation\n')
|
||||
for pr in categories['Documentation']:
|
||||
release_notes.append(f'- {pr["title"]} [#{pr["number"]}]({pr["url"]})\n')
|
||||
release_notes.append('\n')
|
||||
|
||||
# Testing
|
||||
if categories['Testing']:
|
||||
release_notes.append('## Testing\n')
|
||||
for pr in categories['Testing']:
|
||||
release_notes.append(f'- {pr["title"]} [#{pr["number"]}]({pr["url"]})\n')
|
||||
release_notes.append('\n')
|
||||
|
||||
# Build/Packaging
|
||||
if categories['Build/Packaging']:
|
||||
release_notes.append('## Build/Packaging\n')
|
||||
for pr in categories['Build/Packaging']:
|
||||
release_notes.append(f'- {pr["title"]} [#{pr["number"]}]({pr["url"]})\n')
|
||||
release_notes.append('\n')
|
||||
|
||||
with open('RELEASE_NOTES-3.0.4.md', 'w') as f:
|
||||
f.writelines(release_notes)
|
||||
|
||||
print('Generated RELEASE_NOTES-3.0.4.md')
|
||||
@ -0,0 +1,111 @@
|
||||
#!/usr/bin/env python3
|
||||
import subprocess
|
||||
import re
|
||||
import sys
|
||||
|
||||
def run(cmd):
|
||||
return subprocess.check_output(cmd, shell=True, text=True).strip()
|
||||
|
||||
# Get merge commits since v3.0.3
|
||||
merge_log = run("git log v3.0.3..v3.0 --merges --pretty=format:'%H %s'")
|
||||
lines = merge_log.split('\n')
|
||||
prs = []
|
||||
for line in lines:
|
||||
if 'Merge pull request' in line:
|
||||
hash_, subject = line.split(' ', 1)
|
||||
# extract PR number
|
||||
match = re.search(r'#(\d+)', subject)
|
||||
if match:
|
||||
pr_num = match.group(1)
|
||||
prs.append((hash_, pr_num, subject))
|
||||
elif 'Merge branch' in line:
|
||||
# ignore branch merges
|
||||
pass
|
||||
|
||||
print(f'Found {len(prs)} PR merges')
|
||||
|
||||
# For each PR, get commits in PR branch (second parent)
|
||||
pr_commits = {}
|
||||
for hash_, pr_num, subject in prs:
|
||||
cmd = f"git log --oneline --no-merges {hash_}^2"
|
||||
try:
|
||||
output = run(cmd)
|
||||
except subprocess.CalledProcessError as e:
|
||||
# maybe merge commit has only one parent? skip
|
||||
continue
|
||||
commits = output.split('\n') if output else []
|
||||
pr_commits[pr_num] = (subject, commits)
|
||||
|
||||
# Now categorize based on PR subject and commit messages
|
||||
categories = {
|
||||
'Bug Fixes': [],
|
||||
'New Features': [],
|
||||
'Improvements': [],
|
||||
'Documentation': [],
|
||||
'Testing': [],
|
||||
'Build/Packaging': [],
|
||||
'Refactoring': [],
|
||||
'Security': [],
|
||||
'Monitoring': [],
|
||||
'PostgreSQL': [],
|
||||
'MySQL': [],
|
||||
'Other': [],
|
||||
}
|
||||
|
||||
def categorize_pr(subject, commits):
|
||||
subj_lower = subject.lower()
|
||||
# keyword matching
|
||||
if any(word in subj_lower for word in ['fix', 'bug', 'issue', 'crash', 'vulnerability']):
|
||||
return 'Bug Fixes'
|
||||
if any(word in subj_lower for word in ['add', 'new', 'support', 'implement', 'feature', 'introduce', 'enable']):
|
||||
return 'New Features'
|
||||
if any(word in subj_lower for word in ['improve', 'optimize', 'enhance', 'performance', 'better']):
|
||||
return 'Improvements'
|
||||
if any(word in subj_lower for word in ['doc', 'documentation', 'doxygen']):
|
||||
return 'Documentation'
|
||||
if any(word in subj_lower for word in ['test', 'tap', 'regression']):
|
||||
return 'Testing'
|
||||
if any(word in subj_lower for word in ['build', 'package', 'opensuse', 'docker']):
|
||||
return 'Build/Packaging'
|
||||
if any(word in subj_lower for word in ['refactor', 'cleanup', 'restructure']):
|
||||
return 'Refactoring'
|
||||
if any(word in subj_lower for word in ['security', 'injection', 'vulnerability']):
|
||||
return 'Security'
|
||||
if any(word in subj_lower for word in ['monitor', 'metric', 'log']):
|
||||
return 'Monitoring'
|
||||
if any(word in subj_lower for word in ['postgresql', 'pgsql', 'pg']):
|
||||
return 'PostgreSQL'
|
||||
if any(word in subj_lower for word in ['mysql']):
|
||||
return 'MySQL'
|
||||
return 'Other'
|
||||
|
||||
for pr_num, (subject, commits) in pr_commits.items():
|
||||
cat = categorize_pr(subject, commits)
|
||||
categories[cat].append((pr_num, subject, commits))
|
||||
|
||||
# Output markdown
|
||||
output_lines = []
|
||||
output_lines.append('# ProxySQL 3.0.4 Changelog\n')
|
||||
output_lines.append('\n')
|
||||
output_lines.append('This changelog summarizes all changes since ProxySQL 3.0.3.\n')
|
||||
output_lines.append('\n')
|
||||
|
||||
for cat in sorted(categories.keys()):
|
||||
entries = categories[cat]
|
||||
if not entries:
|
||||
continue
|
||||
output_lines.append(f'## {cat}\n')
|
||||
for pr_num, subject, commits in entries:
|
||||
output_lines.append(f'- **PR #{pr_num}**: {subject}\n')
|
||||
if commits:
|
||||
for commit in commits:
|
||||
# commit format: hash message
|
||||
# strip hash
|
||||
msg = commit.split(' ', 1)[1] if ' ' in commit else commit
|
||||
output_lines.append(f' - {msg}\n')
|
||||
output_lines.append('\n')
|
||||
|
||||
with open('CHANGELOG-3.0.4.md', 'w') as f:
|
||||
f.writelines(output_lines)
|
||||
|
||||
print('Generated CHANGELOG-3.0.4.md')
|
||||
@ -0,0 +1,225 @@
|
||||
#!/usr/bin/env python3
|
||||
import subprocess
|
||||
import re
|
||||
import json
|
||||
|
||||
def run(cmd):
|
||||
return subprocess.check_output(cmd, shell=True, text=True).strip()
|
||||
|
||||
# Get merge commits since v3.0.3
|
||||
merge_log = run("git log v3.0.3..v3.0 --merges --pretty=format:'%H %s'")
|
||||
lines = merge_log.split('\n')
|
||||
pr_map = []
|
||||
for line in lines:
|
||||
if 'Merge pull request' in line:
|
||||
hash_, subject = line.split(' ', 1)
|
||||
match = re.search(r'#(\d+)', subject)
|
||||
if match:
|
||||
pr_num = match.group(1)
|
||||
# get second parent commit hash
|
||||
try:
|
||||
second_parent = run(f"git rev-parse {hash_}^2")
|
||||
except subprocess.CalledProcessError:
|
||||
second_parent = hash_
|
||||
pr_map.append((pr_num, hash_, second_parent, subject))
|
||||
|
||||
print(f'Processed {len(pr_map)} PRs')
|
||||
|
||||
# Fetch PR details using gh
|
||||
pr_details = {}
|
||||
for pr_num, merge_hash, head_hash, subject in pr_map:
|
||||
try:
|
||||
data = run(f'gh pr view {pr_num} --json title,body,number,url')
|
||||
pr = json.loads(data)
|
||||
pr['merge_hash'] = merge_hash
|
||||
pr['head_hash'] = head_hash
|
||||
pr_details[pr_num] = pr
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f'Failed to fetch PR {pr_num}: {e}')
|
||||
continue
|
||||
|
||||
# Manual categorization based on PR title and analysis
|
||||
categories = {
|
||||
'New Features': {
|
||||
'PostgreSQL': [],
|
||||
'MySQL': [],
|
||||
'Monitoring': [],
|
||||
'Configuration': [],
|
||||
'Performance': [],
|
||||
'Other': [],
|
||||
},
|
||||
'Bug Fixes': {
|
||||
'MySQL': [],
|
||||
'PostgreSQL': [],
|
||||
'Monitoring': [],
|
||||
'Configuration': [],
|
||||
'Security': [],
|
||||
'Other': [],
|
||||
},
|
||||
'Improvements': {
|
||||
'Performance': [],
|
||||
'Refactoring': [],
|
||||
'Monitoring': [],
|
||||
'Other': [],
|
||||
},
|
||||
'Documentation': [],
|
||||
'Testing': [],
|
||||
'Build/Packaging': [],
|
||||
'Other': [],
|
||||
}
|
||||
|
||||
# Mapping PR numbers to categories (manually curated)
|
||||
category_map = {
|
||||
'5259': ('Bug Fixes', 'MySQL'),
|
||||
'5257': ('New Features', 'MySQL'),
|
||||
'5258': ('Documentation', None),
|
||||
'5254': ('New Features', 'PostgreSQL'),
|
||||
'5237': ('New Features', 'PostgreSQL'),
|
||||
'5250': ('Bug Fixes', 'MySQL'),
|
||||
'5251': ('Testing', None),
|
||||
'4889': ('New Features', 'MySQL'),
|
||||
'5247': ('Bug Fixes', 'Configuration'),
|
||||
'5245': ('Documentation', None),
|
||||
'4901': ('New Features', 'MySQL'),
|
||||
'5199': ('Bug Fixes', 'Monitoring'),
|
||||
'5241': ('Testing', None),
|
||||
'5232': ('Bug Fixes', 'MySQL'),
|
||||
'5240': ('Improvements', 'Performance'),
|
||||
'5225': ('Improvements', 'Performance'),
|
||||
'5230': ('Build/Packaging', None),
|
||||
'5115': ('Documentation', None),
|
||||
'5229': ('Documentation', None),
|
||||
'5215': ('Documentation', None),
|
||||
'5203': ('New Features', 'MySQL'),
|
||||
'5207': ('Testing', None),
|
||||
'5200': ('Other', None),
|
||||
'5198': ('Testing', None),
|
||||
'5228': ('New Features', 'Monitoring'),
|
||||
'5226': ('Improvements', 'Performance'),
|
||||
}
|
||||
|
||||
for pr_num, pr in pr_details.items():
|
||||
if pr_num in category_map:
|
||||
cat, subcat = category_map[pr_num]
|
||||
if subcat:
|
||||
categories[cat][subcat].append(pr)
|
||||
else:
|
||||
if isinstance(categories[cat], list):
|
||||
categories[cat].append(pr)
|
||||
else:
|
||||
categories[cat]['Other'].append(pr)
|
||||
else:
|
||||
categories['Other'].append(pr)
|
||||
|
||||
# Generate release notes
|
||||
out_lines = []
|
||||
out_lines.append('# ProxySQL 3.0.4 Release Notes\n')
|
||||
out_lines.append('\n')
|
||||
out_lines.append('This release of ProxySQL 3.0.4 includes new features, bug fixes, and improvements across PostgreSQL, MySQL, monitoring, and configuration management.\n')
|
||||
out_lines.append('\n')
|
||||
out_lines.append(f'Release commit: {pr_map[0][1][:8]} (faa64a57)\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
# Helper to format entry
|
||||
def format_entry(pr):
|
||||
head_short = pr['head_hash'][:8]
|
||||
title = pr['title']
|
||||
url = pr['url']
|
||||
return f'- {title} ({head_short}, #{pr["number"]})\n'
|
||||
|
||||
# New Features
|
||||
if any(any(subcat) for subcat in categories['New Features'].values()):
|
||||
out_lines.append('## New Features:\n')
|
||||
for subcat in ['PostgreSQL', 'MySQL', 'Monitoring', 'Configuration', 'Performance', 'Other']:
|
||||
entries = categories['New Features'][subcat]
|
||||
if entries:
|
||||
out_lines.append(f'### {subcat}:\n')
|
||||
for pr in entries:
|
||||
out_lines.append(format_entry(pr))
|
||||
out_lines.append('\n')
|
||||
|
||||
# Bug Fixes
|
||||
if any(any(subcat) for subcat in categories['Bug Fixes'].values()):
|
||||
out_lines.append('## Bug Fixes:\n')
|
||||
for subcat in ['MySQL', 'PostgreSQL', 'Monitoring', 'Configuration', 'Security', 'Other']:
|
||||
entries = categories['Bug Fixes'][subcat]
|
||||
if entries:
|
||||
out_lines.append(f'### {subcat}:\n')
|
||||
for pr in entries:
|
||||
out_lines.append(format_entry(pr))
|
||||
out_lines.append('\n')
|
||||
|
||||
# Improvements
|
||||
if any(any(subcat) for subcat in categories['Improvements'].values()):
|
||||
out_lines.append('## Improvements:\n')
|
||||
for subcat in ['Performance', 'Refactoring', 'Monitoring', 'Other']:
|
||||
entries = categories['Improvements'][subcat]
|
||||
if entries:
|
||||
out_lines.append(f'### {subcat}:\n')
|
||||
for pr in entries:
|
||||
out_lines.append(format_entry(pr))
|
||||
out_lines.append('\n')
|
||||
|
||||
# Documentation
|
||||
if categories['Documentation']:
|
||||
out_lines.append('## Documentation:\n')
|
||||
for pr in categories['Documentation']:
|
||||
out_lines.append(format_entry(pr))
|
||||
out_lines.append('\n')
|
||||
|
||||
# Testing
|
||||
if categories['Testing']:
|
||||
out_lines.append('## Testing:\n')
|
||||
for pr in categories['Testing']:
|
||||
out_lines.append(format_entry(pr))
|
||||
out_lines.append('\n')
|
||||
|
||||
# Build/Packaging
|
||||
if categories['Build/Packaging']:
|
||||
out_lines.append('## Build/Packaging:\n')
|
||||
for pr in categories['Build/Packaging']:
|
||||
out_lines.append(format_entry(pr))
|
||||
out_lines.append('\n')
|
||||
|
||||
# Other (if any)
|
||||
if categories['Other']:
|
||||
out_lines.append('## Other Changes:\n')
|
||||
for pr in categories['Other']:
|
||||
out_lines.append(format_entry(pr))
|
||||
out_lines.append('\n')
|
||||
|
||||
out_lines.append('\n')
|
||||
out_lines.append('## Hashes\n')
|
||||
out_lines.append('\n')
|
||||
out_lines.append('The release commit is: `faa64a570d19fe35af43494db0babdee3e3cdc89`\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
with open('RELEASE_NOTES-3.0.4-formatted.md', 'w') as f:
|
||||
f.writelines(out_lines)
|
||||
|
||||
print('Generated RELEASE_NOTES-3.0.4-formatted.md')
|
||||
|
||||
# Also generate a more detailed changelog with commit messages
|
||||
detailed = []
|
||||
detailed.append('# ProxySQL 3.0.4 Detailed Changelog\n')
|
||||
detailed.append('\n')
|
||||
detailed.append('This changelog includes all individual commits since ProxySQL 3.0.3.\n')
|
||||
detailed.append('\n')
|
||||
|
||||
# Get all non-merge commits
|
||||
commits = run("git log v3.0.3..v3.0 --no-merges --pretty=format:'%H|%s|%b'").split('\n')
|
||||
for line in commits:
|
||||
parts = line.split('|', 2)
|
||||
if len(parts) < 2:
|
||||
continue
|
||||
hash_, subject, body = parts[0], parts[1], parts[2] if len(parts) > 2 else ''
|
||||
detailed.append(f'- {hash_[:8]} {subject}\n')
|
||||
if body.strip():
|
||||
for bline in body.strip().split('\n'):
|
||||
if bline.strip():
|
||||
detailed.append(f' {bline.strip()}\n')
|
||||
|
||||
with open('CHANGELOG-3.0.4-commits.md', 'w') as f:
|
||||
f.writelines(detailed)
|
||||
|
||||
print('Generated CHANGELOG-3.0.4-commits.md')
|
||||
@ -0,0 +1,164 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Generate a changelog from git merge commits between two tags/branches.
|
||||
|
||||
This script analyzes merge commits (pull requests) and generates a categorized
|
||||
changelog. It attempts to get commits within each PR by looking at the second
|
||||
parent of merge commits.
|
||||
|
||||
Usage:
|
||||
python generate_changelog.py --from-tag v3.0.3 --to-tag v3.0 --output changelog.md
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
import re
|
||||
import argparse
|
||||
import sys
|
||||
|
||||
|
||||
def run(cmd):
|
||||
"""Run shell command and return output."""
|
||||
return subprocess.check_output(cmd, shell=True, text=True).strip()
|
||||
|
||||
|
||||
def get_merge_commits(from_tag, to_tag):
|
||||
"""Get merge commits between two tags."""
|
||||
merge_log = run(f"git log {from_tag}..{to_tag} --merges --pretty=format:'%H %s'")
|
||||
lines = merge_log.split('\n')
|
||||
prs = []
|
||||
for line in lines:
|
||||
if 'Merge pull request' in line:
|
||||
hash_, subject = line.split(' ', 1)
|
||||
# extract PR number
|
||||
match = re.search(r'#(\d+)', subject)
|
||||
if match:
|
||||
pr_num = match.group(1)
|
||||
prs.append((hash_, pr_num, subject))
|
||||
# ignore branch merges
|
||||
return prs
|
||||
|
||||
|
||||
def get_pr_commits(prs):
|
||||
"""For each PR merge commit, get commits in the PR branch (second parent)."""
|
||||
pr_commits = {}
|
||||
for hash_, pr_num, subject in prs:
|
||||
cmd = f"git log --oneline --no-merges {hash_}^2"
|
||||
try:
|
||||
output = run(cmd)
|
||||
except subprocess.CalledProcessError:
|
||||
# merge commit may have only one parent, skip
|
||||
continue
|
||||
commits = output.split('\n') if output else []
|
||||
pr_commits[pr_num] = (subject, commits)
|
||||
return pr_commits
|
||||
|
||||
|
||||
def categorize_pr(subject, commits):
|
||||
"""Categorize a PR based on its subject and commit messages."""
|
||||
subj_lower = subject.lower()
|
||||
# keyword matching
|
||||
if any(word in subj_lower for word in ['fix', 'bug', 'issue', 'crash', 'vulnerability']):
|
||||
return 'Bug Fixes'
|
||||
if any(word in subj_lower for word in ['add', 'new', 'support', 'implement', 'feature', 'introduce', 'enable']):
|
||||
return 'New Features'
|
||||
if any(word in subj_lower for word in ['improve', 'optimize', 'enhance', 'performance', 'better']):
|
||||
return 'Improvements'
|
||||
if any(word in subj_lower for word in ['doc', 'documentation', 'doxygen']):
|
||||
return 'Documentation'
|
||||
if any(word in subj_lower for word in ['test', 'tap', 'regression']):
|
||||
return 'Testing'
|
||||
if any(word in subj_lower for word in ['build', 'package', 'opensuse', 'docker']):
|
||||
return 'Build/Packaging'
|
||||
if any(word in subj_lower for word in ['refactor', 'cleanup', 'restructure']):
|
||||
return 'Refactoring'
|
||||
if any(word in subj_lower for word in ['security', 'injection', 'vulnerability']):
|
||||
return 'Security'
|
||||
if any(word in subj_lower for word in ['monitor', 'metric', 'log']):
|
||||
return 'Monitoring'
|
||||
if any(word in subj_lower for word in ['postgresql', 'pgsql', 'pg']):
|
||||
return 'PostgreSQL'
|
||||
if any(word in subj_lower for word in ['mysql']):
|
||||
return 'MySQL'
|
||||
return 'Other'
|
||||
|
||||
|
||||
def generate_changelog(from_tag, to_tag, output_file, verbose=False):
|
||||
"""Main function to generate changelog."""
|
||||
if verbose:
|
||||
print(f"Generating changelog from {from_tag} to {to_tag}...")
|
||||
|
||||
prs = get_merge_commits(from_tag, to_tag)
|
||||
if verbose:
|
||||
print(f"Found {len(prs)} PR merges")
|
||||
|
||||
pr_commits = get_pr_commits(prs)
|
||||
|
||||
categories = {
|
||||
'Bug Fixes': [],
|
||||
'New Features': [],
|
||||
'Improvements': [],
|
||||
'Documentation': [],
|
||||
'Testing': [],
|
||||
'Build/Packaging': [],
|
||||
'Refactoring': [],
|
||||
'Security': [],
|
||||
'Monitoring': [],
|
||||
'PostgreSQL': [],
|
||||
'MySQL': [],
|
||||
'Other': [],
|
||||
}
|
||||
|
||||
for pr_num, (subject, commits) in pr_commits.items():
|
||||
cat = categorize_pr(subject, commits)
|
||||
categories[cat].append((pr_num, subject, commits))
|
||||
|
||||
# Build output
|
||||
output_lines = []
|
||||
output_lines.append(f'# Changelog from {from_tag} to {to_tag}\n')
|
||||
output_lines.append('\n')
|
||||
output_lines.append(f'This changelog summarizes all changes between {from_tag} and {to_tag}.\n')
|
||||
output_lines.append('\n')
|
||||
|
||||
for cat in sorted(categories.keys()):
|
||||
entries = categories[cat]
|
||||
if not entries:
|
||||
continue
|
||||
output_lines.append(f'## {cat}\n')
|
||||
for pr_num, subject, commits in entries:
|
||||
output_lines.append(f'- **PR #{pr_num}**: {subject}\n')
|
||||
if commits:
|
||||
for commit in commits:
|
||||
# commit format: hash message
|
||||
msg = commit.split(' ', 1)[1] if ' ' in commit else commit
|
||||
output_lines.append(f' - {msg}\n')
|
||||
output_lines.append('\n')
|
||||
|
||||
with open(output_file, 'w') as f:
|
||||
f.writelines(output_lines)
|
||||
|
||||
if verbose:
|
||||
print(f"Changelog written to {output_file}")
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Generate a changelog from git merge commits.',
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
%(prog)s --from-tag v3.0.3 --to-tag v3.0 --output changelog.md
|
||||
%(prog)s --from-tag v3.0.3 --to-tag HEAD --output changes.md --verbose
|
||||
"""
|
||||
)
|
||||
parser.add_argument('--from-tag', required=True, help='Starting tag/branch (e.g., v3.0.3)')
|
||||
parser.add_argument('--to-tag', required=True, help='Ending tag/branch (e.g., v3.0 or HEAD)')
|
||||
parser.add_argument('--output', '-o', default='changelog.md', help='Output file (default: changelog.md)')
|
||||
parser.add_argument('--verbose', '-v', action='store_true', help='Verbose output')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
generate_changelog(args.from_tag, args.to_tag, args.output, args.verbose)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@ -0,0 +1,362 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Generate release notes from GitHub pull requests between two git tags.
|
||||
|
||||
This script uses the GitHub CLI (gh) to fetch PR details and generates
|
||||
formatted release notes similar to ProxySQL's release notes format.
|
||||
|
||||
Features:
|
||||
- Automatic categorization based on PR labels and titles
|
||||
- Optional manual categorization mapping via JSON file
|
||||
- Support for subcategories (PostgreSQL, MySQL, Monitoring, etc.)
|
||||
- Outputs both release notes and detailed changelog
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
import re
|
||||
import json
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def run(cmd):
|
||||
"""Run shell command and return output."""
|
||||
return subprocess.check_output(cmd, shell=True, text=True).strip()
|
||||
|
||||
|
||||
def get_merge_commits(from_tag, to_tag):
|
||||
"""Get merge commits between two tags and extract PR numbers."""
|
||||
merge_log = run(f"git log {from_tag}..{to_tag} --merges --pretty=format:'%H %s'")
|
||||
lines = merge_log.split('\n')
|
||||
pr_map = []
|
||||
for line in lines:
|
||||
if 'Merge pull request' in line:
|
||||
hash_, subject = line.split(' ', 1)
|
||||
match = re.search(r'#(\d+)', subject)
|
||||
if match:
|
||||
pr_num = match.group(1)
|
||||
# get second parent commit hash (PR head)
|
||||
try:
|
||||
second_parent = run(f"git rev-parse {hash_}^2")
|
||||
except subprocess.CalledProcessError:
|
||||
second_parent = hash_
|
||||
pr_map.append((pr_num, hash_, second_parent, subject))
|
||||
return pr_map
|
||||
|
||||
|
||||
def fetch_pr_details(pr_numbers, verbose=False):
|
||||
"""Fetch PR details using GitHub CLI."""
|
||||
pr_details = {}
|
||||
for pr_num in pr_numbers:
|
||||
if verbose:
|
||||
print(f"Fetching PR #{pr_num}...")
|
||||
try:
|
||||
data = run(f'gh pr view {pr_num} --json title,body,number,url,labels,state,createdAt,mergedAt')
|
||||
pr = json.loads(data)
|
||||
pr_details[pr_num] = pr
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"Failed to fetch PR {pr_num}: {e}", file=sys.stderr)
|
||||
continue
|
||||
return pr_details
|
||||
|
||||
|
||||
def load_category_mapping(config_file):
|
||||
"""Load manual category mapping from JSON file."""
|
||||
if not config_file or not Path(config_file).exists():
|
||||
return {}
|
||||
try:
|
||||
with open(config_file, 'r') as f:
|
||||
return json.load(f)
|
||||
except json.JSONDecodeError as e:
|
||||
print(f"Error parsing config file {config_file}: {e}", file=sys.stderr)
|
||||
return {}
|
||||
|
||||
|
||||
def auto_categorize(pr, mapping):
|
||||
"""Categorize a PR based on mapping, labels, and title."""
|
||||
pr_num = str(pr['number'])
|
||||
|
||||
# First, check manual mapping
|
||||
if pr_num in mapping:
|
||||
cat_info = mapping[pr_num]
|
||||
if isinstance(cat_info, dict):
|
||||
return cat_info.get('category', 'Other'), cat_info.get('subcategory')
|
||||
elif isinstance(cat_info, str):
|
||||
return cat_info, None
|
||||
elif isinstance(cat_info, list) and len(cat_info) >= 1:
|
||||
cat = cat_info[0]
|
||||
subcat = cat_info[1] if len(cat_info) > 1 else None
|
||||
return cat, subcat
|
||||
|
||||
# Auto-categorize based on labels
|
||||
title = pr['title'].lower()
|
||||
labels = [l['name'].lower() for l in pr.get('labels', [])]
|
||||
|
||||
# Label-based categorization
|
||||
for label in labels:
|
||||
if 'bug' in label:
|
||||
return 'Bug Fixes', None
|
||||
if 'feature' in label:
|
||||
return 'New Features', None
|
||||
if 'documentation' in label:
|
||||
return 'Documentation', None
|
||||
if 'test' in label:
|
||||
return 'Testing', None
|
||||
if 'security' in label:
|
||||
return 'Security', None
|
||||
if 'refactor' in label:
|
||||
return 'Improvements', 'Refactoring'
|
||||
if 'improvement' in label:
|
||||
return 'Improvements', None
|
||||
|
||||
# Title keyword-based categorization
|
||||
if any(word in title for word in ['fix', 'bug', 'issue', 'crash', 'vulnerability', 'error']):
|
||||
return 'Bug Fixes', None
|
||||
if any(word in title for word in ['add', 'new', 'support', 'implement', 'feature', 'introduce', 'enable']):
|
||||
return 'New Features', None
|
||||
if any(word in title for word in ['improve', 'optimize', 'enhance', 'performance', 'better']):
|
||||
return 'Improvements', None
|
||||
if any(word in title for word in ['doc', 'documentation', 'doxygen']):
|
||||
return 'Documentation', None
|
||||
if any(word in title for word in ['test', 'tap', 'regression']):
|
||||
return 'Testing', None
|
||||
if any(word in title for word in ['build', 'package', 'opensuse', 'docker']):
|
||||
return 'Build/Packaging', None
|
||||
if any(word in title for word in ['refactor', 'cleanup', 'restructure']):
|
||||
return 'Improvements', 'Refactoring'
|
||||
if any(word in title for word in ['security', 'injection', 'vulnerability']):
|
||||
return 'Security', None
|
||||
if any(word in title for word in ['monitor', 'metric', 'log']):
|
||||
return 'Monitoring', None
|
||||
if any(word in title for word in ['postgresql', 'pgsql', 'pg']):
|
||||
return 'New Features', 'PostgreSQL'
|
||||
if any(word in title for word in ['mysql']):
|
||||
return 'New Features', 'MySQL'
|
||||
|
||||
return 'Other', None
|
||||
|
||||
|
||||
def organize_prs(pr_details, mapping, verbose=False):
|
||||
"""Organize PRs into categorized structure."""
|
||||
categories = {
|
||||
'New Features': {
|
||||
'PostgreSQL': [],
|
||||
'MySQL': [],
|
||||
'Monitoring': [],
|
||||
'Configuration': [],
|
||||
'Performance': [],
|
||||
'Other': [],
|
||||
},
|
||||
'Bug Fixes': {
|
||||
'MySQL': [],
|
||||
'PostgreSQL': [],
|
||||
'Monitoring': [],
|
||||
'Configuration': [],
|
||||
'Security': [],
|
||||
'Other': [],
|
||||
},
|
||||
'Improvements': {
|
||||
'Performance': [],
|
||||
'Refactoring': [],
|
||||
'Monitoring': [],
|
||||
'Other': [],
|
||||
},
|
||||
'Documentation': [],
|
||||
'Testing': [],
|
||||
'Build/Packaging': [],
|
||||
'Other': [],
|
||||
}
|
||||
|
||||
for pr_num, pr in pr_details.items():
|
||||
cat, subcat = auto_categorize(pr, mapping)
|
||||
|
||||
if cat in categories:
|
||||
if isinstance(categories[cat], dict):
|
||||
# Has subcategories
|
||||
if subcat and subcat in categories[cat]:
|
||||
categories[cat][subcat].append(pr)
|
||||
else:
|
||||
categories[cat]['Other'].append(pr)
|
||||
else:
|
||||
# Simple list category
|
||||
categories[cat].append(pr)
|
||||
else:
|
||||
categories['Other'].append(pr)
|
||||
|
||||
if verbose:
|
||||
for cat, contents in categories.items():
|
||||
if isinstance(contents, dict):
|
||||
total = sum(len(sub) for sub in contents.values())
|
||||
print(f"{cat}: {total} PRs")
|
||||
for subcat, prs in contents.items():
|
||||
if prs:
|
||||
print(f" {subcat}: {len(prs)}")
|
||||
else:
|
||||
if contents:
|
||||
print(f"{cat}: {len(contents)} PRs")
|
||||
|
||||
return categories
|
||||
|
||||
|
||||
def generate_release_notes(categories, from_tag, to_tag, output_file):
|
||||
"""Generate formatted release notes."""
|
||||
out_lines = []
|
||||
out_lines.append(f'# ProxySQL {to_tag} Release Notes\n')
|
||||
out_lines.append('\n')
|
||||
out_lines.append(f'This release of ProxySQL {to_tag} includes new features, bug fixes, and improvements.\n')
|
||||
out_lines.append('\n')
|
||||
out_lines.append(f'Release range: {from_tag} to {to_tag}\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
def format_pr(pr):
|
||||
title = pr['title']
|
||||
url = pr['url']
|
||||
number = pr['number']
|
||||
return f'- {title} (#{number})\n'
|
||||
|
||||
# New Features
|
||||
if any(any(subcat) for subcat in categories['New Features'].values()):
|
||||
out_lines.append('## New Features:\n')
|
||||
for subcat in ['PostgreSQL', 'MySQL', 'Monitoring', 'Configuration', 'Performance', 'Other']:
|
||||
entries = categories['New Features'][subcat]
|
||||
if entries:
|
||||
out_lines.append(f'### {subcat}:\n')
|
||||
for pr in entries:
|
||||
out_lines.append(format_pr(pr))
|
||||
out_lines.append('\n')
|
||||
|
||||
# Bug Fixes
|
||||
if any(any(subcat) for subcat in categories['Bug Fixes'].values()):
|
||||
out_lines.append('## Bug Fixes:\n')
|
||||
for subcat in ['MySQL', 'PostgreSQL', 'Monitoring', 'Configuration', 'Security', 'Other']:
|
||||
entries = categories['Bug Fixes'][subcat]
|
||||
if entries:
|
||||
out_lines.append(f'### {subcat}:\n')
|
||||
for pr in entries:
|
||||
out_lines.append(format_pr(pr))
|
||||
out_lines.append('\n')
|
||||
|
||||
# Improvements
|
||||
if any(any(subcat) for subcat in categories['Improvements'].values()):
|
||||
out_lines.append('## Improvements:\n')
|
||||
for subcat in ['Performance', 'Refactoring', 'Monitoring', 'Other']:
|
||||
entries = categories['Improvements'][subcat]
|
||||
if entries:
|
||||
out_lines.append(f'### {subcat}:\n')
|
||||
for pr in entries:
|
||||
out_lines.append(format_pr(pr))
|
||||
out_lines.append('\n')
|
||||
|
||||
# Simple categories
|
||||
simple_cats = ['Documentation', 'Testing', 'Build/Packaging', 'Other']
|
||||
for cat in simple_cats:
|
||||
entries = categories[cat]
|
||||
if entries:
|
||||
out_lines.append(f'## {cat}:\n')
|
||||
for pr in entries:
|
||||
out_lines.append(format_pr(pr))
|
||||
out_lines.append('\n')
|
||||
|
||||
out_lines.append('\n')
|
||||
out_lines.append('## Hashes\n')
|
||||
out_lines.append('\n')
|
||||
out_lines.append(f'The release range is: `{from_tag}` to `{to_tag}`\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
with open(output_file, 'w') as f:
|
||||
f.writelines(out_lines)
|
||||
|
||||
return output_file
|
||||
|
||||
|
||||
def generate_detailed_changelog(pr_details, from_tag, to_tag, output_file):
|
||||
"""Generate detailed changelog with PR descriptions."""
|
||||
out_lines = []
|
||||
out_lines.append(f'# Detailed Changelog from {from_tag} to {to_tag}\n')
|
||||
out_lines.append('\n')
|
||||
out_lines.append(f'This changelog lists all pull requests merged between {from_tag} and {to_tag}.\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
for pr in pr_details.values():
|
||||
out_lines.append(f'## PR #{pr["number"]}: {pr["title"]}\n')
|
||||
out_lines.append(f'- URL: {pr["url"]}\n')
|
||||
if pr.get('labels'):
|
||||
labels = ', '.join([l['name'] for l in pr['labels']])
|
||||
out_lines.append(f'- Labels: {labels}\n')
|
||||
if pr.get('body'):
|
||||
# Take first paragraph as summary
|
||||
lines = pr['body'].split('\n')
|
||||
summary = ''
|
||||
for line in lines:
|
||||
if line.strip() and not line.strip().startswith('#'):
|
||||
summary = line.strip()
|
||||
break
|
||||
if summary:
|
||||
out_lines.append(f'- Summary: {summary}\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
with open(output_file, 'w') as f:
|
||||
f.writelines(out_lines)
|
||||
|
||||
return output_file
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Generate release notes from GitHub pull requests.',
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
%(prog)s --from-tag v3.0.3 --to-tag v3.0 --output release-notes.md
|
||||
%(prog)s --from-tag v3.0.3 --to-tag HEAD --config mapping.json --verbose
|
||||
"""
|
||||
)
|
||||
parser.add_argument('--from-tag', required=True, help='Starting tag/branch')
|
||||
parser.add_argument('--to-tag', required=True, help='Ending tag/branch')
|
||||
parser.add_argument('--output', '-o', default='release-notes.md',
|
||||
help='Output file for release notes (default: release-notes.md)')
|
||||
parser.add_argument('--changelog', '-c', help='Output file for detailed changelog')
|
||||
parser.add_argument('--config', help='JSON file with manual category mapping')
|
||||
parser.add_argument('--verbose', '-v', action='store_true', help='Verbose output')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.verbose:
|
||||
print(f"Generating release notes from {args.from_tag} to {args.to_tag}")
|
||||
|
||||
# Get merge commits
|
||||
pr_map = get_merge_commits(args.from_tag, args.to_tag)
|
||||
pr_numbers = [pr_num for pr_num, _, _, _ in pr_map]
|
||||
|
||||
if args.verbose:
|
||||
print(f"Found {len(pr_numbers)} PR merges")
|
||||
|
||||
# Fetch PR details
|
||||
pr_details = fetch_pr_details(pr_numbers, args.verbose)
|
||||
|
||||
if not pr_details:
|
||||
print("No PR details fetched. Check GitHub CLI authentication.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
# Load manual mapping
|
||||
mapping = load_category_mapping(args.config)
|
||||
|
||||
# Organize PRs into categories
|
||||
categories = organize_prs(pr_details, mapping, args.verbose)
|
||||
|
||||
# Generate release notes
|
||||
release_file = generate_release_notes(categories, args.from_tag, args.to_tag, args.output)
|
||||
print(f"Release notes written to {release_file}")
|
||||
|
||||
# Generate detailed changelog if requested
|
||||
if args.changelog:
|
||||
changelog_file = generate_detailed_changelog(pr_details, args.from_tag, args.to_tag, args.changelog)
|
||||
print(f"Detailed changelog written to {changelog_file}")
|
||||
|
||||
if args.verbose:
|
||||
print("Done!")
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@ -0,0 +1,284 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Generate structured release notes from PR data.
|
||||
|
||||
This script reads PR data collected by collect_pr_data.py and generates
|
||||
release notes in the style of ProxySQL v3.0.3 release notes, with
|
||||
descriptive grouping and individual commit references.
|
||||
"""
|
||||
|
||||
import json
|
||||
import argparse
|
||||
import sys
|
||||
from collections import defaultdict
|
||||
|
||||
|
||||
def load_pr_data(filename):
|
||||
"""Load PR data from JSON file."""
|
||||
with open(filename, 'r') as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def categorize_pr(pr):
|
||||
"""Categorize PR based on title, labels, and content."""
|
||||
title = pr['title'].lower()
|
||||
body = pr.get('body', '').lower()
|
||||
labels = [l['name'].lower() for l in pr.get('labels', [])]
|
||||
|
||||
# Check for documentation PRs
|
||||
if any(word in title for word in ['doc', 'documentation', 'doxygen']):
|
||||
return 'Documentation'
|
||||
if any(word in body for word in ['doc', 'documentation', 'doxygen']):
|
||||
return 'Documentation'
|
||||
|
||||
# Check for test PRs
|
||||
if any(word in title for word in ['test', 'tap', 'regression']):
|
||||
return 'Testing'
|
||||
if any(word in body for word in ['test', 'tap', 'regression']):
|
||||
return 'Testing'
|
||||
|
||||
# Check for build/packaging
|
||||
if any(word in title for word in ['build', 'package', 'opensuse', 'docker', 'rpm', 'deb']):
|
||||
return 'Build/Packaging'
|
||||
|
||||
# Check for bug fixes
|
||||
if any(word in title for word in ['fix', 'bug', 'issue', 'crash', 'vulnerability', 'error']):
|
||||
return 'Bug Fixes'
|
||||
|
||||
# Check for PostgreSQL
|
||||
if any(word in title for word in ['postgresql', 'pgsql', 'pg']):
|
||||
return 'PostgreSQL'
|
||||
if any(word in body for word in ['postgresql', 'pgsql', 'pg']):
|
||||
return 'PostgreSQL'
|
||||
|
||||
# Check for MySQL
|
||||
if any(word in title for word in ['mysql']):
|
||||
return 'MySQL'
|
||||
if any(word in body for word in ['mysql']):
|
||||
return 'MySQL'
|
||||
|
||||
# Check for monitoring
|
||||
if any(word in title for word in ['monitor', 'metric', 'log', 'ping', 'keepalive']):
|
||||
return 'Monitoring'
|
||||
|
||||
# Check for performance
|
||||
if any(word in title for word in ['performance', 'optimize', 'refactor', 'lock-free', 'gtid']):
|
||||
return 'Performance'
|
||||
|
||||
# Default
|
||||
return 'Other'
|
||||
|
||||
|
||||
def extract_commits_info(pr):
|
||||
"""Extract structured commit information from PR."""
|
||||
commits_info = []
|
||||
for commit in pr.get('commits', []):
|
||||
headline = commit.get('messageHeadline', '')
|
||||
oid = commit.get('oid', '')[:8] # Short hash
|
||||
if headline:
|
||||
commits_info.append((oid, headline))
|
||||
return commits_info
|
||||
|
||||
|
||||
def generate_release_notes(pr_data, output_file):
|
||||
"""Generate structured release notes."""
|
||||
# Categorize PRs
|
||||
categories = defaultdict(list)
|
||||
for pr in pr_data:
|
||||
cat = categorize_pr(pr)
|
||||
categories[cat].append(pr)
|
||||
|
||||
out_lines = []
|
||||
out_lines.append('# ProxySQL 3.0.4 Release Notes\n')
|
||||
out_lines.append('\n')
|
||||
out_lines.append('This release of ProxySQL 3.0.4 includes significant improvements ')
|
||||
out_lines.append('to PostgreSQL support, MySQL protocol handling, monitoring accuracy, ')
|
||||
out_lines.append('and security.\n')
|
||||
out_lines.append('\n')
|
||||
out_lines.append('Release commit: `faa64a570d19fe35af43494db0babdee3e3cdc89`\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
# Helper to format with backticks
|
||||
def add_backticks(text):
|
||||
# Simple heuristic: add backticks around likely technical terms
|
||||
# This is a placeholder - in production, use more sophisticated detection
|
||||
return text
|
||||
|
||||
# New Features section
|
||||
out_lines.append('## New Features:\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
# PostgreSQL features
|
||||
postgres_prs = [p for p in categories['PostgreSQL'] if 'fix' not in p['title'].lower()]
|
||||
if postgres_prs:
|
||||
out_lines.append('### PostgreSQL Improvements:\n')
|
||||
out_lines.append('\n')
|
||||
for pr in postgres_prs:
|
||||
commits = extract_commits_info(pr)
|
||||
if commits:
|
||||
for oid, headline in commits:
|
||||
out_lines.append(f'- {headline} ({oid}, #{pr["number"]})\n')
|
||||
else:
|
||||
out_lines.append(f'- {pr["title"]} ({pr.get("merge_hash", "")[:8]}, #{pr["number"]})\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
# MySQL features
|
||||
mysql_prs = [p for p in categories['MySQL'] if 'fix' not in p['title'].lower()]
|
||||
if mysql_prs:
|
||||
out_lines.append('### MySQL Protocol Enhancements:\n')
|
||||
out_lines.append('\n')
|
||||
for pr in mysql_prs:
|
||||
commits = extract_commits_info(pr)
|
||||
if commits:
|
||||
for oid, headline in commits:
|
||||
out_lines.append(f'- {headline} ({oid}, #{pr["number"]})\n')
|
||||
else:
|
||||
out_lines.append(f'- {pr["title"]} ({pr.get("merge_hash", "")[:8]}, #{pr["number"]})\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
# Monitoring features
|
||||
monitoring_prs = [p for p in categories['Monitoring'] if 'fix' not in p['title'].lower()]
|
||||
if monitoring_prs:
|
||||
out_lines.append('### Monitoring & Diagnostics:\n')
|
||||
out_lines.append('\n')
|
||||
for pr in monitoring_prs:
|
||||
commits = extract_commits_info(pr)
|
||||
if commits:
|
||||
for oid, headline in commits:
|
||||
out_lines.append(f'- {headline} ({oid}, #{pr["number"]})\n')
|
||||
else:
|
||||
out_lines.append(f'- {pr["title"]} ({pr.get("merge_hash", "")[:8]}, #{pr["number"]})\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
# Bug Fixes section
|
||||
bug_fixes = []
|
||||
for cat in ['PostgreSQL', 'MySQL', 'Monitoring', 'Other']:
|
||||
if cat in categories:
|
||||
bug_fixes.extend([p for p in categories[cat] if 'fix' in p['title'].lower()])
|
||||
|
||||
if bug_fixes:
|
||||
out_lines.append('## Bug Fixes:\n')
|
||||
out_lines.append('\n')
|
||||
for pr in bug_fixes:
|
||||
commits = extract_commits_info(pr)
|
||||
if commits:
|
||||
for oid, headline in commits:
|
||||
out_lines.append(f'- {headline} ({oid}, #{pr["number"]})\n')
|
||||
else:
|
||||
out_lines.append(f'- {pr["title"]} ({pr.get("merge_hash", "")[:8]}, #{pr["number"]})\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
# Performance improvements
|
||||
perf_prs = categories.get('Performance', [])
|
||||
if perf_prs:
|
||||
out_lines.append('## Improvements:\n')
|
||||
out_lines.append('\n')
|
||||
out_lines.append('### Performance Optimizations:\n')
|
||||
out_lines.append('\n')
|
||||
for pr in perf_prs:
|
||||
commits = extract_commits_info(pr)
|
||||
if commits:
|
||||
for oid, headline in commits:
|
||||
out_lines.append(f'- {headline} ({oid}, #{pr["number"]})\n')
|
||||
else:
|
||||
out_lines.append(f'- {pr["title"]} ({pr.get("merge_hash", "")[:8]}, #{pr["number"]})\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
# Documentation
|
||||
doc_prs = categories.get('Documentation', [])
|
||||
if doc_prs:
|
||||
out_lines.append('## Documentation:\n')
|
||||
out_lines.append('\n')
|
||||
for pr in doc_prs:
|
||||
commits = extract_commits_info(pr)
|
||||
if commits:
|
||||
for oid, headline in commits:
|
||||
out_lines.append(f'- {headline} ({oid}, #{pr["number"]})\n')
|
||||
else:
|
||||
out_lines.append(f'- {pr["title"]} ({pr.get("merge_hash", "")[:8]}, #{pr["number"]})\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
# Testing
|
||||
test_prs = categories.get('Testing', [])
|
||||
if test_prs:
|
||||
out_lines.append('## Testing:\n')
|
||||
out_lines.append('\n')
|
||||
for pr in test_prs:
|
||||
commits = extract_commits_info(pr)
|
||||
if commits:
|
||||
for oid, headline in commits:
|
||||
out_lines.append(f'- {headline} ({oid}, #{pr["number"]})\n')
|
||||
else:
|
||||
out_lines.append(f'- {pr["title"]} ({pr.get("merge_hash", "")[:8]}, #{pr["number"]})\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
# Build/Packaging
|
||||
build_prs = categories.get('Build/Packaging', [])
|
||||
if build_prs:
|
||||
out_lines.append('## Build/Packaging:\n')
|
||||
out_lines.append('\n')
|
||||
for pr in build_prs:
|
||||
commits = extract_commits_info(pr)
|
||||
if commits:
|
||||
for oid, headline in commits:
|
||||
out_lines.append(f'- {headline} ({oid}, #{pr["number"]})\n')
|
||||
else:
|
||||
out_lines.append(f'- {pr["title"]} ({pr.get("merge_hash", "")[:8]}, #{pr["number"]})\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
# Other
|
||||
other_prs = categories.get('Other', [])
|
||||
if other_prs:
|
||||
out_lines.append('## Other Changes:\n')
|
||||
out_lines.append('\n')
|
||||
for pr in other_prs:
|
||||
commits = extract_commits_info(pr)
|
||||
if commits:
|
||||
for oid, headline in commits:
|
||||
out_lines.append(f'- {headline} ({oid}, #{pr["number"]})\n')
|
||||
else:
|
||||
out_lines.append(f'- {pr["title"]} ({pr.get("merge_hash", "")[:8]}, #{pr["number"]})\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
out_lines.append('## Hashes\n')
|
||||
out_lines.append('\n')
|
||||
out_lines.append('The release commit is: `faa64a570d19fe35af43494db0babdee3e3cdc89`\n')
|
||||
out_lines.append('\n')
|
||||
|
||||
with open(output_file, 'w') as f:
|
||||
f.writelines(out_lines)
|
||||
|
||||
return output_file
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Generate structured release notes from PR data.',
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
%(prog)s --input pr-data.json --output release-notes.md
|
||||
"""
|
||||
)
|
||||
parser.add_argument('--input', '-i', required=True, help='Input JSON file with PR data')
|
||||
parser.add_argument('--output', '-o', default='release-notes-structured.md', help='Output file')
|
||||
parser.add_argument('--verbose', '-v', action='store_true', help='Verbose output')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.verbose:
|
||||
print(f"Loading PR data from {args.input}")
|
||||
|
||||
pr_data = load_pr_data(args.input)
|
||||
|
||||
if args.verbose:
|
||||
print(f"Processing {len(pr_data)} PRs")
|
||||
|
||||
output_file = generate_release_notes(pr_data, args.output)
|
||||
|
||||
if args.verbose:
|
||||
print(f"Structured release notes written to {output_file}")
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@ -0,0 +1,274 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Orchestrate release notes generation workflow.
|
||||
|
||||
This script runs all required steps to prepare data for LLM analysis,
|
||||
then generates a comprehensive prompt for the LLM to create release notes
|
||||
and changelogs.
|
||||
|
||||
Workflow:
|
||||
1. Collect PR data from GitHub
|
||||
2. Generate structured commit data
|
||||
3. Create analysis files
|
||||
4. Generate LLM prompt with instructions
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
import json
|
||||
import argparse
|
||||
import sys
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def run_cmd(cmd, verbose=False):
|
||||
"""Run command and return output."""
|
||||
if verbose:
|
||||
print(f"Running: {cmd}")
|
||||
try:
|
||||
result = subprocess.run(cmd, shell=True, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
print(f"Error running command: {cmd}", file=sys.stderr)
|
||||
print(f"Error output: {result.stderr}", file=sys.stderr)
|
||||
return None
|
||||
return result.stdout.strip()
|
||||
except Exception as e:
|
||||
print(f"Exception running command {cmd}: {e}", file=sys.stderr)
|
||||
return None
|
||||
|
||||
|
||||
def generate_llm_prompt(from_tag, to_tag, data_files, output_dir, verbose=False):
|
||||
"""Generate comprehensive prompt for LLM."""
|
||||
|
||||
# Read some data to include in prompt
|
||||
pr_summary = ""
|
||||
try:
|
||||
with open(data_files['pr_summary'], 'r') as f:
|
||||
pr_summary = f.read()[:2000] # First 2000 chars
|
||||
except:
|
||||
pass
|
||||
|
||||
prompt = f"""
|
||||
# ProxySQL Release Notes Generation Task
|
||||
|
||||
## Context
|
||||
You need to generate release notes and changelogs for ProxySQL version {to_tag} (changes since {from_tag}).
|
||||
|
||||
## Available Data Files
|
||||
The following files have been prepared for your analysis:
|
||||
|
||||
1. **PR Data**: `{data_files['pr_data']}` - JSON with all PR details (titles, descriptions, labels, commits)
|
||||
2. **PR Summary**: `{data_files['pr_summary']}` - Markdown summary of all PRs
|
||||
3. **Structured Notes**: `{data_files['structured_notes']}` - Commit-level organized data
|
||||
4. **Commit Categorization**: `{data_files['commit_categories']}` - Commits categorized by type
|
||||
|
||||
## Task Requirements
|
||||
|
||||
### 1. Generate Release Notes (like ProxySQL v3.0.3 format)
|
||||
- **Descriptive content**: Not just PR titles, but explanations of what each feature/fix does and why it matters
|
||||
- **Logical grouping**: Organize under categories like:
|
||||
- PostgreSQL Improvements
|
||||
- MySQL Protocol Enhancements
|
||||
- Monitoring & Diagnostics
|
||||
- Bug Fixes (with subcategories: MySQL, PostgreSQL, Monitoring, Security)
|
||||
- Performance Optimizations
|
||||
- Documentation
|
||||
- Testing
|
||||
- Build/Packaging
|
||||
- Other Changes
|
||||
- **Backtick formatting**: Use `backticks` around all technical terms:
|
||||
- Function names: `Read_Global_Variables_from_configfile()`
|
||||
- Variable names: `wait_timeout`, `cur_cmd_cmnt`
|
||||
- SQL queries: `SELECT @@version`, `SELECT VERSION()`
|
||||
- Protocol commands: `COM_PING`, `CLIENT_DEPRECATE_EOF`
|
||||
- Configuration options: `cache_empty_result=0`
|
||||
- Metrics: `PgSQL_Monitor_ssl_connections_OK`
|
||||
- **Commit references**: Include relevant commit hashes (short form) and PR numbers
|
||||
- **Remove WIP/skip-ci tags**: Make all entries production-ready
|
||||
- **Include release hash**: The final commit is `faa64a570d19fe35af43494db0babdee3e3cdc89`
|
||||
|
||||
### 2. Generate Detailed Changelog
|
||||
- List all changes with commit hashes and PR references
|
||||
- Include brief descriptions from commit messages
|
||||
- Categorize changes for easy reference
|
||||
|
||||
### 3. Generate Commit List (optional)
|
||||
- Complete list of all commits since {from_tag}
|
||||
|
||||
## Example Structure (from ProxySQL 3.0.3)
|
||||
```
|
||||
# ProxySQL 3.0.3 Release Notes
|
||||
|
||||
This release of ProxySQL 3.0.3 includes a significant number of new features...
|
||||
|
||||
## New Features:
|
||||
|
||||
### PostgreSQL Extended Query Protocol Support:
|
||||
- Add PostgreSQL extended query (prepared statement) support (24fecc1f, #5044)
|
||||
- Lays groundwork for handling PostgreSQL extended query protocol...
|
||||
- Added `Describe` message handling (a741598a, #5044)
|
||||
- Added `Close` statement handling (4d0618c2, #5044)
|
||||
|
||||
### Build System & Dependencies:
|
||||
- Upgrade `coredumper` to Percona fork hash `8f2623b` (a315f128, #5171)
|
||||
- Upgrade `curl` to v8.16.0 (40414de1, #5154)
|
||||
```
|
||||
|
||||
## Your Output Should Be:
|
||||
1. **`ProxySQL-{to_tag}-Release-Notes.md`** - Main release notes
|
||||
2. **`CHANGELOG-{to_tag}-detailed.md`** - Detailed changelog
|
||||
3. **`CHANGELOG-{to_tag}-commits.md`** - Complete commit list (optional)
|
||||
|
||||
## Analysis Approach
|
||||
1. Review the PR data to understand scope and significance of changes
|
||||
2. Identify major themes and group related changes
|
||||
3. Write descriptive explanations, not just copy titles
|
||||
4. Apply backtick formatting consistently
|
||||
5. Verify all technical terms are properly formatted
|
||||
|
||||
## Available Data Preview
|
||||
{pr_summary[:500]}...
|
||||
"""
|
||||
|
||||
prompt_file = os.path.join(output_dir, f"llm-prompt-{to_tag}.md")
|
||||
with open(prompt_file, 'w') as f:
|
||||
f.write(prompt)
|
||||
|
||||
if verbose:
|
||||
print(f"LLM prompt written to {prompt_file}")
|
||||
|
||||
return prompt_file
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Orchestrate release notes generation workflow.',
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
%(prog)s --from-tag v3.0.3 --to-tag v3.0.4 --output-dir release-data
|
||||
%(prog)s --from-tag v3.0.3 --to-tag v3.0.4 --verbose
|
||||
"""
|
||||
)
|
||||
parser.add_argument('--from-tag', required=True, help='Starting tag/branch')
|
||||
parser.add_argument('--to-tag', required=True, help='Ending tag/branch')
|
||||
parser.add_argument('--output-dir', default='release-data', help='Output directory for data files')
|
||||
parser.add_argument('--verbose', '-v', action='store_true', help='Verbose output')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Create output directory
|
||||
output_dir = Path(args.output_dir)
|
||||
output_dir.mkdir(exist_ok=True)
|
||||
|
||||
if args.verbose:
|
||||
print(f"Starting release notes workflow for {args.from_tag} to {args.to_tag}")
|
||||
print(f"Output directory: {output_dir}")
|
||||
|
||||
data_files = {}
|
||||
|
||||
# Step 1: Collect PR data
|
||||
if args.verbose:
|
||||
print("\n1. Collecting PR data...")
|
||||
|
||||
pr_data_file = output_dir / f"pr-data-{args.to_tag}.json"
|
||||
pr_summary_file = output_dir / f"pr-summary-{args.to_tag}.md"
|
||||
|
||||
cmd = f"python {Path(__file__).parent}/collect_pr_data.py --from-tag {args.from_tag} --to-tag {args.to_tag} --output {pr_data_file} --verbose"
|
||||
if not args.verbose:
|
||||
cmd = cmd.replace(" --verbose", "")
|
||||
|
||||
result = run_cmd(cmd, args.verbose)
|
||||
if result is None:
|
||||
print("Failed to collect PR data", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
data_files['pr_data'] = str(pr_data_file)
|
||||
data_files['pr_summary'] = str(pr_summary_file)
|
||||
|
||||
# Step 2: Generate structured notes
|
||||
if args.verbose:
|
||||
print("\n2. Generating structured notes...")
|
||||
|
||||
structured_file = output_dir / f"structured-notes-{args.to_tag}.md"
|
||||
cmd = f"python {Path(__file__).parent}/generate_structured_notes.py --input {pr_data_file} --output {structured_file}"
|
||||
if args.verbose:
|
||||
cmd += " --verbose"
|
||||
|
||||
result = run_cmd(cmd, args.verbose)
|
||||
if result is None:
|
||||
print("Failed to generate structured notes", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
data_files['structured_notes'] = str(structured_file)
|
||||
|
||||
# Step 3: Categorize commits
|
||||
if args.verbose:
|
||||
print("\n3. Categorizing commits...")
|
||||
|
||||
commit_cat_file = output_dir / f"commit-categories-{args.to_tag}.md"
|
||||
cmd = f"python {Path(__file__).parent}/categorize_commits.py --from-tag {args.from_tag} --to-tag {args.to_tag} --output-format markdown > {commit_cat_file}"
|
||||
|
||||
result = run_cmd(cmd, args.verbose)
|
||||
if result is None:
|
||||
print("Failed to categorize commits", file=sys.stderr)
|
||||
|
||||
data_files['commit_categories'] = str(commit_cat_file)
|
||||
|
||||
# Step 4: Generate LLM prompt
|
||||
if args.verbose:
|
||||
print("\n4. Generating LLM prompt...")
|
||||
|
||||
prompt_file = generate_llm_prompt(
|
||||
args.from_tag,
|
||||
args.to_tag,
|
||||
data_files,
|
||||
output_dir,
|
||||
args.verbose
|
||||
)
|
||||
|
||||
# Step 5: Create workflow summary
|
||||
summary = f"""
|
||||
# Release Notes Workflow Summary
|
||||
|
||||
## Generated Files
|
||||
- PR Data: {data_files['pr_data']}
|
||||
- PR Summary: {data_files['pr_summary']}
|
||||
- Structured Notes: {data_files['structured_notes']}
|
||||
- Commit Categories: {data_files.get('commit_categories', 'N/A')}
|
||||
- LLM Prompt: {prompt_file}
|
||||
|
||||
## Next Steps
|
||||
1. Review the generated files in {output_dir}
|
||||
2. Use the LLM prompt to generate release notes
|
||||
3. The LLM should create:
|
||||
- `ProxySQL-{args.to_tag}-Release-Notes.md`
|
||||
- `CHANGELOG-{args.to_tag}-detailed.md`
|
||||
- `CHANGELOG-{args.to_tag}-commits.md` (optional)
|
||||
|
||||
## LLM Instructions
|
||||
Provide the LLM with:
|
||||
1. Access to all files in {output_dir}
|
||||
2. The prompt: {prompt_file}
|
||||
3. Instructions to write descriptive release notes with backtick formatting
|
||||
|
||||
## Verification Checklist
|
||||
- [ ] All technical terms have backticks
|
||||
- [ ] No WIP/skip-ci tags remain
|
||||
- [ ] Changes are grouped logically
|
||||
- [ ] Descriptions explain what/why, not just copy titles
|
||||
- [ ] Commit hashes and PR numbers are referenced
|
||||
"""
|
||||
|
||||
summary_file = output_dir / "workflow-summary.md"
|
||||
with open(summary_file, 'w') as f:
|
||||
f.write(summary)
|
||||
|
||||
if args.verbose:
|
||||
print(f"\nWorkflow summary written to {summary_file}")
|
||||
print("\n✅ Workflow completed!")
|
||||
print(f"\nNext: Provide the LLM with files in {output_dir} and prompt: {prompt_file}")
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@ -0,0 +1,80 @@
|
||||
#!/usr/bin/env python3
|
||||
import sys
|
||||
import re
|
||||
|
||||
# Categories mapping keywords
|
||||
categories = {
|
||||
'Bug Fix': ['fix', 'bug', 'issue', 'crash', 'vulnerability', 'error', 'wrong', 'incorrect', 'failure', 'broken'],
|
||||
'New Feature': ['add', 'new', 'support', 'implement', 'feature', 'introduce', 'enable'],
|
||||
'Improvement': ['improve', 'optimize', 'enhance', 'speed', 'performance', 'better', 'reduce', 'faster', 'efficient'],
|
||||
'Documentation': ['doc', 'documentation', 'comment', 'doxygen', 'readme'],
|
||||
'Testing': ['test', 'tap', 'regression', 'validation'],
|
||||
'Build/Packaging': ['build', 'package', 'makefile', 'cmake', 'docker', 'opensuse', 'deb', 'rpm'],
|
||||
'Refactoring': ['refactor', 'cleanup', 'restructure', 'reorganize', 'rename'],
|
||||
'Security': ['security', 'injection', 'vulnerability', 'secure', 'sanitize'],
|
||||
'Monitoring': ['monitor', 'metric', 'log', 'warning', 'alert'],
|
||||
'PostgreSQL': ['postgresql', 'pgsql', 'pg'],
|
||||
'MySQL': ['mysql'],
|
||||
}
|
||||
|
||||
def categorize_commit(message):
|
||||
msg_lower = message.lower()
|
||||
scores = {}
|
||||
for cat, keywords in categories.items():
|
||||
score = 0
|
||||
for kw in keywords:
|
||||
if re.search(r'\b' + re.escape(kw) + r'\b', msg_lower):
|
||||
score += 1
|
||||
if score:
|
||||
scores[cat] = score
|
||||
if scores:
|
||||
# return max score category
|
||||
return max(scores.items(), key=lambda x: x[1])[0]
|
||||
return 'Other'
|
||||
|
||||
def main():
|
||||
with open('/tmp/commits.txt', 'r') as f:
|
||||
lines = f.readlines()
|
||||
|
||||
commits = []
|
||||
current = []
|
||||
for line in lines:
|
||||
line = line.rstrip('\n')
|
||||
if line and '|' in line and line.count('|') >= 2:
|
||||
if current:
|
||||
commits.append(''.join(current))
|
||||
current = []
|
||||
current.append(line + '\n')
|
||||
if current:
|
||||
commits.append(''.join(current))
|
||||
|
||||
categorized = {}
|
||||
for commit in commits:
|
||||
parts = commit.split('|', 2)
|
||||
if len(parts) < 3:
|
||||
continue
|
||||
hash_, subject, body = parts[0], parts[1], parts[2]
|
||||
full_msg = subject + ' ' + body
|
||||
cat = categorize_commit(full_msg)
|
||||
categorized.setdefault(cat, []).append((hash_, subject, body))
|
||||
|
||||
# Print categorized
|
||||
for cat in sorted(categorized.keys()):
|
||||
print(f'\n## {cat}\n')
|
||||
for hash_, subject, body in categorized[cat]:
|
||||
# truncate subject if too long
|
||||
print(f'- {hash_[:8]} {subject}')
|
||||
# print body lines indented
|
||||
if body.strip():
|
||||
for line in body.strip().split('\n'):
|
||||
if line.strip():
|
||||
print(f' {line.strip()}')
|
||||
print()
|
||||
|
||||
# Print stats
|
||||
print('\n---\n')
|
||||
for cat in sorted(categorized.keys()):
|
||||
print(f'{cat}: {len(categorized[cat])}')
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Loading…
Reference in new issue