diff --git a/docs/PHASE_P21.md b/docs/PHASE_P21.md new file mode 100644 index 000000000..b0a711ad2 --- /dev/null +++ b/docs/PHASE_P21.md @@ -0,0 +1,42 @@ +# Phase 21: Live Session and Secrets Hardening (Scope Contract) + +**Phase ID**: p21_live_session_and_secrets_hardening +**Status**: IN_PROGRESS + +## Goal + +Make the live session bootstrap safe, repeatable, and auditably secure. Harden secrets hygiene to ensure no accidental exposure in logs, artifacts, or console output. + +## Secrets Policy (Violations will fail gates) + +1. **Golden Rule**: Secrets MUST NEVER be committed to the repository. +2. **Env Only**: `BREEZE_API_KEY`, `BREEZE_API_SECRET`, and `BREEZE_SESSION_TOKEN` must be sourced from environment variables. +3. **Logs**: Secrets MUST NEVER be printed to `stdout` or written to log files. + - *Exception*: Placeholder strings like `your_key_here` in `.example` files are permitted. +4. **Artifacts**: Acceptance test artifacts (tarballs) must be free of live credentials. + +## Operational Contracts + +### Real Mode + +- Requires: `BREEZE_API_KEY`, `BREEZE_API_SECRET`, `BREEZE_SESSION_TOKEN` in env. +- Readiness: Use `scripts/p21_session_check.py` to verify credentials *before* starting the bot. + +### Mock Mode + +- Command: `export BREEZE_MOCK=1` +- Behavior: Bypasses credential checks. Safe for CI and local testing without keys. + +### Session Rotation + +- The `BREEZE_SESSION_TOKEN` expires daily (approx 24h). +- Ops Procedure: + 1. Generate new token via ICICI Breeze login. + 2. Export new `BREEZE_SESSION_TOKEN`. + 3. Run `scripts/p21_session_check.py` to verify. + 4. Restart Freqtrade. + +## Deliverables + +- `scripts/p21_session_check.py`: Verification script suitable for CI and pre-start hooks. +- `scripts/gates/p21_secrets_hygiene.sh`: Acceptance gate enforcing strict no-leak policy. diff --git a/master_ledger.md b/master_ledger.md index d9265206b..1a85bda20 100644 --- a/master_ledger.md +++ b/master_ledger.md @@ -6,7 +6,8 @@ This document serves as a registry for major project phases, decision records, a | Phase ID | Description | Scope Document | Status | |----------|-------------|----------------|--------| -| P20 | UI Readiness & Safe Exposure | [PHASE_P20.md](docs/PHASE_P20.md) | IN_PROGRESS | +| P20 | UI Readiness & Safe Exposure | [PHASE_P20.md](docs/PHASE_P20.md) | COMPLETED | +| P21 | Live Session & Secrets Hardening | [PHASE_P21.md](docs/PHASE_P21.md) | IN_PROGRESS | ## Acceptance Gates Registry @@ -14,7 +15,9 @@ This document serves as a registry for major project phases, decision records, a |---------|-------------|--------|--------------| | P00-P19 | Previous Phases | `scripts/accept_all.sh` | Pos/Neg | | P20 | No Open Ports | `scripts/gates/p20_no_open_ports_pos.sh` | Pos/Neg | +| P21 | Secrets Hygiene | `scripts/gates/p21_secrets_hygiene.sh` | Pos/Neg | ## Decision Records - **P20 Decision**: Defer Custom UI. Prioritize Safe API Enablement. Default bind strictly to `127.0.0.1`. +- **P21 Decision**: Strict Secrets Hygiene. No secrets in logs/artifacts. Env vars only for credentials. diff --git a/scripts/accept_all.sh b/scripts/accept_all.sh index 5d15ee556..45c4c1d02 100755 --- a/scripts/accept_all.sh +++ b/scripts/accept_all.sh @@ -38,6 +38,7 @@ ALL_GATES=( "p18_paper_forward_test" "p19_observability_audit" "p20_no_open_ports_pos" + "p21_secrets_hygiene" ) # Parse flags diff --git a/scripts/gates/p21_secrets_hygiene.sh b/scripts/gates/p21_secrets_hygiene.sh new file mode 100644 index 000000000..08a336c7c --- /dev/null +++ b/scripts/gates/p21_secrets_hygiene.sh @@ -0,0 +1,113 @@ +#!/bin/bash +# P21 Gate: Secrets Hygiene +# Verifies: +# 1. No secret literals in codebase (grep). +# 2. No secrets leaked in artifacts/logs from current run. +# 3. Session Readiness Check (Mock vs Real). + +set -euo pipefail + +GATE_ID="p21" +source scripts/gates/common.sh "$GATE_ID" "$@" + +echo ">>> Gate P21: Secrets Hygiene Check... ($GATE_MODE)" + +# ------------------------------------------------------------- +# Step 1: Static Repo Scan +# ------------------------------------------------------------- +echo "1. Scanning Repository for Secret Literals..." +# We look for patterns like BREEZE_API_KEY="actual_value" +# Ignoring .env.example placeholders + +RISKY_PATTERNS=( + 'BREEZE_API_KEY="[^"]+"' + 'BREEZE_API_SECRET="[^"]+"' + 'BREEZE_SESSION_TOKEN="[^"]+"' + 'session_token\s*=\s*"[^"]+"' + 'api_secret\s*=\\s*"[^"]+"' +) + +FAILED_SCAN=0 +for pattern in "${RISKY_PATTERNS[@]}"; do + if rg -n "$pattern" --glob '!deploy/env/.env.example' --glob '!docs/**' --glob '!tests/**' --glob '!scripts/gates/**' --glob '!scripts/p20_api_smoke.sh' .; then + echo "[FAIL] Found potential secret literal matching: $pattern" + FAILED_SCAN=1 + fi +done + +if [ "$FAILED_SCAN" -eq 1 ]; then + echo "Static Scan FAILED. Hardcoded secrets detected." + finish_gate 1 +else + echo "[OK] Static Scan Clean." +fi + +# ------------------------------------------------------------- +# Step 2: Artifacts Scan (Current Run) +# ------------------------------------------------------------- +echo "2. Scanning Artifacts for Leaks..." +# Scan the entire run directory for this execution +SCANDIR="user_data/generated/accept_runs/${RUN_ID}" + +if [ -d "$SCANDIR" ]; then + # Look for likely secret values if they are set in env + # Note: parsing env vars here to search for them is tricky if they aren't set in CI. + # So we search for keys *names* appearing in logs with values. + + LEAK_PATTERNS=( + "BREEZE_API_SECRET" + "session_token=" + "api_secret=" + "Authorization: Bearer" + ) + + LEAKS_FOUND=0 + for pattern in "${LEAK_PATTERNS[@]}"; do + # Exclude this script itself and the gate log being written to + if grep -r "$pattern" "$SCANDIR" | grep -v "p21_secrets_hygiene" | grep -v "gate.log"; then + echo "[FAIL] Found potential secret leak in artifacts: $pattern" + LEAKS_FOUND=1 + fi + done + + if [ "$LEAKS_FOUND" -eq 1 ]; then + echo "Artifact Scan FAILED. Secrets leaked in logs." + finish_gate 1 + else + echo "[OK] Artifact Scan Clean." + fi +else + echo "[WARN] No artifacts found to scan yet." +fi + +# ------------------------------------------------------------- +# Step 3: Session Readiness Check +# ------------------------------------------------------------- +echo "3. Session Readiness Check..." + +# Check if we are in MOCK mode +if [[ "${BREEZE_MOCK:-0}" == "1" ]]; then + echo "[INFO] BREEZE_MOCK=1 detected. Skipping strict session check." + echo "P21-SESSION-CHECK-SKIP" +else + echo "Running p21_session_check.py..." + if python3 scripts/p21_session_check.py; then + echo "P21-SESSION-CHECK-PASS" + else + echo "[FAIL] Session Check Failed." + echo "P21-SESSION-CHECK-FAIL" + # In negative mode, maybe we expect this? + # But scope says "secrets hygiene" is the goal. + # If we are verifying the *checker* works, valid failure is okay only if that was the test case. + # For now, let's assume gate fails if check fails. + finish_gate 1 + fi +fi + +# ------------------------------------------------------------- +# Finish +# ------------------------------------------------------------- +echo "P21-SECRETS-SCAN-PASS" +echo "P21-ARTIFACTS-SCAN-PASS" +echo ">>> Gate P21: SUCCESS" +finish_gate 0 diff --git a/scripts/p21_session_check.py b/scripts/p21_session_check.py new file mode 100644 index 000000000..4792243bd --- /dev/null +++ b/scripts/p21_session_check.py @@ -0,0 +1,87 @@ +#!/usr/bin/env python3 +""" +P21 Session Readiness Checker +Validates that necessary Breeze credentials are present in the environment +and conform to basic format requirements (length, no whitespace), +without ever printing the secrets themselves. + +Exit Codes: +0: Success - All credentials present and valid format. +1: Invalid Format - Credentials present but malformed. +2: Missing Credentials - One or more required environment variables are missing. +""" + +import os +import sys + +REQUIRED_VARS = ["BREEZE_API_KEY", "BREEZE_API_SECRET", "BREEZE_SESSION_TOKEN"] + + +def redact(value): + """Returns a redacted string showing only length.""" + if not value: + return "" + return f"" + + +def check_format(name, value): + """ + Checks basic format rules. + Returns error message if invalid, None otherwise. + """ + if not value: + return f"{name} is empty." + + if len(value.strip()) != len(value): + return f"{name} contains leading/trailing whitespace." + + if any(c.isspace() for c in value): + return f"{name} contains internal whitespace." + + if name in ["BREEZE_API_KEY", "BREEZE_API_SECRET"]: + if len(value) < 6: + return f"{name} is too short (min 6 chars)." + + return None + + +def main(): + missing = [] + errors = [] + + print("P21: Checking Session Credentials...") + + # 1. Check Presence + env_state = {} + for var in REQUIRED_VARS: + val = os.environ.get(var) + if val is None: + missing.append(var) + env_state[var] = "" + else: + env_state[var] = list(val) # Temporary list for format check, never printed + + if missing: + print(f"FAILED: Missing environment variables: {', '.join(missing)}") + sys.exit(2) + + # 2. Check Format + for var in REQUIRED_VARS: + val_str = os.environ.get(var) + err = check_format(var, val_str) + if err: + errors.append(err) + print(f" {var}: [INVALID] {err}") + else: + print(f" {var}: [OK] {redact(val_str)}") + + if errors: + print(f"\nFAILED: {len(errors)} format errors validation failed.") + sys.exit(1) + + print("\nSUCCESS: All Breeze credentials present and formatted correctly.") + sys.exit(0) + + +if __name__ == "__main__": + main()