aherman
/
freqtrade
mirror of https://github.com/freqtrade/freqtrade.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

edits to clarify backtesting analysis

Browse Source
pull/2108/head
Jonathan Raviotta 7 years ago
parent 2bc67b4a96
commit ccf3c69874
3 changed files with 151 additions and 76 deletions
Show Stats Download Patch File Download Diff File

66
docs/data-analysis.md
Unescape View File

@ -2,11 +2,33 @@
You can analyze the results of backtests and trading history easily using Jupyter notebooks. A sample notebook is located at `user_data/notebooks/analysis_example.ipynb`. For usage instructions, see [jupyter.org](https://jupyter.org/documentation).
## Example snippets
### Load backtest results into a pandas dataframe
```python
# Load backtest results
df = load_backtest_data("user_data/backtest_data/backtest-result.json")
# Show value-counts per pair
df.groupby("pair")["sell_reason"].value_counts()
```
### Load live trading results into a pandas dataframe
``` python
# Fetch trades from database
df = load_trades_from_db("sqlite:///tradesv3.sqlite")
# Display results
df.groupby("pair")["sell_reason"].value_counts()
```
## Strategy debugging example
Debugging a strategy can be time-consuming. FreqTrade offers helper functions to visualize raw data.
### Import requirements and define variables used in the script
### Import requirements and define variables used in analyses
```python
# Imports
@ -47,12 +69,6 @@ print("Loaded " + str(len(bt_data)) + f" rows of data for {pair} from {data_loca
### Load and run strategy
* Rerun each time the strategy file is changed
* Display the trade details. Note that using `data.head()` would also work, however most indicators have some "startup" data at the top of the dataframe.
Some possible problems:
* Columns with NaN values at the end of the dataframe
* Columns used in `crossed*()` functions with completely different units
```python
# Load strategy using values set above
@ -60,33 +76,31 @@ strategy = StrategyResolver({'strategy': strategy_name,
'user_data_dir': user_data_dir,
'strategy_path': strategy_location}).strategy
# Run strategy (just like in backtesting)
# Generate buy/sell signals using strategy
df = strategy.analyze_ticker(bt_data, {'pair': pair})
# Report results
print(f"Generated {df['buy'].sum()} buy signals")
data = df.set_index('date', drop=True)
data.tail()
```
### Load backtest results into a pandas dataframe
### Display the trade details
```python
# Load backtest results
df = load_backtest_data("user_data/backtest_data/backtest-result.json")
* Note that using `data.head()` would also work, however most indicators have some "startup" data at the top of the dataframe.
# Show value-counts per pair
df.groupby("pair")["sell_reason"].value_counts()
```
#### Some possible problems
### Load live trading results into a pandas dataframe
* Columns with NaN values at the end of the dataframe
* Columns used in `crossed*()` functions with completely different units
``` python
# Fetch trades from database
df = load_trades_from_db("sqlite:///tradesv3.sqlite")
#### Comparison with full backtest
# Display results
df.groupby("pair")["sell_reason"].value_counts()
having 200 buy signals as output for one pair from `analyze_ticker()` does not necessarily mean that 200 trades will be made during backtesting.
Assuming you use only one condition such as, `df['rsi'] < 30` as buy condition, this will generate multiple "buy" signals for each pair in sequence (until rsi returns > 29).
The bot will only buy on the first of these signals (and also only if a trade-slot ("max_open_trades") is still available), or on one of the middle signals, as soon as a "slot" becomes available.
```python
# Report results
print(f"Generated {df['buy'].sum()} buy signals")
data = df.set_index('date', drop=True)
data.tail()
```
Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data.

12
setup.py
Unescape View File

@ -25,7 +25,15 @@ develop = [
'pytest-random-order',
]
all_extra = api + plot + develop
jupyter = [
'jupyter',
'nbstripout',
'ipykernel',
'isort',
'yapf',
]
all_extra = api + plot + develop + jupyter
setup(name='freqtrade',
version=__version__,
@ -68,7 +76,7 @@ setup(name='freqtrade',
'dev': all_extra,
'plot': plot,
'all': all_extra,
'jupyter': [],
'jupyter': jupyter,
},
include_package_data=True,

149
user_data/notebooks/analysis_example.ipynb
Unescape View File

@ -4,7 +4,24 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Strategy debugging example"
"# Analyzing bot data\n",
"\n",
"You can analyze the results of backtests and trading history easily using Jupyter notebooks. A sample notebook is located at `user_data/notebooks/analysis_example.ipynb`. For usage instructions, see [jupyter.org](https://jupyter.org/documentation)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Imports\n",
"from pathlib import Path\n",
"import os\n",
"from freqtrade.data.history import load_pair_history\n",
"from freqtrade.resolvers import StrategyResolver\n",
"from freqtrade.data.btanalysis import load_backtest_data\n",
"from freqtrade.data.btanalysis import load_trades_from_db"
]
},
{
@ -15,11 +32,9 @@
"source": [
"# Change directory\n",
"# Define all paths relative to the project root shown in the cell output\n",
"import os\n",
"from pathlib import Path\n",
"try:\n",
"\tos.chdir(Path(os.getcwd(), '../..'))\n",
"\tprint(os.getcwd())\n",
"\tos.chdir(Path(Path.cwd(), '../..'))\n",
"\tprint(Path.cwd())\n",
"except:\n",
"\tpass"
]
@ -28,7 +43,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Import requirements and define variables used in the script"
"## Example snippets"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load backtest results into a pandas dataframe"
]
},
{
@ -37,13 +59,55 @@
"metadata": {},
"outputs": [],
"source": [
"# Imports\n",
"from pathlib import Path\n",
"from freqtrade.data.history import load_pair_history\n",
"from freqtrade.resolvers import StrategyResolver\n",
"from freqtrade.data.btanalysis import load_backtest_data\n",
"from freqtrade.data.btanalysis import load_trades_from_db\n",
"# Load backtest results\n",
"df = load_backtest_data(\"user_data/backtest_data/backtest-result.json\")\n",
"\n",
"# Show value-counts per pair\n",
"df.groupby(\"pair\")[\"sell_reason\"].value_counts()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load live trading results into a pandas dataframe"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Fetch trades from database\n",
"df = load_trades_from_db(\"sqlite:///tradesv3.sqlite\")\n",
"\n",
"# Display results\n",
"df.groupby(\"pair\")[\"sell_reason\"].value_counts()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Strategy debugging example\n",
"\n",
"Debugging a strategy can be time-consuming. FreqTrade offers helper functions to visualize raw data."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Import requirements and define variables used in analyses"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Define some constants\n",
"ticker_interval = \"1m\"\n",
"# Name of the strategy class\n",
@ -51,9 +115,9 @@
"# Path to user data\n",
"user_data_dir = 'user_data'\n",
"# Location of the strategy\n",
"strategy_location = Path(user_data_dir, 'strategies')\n",
"strategy_location = os.path.join(user_data_dir, 'strategies')\n",
"# Location of the data\n",
"data_location = Path(user_data_dir, 'data', 'binance')\n",
"data_location = os.path.join(user_data_dir, 'data', 'binance')\n",
"# Pair to analyze \n",
"# Only use one pair here\n",
"pair = \"BTC_USDT\""
@ -85,15 +149,8 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load and run strategy \n",
"\n",
"* Rerun each time the strategy file is changed\n",
"* Display the trade details. Note that using `data.head()` would also work, however most indicators have some \"startup\" data at the top of the dataframe.\n",
"\n",
"Some possible problems:\n",
"\n",
"* Columns with NaN values at the end of the dataframe\n",
"* Columns used in `crossed*()` functions with completely different units"
"### Load and run strategy\n",
"* Rerun each time the strategy file is changed"
]
},
{
@ -107,20 +164,28 @@
" 'user_data_dir': user_data_dir,\n",
" 'strategy_path': strategy_location}).strategy\n",
"\n",
"# Run strategy (just like in backtesting)\n",
"df = strategy.analyze_ticker(bt_data, {'pair': pair})\n",
"\n",
"# Report results\n",
"print(f\"Generated {df['buy'].sum()} buy signals\")\n",
"data = df.set_index('date', drop=True)\n",
"data.tail()"
"# Generate buy/sell signals using strategy\n",
"df = strategy.analyze_ticker(bt_data, {'pair': pair})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load backtest results into a pandas dataframe"
"### Display the trade details\n",
"* Note that using `data.head()` would also work, however most indicators have some \"startup\" data at the top of the dataframe.\n",
"\n",
"#### Some possible problems\n",
"\n",
"* Columns with NaN values at the end of the dataframe\n",
"* Columns used in `crossed*()` functions with completely different units\n",
"\n",
"#### Comparison with full backtest\n",
"\n",
"having 200 buy signals as output for one pair from `analyze_ticker()` does not necessarily mean that 200 trades will be made during backtesting.\n",
"\n",
"Assuming you use only one condition such as, `df['rsi'] < 30` as buy condition, this will generate multiple \"buy\" signals for each pair in sequence (until rsi returns > 29).\n",
"The bot will only buy on the first of these signals (and also only if a trade-slot (\"max_open_trades\") is still available), or on one of the middle signals, as soon as a \"slot\" becomes available.\n"
]
},
{
@ -129,18 +194,10 @@
"metadata": {},
"outputs": [],
"source": [
"# Load backtest results\n",
"df = load_backtest_data(\"user_data/backtest_data/backtest-result.json\")\n",
"\n",
"# Show value-counts per pair\n",
"df.groupby(\"pair\")[\"sell_reason\"].value_counts()\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load live trading results into a pandas dataframe"
"# Report results\n",
"print(f\"Generated {df['buy'].sum()} buy signals\")\n",
"data = df.set_index('date', drop=True)\n",
"data.tail()"
]
},
{
@ -149,11 +206,7 @@
"metadata": {},
"outputs": [],
"source": [
"# Fetch trades from database\n",
"df = load_trades_from_db(\"sqlite:///tradesv3.sqlite\")\n",
"\n",
"# Display results\n",
"df.groupby(\"pair\")[\"sell_reason\"].value_counts()"
"Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data."
]
}
],

Write Preview
Loading…
Cancel
Save