Features 
¶
On top of the features offered by the open-source vectorbt, VectorBT PRO implements the following major enhancements.
Info
To keep the page short, only some of the most interesting features of each release are showcased. The detailed release notes are available to subscribers only. If you're on the private website, go to Getting started → Release notes.
Tags are releases where features were introduced for the first time. Note that most features are continuously updated, thus the following examples are meant to be run with the latest vectorbt PRO version installed 
Imports required by the code snippets below:
>>> import numpy as np
>>> import pandas as pd
>>> from numba import njit
>>> from collections import namedtuple
>>> from itertools import combinations
>>> import vectorbtpro as vbt
>>> vbt.settings.set_theme("dark")
Index records ¶
- How do you backtest time and asset-anchored queries such as "Order X units of asset Y on date Z"? Normally, you'd have to construct a full array and set information manually. But now there's a less stressful way: thanks to the preparers and redesigned smart indexing you can provide all information in a compressed record format! Under the hood, the record array will be translated into a set of index dictionaries - one per argument.
Define an entry ladder using records
>>> data = vbt.YFData.fetch(["BTC-USD", "ETH-USD"], missing_index="drop")
>>> records = [
... dict(date="2022", symbol="BTC-USD", long_entry=True), # (1)!
... dict(date="2022", symbol="ETH-USD", short_entry=True),
... dict(row=-1, exit=True),
... ]
>>> pf = vbt.PF.from_signals(data, records=records) # (2)!
>>> pf.orders.records_readable
Order Id Column Signal Index Creation Index
0 0 BTC-USD 2022-01-01 00:00:00+00:00 2022-01-01 00:00:00+00:00 \
1 1 BTC-USD 2023-04-25 00:00:00+00:00 2023-04-25 00:00:00+00:00
2 0 ETH-USD 2022-01-01 00:00:00+00:00 2022-01-01 00:00:00+00:00
3 1 ETH-USD 2023-04-25 00:00:00+00:00 2023-04-25 00:00:00+00:00
Fill Index Size Price Fees Side Type
0 2022-01-01 00:00:00+00:00 0.002097 47686.812500 0.0 Buy Market \
1 2023-04-25 00:00:00+00:00 0.002097 27534.675781 0.0 Sell Market
2 2022-01-01 00:00:00+00:00 0.026527 3769.697021 0.0 Sell Market
3 2023-04-25 00:00:00+00:00 0.026527 1834.759644 0.0 Buy Market
Stop Type
0 None
1 None
2 None
3 None
- Every broadcastable argument is supported. Rows can be provided via "row", "index", "date", or "datetime". Columns can be provided via "col", "column", or "symbol". If a row or column is not provided, will set the entire row or column respectively. If neither of them is provided, will set the entire array. Rows and columns can be provided as integer positions, labels, datetimes, or even complex indexers!
- Arguments not used in records can still be provided as usual. Arguments used in records can also be provided to be used as defaults.
Portfolio preparers ¶
- When you pass an argument to a simulation method such as
Portfolio.from_signals, it must go through a complex preparation pipeline first to get a format suitable for Numba. Such a pipeline usually involves mapping of enums, broadcasting, data type checks, template substitution, and many other steps. To make vectorbtpro more transparent, this pipeline has been outsourced to a separate class so you now have the full control of arguments that are getting passed to the Numba functions! Moreover, you can extend the preparers to automatically prepare arguments for your own simulators.
Get a prepared argument, modify it, and use in a new simulation
>>> data = vbt.YFData.fetch("BTC-USD", end="2017-01")
>>> prep_result = vbt.PF.from_holding(
... data,
... stop_ladder="uniform",
... tp_stop=vbt.Param([
... [0.1, 0.2, 0.3, 0.4, 0.5],
... [0.4, 0.5, 0.6],
... ], keys=["tp_ladder_1", "tp_ladder_2"]),
... return_prep_result=True # (1)!
... )
>>> prep_result.target_args["tp_stop"] # (2)!
array([[0.1, 0.4],
[0.2, 0.5],
[0.3, 0.6],
[0.4, nan],
[0.5, nan]])
>>> new_tp_stop = prep_result.target_args["tp_stop"] + 0.1
>>> new_prep_result = prep_result.replace(target_args=dict(tp_stop=new_tp_stop), nested_=True) # (3)!
>>> new_prep_result.target_args["tp_stop"]
array([[0.2, 0.5],
[0.3, 0.6],
[0.4, 0.7],
[0.5, nan],
[0.6, nan]])
>>> pf = vbt.PF.from_signals(new_prep_result) # (4)!
>>> pf.total_return
tp_stop
tp_ladder_1 0.4
tp_ladder_2 0.6
Name: total_return, dtype: float64
>>> sim_out = new_prep_result.target_func(**new_prep_result.target_args) # (5)!
>>> pf = vbt.PF(order_records=sim_out, **new_prep_result.pf_args)
>>> pf.total_return
tp_stop
tp_ladder_1 0.4
tp_ladder_2 0.6
Name: total_return, dtype: float64
- Return the result of the preparation
- Contains two attributes: simulation arguments as
sim_argsand portfolio arguments aspf_args - Replace the argument. Since it resides in another dictionary (
sim_args), we need to enablenested_. The result is a new instance ofPFPrepResult. - Pass the new preparation result as the first argument to the base simulation method
- Or simulate manually!
Stop laddering ¶
- Stop laddering is a technique to incrementally move out of a position. Instead of providing a single stop value to close out the position, you can now provide an array of stop values, each removing a certain amount of the position when hit. This amount can be controlled by providing a differnet ladder mode. Furthermore, thanks to a new broadcasting feature that allows arrays to broadcast along just one axis, the stop values don't need to have the same shape as data. You can even provide stop arrays of different shapes as parameters!
Test two TP ladders
>>> data = vbt.YFData.fetch("BTC-USD", end="2017-01")
>>> pf = vbt.PF.from_holding(
... data,
... stop_ladder="uniform",
... tp_stop=vbt.Param([
... [0.1, 0.2, 0.3, 0.4, 0.5],
... [0.4, 0.5, 0.6],
... ], keys=["tp_ladder_1", "tp_ladder_2"])
... )
>>> pf.trades.plot(column="tp_ladder_1").show()
Indicator search¶
- VectorBT PRO implements or integrates a total of over 500 indicators, which makes it increasingly difficult to keep an overview. To make them more discoverable, there is a bunch of new methods to globally search for indicators.
List all moving average indicators
>>> vbt.IF.list_indicators("*ma")
[
'vbt:MA',
'talib:DEMA',
'talib:EMA',
'talib:KAMA',
'talib:MA',
...
'technical:ZEMA',
'technical:ZLEMA',
'technical:ZLHMA',
'technical:ZLMA'
]
>>> vbt.indicator("technical:ZLMA") # (1)!
vectorbtpro.indicators.factory.technical.ZLMA
- Same as
vbt.IF.get_indicator
Staticization¶
- One of the biggest limitations of Numba is that functions passed as arguments (i.e., callbacks) make the main function uncacheable, such that it has to be re-compiled in each new runtime, over and over again. Especially the performance of simulator functions takes a hit as they usually take up to a minute to compile. Gladly, we've got a new neat trick in our arsenal: "staticization". It works as follows. First, the source code of a function gets annotated with a special syntax. The annotated code then gets extracted (a.k.a. "cutting"
), modified into a cacheable version by removing any callbacks from the arguments, and saved to a Python file. Once the function gets called again, the cacheable version gets executed. Sounds complex? See below!
Define the signal function in a Python file, here signal_func_nb.py
import vectorbtpro as vbt
from numba import njit
@njit
def signal_func_nb(c, fast_sma, slow_sma): # (1)!
long = vbt.pf_nb.iter_crossed_above_nb(c, fast_sma, slow_sma)
short = vbt.pf_nb.iter_crossed_below_nb(c, fast_sma, slow_sma)
return long, False, short, False
- Make sure that the function is named exactly as the callback argument
Run a staticized simulation. Running the script again won't re-compile.
>>> data = vbt.YFData.fetch("BTC-USD")
>>> pf = vbt.PF.from_signals(
... data,
... signal_func_nb="signal_func_nb.py", # (1)!
... signal_args=(vbt.Rep("fast_sma"), vbt.Rep("slow_sma")),
... broadcast_named_args=dict(
... fast_sma=data.run("sma", 20, hide_params=True, unpack=True),
... slow_sma=data.run("sma", 50, hide_params=True, unpack=True)
... ),
... staticized=True # (2)!
... )
- Path to the module the function resides in
- Does all the magic
Position info¶
- Dynamic signal functions have now access to the current position information such as (open) P&L.
Enter randomly, exit randomly but only if in profit
>>> @njit
... def signal_func_nb(c, entries, exits):
... is_entry = vbt.pf_nb.select_nb(c, entries)
... is_exit = vbt.pf_nb.select_nb(c, exits)
... if is_entry:
... return True, False, False, False
... if is_exit:
... pos_info = c.last_pos_info[c.col]
... if pos_info["status"] == vbt.pf_enums.TradeStatus.Open:
... if pos_info["pnl"] >= 0:
... return False, True, False, False
... return False, False, False, False
>>> data = vbt.YFData.fetch("BTC-USD")
>>> entries, exits = data.run("RANDNX", n=10, unpack=True)
>>> pf = vbt.Portfolio.from_signals(
... data,
... signal_func_nb=signal_func_nb,
... signal_args=(vbt.Rep("entries"), vbt.Rep("exits")),
... broadcast_named_args=dict(entries=entries, exits=exits),
... jitted=False # (1)!
... )
>>> pf.trades.records_readable[["Entry Index", "Exit Index", "PnL"]]
Entry Index Exit Index PnL
0 2014-11-01 00:00:00+00:00 2016-01-08 00:00:00+00:00 39.134739
1 2016-03-27 00:00:00+00:00 2016-09-07 00:00:00+00:00 61.220063
2 2016-12-24 00:00:00+00:00 2016-12-31 00:00:00+00:00 14.471414
3 2017-03-16 00:00:00+00:00 2017-08-05 00:00:00+00:00 373.492028
4 2017-09-12 00:00:00+00:00 2018-05-05 00:00:00+00:00 815.699284
5 2019-02-15 00:00:00+00:00 2019-11-10 00:00:00+00:00 2107.383227
6 2019-12-04 00:00:00+00:00 2019-12-10 00:00:00+00:00 12.630214
7 2020-07-12 00:00:00+00:00 2021-11-14 00:00:00+00:00 21346.035444
8 2022-01-15 00:00:00+00:00 2023-03-06 00:00:00+00:00 -11925.133817
- Disable Numba when testing to avoid compilation
Time stops¶
- Joining the ranks of stop orders, time stop orders can close out a position after a period of time or also on a specific date.
Enter randomly, exit before the end of the month
>>> data = vbt.YFData.fetch("BTC-USD", start="2022-01", end="2022-04")
>>> entries = vbt.pd_acc.signals.generate_random(data.symbol_wrapper, n=10)
>>> pf = vbt.PF.from_signals(data, entries, dt_stop="M") # (1)!
>>> pf.orders.records_readable[["Fill Index", "Side", "Stop Type"]]
Fill Index Side Stop Type
0 2022-01-19 00:00:00+00:00 Buy None
1 2022-01-31 00:00:00+00:00 Sell DT
2 2022-02-25 00:00:00+00:00 Buy None
3 2022-02-28 00:00:00+00:00 Sell DT
4 2022-03-11 00:00:00+00:00 Buy None
5 2022-03-31 00:00:00+00:00 Sell DT
dt_stopfor datetime-based stops,td_stopfor timedelta-based stops. Datetime-based stops can be periods ("D"), timestamps ("2023-01-01"), and even times ("18:00").
Expanding trade metrics¶
- Regular metrics such as MAE and MFE represent only the final point of each trade, but what if we would like to see their development during each trade? You can now analyze expanding trade metrics as DataFrames!
Visualize the expanding MFE using projections
>>> data = vbt.YFData.fetch("BTC-USD")
>>> pf = vbt.PF.from_random_signals(data, n=50, tp_stop=0.5)
>>> pf.trades.plot_expanding_mfe_returns().show()
Compression¶
- Serialized vectorbtpro objects may sometimes take a lot of disk space. With this update, there's now support for a variety of compression algorithms to make files as light as possible!
Save data without and with compression
>>> data = vbt.RandomOHLCData.fetch("RAND", start="2022", end="2023", freq="1 minute")
>>> file_path = data.save()
>>> print(vbt.file_size(file_path))
21.0 MB
>>> file_path = data.save(compression="blosc")
>>> print(vbt.file_size(file_path))
13.3 MB
Parallel data¶
- Data fetching and updating can be easily parallelized.
Benchmark fetching multiple symbols serially and concurrently
>>> symbols = ["SPY", "TLT", "XLF", "XLE", "XLU", "XLK", "XLB", "XLP", "XLY", "XLI", "XLV"]
>>> with vbt.Timer() as timer:
... data = vbt.YFData.fetch(symbols)
>>> print(timer.elapsed())
4.52 seconds
>>> with vbt.Timer() as timer:
... data = vbt.YFData.fetch(symbols, execute_kwargs=dict(engine="threadpool"))
>>> print(timer.elapsed())
918.54 milliseconds
Faster loading¶
- If your pipeline doesn't need accessors, Plotly graphs, and most of other optional functionalities, you can disable the auto-import feature entirely to bring down the loading time of vectorbtpro to under a second
Measure the loading time
>>> import time
>>> start = time.time()
>>> import vectorbtpro as vbt
>>> end = time.time()
>>> end - start
0.580937910079956
Target size to signals¶
- Target size can be translated to signals using a special signal function to gain access to the stop and limit order functionality, for example, when performing a portfolio optimization.
Perform the Mean-Variance Optimization with SL and TP
>>> data = vbt.YFData.fetch(
... ["SPY", "TLT", "XLF", "XLE", "XLU", "XLK", "XLB", "XLP", "XLY", "XLI", "XLV"],
... start="2022",
... end="2023",
... missing_index="drop"
... )
>>> pf_opt = vbt.PFO.from_riskfolio(data.returns, every="MS")
>>> pf = pf_opt.simulate(
... data,
... pf_method="from_signals",
... sl_stop=0.05,
... tp_stop=0.1,
... stop_exit_price="close" # (1)!
... )
>>> pf.plot_allocations().show()
- Otherwise user and stop signals will happen at different times during the same bar, which makes establishing a correctly ordered sequence of orders impossible
Target price¶
- Limit and stop orders can also be defined using a target price rather than a delta.
Set the SL to the previous low in a random portfolio
>>> data = vbt.YFData.fetch("BTC-USD")
>>> pf = vbt.PF.from_random_signals(
... data,
... n=100,
... sl_stop=data.low.vbt.ago(1),
... delta_format="target"
... )
>>> sl_orders = pf.orders.stop_type_sl
>>> signal_index = pf.wrapper.index[sl_orders.signal_idx.values]
>>> hit_index = pf.wrapper.index[sl_orders.idx.values]
>>> hit_after = hit_index - signal_index
>>> hit_after
TimedeltaIndex([ '7 days', '3 days', '1 days', '5 days', '4 days',
'1 days', '28 days', '1 days', '1 days', '1 days',
'1 days', '1 days', '13 days', '10 days', '5 days',
'1 days', '3 days', '4 days', '1 days', '9 days',
'5 days', '1 days', '1 days', '2 days', '1 days',
'1 days', '1 days', '3 days', '1 days', '1 days',
'1 days', '1 days', '1 days', '2 days', '2 days',
'1 days', '12 days', '3 days', '1 days', '1 days',
'1 days', '1 days', '1 days', '3 days', '1 days',
'1 days', '1 days', '4 days', '1 days', '1 days',
'2 days', '6 days', '11 days', '1 days', '2 days',
'1 days', '1 days', '1 days', '1 days', '1 days',
'4 days', '10 days', '1 days', '1 days', '1 days',
'2 days', '3 days', '1 days'],
dtype='timedelta64[ns]', name='Date', freq=None)
Leverage¶
- Leverage has become an integral part of the portfolio simulation. Supports two different leverage modes:
lazy(enables leverage only if there's not enough cash available) andeager(enables leverage while using only a fraction of the available cash). Allows setting leverage per order. Can also determine an optimal leverage value automatically to satisfy any order requirement!
Explore how leverage affects the equity curve in a random portfolio
>>> data = vbt.YFData.fetch("BTC-USD", start="2020", end="2022")
>>> pf = vbt.PF.from_random_signals(
... data,
... n=100,
... leverage=vbt.Param([0.5, 1, 2, 3]),
... )
>>> pf.value.vbt.plot().show()
Configuration files¶
- VectorBT PRO extends configparser to define its own configuration format that lets the user save, introspect, modify, and load back any complex in-house object. The main advantages of this format are readability and round-tripping: any object can be encoded and then decoded back without information loss. The main features include nested structures, references, parsing of literals, as well as evaluation of arbitrary Python expressions. Additionally, you can now create a configuration file for vectorbtpro and put it into the working directory - it will be used to update the default settings whenever the package is imported!
Define global settings in vbt.ini
[plotting]
default_theme = dark
[portfolio]
init_cash = 5000
[data.custom.binance.client_config]
api_key = YOUR_API_KEY
api_secret = YOUR_API_SECRET
[data.custom.ccxt.exchanges.binance.exchange_config]
apiKey = &data.custom.binance.client_config.api_key
secret = &data.custom.binance.client_config.api_secret
Verify that the settings have been loaded correctly
>>> import vectorbtpro as vbt
>>> vbt.settings.portfolio["init_cash"]
5000
Serialization¶
- Just like machine learning models, every native vectorbtpro object can be serialized and saved to a binary file - it's never been easier to share data and insights! Another benefit is that only the actual content of each object is serialized, and not its class definition, such that the loaded object uses only the most up-to-date class definition. There's also a special logic implemented that can help you "reconstruct" objects if vectorbtpro has introduced some breaking API changes
Backtest each month of data and save the results for later
>>> data = vbt.YFData.fetch("BTC-USD", start="2022-01-01", end="2022-06-01")
>>> def backtest_month(close):
... return vbt.PF.from_random_signals(close, n=10)
>>> month_pfs = data.close.resample("MS").apply(backtest_month)
>>> month_pfs
Date
2022-01-01 00:00:00+00:00 Portfolio(\n wrapper=ArrayWrapper(\n ...
2022-02-01 00:00:00+00:00 Portfolio(\n wrapper=ArrayWrapper(\n ...
2022-03-01 00:00:00+00:00 Portfolio(\n wrapper=ArrayWrapper(\n ...
2022-04-01 00:00:00+00:00 Portfolio(\n wrapper=ArrayWrapper(\n ...
2022-05-01 00:00:00+00:00 Portfolio(\n wrapper=ArrayWrapper(\n ...
Freq: MS, Name: Close, dtype: object
>>> vbt.save(month_pfs, "month_pfs") # (1)!
>>> month_pfs = vbt.load("month_pfs") # (2)!
>>> month_pfs.apply(lambda pf: pf.total_return)
Date
2022-01-01 00:00:00+00:00 -0.048924
2022-02-01 00:00:00+00:00 0.168370
2022-03-01 00:00:00+00:00 0.016087
2022-04-01 00:00:00+00:00 -0.120525
2022-05-01 00:00:00+00:00 0.110751
Freq: MS, Name: Close, dtype: float64
- Save to disk
- Load from disk later
Trading View¶
- Welcome a new class specialized in pulling data from TradingView!
>>> data = vbt.TVData.fetch(
... "NASDAQ:AAPL",
... timeframe="1 minute",
... tz="US/Eastern"
... )
>>> data.get()
Open High Low Close Volume
datetime
2022-12-05 09:30:00-05:00 147.75 148.31 147.50 148.28 37769.0
2022-12-05 09:31:00-05:00 148.28 148.67 148.28 148.49 10525.0
2022-12-05 09:32:00-05:00 148.50 148.73 148.30 148.30 4860.0
2022-12-05 09:33:00-05:00 148.25 148.73 148.25 148.64 5306.0
2022-12-05 09:34:00-05:00 148.62 148.97 148.52 148.97 5808.0
... ... ... ... ... ...
2023-01-17 15:55:00-05:00 135.80 135.91 135.80 135.86 37573.0
2023-01-17 15:56:00-05:00 135.85 135.88 135.80 135.88 18796.0
2023-01-17 15:57:00-05:00 135.88 135.93 135.85 135.91 21019.0
2023-01-17 15:58:00-05:00 135.90 135.97 135.89 135.95 20934.0
2023-01-17 15:59:00-05:00 135.94 136.00 135.84 135.94 86696.0
[11310 rows x 5 columns]
Trade signals¶
- New method for plotting trades that categorizes entry and exit trades into long entries, long exits, short entries, and short exits. Supports different styles for positions.
Plot trade signals of a Bollinger Bands strategy
>>> data = vbt.YFData.fetch("BTC-USD")
>>> bb = data.run("bbands")
>>> entries = data.hlc3.vbt.crossed_above(bb.upper) & (bb.bandwidth < 0.1)
>>> exits = data.hlc3.vbt.crossed_below(bb.upper) & (bb.bandwidth > 0.5)
>>> short_entries = data.hlc3.vbt.crossed_below(bb.lower) & (bb.bandwidth < 0.1)
>>> short_exits = data.hlc3.vbt.crossed_above(bb.lower) & (bb.bandwidth > 0.5)
>>> pf = vbt.PF.from_signals(data, entries, exits, short_entries, short_exits)
>>> pf.plot_trade_signals().show()
Indicators for ML¶
- Want to feed indicators as features to a machine-learning model? No more need to run them individually: you can instruct vectorbtpro to run all indicators of an indicator package on the given data instance; the data instance will recognize input names of each indicator and pass the required data. You can also easily change the defaults of each indicator.
Run all talib indicators on entire BTC-USD history
>>> data = vbt.YFData.fetch("BTC-USD")
>>> features = data.run("talib", periods=vbt.run_func_dict(mavp=14))
>>> features.shape
(3046, 175)
CV decorator¶
- Most cross-validation tasks involve testing a grid of parameter combinations on the training data, selecting the best parameter combination, and validating it on the test data. This procedure needs to be repeated on each split. The cross-validation decorator combines the parameterized and split decorators to automate such a task.
Tutorial
Learn more in the Cross-validation tutorial.
Cross-validate a SMA crossover using random search
>>> @vbt.cv_split(
... splitter="from_rolling",
... splitter_kwargs=dict(length=365, split=0.5, set_labels=["train", "test"]),
... takeable_args=["data"],
... execute_kwargs=dict(show_progress=True),
... parameterized_kwargs=dict(random_subset=100),
... merge_func="concat"
... )
... def sma_crossover_cv(data, fast_period, slow_period, metric):
... fast_sma = data.run("sma", fast_period, hide_params=True)
... slow_sma = data.run("sma", slow_period, hide_params=True)
... entries = fast_sma.real_crossed_above(slow_sma)
... exits = fast_sma.real_crossed_below(slow_sma)
... pf = vbt.PF.from_signals(data, entries, exits, direction="both")
... return pf.deep_getattr(metric)
>>> sma_crossover_cv(
... vbt.YFData.fetch("BTC-USD", start="4 years ago"),
... vbt.Param(np.arange(20, 50), condition="x < slow_period"),
... vbt.Param(np.arange(20, 50)),
... "trades.expectancy"
... )
split set fast_period slow_period
0 train 20 25 8.015725
test 20 23 0.573465
1 train 40 48 -4.356317
test 39 40 5.666271
2 train 24 45 18.253340
test 22 36 111.202831
3 train 20 31 54.626024
test 20 25 -1.596945
4 train 25 48 41.328588
test 25 30 6.620254
5 train 26 32 7.178085
test 24 29 4.087456
6 train 22 23 -0.581255
test 22 31 -2.494519
dtype: float64
Split decorator¶
- Normally, to run a function on each split, you need to build a splitter specifically targeted at the input data passed to the function. That is, each time the input data changes, you need to rebuild the splitter. This process is automated by the split decorator, which wraps a function and thus gets access to all the arguments the function receives to do various splitting decisions. Basically, it can "infect" any Python function with splitting functionality
Tutorial
Learn more in the Cross-validation tutorial.
Get total return from holding in each quarter
>>> @vbt.split(
... splitter="from_grouper",
... splitter_kwargs=dict(by="Q"),
... takeable_args=["data"],
... merge_func="concat"
... )
... def get_quarter_return(data):
... return data.returns.vbt.returns.total()
>>> data = vbt.YFData.fetch("BTC-USD")
>>> get_quarter_return(data.loc["2021"])
Date
2021Q1 1.005805
2021Q2 -0.407050
2021Q3 0.304383
2021Q4 -0.037627
Freq: Q-DEC, dtype: float64
>>> get_quarter_return(data.loc["2022"])
Date
2022Q1 -0.045047
2022Q2 -0.572515
2022Q3 0.008429
2022Q4 -0.143154
Freq: Q-DEC, dtype: float64
Edge ratio¶
- Edge ratio is a unique way to quantify entry profitability. Unlike most performance metrics, the edge ratio takes both open profits and losses into account. This can help you determine optimal trade exits.
Compare the edge ratio of an EMA crossover to a random strategy
>>> data = vbt.YFData.fetch("BTC-USD")
>>> fast_ema = data.run("ema", 10, hide_params=True)
>>> slow_ema = data.run("ema", 20, hide_params=True)
>>> entries = fast_ema.real_crossed_above(slow_ema)
>>> exits = fast_ema.real_crossed_below(slow_ema)
>>> pf = vbt.PF.from_signals(data, entries, exits, direction="both")
>>> rand_pf = vbt.PF.from_random_signals(data, n=pf.orders.count() // 2) # (1)!
>>> fig = pf.trades.plot_running_edge_ratio(
... trace_kwargs=dict(line_color="limegreen", name="Edge Ratio (S)")
... )
>>> fig = rand_pf.trades.plot_running_edge_ratio(
... trace_kwargs=dict(line_color="mediumslateblue", name="Edge Ratio (R)"),
... fig=fig
... )
>>> fig.show()
- Random strategy should have a similar number of orders to be comparable
Trade history¶
- Trade history is a human-readable DataFrame that lists orders and extends them with valuable information on entry trades, exit trades, and positions.
Get the trade history of a random portfolio with one signal
>>> data = vbt.YFData.fetch(["BTC-USD", "ETH-USD"], missing_index="drop")
>>> pf = vbt.PF.from_random_signals(
... data,
... n=1,
... run_kwargs=dict(hide_params=True),
... tp_stop=0.5,
... sl_stop=0.1
... )
>>> pf.trade_history
Order Id Column Signal Index Creation Index \
0 0 BTC-USD 2016-02-20 00:00:00+00:00 2016-02-20 00:00:00+00:00
1 1 BTC-USD 2016-02-20 00:00:00+00:00 2016-06-12 00:00:00+00:00
2 0 ETH-USD 2019-05-25 00:00:00+00:00 2019-05-25 00:00:00+00:00
3 1 ETH-USD 2019-05-25 00:00:00+00:00 2019-07-15 00:00:00+00:00
Fill Index Side Type Stop Type Size Price \
0 2016-02-20 00:00:00+00:00 Buy Market None 0.228747 437.164001
1 2016-06-12 00:00:00+00:00 Sell Market TP 0.228747 655.746002
2 2019-05-25 00:00:00+00:00 Buy Market None 0.397204 251.759872
3 2019-07-15 00:00:00+00:00 Sell Market SL 0.397204 226.583885
Fees PnL Return Direction Status Entry Trade Id Exit Trade Id \
0 0.0 50.0 0.5 Long Closed 0 -1
1 0.0 50.0 0.5 Long Closed -1 0
2 0.0 -10.0 -0.1 Long Closed 0 -1
3 0.0 -10.0 -0.1 Long Closed -1 0
Position Id
0 0
1 0
2 0
3 0
Conditional parameters¶
- Parameters can depend on each other. For instance, when testing a crossover of moving averages, it makes no sense to test a fast window that has a bigger length than the slow window. By filtering such cases out, you only need to evaluate half as many parameter combinations.
Test slow windows being longer than fast windows by at least 5
>>> @vbt.parameterized(merge_func="column_stack")
... def ma_crossover_signals(data, fast_window, slow_window):
... fast_sma = data.run("sma", fast_window, short_name="fast_sma")
... slow_sma = data.run("sma", slow_window, short_name="slow_sma")
... entries = fast_sma.real_crossed_above(slow_sma.real)
... exits = fast_sma.real_crossed_below(slow_sma.real)
... return entries, exits
>>> entries, exits = ma_crossover_signals(
... vbt.YFData.fetch("BTC-USD", start="one year ago UTC"),
... vbt.Param(np.arange(5, 50), condition="slow_window - fast_window >= 5"),
... vbt.Param(np.arange(5, 50))
... )
>>> entries.columns
MultiIndex([( 5, 10),
( 5, 11),
( 5, 12),
( 5, 13),
( 5, 14),
...
(42, 48),
(42, 49),
(43, 48),
(43, 49),
(44, 49)],
names=['fast_window', 'slow_window'], length=820)
Splitter¶
- Splitters in scikit-learn are a poor fit for validating ML-based and rule-based trading strategies; vectorbtpro has a juggernaut class that supports various splitting schemes safe for backtesting, including rolling windows, expanding windows, time-anchored windows, random windows for block bootstraps, and even Pandas-native
groupbyandresampleinstructions such as "M" for monthly frequency. As the cherry on the cake, the produced splits can be easily analyzed and visualized too! For example, you can detect any split or set overlaps, convert all the splits into a single boolean mask for custom analysis, group splits and sets, and analyze their distribution relative to each other. The class has more lines of code than the entire backtesting.py package, don't underestimate the new king in town!
Tutorial
Learn more in the Cross-validation tutorial.
Roll a 360-day window and split it equally into train and test sets
>>> data = vbt.YFData.fetch("BTC-USD", start="4 years ago")
>>> splitter = vbt.Splitter.from_rolling(
... data.index,
... length="360 days",
... split=0.5,
... set_labels=["train", "test"],
... freq="daily"
... )
>>> splitter.plots().show()
Signal detection¶
- VectorBT PRO implements an indicator based on a robust peak detection algorithm using z-scores. This indicator can be used to identify outbreaks and outliers in any time-series data.
Detect sudden changes in the bandwidth of a Bollinger Bands indicator
>>> data = vbt.YFData.fetch("BTC-USD")
>>> fig = vbt.make_subplots(rows=2, cols=1, shared_xaxes=True)
>>> bbands = data.run("bbands")
>>> bbands.loc["2022"].plot(add_trace_kwargs=dict(row=1, col=1), fig=fig)
>>> sigdet = vbt.SIGDET.run(bbands.bandwidth, factor=5)
>>> sigdet.loc["2022"].plot(add_trace_kwargs=dict(row=2, col=1), fig=fig)
>>> fig.show()
Pivot detection¶
- Pivot detection indicator is a tool that can be used to find out when the price's trend is reversing. By determining the support and resistance areas, it helps to identify significant changes in price while filtering out short-term fluctuations, thus eliminating the noise. The workings are rather simple: register a peak when the price jumps above one threshold and a valley when the price falls below another threshold. Another advantage: in contrast to the regular Zig Zag indicator, which tends to look into the future, our indicator returns only the confirmed pivot points and is safe to use in backtesting.
Plot the last pivot value
>>> data = vbt.YFData.fetch("BTC-USD", start="2020", end="2023")
>>> fig = data.plot(plot_volume=False)
>>> pivot_info = data.run("pivotinfo", up_th=1.0, down_th=0.5)
>>> pivot_info.plot(fig=fig, conf_value_trace_kwargs=dict(visible=False))
>>> fig.show()
Random search¶
- While grid search looks at every possible combination of hyperparameters, random search only selects and tests a random combination of hyperparameters. This is especially useful when the number of parameter combinations is huge. Also, random search has shown to find equal or better values than grid search within fewer function evaluations. The indicator factory, parameterized decorator, and any method that does broadcasting now supports random search out of the box.
Test a random subset of SL, TSL, and TP combinations
>>> data = vbt.YFData.fetch("BTC-USD", start="2020")
>>> stop_values = np.arange(1, 100) / 100 # (1)!
>>> pf = vbt.PF.from_random_signals(
... data,
... n=100,
... sl_stop=vbt.Param(stop_values),
... tsl_stop=vbt.Param(stop_values),
... tp_stop=vbt.Param(stop_values),
... broadcast_kwargs=dict(random_subset=1000) # (2)!
... )
>>> pf.total_return.sort_values(ascending=False)
sl_stop tsl_stop tp_stop
0.06 0.85 0.43 2.291260
0.74 0.40 2.222212
0.97 0.22 2.149849
0.40 0.10 0.23 2.082935
0.47 0.09 0.25 2.030105
...
0.51 0.36 0.01 -0.618805
0.53 0.37 0.01 -0.624761
0.35 0.60 0.02 -0.662992
0.29 0.13 0.02 -0.671376
0.46 0.72 0.02 -0.720024
Name: total_return, Length: 1000, dtype: float64
- 100 combinations of each parameter = 100 ^ 3 = 1,000,000 combinations
- Indicator factory and parameterized decorator take this argument directly
Technical indicators¶
- VectorBT PRO integrates most of the indicators and consensus classes from the freqtrade's technical library.
Compute and plot the summary consensus with a one-liner
>>> vbt.YFData.fetch("BTC-USD").run("sumcon", smooth=100).plot().show()
Renko chart¶
- In contrast to regular charts, Renko chart is built using price movements. Each "brick" in this chart is created when the price moves a specified price amount. Since the output has irregular time intervals, only one column can be processed at a time. As with everything, the vectorbtpro's implementation can translate a huge number of data points very fast thanks to Numba.
Resample closing price into a Renko format
>>> data = vbt.YFData.fetch("BTC-USD", start="2021", end="2022")
>>> renko_ohlc = data.close.vbt.to_renko_ohlc(1000, reset_index=True) # (1)!
>>> renko_ohlc.vbt.ohlcv.plot().show()
- Bitcoin is very volatile, hence the brick size of 1000
Parameterized decorator¶
- There is a special decorator that can make any Python function accept multiple parameter combinations, even if the function itself can handle only one! The decorator wraps the function, thus getting access to its arguments; it then identifies all the arguments that act as parameters, builds a grid of them, and calls the underlying function on each parameter combination from that grid. The execution part can be easily parallelized. After all the outputs are ready, it merges them into a single object. Use cases are endless: from running indicators that cannot be wrapped with the indicator factory, to parameterizing entire pipelines!
Parameterize a Bollinger Bands pipeline
>>> @vbt.parameterized(merge_func="concat", show_progress=True) # (1)!
... def bbands_sharpe(data, timeperiod=14, nbdevup=2, nbdevdn=2, thup=0.3, thdn=0.1):
... bb = data.run(
... "talib_bbands",
... timeperiod=timeperiod,
... nbdevup=nbdevup,
... nbdevdn=nbdevdn
... )
... bandwidth = (bb.upperband - bb.lowerband) / bb.middleband
... cond1 = data.low < bb.lowerband
... cond2 = bandwidth > thup
... cond3 = data.high > bb.upperband
... cond4 = bandwidth < thdn
... entries = (cond1 & cond2) | (cond3 & cond4)
... exits = (cond1 & cond4) | (cond3 & cond2)
... pf = vbt.PF.from_signals(data, entries, exits)
... return pf.sharpe_ratio
>>> bbands_sharpe(
... vbt.YFData.fetch("BTC-USD"),
... nbdevup=vbt.Param([1, 2]), # (2)!
... nbdevdn=vbt.Param([1, 2]),
... thup=vbt.Param([0.4, 0.5]),
... thdn=vbt.Param([0.1, 0.2])
... )
- Use
concatfor merging metrics in form of scalars and Series, andcolumn_stackfor merging time series in form of DataFrames and complex vectorbt objects such as portfolios - Build the Cartesian product of 4 parameters
nbdevup nbdevdn thup thdn
1 1 0.4 0.1 1.681532
0.2 1.617400
0.5 0.1 1.424175
0.2 1.563520
2 0.4 0.1 1.218554
0.2 1.520852
0.5 0.1 1.242523
0.2 1.317883
2 1 0.4 0.1 1.174562
0.2 1.469828
0.5 0.1 1.427940
0.2 1.460635
2 0.4 0.1 1.000210
0.2 1.378108
0.5 0.1 1.196087
0.2 1.782502
dtype: float64
Riskfolio-Lib¶
- Riskfolio-Lib is another increasingly popular library for portfolio optimization that has been integrated into vectorbtpro. The integration was done by automating typical workflows inside Riskfolio-Lib and putting them into a single function, such that many portfolio optimization problems can be expressed using a single set of keyword arguments and thus parameterized easily.
Tutorial
Learn more in the Portfolio optimization tutorial.
Run Nested Clustered Optimization (NCO) on a monthly basis
>>> data = vbt.YFData.fetch(
... ["SPY", "TLT", "XLF", "XLE", "XLU", "XLK", "XLB", "XLP", "XLY", "XLI", "XLV"],
... start="2020",
... end="2023",
... missing_index="drop"
... )
>>> pfo = vbt.PFO.from_riskfolio(
... returns=data.close.vbt.to_returns(),
... port_cls="hc",
... every="MS"
... )
>>> pfo.plot().show()
Symbol search¶
- Most data classes can retrieve the full list of symbols available at an exchange and optionally filter the list either using a globbing or regular expression pattern. This works for local data classes as well!
Get all XRP pairs listed on Binance
>>> vbt.BinanceData.list_symbols("XRP*")
{'XRPAUD',
'XRPBEARBUSD',
'XRPBEARUSDT',
'XRPBIDR',
'XRPBKRW',
'XRPBNB',
'XRPBRL',
'XRPBTC',
'XRPBULLBUSD',
'XRPBULLUSDT',
'XRPBUSD',
'XRPDOWNUSDT',
'XRPETH',
'XRPEUR',
'XRPGBP',
'XRPNGN',
'XRPPAX',
'XRPRUB',
'XRPTRY',
'XRPTUSD',
'XRPUPUSDT',
'XRPUSDC',
'XRPUSDT'}
Symbol classes¶
- Thanks to vectorbtpro taking advantage of multi-indexes in Pandas, you can associate each symbol with one to multiple classes, such as sectors. This can allow you to analyze the performance of a trading strategy relative to each class.
Compare equal-weighted portfolios for three sectors
>>> symbol_classes = vbt.symbol_dict({
... "MSFT": dict(sector="Technology"),
... "GOOGL": dict(sector="Technology"),
... "META": dict(sector="Technology"),
... "JPM": dict(sector="Finance"),
... "BAC": dict(sector="Finance"),
... "WFC": dict(sector="Finance"),
... "AMZN": dict(sector="Retail"),
... "WMT": dict(sector="Retail"),
... "BABA": dict(sector="Retail"),
... })
>>> data = vbt.YFData.fetch(
... list(symbol_classes.keys()),
... symbol_classes=symbol_classes,
... missing_index="drop"
... )
>>> pf = vbt.PF.from_orders(
... data,
... size=vbt.index_dict({0: 1 / 3}), # (1)!
... size_type="targetpercent",
... group_by="sector",
... cash_sharing=True
... )
>>> pf.value.vbt.plot().show()
- There are three assets in each group - allocate 33.3% to each asset at the first bar
Patterns¶
- Patterns are the distinctive formations created by the movements of security prices on a chart and are the foundation of technical analysis. There are new designated functions and classes for detecting patterns of any complexity in any time series data. The concept is simple: fit a pattern to match the scale and period of the selected data window, and compute their element-wise distance to each other to derive a single similarity score. You can then tweak the threshold for this score above which a data window should be marked as "matched". Thanks to Numba, this operation can be done hundreds of thousands of times per second!
Tutorial
Learn more in the Patterns and projections tutorial.
Find and plot a descending triangle pattern
>>> data = vbt.YFData.fetch("BTC-USD")
>>> data.hlc3.vbt.find_pattern(
... pattern=[5, 1, 3, 1, 2, 1],
... window=100,
... max_window=700,
... ).loc["2017":"2019"].plot().show()
Projections¶
- There are more clean ways of analyzing events and their impact on the price than conventional backtesting. Meet projections!
Not only they can allow you to assess the performance of events visually and quantitatively, but they also can enable you to project events into the future for assistance in trading. That is possible by extracting the price range after each event, putting all the price ranges into a multidimensional array, and deriving the confidence intervals and other meaningful statistics from that array. Combined with patterns, these are a quantitative analyst's dream tools! 
Tutorial
Learn more in the Patterns and projections tutorial.
Find occurrences of the price moving similarly to the last week and project them
>>> data = vbt.YFData.fetch("ETH-USD")
>>> pattern_ranges = data.hlc3.vbt.find_pattern(
... pattern=data.close.iloc[-7:],
... rescale_mode="rebase"
... )
>>> delta_ranges = pattern_ranges.with_delta(7)
>>> fig = data.iloc[-7:].plot(plot_volume=False)
>>> delta_ranges.plot_projections(fig=fig)
>>> fig.show()
Array-like parameters¶
- Broadcasting mechanism has been completely refactored and now supports parameters. Many parameters in vectorbtpro, such as SL and TP, are array-like and can be provided per row, column, and even element. Internally, even a scalar is treated like a regular time series and broadcasted along other proper time series. Thus, to test multiple parameter combinations, one had to tile other time series such that all shapes perfectly match. With this feature, the tiling procedure is performed automatically!
Write a steep slope indicator without indicator factory
>>> def steep_slope(close, up_th):
... r = vbt.broadcast(dict(close=close, up_th=up_th))
... return r["close"].pct_change() >= r["up_th"]
>>> data = vbt.YFData.fetch("BTC-USD", start="2020", end="2022")
>>> fig = data.plot(plot_volume=False)
>>> sma = vbt.talib("SMA").run(data.close, timeperiod=50).real
>>> sma.rename("SMA").vbt.plot(fig=fig)
>>> mask = steep_slope(sma, vbt.Param([0.005, 0.01, 0.015])) # (1)!
>>> def plot_mask_ranges(column, color):
... mask.vbt.ranges.plot_shapes(
... column=column,
... plot_close=False,
... shape_kwargs=dict(fillcolor=color),
... fig=fig
... )
>>> plot_mask_ranges(0.005, "orangered")
>>> plot_mask_ranges(0.010, "orange")
>>> plot_mask_ranges(0.015, "yellow")
>>> fig.update_xaxes(showgrid=False)
>>> fig.update_yaxes(showgrid=False)
>>> fig.show()
- Test three parameters and generate a mask with three columns - one per parameter
Parameters¶
- There is a new module addition for working with parameters.
Generate 10,000 random parameter combinations for MACD
>>> window_space = np.arange(100)
>>> fastk_windows, slowk_windows = list(zip(*combinations(window_space, 2))) # (1)!
>>> window_type_space = list(vbt.enums.WType)
>>> param_product = vbt.combine_params(
... dict(
... fast_window=vbt.Param(fastk_windows, level=0), # (2)!
... slow_window=vbt.Param(slowk_windows, level=0),
... signal_window=vbt.Param(window_space, level=1),
... macd_wtype=vbt.Param(window_type_space, level=2), # (3)!
... signal_wtype=vbt.Param(window_type_space, level=2),
... ),
... random_subset=10_000,
... build_index=False
... )
>>> pd.DataFrame(param_product)
fast_window slow_window signal_window macd_wtype signal_wtype
0 0 1 47 3 3
1 0 2 21 2 2
2 0 2 33 1 1
3 0 2 42 1 1
4 0 3 52 1 1
... ... ... ... ... ...
9995 97 99 19 1 1
9996 97 99 92 4 4
9997 98 99 2 2 2
9998 98 99 12 1 1
9999 98 99 81 2 2
[10000 rows x 5 columns]
- Fast windows should be shorter than slow windows
- We already combined fast and slow windows, thus make them share the same product level
- We don't want to combine window types, thus make them share the same product level
Runnable data¶
- Tired of figuring out which arguments are required by an indicator? Data instances can now recognize the arguments of an indicator and just any function in general, map them to the column names, and run the function by passing the required columns. You can also change the mapping, override indicator parameters, and also query indicators by their names - the data instance will search for it in all integrated indicator packages and return the first (and best) one found!
>>> data = vbt.YFData.fetch("BTC-USD")
>>> stochrsi = data.run("stochrsi")
>>> stochrsi.fastd
Date
2014-09-17 00:00:00+00:00 NaN
2014-09-18 00:00:00+00:00 NaN
2014-09-19 00:00:00+00:00 NaN
2014-09-20 00:00:00+00:00 NaN
2014-09-21 00:00:00+00:00 NaN
...
2023-01-15 00:00:00+00:00 96.168788
2023-01-16 00:00:00+00:00 91.733393
2023-01-17 00:00:00+00:00 78.295255
2023-01-18 00:00:00+00:00 48.793133
2023-01-20 00:00:00+00:00 26.242474
Name: Close, Length: 3047, dtype: float64
Order delays¶
- By default, vectorbtpro executes every order at the end of the current bar. To delay the execution to the next bar, one had to manually shift all the order-related arrays by one bar, which makes the operation prone to user mistakes. To avoid them, you can now simply specify the number of bars in the past you want the order information to be taken from. Moreover, the price argument has got the new options "nextopen" and "nextclose" for a one-line solution.
Compare orders without and with a delay in a random portfolio
>>> pf = vbt.PF.from_random_signals(
... vbt.YFData.fetch("BTC-USD", start="2021-01", end="2021-02"),
... n=3,
... price=vbt.Param(["close", "nextopen"])
... )
>>> fig = pf.orders["close"].plot(
... buy_trace_kwargs=dict(name="Buy (close)", marker=dict(symbol="triangle-up-open")),
... sell_trace_kwargs=dict(name="Buy (close)", marker=dict(symbol="triangle-down-open"))
... )
>>> pf.orders["nextopen"].plot(
... plot_ohlc=False,
... plot_close=False,
... buy_trace_kwargs=dict(name="Buy (nextopen)"),
... sell_trace_kwargs=dict(name="Sell (nextopen)"),
... fig=fig
... )
>>> fig.show()
Signal callbacks¶
- Want to hack into the simulation based on signals, or even generate signals dynamically based on the current backtesting environment? There are two new callbacks that push flexibility of the simulator to the limits: one that lets the user generate or override signals for each asset at the current bar, and another one that lets the user compute any user-defined metrics for the entire group at the end of the bar. Both are taking a so-called "context" that contains the information on the current simulation state based on which trading decisions can be made - very similar to how event-driven backtesters work.
Backtest SMA crossover iteratively
>>> InOutputs = namedtuple("InOutputs", ["fast_sma", "slow_sma"])
>>> def initialize_in_outputs(target_shape):
... return InOutputs(
... fast_sma=np.full(target_shape, np.nan),
... slow_sma=np.full(target_shape, np.nan)
... )
>>> @njit
... def signal_func_nb(c, fast_window, slow_window):
... fast_sma = c.in_outputs.fast_sma
... slow_sma = c.in_outputs.slow_sma
... fast_start_i = c.i - fast_window + 1
... slow_start_i = c.i - slow_window + 1
... if fast_start_i >= 0 and slow_start_i >= 0:
... fast_sma[c.i, c.col] = np.nanmean(c.close[fast_start_i : c.i + 1])
... slow_sma[c.i, c.col] = np.nanmean(c.close[slow_start_i : c.i + 1])
... is_entry = vbt.pf_nb.iter_crossed_above_nb(c, fast_sma, slow_sma)
... is_exit = vbt.pf_nb.iter_crossed_below_nb(c, fast_sma, slow_sma)
... return is_entry, is_exit, False, False
... return False, False, False, False
>>> pf = vbt.PF.from_signals(
... vbt.YFData.fetch("BTC-USD"),
... signal_func_nb=signal_func_nb,
... signal_args=(50, 200),
... in_outputs=vbt.RepFunc(initialize_in_outputs),
... )
>>> fig = pf.get_in_output("fast_sma").vbt.plot()
>>> pf.get_in_output("slow_sma").vbt.plot(fig=fig)
>>> pf.orders.plot(plot_ohlc=False, plot_close=False, fig=fig)
>>> fig.show()
Limit orders¶
- Long-awaited support for limit orders in simulation based on signals! Features time-in-force (TIF) orders, such as DAY, GTC, GTD, LOO, and FOK orders
You can also reverse a limit order as well as create it using a delta for easier testing.
Explore how limit delta affects number of orders in a random portfolio
>>> pf = vbt.PF.from_random_signals(
... vbt.YFData.fetch("BTC-USD"),
... n=100,
... order_type="limit",
... limit_delta=vbt.Param(np.arange(0.001, 0.1, 0.001)), # (1)!
... )
>>> pf.orders.count().vbt.plot(
... xaxis_title="Limit delta",
... yaxis_title="Order count"
... ).show()
- Limit delta is the distance between the close (or any other price) and the target limit price in percentage terms. The highest the delta, the less is the chance that it will be eventually hit.
Delta formats¶
- Previously, stop orders had to be provided strictly as percentages, which works well for single values but may require some transformations for arrays. For example, to set SL to ATR, one would need to know the entry price. Generally, to lock in a specific dollar amount of a trade, you may prefer to utilize a fixed price trailing stop. To address this, vectorbtpro has introduced multiple stop value formats (so called "delta formats") to choose from.
Use ATR as SL
>>> data = vbt.YFData.fetch("BTC-USD")
>>> atr = vbt.talib("ATR").run(data.high, data.low, data.close).real
>>> period = slice("2022-01-01", "2022-01-07")
>>> pf = vbt.PF.from_holding(
... data.loc[period],
... sl_stop=atr.loc[period],
... delta_format="absolute"
... )
>>> pf.orders.plot().show()
Bar skipping¶
- Simulation based on orders and signals can (partially) skip bars that do not define any orders - a change that often results in noticeable speedups for sparsely distributed orders.
Benchmark bar skipping in a buy-and-hold portfolio
>>> data = vbt.BinanceData.fetch("BTCUSDT", start="one month ago UTC", timeframe="minute")
>>> size = data.symbol_wrapper.fill(np.nan)
>>> size[0] = np.inf
>>> %%timeit
>>> vbt.PF.from_orders(data, size, ffill_val_price=True) # (1)!
5.92 ms ± 300 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)
- When enabled (default), this argument forces the simulation to process each bar
>>> %%timeit
>>> vbt.PF.from_orders(data, size, ffill_val_price=False)
2.75 ms ± 16 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
Rolling OLS¶
- Rolling regressions are one of the models for analyzing changing relationships among variables over time. In vectorbtpro, it's implemented as an indicator that takes two time series and returns the slope, intercept, prediction, error, and the z-score of the error at each time step. Not only this indicator can be used for cointegration tests, such as to determine optimal rebalancing timings in pairs trading, but it's also (literally) 1000x faster than the statsmodels' equivalent RollingOLS
Determine the spread between BTC and ETH
>>> data = vbt.YFData.fetch(
... ["BTC-USD", "ETH-USD"],
... start="2022",
... end="2023",
... missing_index="drop"
... )
>>> ols = vbt.OLS.run(
... data.get("Close", "BTC-USD"),
... data.get("Close", "ETH-USD")
... )
>>> ols.plot_zscore().show()
MAE and MFE¶
- Maximum Adverse Excursion (MAE) helps you to identify what the maximum loss was during a trade. This is also known as maximum drawdown of the position. Maximum Favorable Excursion (MFE), on the other hand, shows you what the highest profit was during a trade. Analyzing MAE and MFE statistics can help you optimize your exit strategies.
Analyze the MAE of a random portfolio without SL
>>> data = vbt.YFData.fetch("BTC-USD")
>>> pf = vbt.PF.from_random_signals(data, n=50)
>>> pf.trades.plot_mae_returns().show()
Analyze the MAE of a random portfolio with SL
>>> pf = vbt.PF.from_random_signals(data, n=50, sl_stop=0.1)
>>> pf.trades.plot_mae_returns().show()
OHLC-native classes¶
- Previously, OHLC was used for simulation but only the close price was used for analysis purposes. With this update, most classes will allow you to keep track of the entire OHLC data for a more accurate quantitative and qualitative analysis.
Plot trades of a random portfolio
>>> data = vbt.YFData.fetch("BTC-USD", start="2020-01", end="2020-03")
>>> pf = vbt.PF.from_random_signals(
... open=data.open,
... high=data.high,
... low=data.low,
... close=data.close,
... n=10
... )
>>> pf.trades.plot().show()
Data parsing¶
- Tired of passing open, high, low, and close as separate time series? Portfolio class methods have been extended to take a data instance instead of close and extract the contained OHLC data automatically - a small but timesaving feature!
Run the example above using the new approach
>>> data = vbt.YFData.fetch("BTC-USD", start="2020-01", end="2020-03")
>>> pf = vbt.PF.from_random_signals(data, n=10)
Index dictionaries¶
Manually constructing arrays and setting their data with Pandas is often painful. Gladly, there is a new functionality that provides a much needed help! Any broadcastable argument can become an index dictionary, which contains instructions on where to set values in the array and does the filling job for you. It knows exactly which axis has to be modified and doesn't create a full array if not necessary - with much love to RAM 
1) Accumulate daily and exit on Sunday vs 2) accumulate weekly and exit on month end
>>> data = vbt.YFData.fetch(["BTC-USD", "ETH-USD"])
>>> tile = pd.Index(["daily", "weekly"], name="strategy") # (1)!
>>> pf = vbt.PF.from_orders(
... data.close,
... size=vbt.index_dict({ # (2)!
... vbt.idx(
... vbt.pointidx(every="D"),
... vbt.colidx("daily", level="strategy")): 100, # (3)!
... vbt.idx(
... vbt.pointidx(every="W-SUN"),
... vbt.colidx("daily", level="strategy")): -np.inf, # (4)!
... vbt.idx(
... vbt.pointidx(every="W-MON"),
... vbt.colidx("weekly", level="strategy")): 100,
... vbt.idx(
... vbt.pointidx(every="M"),
... vbt.colidx("weekly", level="strategy")): -np.inf,
... }),
... size_type="value",
... direction="longonly",
... init_cash="auto",
... broadcast_kwargs=dict(tile=tile)
... )
>>> pf.sharpe_ratio
strategy symbol
daily BTC-USD 0.702259
ETH-USD 0.782296
weekly BTC-USD 0.838895
ETH-USD 0.524215
Name: sharpe_ratio, dtype: float64
- To represent two strategies, you need to tile the same data twice. For this, create a parameter with strategy names and pass it as
tileto the broadcaster for it to tile the columns of each array (such as price) by two times. - Index dictionary contains index instructions as keys and data as values to be set. Keys can be anything from row indices and labels, to custom indexer classes such as PointIdxr.
- Find the indices of the rows that correspond to the beginning of each day and the index of the column "daily", and set each element under those indices to 100 (= accumulate)
- Find the indices of the rows that correspond to Sunday. If any value under those indices has already been set with any previous instruction, it will be overridden.
Slicing¶
- Similarly to selecting columns, each vectorbtpro object is now capable of slicing rows, using the exact same mechanism as in Pandas
This makes it supereasy to analyze and plot any subset of simulated data, without the need of re-simulation!
Analyze multiple date ranges of the same portfolio
>>> data = vbt.YFData.fetch("BTC-USD")
>>> pf = vbt.PF.from_holding(data, freq="d")
>>> pf.sharpe_ratio
1.116727709477293
>>> pf.loc[:"2020"].sharpe_ratio # (1)!
1.2699801554196481
>>> pf.loc["2021": "2021"].sharpe_ratio # (2)!
0.9825161170278687
>>> pf.loc["2022":].sharpe_ratio # (3)!
-1.0423271337174647
- Get the Sharpe during the year 2020 and before
- Get the Sharpe during the year 2021
- Get the Sharpe during the year 2022 and after
Column stacking¶
- Complex vectorbtpro objects of the same type can be easily stacked along columns. For instance, you can combine multiple totally-unrelated trading strategies into the same portfolio for analysis. Under the hood, the final object is still represented as a monolithic multi-dimensional structure that can be processed even faster than merged objects separately
Analyze two trading strategies separately and then jointly
>>> def strategy1(data):
... fast_ma = vbt.MA.run(data.close, 50, short_name="fast_ma")
... slow_ma = vbt.MA.run(data.close, 200, short_name="slow_ma")
... entries = fast_ma.ma_crossed_above(slow_ma)
... exits = fast_ma.ma_crossed_below(slow_ma)
... return vbt.PF.from_signals(
... data.close,
... entries,
... exits,
... size=100,
... size_type="value",
... init_cash="auto"
... )
>>> def strategy2(data):
... bbands = vbt.BBANDS.run(data.close, window=14)
... entries = bbands.close_crossed_below(bbands.lower)
... exits = bbands.close_crossed_above(bbands.upper)
... return vbt.PF.from_signals(
... data.close,
... entries,
... exits,
... init_cash=200
... )
>>> data1 = vbt.BinanceData.fetch("BTCUSDT")
>>> pf1 = strategy1(data1) # (1)!
>>> pf1.sharpe_ratio
0.9100317671866922
>>> data2 = vbt.BinanceData.fetch("ETHUSDT")
>>> pf2 = strategy2(data2) # (2)!
>>> pf2.sharpe_ratio
-0.11596286232734827
>>> pf_sep = vbt.PF.column_stack((pf1, pf2)) # (3)!
>>> pf_sep.sharpe_ratio
0 0.910032
1 -0.115963
Name: sharpe_ratio, dtype: float64
>>> pf_join = vbt.PF.column_stack((pf1, pf2), group_by=True) # (4)!
>>> pf_join.sharpe_ratio
0.42820898354646514
- Analyze the first strategy in a separate portfolio
- Analyze the second strategy in a separate portfolio
- Analyze both strategies in the same portfolio separately
- Analyze both strategies in the same portfolio jointly
Row stacking¶
- Complex vectorbtpro objects of the same type can be easily stacked along rows. For instance, you can append new data to an existing portfolio, or even concatenate in-sample portfolios with their out-of-sample counterparts
Analyze two date ranges separately and then jointly
>>> def strategy(data, start=None, end=None):
... fast_ma = vbt.MA.run(data.close, 50, short_name="fast_ma")
... slow_ma = vbt.MA.run(data.close, 200, short_name="slow_ma")
... entries = fast_ma.ma_crossed_above(slow_ma)
... exits = fast_ma.ma_crossed_below(slow_ma)
... return vbt.PF.from_signals(
... data.close[start:end],
... entries[start:end],
... exits[start:end],
... size=100,
... size_type="value",
... init_cash="auto"
... )
>>> data = vbt.BinanceData.fetch("BTCUSDT")
>>> pf_whole = strategy(data) # (1)!
>>> pf_whole.sharpe_ratio
0.9100317671866922
>>> pf_sub1 = strategy(data, end="2019-12-31") # (2)!
>>> pf_sub1.sharpe_ratio
0.7810397448678937
>>> pf_sub2 = strategy(data, start="2020-01-01") # (3)!
>>> pf_sub2.sharpe_ratio
1.070339534746574
>>> pf_join = vbt.PF.row_stack((pf_sub1, pf_sub2)) # (4)!
>>> pf_join.sharpe_ratio
0.9100317671866922
- Analyze the entire range
- Analyze the first date range
- Analyze the second date range
- Join both date ranges and analyze as a whole
Index alignment¶
- There is no more limitation of each Pandas array being required to have the same index. Indexes of all arrays that should broadcast against each other are automatically aligned, as long as they have the same data type.
Predict ETH price with BTC price using linear regression
>>> btc_data = vbt.YFData.fetch("BTC-USD")
>>> btc_data.wrapper.shape
(2817, 7)
>>> eth_data = vbt.YFData.fetch("ETH-USD") # (1)!
>>> eth_data.wrapper.shape
(1668, 7)
>>> ols = vbt.OLS.run( # (2)!
... btc_data.close,
... eth_data.close
... )
>>> ols.pred
Date
2014-09-17 00:00:00+00:00 NaN
2014-09-18 00:00:00+00:00 NaN
2014-09-19 00:00:00+00:00 NaN
2014-09-20 00:00:00+00:00 NaN
2014-09-21 00:00:00+00:00 NaN
... ...
2022-05-30 00:00:00+00:00 2109.769242
2022-05-31 00:00:00+00:00 2028.856767
2022-06-01 00:00:00+00:00 1911.555689
2022-06-02 00:00:00+00:00 1930.169725
2022-06-03 00:00:00+00:00 1882.573170
Freq: D, Name: Close, Length: 2817, dtype: float64
- ETH-USD history is shorter than BTC-USD history
- This now works! Just make sure that all arrays share the same timeframe and timezone.
Numba datetime¶
- There is no support for datetime indexes (and any other Pandas objects) in Numba. There are also no built-in Numba functions for working with datetime. So, how to connect data to time? vectorbtpro closes this loophole by implementing a collection of functions to extract various information from each timestamp, such as the current time and day of the week to determine whether the bar happens during trading hours.
Tutorial
Learn more in the Signal development tutorial.
Plot the percentage change from the start of the month to now
>>> @njit
... def month_start_pct_change_nb(arr, index):
... out = np.full(arr.shape, np.nan)
... for col in range(arr.shape[1]):
... for i in range(arr.shape[0]):
... if i == 0 or vbt.dt_nb.month_nb(index[i - 1]) != vbt.dt_nb.month_nb(index[i]):
... month_start_value = arr[i, col]
... else:
... out[i, col] = (arr[i, col] - month_start_value) / month_start_value
... return out
>>> data = vbt.YFData.fetch(["BTC-USD", "ETH-USD"], start="2022", end="2023")
>>> pct_change = month_start_pct_change_nb(
... vbt.to_2d_array(data.close),
... data.index.vbt.to_ns() # (1)!
... )
>>> pct_change = data.symbol_wrapper.wrap(pct_change)
>>> pct_change.vbt.plot().show()
- Convert the datetime index to the nanosecond format
Periods ago¶
- Instead of writing Numba functions, comparing values at different bars can be also done in a vectorized manner with Pandas. The problem is that there are no-built in functions to easily shift values based on timedeltas, neither there are rolling functions to check whether an event happened during a period time in the past. This gap is closed by various new accessor methods.
Tutorial
Learn more in the Signal development tutorial.
Check whether the price dropped for 5 consecutive bars
>>> data = vbt.YFData.fetch("BTC-USD", start="2022-05", end="2022-08")
>>> mask = (data.close < data.close.vbt.ago(1)).vbt.all_ago(5)
>>> fig = data.plot(plot_volume=False)
>>> mask.vbt.signals.ranges.plot_shapes(
... plot_close=False,
... fig=fig,
... shape_kwargs=dict(fillcolor="orangered")
... )
>>> fig.show()
Signal contexts¶
- Signal generation functions have been redesigned from the ground up to operate on contexts. This allows for designing more complex signal strategies with less namespace polution.
Tutorial
Learn more in the Signal development tutorial.
Entry at the first bar of the week, exit at the last bar of the week
>>> @njit
... def entry_place_func_nb(c, index):
... for i in range(c.from_i, c.to_i): # (1)!
... if i == 0:
... return i - c.from_i # (2)!
... else:
... index_before = index[i - 1]
... index_now = index[i]
... index_next_week = vbt.dt_nb.future_weekday_nb(index_before, 0)
... if index_now >= index_next_week: # (3)!
... return i - c.from_i
... return -1
>>> @njit
... def exit_place_func_nb(c, index):
... for i in range(c.from_i, c.to_i):
... if i == len(index) - 1:
... return i - c.from_i
... else:
... index_now = index[i]
... index_after = index[i + 1]
... index_next_week = vbt.dt_nb.future_weekday_nb(index_now, 0)
... if index_after >= index_next_week: # (4)!
... return i - c.from_i
... return -1
>>> data = vbt.YFData.fetch("BTC-USD", start="2020-01", end="2020-14")
>>> entries, exits = vbt.pd_acc.signals.generate_both(
... data.symbol_wrapper.shape,
... entry_place_func_nb=entry_place_func_nb,
... entry_args=(data.index.vbt.to_ns(),),
... exit_place_func_nb=exit_place_func_nb,
... exit_args=(data.index.vbt.to_ns(),),
... wrapper=data.symbol_wrapper
... )
>>> pd.concat((
... entries.rename("Entries"),
... exits.rename("Exits")
... ), axis=1).to_period("W")
Entries Exits
Date
2019-12-30/2020-01-05 True False
2019-12-30/2020-01-05 False False
2019-12-30/2020-01-05 False False
2019-12-30/2020-01-05 False False
2019-12-30/2020-01-05 False True
2020-01-06/2020-01-12 True False
2020-01-06/2020-01-12 False False
2020-01-06/2020-01-12 False False
2020-01-06/2020-01-12 False False
2020-01-06/2020-01-12 False False
2020-01-06/2020-01-12 False False
2020-01-06/2020-01-12 False True
2020-01-13/2020-01-19 True False
- Iterate over the bars in the current period segment
- If a signal should be placed, return an index relative to the segment
- Place a signal if the current bar crosses Monday
- Place a signal if the next bar crosses Monday
Data transformation¶
- You've fetched some data, how do you change it? There's a new method that puts all symbols into one single DataFrame and passes this DataFrame to a UDF for transformation.
Remove weekends
>>> data = vbt.YFData.fetch(["BTC-USD", "ETH-USD"], start="2020-01", end="2020-14")
>>> new_data = data.transform(lambda df: df[~df.index.weekday.isin([5, 6])])
>>> new_data.close
symbol BTC-USD ETH-USD
Date
2020-01-01 00:00:00+00:00 7200.174316 130.802002
2020-01-02 00:00:00+00:00 6985.470215 127.410179
2020-01-03 00:00:00+00:00 7344.884277 134.171707
2020-01-06 00:00:00+00:00 7769.219238 144.304153
2020-01-07 00:00:00+00:00 8163.692383 143.543991
2020-01-08 00:00:00+00:00 8079.862793 141.258133
2020-01-09 00:00:00+00:00 7879.071289 138.979202
2020-01-10 00:00:00+00:00 8166.554199 143.963776
2020-01-13 00:00:00+00:00 8144.194336 144.226593
Synthetic OHLC¶
- There are new basic models for synthetic OHLC data generation - especially useful for leakage detection.
Generate 3 months of synthetic data using Geometric Brownian Motion
>>> data = vbt.GBMOHLCData.fetch("R", start="2022-01", end="2022-04")
>>> data.plot().show()
TA-Lib time frames¶
- Comparing indicators on different time frames involves a lot of nuances. Thankfully, all TA-Lib indicators now support a parameter that resamples the input arrays to a target time frame, calculates the indicator, and then resamples the output arrays back to the original time frame - a parameterized MTF analysis has never been easier!
Tutorial
Learn more in the MTF analysis tutorial.
Run SMA on multiple time frames and display the whole thing as a heatmap
>>> h1_data = vbt.BinanceData.fetch(
... "BTCUSDT",
... start="3 months ago UTC",
... timeframe="1h"
... )
>>> mtf_sma = vbt.talib("SMA").run(
... h1_data.close,
... timeperiod=14,
... timeframe=["1d", "4h", "1h"],
... skipna=True
... )
>>> mtf_sma.real.vbt.ts_heatmap().show()
Portfolio optimization¶
- Portfolio optimization is the process of creating a portfolio of assets, for which your investment has the maximum return and minimum risk. Usually, this process is performed periodically and involves generating new weights to rebalance an existing portfolio. As most things in vectorbtpro, the weight generation step is implemented as a callback by the user while the optimizer calls that callback periodically. The final result is a collection of the returned weight allocations that can be analyzed, visualized, and used in an actual simulation
Tutorial
Learn more in the Portfolio optimization tutorial.
Allocate assets inversely to their total return in the last month
>>> def regime_change_optimize_func(data):
... returns = data.returns
... total_return = returns.vbt.returns.total()
... weights = data.symbol_wrapper.fill_reduced(0)
... pos_mask = total_return > 0
... if pos_mask.any():
... weights[pos_mask] = total_return[pos_mask] / total_return.abs().sum()
... neg_mask = total_return < 0
... if neg_mask.any():
... weights[neg_mask] = total_return[neg_mask] / total_return.abs().sum()
... return -1 * weights
>>> data = vbt.YFData.fetch(
... ["SPY", "TLT", "XLF", "XLE", "XLU", "XLK", "XLB", "XLP", "XLY", "XLI", "XLV"],
... start="2020",
... end="2023",
... missing_index="drop"
... )
>>> pfo = vbt.PFO.from_optimize_func(
... data.symbol_wrapper,
... regime_change_optimize_func,
... vbt.RepEval("data[index_slice]", context=dict(data=data)),
... every="MS"
... )
>>> pfo.plot().show()
PyPortfolioOpt¶
- PyPortfolioOpt is a popular financial portfolio optimization package that includes both classical methods (Markowitz 1952 and Black-Litterman), suggested best practices (e.g covariance shrinkage), along with many recent developments and novel features, like L2 regularisation, shrunk covariance, and hierarchical risk parity.
Tutorial
Learn more in the Portfolio optimization tutorial.
Run Nested Clustered Optimization (NCO) on a monthly basis
>>> data = vbt.YFData.fetch(
... ["SPY", "TLT", "XLF", "XLE", "XLU", "XLK", "XLB", "XLP", "XLY", "XLI", "XLV"],
... start="2020",
... end="2023",
... missing_index="drop"
... )
>>> pfo = vbt.PFO.from_pypfopt(
... returns=data.returns,
... optimizer="hrp",
... target="optimize",
... every="MS"
... )
>>> pfo.plot().show()
Universal Portfolios¶
- Universal Portfolios is a package putting together different Online Portfolio Selection (OLPS) algorithms.
Tutorial
Learn more in the Portfolio optimization tutorial.
Simulate an online minumum-variance portfolio on a weekly time frame
>>> data = vbt.YFData.fetch(
... ["SPY", "TLT", "XLF", "XLE", "XLU", "XLK", "XLB", "XLP", "XLY", "XLI", "XLV"],
... start="2020",
... end="2023",
... missing_index="drop"
... )
>>> pfo = vbt.PFO.from_universal_algo(
... "MPT",
... data.resample("W").close,
... window=52,
... min_history=4,
... mu_estimator='historical',
... cov_estimator='empirical',
... method='mpt',
... q=0
... )
>>> pfo.plot().show()
Safe resampling¶
- The look-ahead bias is an ongoing threat when working with array data, especially on multiple time frames. Using Pandas alone is strongly discouraged because it's not aware that financial data mainly involves bars where timestamps are opening times and events can happen at any time between bars, and thus falsely assumes that timestamps denote the exact time of an event. In vectorbtpro, there is an entire collection of functions and classes for resampling and analyzing data in a safe way!
Tutorial
Learn more in the MTF analysis tutorial.
Calculate SMA on multiple time frames and display on the same chart
>>> def mtf_sma(close, close_freq, target_freq, timeperiod=5):
... target_close = close.vbt.resample_closing(target_freq) # (1)!
... target_sma = vbt.talib("SMA").run(target_close, timeperiod=timeperiod).real # (2)!
... target_sma = target_sma.rename(f"SMA ({target_freq})")
... return target_sma.vbt.resample_closing(close.index, freq=close_freq) # (3)!
>>> data = vbt.YFData.fetch("BTC-USD", start="2020", end="2023")
>>> fig = mtf_sma(data.close, "D", "D").vbt.plot()
>>> mtf_sma(data.close, "D", "W-MON").vbt.plot(fig=fig)
>>> mtf_sma(data.close, "D", "MS").vbt.plot(fig=fig)
>>> fig.show()
- Resample the source frequency to the target frequency. Close happens at the end of the bar, thus resample as a "closing event".
- Calculate SMA on the target frequency
- Resample the target frequency back to the source frequency to be able to display multiple time frames on the same chart. Since
closecontains gaps, we cannot resample toclose_freqbecause it may result in unaligned series - resample directly to the index ofcloseinstead.
Resamplable objects¶
- Not only you can resample time series, but also complex vectorbtpro objects! Under the hood, each object comprises of a bunch of array-like attributes, thus resampling here simply means aggregating all the accompanied information in one go. This is very convenient when you want to simulate on higher frequency for best accuracy, and then analyze on lower frequency for best speed.
Tutorial
Learn more in the MTF analysis tutorial.
Plot the monthly return heatmap of a random portfolio
>>> import calendar
>>> data = vbt.YFData.fetch("BTC-USD", start="2018", end="2023")
>>> pf = vbt.PF.from_random_signals(data, n=100, direction="both")
>>> mo_returns = pf.resample("MS").returns # (1)!
>>> mo_return_matrix = pd.Series(
... mo_returns.values,
... index=pd.MultiIndex.from_arrays([
... mo_returns.index.year,
... mo_returns.index.month
... ], names=["year", "month"])
... ).unstack("month")
>>> mo_return_matrix.columns = mo_return_matrix.columns.map(lambda x: calendar.month_abbr[x])
>>> mo_return_matrix.vbt.heatmap(
... is_x_category=True,
... trace_kwargs=dict(zmid=0, colorscale="Spectral")
... ).show()
- Resample the entire portfolio to the monthly frequency and compute the returns
Data saver¶
- Imagine a script that can periodically pull the latest data from an exchange and save it to disk, all without your intervention? vectorbtpro implements two classes that can do just this: one that saves to CSV and another one that saves to HDF.
BTCUSDT_1m_saver.py
import vectorbtpro as vbt
import logging
logging.basicConfig(level=logging.INFO)
if __name__ == "__main__":
if vbt.CSVDataSaver.file_exists():
csv_saver = vbt.CSVDataSaver.load()
csv_saver.update()
init_save = False
else:
data = vbt.BinanceData.fetch(
"BTCUSDT",
start="10 minutes ago UTC",
timeframe="1 minute"
)
csv_saver = vbt.CSVDataSaver(data)
init_save = True
csv_saver.update_every(1, "minute", init_save=init_save)
csv_saver.save() # (1)!
- CSV data saver stores only the latest data update, which acts as a starting point of the next update, thus save it and re-use in the next runtime
Run in console and then interrupt
$ python BTCUSDT_1m_saver.py
2023-02-01 12:26:36.744000+00:00 - 2023-02-01 12:36:00+00:00: : 1it [00:01, 1.22s/it]
INFO:vectorbtpro.data.saver:Saved initial 10 rows from 2023-02-01 12:27:00+00:00 to 2023-02-01 12:36:00+00:00
INFO:vectorbtpro.utils.schedule_:Starting schedule manager with jobs [Every 1 minute do update(save_kwargs=None) (last run: [never], next run: 2023-02-01 13:37:38)]
INFO:vectorbtpro.data.saver:Saved 2 rows from 2023-02-01 12:36:00+00:00 to 2023-02-01 12:37:00+00:00
INFO:vectorbtpro.data.saver:Saved 2 rows from 2023-02-01 12:37:00+00:00 to 2023-02-01 12:38:00+00:00
INFO:vectorbtpro.utils.schedule_:Stopping schedule manager
Run in console again to continue
$ python BTCUSDT_1m_saver.py
INFO:vectorbtpro.utils.schedule_:Starting schedule manager with jobs [Every 1 minute do update(save_kwargs=None) (last run: [never], next run: 2023-02-01 13:42:08)]
INFO:vectorbtpro.data.saver:Saved 5 rows from 2023-02-01 12:38:00+00:00 to 2023-02-01 12:42:00+00:00
INFO:vectorbtpro.utils.schedule_:Stopping schedule manager
Polygon.io¶
- Welcome a new class specialized in pulling data from Polygon.io!
Get one month of 30-minute AAPL data from Polygon.io
>>> vbt.PolygonData.set_custom_settings(
... client_config=dict(
... api_key="YOUR_API_KEY"
... )
... )
>>> data = vbt.PolygonData.fetch(
... "AAPL",
... start="2022-12-01", # (1)!
... end="2023-01-01",
... timeframe="30 minutes",
... tz="US/Eastern"
... )
>>> data.get()
Open High Low Close Volume \
Open time
2022-12-01 04:00:00-05:00 148.08 148.08 147.04 147.3700 50886.0
2022-12-01 04:30:00-05:00 147.37 147.37 147.12 147.2600 16575.0
2022-12-01 05:00:00-05:00 147.31 147.51 147.20 147.3800 20753.0
2022-12-01 05:30:00-05:00 147.43 147.56 147.38 147.3800 7388.0
2022-12-01 06:00:00-05:00 147.30 147.38 147.24 147.2400 7416.0
... ... ... ... ... ...
2022-12-30 17:30:00-05:00 129.94 130.05 129.91 129.9487 35694.0
2022-12-30 18:00:00-05:00 129.95 130.00 129.94 129.9500 15595.0
2022-12-30 18:30:00-05:00 129.94 130.05 129.94 130.0100 20287.0
2022-12-30 19:00:00-05:00 129.99 130.04 129.99 130.0000 12490.0
2022-12-30 19:30:00-05:00 130.00 130.04 129.97 129.9700 28271.0
Trade count VWAP
Open time
2022-12-01 04:00:00-05:00 1024 147.2632
2022-12-01 04:30:00-05:00 412 147.2304
2022-12-01 05:00:00-05:00 306 147.3466
2022-12-01 05:30:00-05:00 201 147.4818
2022-12-01 06:00:00-05:00 221 147.2938
... ... ...
2022-12-30 17:30:00-05:00 350 129.9672
2022-12-30 18:00:00-05:00 277 129.9572
2022-12-30 18:30:00-05:00 312 130.0034
2022-12-30 19:00:00-05:00 176 130.0140
2022-12-30 19:30:00-05:00 366 129.9941
[672 rows x 7 columns]
- In the timezone provided via
tz
Alpha Vantage¶
- Welcome a new class specialized in pulling data from Alpha Vantage!
Get Stochastic RSI of IBM from Alpha Vantage
>>> data = vbt.AVData.fetch(
... "IBM",
... category="technical-indicators",
... function="STOCHRSI",
... params=dict(fastkperiod=14)
... )
>>> data.get()
FastD FastK
1999-12-07 00:00:00+00:00 100.0000 100.0000
1999-12-08 00:00:00+00:00 100.0000 100.0000
1999-12-09 00:00:00+00:00 77.0255 31.0765
1999-12-10 00:00:00+00:00 43.6922 0.0000
1999-12-13 00:00:00+00:00 12.0197 4.9826
... ... ...
2023-01-26 00:00:00+00:00 11.7960 0.0000
2023-01-27 00:00:00+00:00 3.7773 0.0000
2023-01-30 00:00:00+00:00 4.4824 13.4471
2023-01-31 00:00:00+00:00 7.8258 10.0302
2023-02-01 16:00:01+00:00 13.0966 15.8126
[5826 rows x 2 columns]
Nasdaq Data Link¶
- Welcome a new class specialized in pulling data from Nasdaq Data Link!
Get Index of Consumer Sentiment from Nasdaq Data Link
>>> data = vbt.NDLData.fetch("UMICH/SOC1")
>>> data.get()
Index
Date
1952-11-30 00:00:00+00:00 86.2
1953-02-28 00:00:00+00:00 90.7
1953-08-31 00:00:00+00:00 80.8
1953-11-30 00:00:00+00:00 80.7
1954-02-28 00:00:00+00:00 82.0
... ...
2022-08-31 00:00:00+00:00 58.2
2022-09-30 00:00:00+00:00 58.6
2022-10-31 00:00:00+00:00 59.9
2022-11-30 00:00:00+00:00 56.8
2022-12-31 00:00:00+00:00 59.7
[632 rows x 1 columns]
Data merging¶
- Often, there's a need to backtest symbols that are coming from different exchanges by putting them into the same basket. For this, vectorbtpro has got a class method that can merge multiple data instances into a single one. Not only you can combine multiple symbols, but also merge datasets that correspond to a single symbol - all done automatically!
Pull BTC datasets from various exchanges and plot them relative to their mean
>>> binance_data = vbt.CCXTData.fetch("BTCUSDT", exchange="binance")
>>> bybit_data = vbt.CCXTData.fetch("BTCUSDT", exchange="bybit")
>>> bitfinex_data = vbt.CCXTData.fetch("BTC/USDT", exchange="bitfinex")
>>> kucoin_data = vbt.CCXTData.fetch("BTC-USDT", exchange="kucoin")
>>> data = vbt.Data.merge([
... binance_data.rename({"BTCUSDT": "Binance"}),
... bybit_data.rename({"BTCUSDT": "Bybit"}),
... bitfinex_data.rename({"BTC/USDT": "Bitfinex"}),
... kucoin_data.rename({"BTC-USDT": "KuCoin"}),
... ], missing_index="drop", silence_warnings=True)
>>> @njit
... def rescale_nb(x):
... return (x - x.mean()) / x.mean()
>>> rescaled_close = data.close.vbt.row_apply(rescale_nb)
>>> rescaled_close = rescaled_close.vbt.rolling_mean(30)
>>> rescaled_close.loc["2020":"2020"].vbt.plot().show()
Pre-computation¶
- There is a tradeoff between memory consumption and execution speed: a 1000-column dataset is usually processed much faster than a 1-column dataset 1000 times. But the first dataset also requires 1000 times more memory than the second one. That's why during the simulation phase, vectorbtpro mainly generates orders while other portfolio attributes such as various balances, equity, and returns are then later reconstructed during the analysis phase if it's needed by the user. For use cases where performance is the main criteria, there are now arguments that allow pre-computing these attributes at the simulation time!
Benchmark a random portfolio with 1000 columns without and with pre-computation
>>> data = vbt.YFData.fetch("BTC-USD")
>>> %%timeit # (1)!
>>> for n in range(1000):
... pf = vbt.PF.from_random_signals(data, n=n, save_returns=False)
... pf.sharpe_ratio
15 s ± 829 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
- Good for RAM, bad for performance
>>> %%timeit
>>> pf = vbt.PF.from_random_signals(data, n=np.arange(1000).tolist(), save_returns=False)
>>> pf.sharpe_ratio
855 ms ± 6.26 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
>>> %%timeit
>>> pf = vbt.PF.from_random_signals(data, n=np.arange(1000).tolist(), save_returns=True)
>>> pf.sharpe_ratio
593 ms ± 7.07 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
Accumulators¶
- Most rolling indicators implemented with Pandas and NumPy require running over data more than once. For example, a simple sum of three arrays involves at least two passes over data. Moreover, if you want to calculate such an indicator iterativelly (i.e., bar by bar), you either need to pre-calculate it entirely and store in memory, or re-calculate each window, which may dramatically hit performance. Accumulators, on the other hand, keep an internal state that allows you to calculate an indicator value every time a new data point arrives, leading to the best performance possible.
Design a one-pass rolling z-score
>>> @njit
... def fastest_rolling_zscore_1d_nb(arr, window, minp=None, ddof=1):
... if minp is None:
... minp = window
... out = np.full(arr.shape, np.nan)
... cumsum = 0.0
... cumsum_sq = 0.0
... nancnt = 0
...
... for i in range(len(arr)):
... pre_window_value = arr[i - window] if i - window >= 0 else np.nan
... mean_in_state = vbt.nb.RollMeanAIS(
... i, arr[i], pre_window_value, cumsum, nancnt, window, minp
... )
... mean_out_state = vbt.nb.rolling_mean_acc_nb(mean_in_state)
... _, _, _, mean = mean_out_state
... std_in_state = vbt.nb.RollStdAIS(
... i, arr[i], pre_window_value, cumsum, cumsum_sq, nancnt, window, minp, ddof
... )
... std_out_state = vbt.nb.rolling_std_acc_nb(std_in_state)
... cumsum, cumsum_sq, nancnt, _, std = std_out_state
... out[i] = (arr[i] - mean) / std
... return out
>>> data = vbt.YFData.fetch("BTC-USD")
>>> rolling_zscore = fastest_rolling_zscore_1d_nb(data.returns.values, 14)
>>> data.symbol_wrapper.wrap(rolling_zscore)
Date
2014-09-17 00:00:00+00:00 NaN
2014-09-18 00:00:00+00:00 NaN
2014-09-19 00:00:00+00:00 NaN
...
2023-02-01 00:00:00+00:00 0.582381
2023-02-02 00:00:00+00:00 -0.705441
2023-02-03 00:00:00+00:00 -0.217880
Freq: D, Name: BTC-USD, Length: 3062, dtype: float64
>>> (data.returns - data.returns.rolling(14).mean()) / data.returns.rolling(14).std()
Date
2014-09-17 00:00:00+00:00 NaN
2014-09-18 00:00:00+00:00 NaN
2014-09-19 00:00:00+00:00 NaN
...
2023-02-01 00:00:00+00:00 0.582381
2023-02-02 00:00:00+00:00 -0.705441
2023-02-03 00:00:00+00:00 -0.217880
Freq: D, Name: Close, Length: 3062, dtype: float64
1D-native indicators¶
- Previously, own indicators could be created only by accepting two-dimensional input arrays, which forced the user to adapt all functions accordingly. With the new feature, the indicator factory can split each input array along columns and pass one column at once, making it super-easy to design indicators that are meant to be natively run on one-dimensional data (such as TA-Lib!).
Create a TA-Lib powered STOCHRSI indicator
>>> import talib
>>> params = dict(
... rsi_period=14,
... fastk_period=5,
... slowk_period=3,
... slowk_matype=0,
... slowd_period=3,
... slowd_matype=0
... )
>>> def stochrsi_1d(close, *args):
... rsi = talib.RSI(close, args[0])
... k, d = talib.STOCH(rsi, rsi, rsi, *args[1:])
... return rsi, k, d
>>> STOCHRSI = vbt.IF(
... input_names=["close"],
... param_names=list(params.keys()),
... output_names=["rsi", "k", "d"]
... ).with_apply_func(stochrsi_1d, takes_1d=True, **params)
>>> data = vbt.YFData.fetch("BTC-USD", start="2022-01", end="2022-06")
>>> stochrsi = STOCHRSI.run(data.close)
>>> fig = stochrsi.k.rename("%K").vbt.plot()
>>> stochrsi.d.rename("%D").vbt.plot(fig=fig)
>>> fig.show()
Parallelizable indicators¶
- Processing of parameter combinations by the indicator factory can be distributed over multiple threads, processes, or even in the cloud. This helps immensely when working with slow indicators
Benchmark a serial and multithreaded rolling min-max indicator
>>> @njit
... def minmax_nb(close, window):
... return (
... vbt.nb.rolling_min_nb(close, window),
... vbt.nb.rolling_max_nb(close, window)
... )
>>> MINMAX = vbt.IF(
... class_name="MINMAX",
... input_names=["close"],
... param_names=["window"],
... output_names=["min", "max"]
... ).with_apply_func(minmax_nb, window=14)
>>> data = vbt.YFData.fetch("BTC-USD")
>>> %%timeit
>>> minmax = MINMAX.run(
... data.close,
... np.arange(2, 200),
... jitted_loop=True
... )
420 ms ± 2.05 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
>>> %%timeit
>>> minmax = MINMAX.run(
... data.close,
... np.arange(2, 200),
... jitted_loop=True,
... jitted_warmup=True, # (1)!
... execute_kwargs=dict(engine="threadpool", n_chunks="auto") # (2)!
... )
120 ms ± 355 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
- Run one parameter combination to compile the indicator before running others in a multithreaded fashion
- One Numba loop per thread, the same number of threads as there are cores
TA-Lib plotting¶
- Every TA-Lib indicator knows how to be plotted - fully automatically based on output flags!
>>> data = vbt.YFData.fetch("BTC-USD", start="2020", end="2021")
>>> vbt.talib("MACD").run(data.close).plot().show()
Indicator expressions¶
- Indicators can now be parsed from expressions. An indicator expression is a regular string that represents a Python code enhanced through various extensions. The indicator factory can derive all the required information such as inputs, parameters, outputs, NumPy, vectorbtpro, and TA-Lib functions, and even complex indicators thanks to a unique format and a built-in matching mechanism. Designing indicators has never been easier!
Build a MACD indicator from an expression
>>> data = vbt.YFData.fetch("BTC-USD", start="2020", end="2021")
>>> expr = """
... MACD:
... fast_ema = @talib_ema(close, @p_fast_w)
... slow_ema = @talib_ema(close, @p_slow_w)
... macd = fast_ema - slow_ema
... signal = @talib_ema(macd, @p_signal_w)
... macd, signal
... """
>>> MACD = vbt.IF.from_expr(expr, fast_w=12, slow_w=26, signal_w=9) # (1)!
>>> macd = MACD.run(data.close)
>>> fig = macd.macd.rename("MACD").vbt.plot()
>>> macd.signal.rename("Signal").vbt.plot(fig=fig)
>>> fig.show()
- No more manually passing
input_names,param_names, and other information!
WorldQuant Alphas¶
- Each of the WorldQuant's 101 Formulaic Alphas is now an indicator
Run the first alpha
>>> data = vbt.YFData.fetch(["BTC-USD", "ETH-USD", "XRP-USD"], missing_index="drop")
>>> vbt.wqa101(1).run(data.close).out
symbol BTC-USD ETH-USD XRP-USD
Date
2017-11-09 00:00:00+00:00 0.166667 0.166667 0.166667
2017-11-10 00:00:00+00:00 0.166667 0.166667 0.166667
2017-11-11 00:00:00+00:00 0.166667 0.166667 0.166667
2017-11-12 00:00:00+00:00 0.166667 0.166667 0.166667
2017-11-13 00:00:00+00:00 0.166667 0.166667 0.166667
... ... ... ...
2023-01-31 00:00:00+00:00 0.166667 0.166667 0.166667
2023-02-01 00:00:00+00:00 0.000000 0.000000 0.500000
2023-02-02 00:00:00+00:00 0.000000 0.000000 0.500000
2023-02-03 00:00:00+00:00 0.000000 0.500000 0.000000
2023-02-04 00:00:00+00:00 -0.166667 0.333333 0.333333
[1914 rows x 3 columns]
Benchmark¶
- Benchmark can be easily set for the entire portfolio.
Compare Microsoft to S&P 500
>>> data = vbt.YFData.fetch(["SPY", "MSFT"], start="2010")
>>> pf = vbt.PF.from_holding(
... close=data.data["MSFT"]["Close"],
... bm_close=data.data["SPY"]["Close"]
... )
>>> pf.plot_cum_returns().show()
Alpaca¶
- Welcome a new class specialized in pulling data from Alpaca!
Get one week of adjusted 1-minute AAPL data from Alpaca
>>> vbt.AlpacaData.set_custom_settings(
... client_config=dict(
... api_key="YOUR_API_KEY",
... secret_key="YOUR_API_SECRET"
... )
... )
>>> data = vbt.AlpacaData.fetch(
... "AAPL",
... start="one week ago 00:00", # (1)!
... end="15 minutes ago", # (2)!
... timeframe="1 minute",
... adjustment="all",
... tz="US/Eastern"
... )
>>> data.get()
Open High Low Close Volume \
Open time
2023-01-30 04:00:00-05:00 145.5400 145.5400 144.0100 144.0200 5452.0
2023-01-30 04:01:00-05:00 144.0800 144.0800 144.0000 144.0500 3616.0
2023-01-30 04:02:00-05:00 144.0300 144.0400 144.0100 144.0100 1671.0
2023-01-30 04:03:00-05:00 144.0100 144.0300 144.0000 144.0300 4721.0
2023-01-30 04:04:00-05:00 144.0200 144.0200 144.0200 144.0200 1343.0
... ... ... ... ... ...
2023-02-03 19:54:00-05:00 154.3301 154.3301 154.3301 154.3301 347.0
2023-02-03 19:55:00-05:00 154.3300 154.3400 154.3200 154.3400 1438.0
2023-02-03 19:56:00-05:00 154.3400 154.3400 154.3300 154.3300 588.0
2023-02-03 19:58:00-05:00 154.3500 154.3500 154.3500 154.3500 555.0
2023-02-03 19:59:00-05:00 154.3400 154.3900 154.3300 154.3900 3835.0
Trade count VWAP
Open time
2023-01-30 04:00:00-05:00 165 144.376126
2023-01-30 04:01:00-05:00 81 144.036336
2023-01-30 04:02:00-05:00 52 144.035314
2023-01-30 04:03:00-05:00 56 144.012680
2023-01-30 04:04:00-05:00 40 144.021854
... ... ...
2023-02-03 19:54:00-05:00 21 154.331340
2023-02-03 19:55:00-05:00 38 154.331756
2023-02-03 19:56:00-05:00 17 154.338971
2023-02-03 19:58:00-05:00 27 154.343090
2023-02-03 19:59:00-05:00 58 154.357219
[4224 rows x 7 columns]
- In the timezone provided via
tz - Remove if you have a paid plan
Formatting engine¶
- VectorBT PRO is a very extensive library that defines thousands of classes, functions, and objects. Thus, when working with any of them, you may want to "see through" the object to gain a better understanding of its attributes and contents. Gladly, there is a new formatting engine that can accurately format any in-house object as a human-readable string. Did you know that the API documentation is partially powered by this engine?
Introspect a data instance
>>> data = vbt.YFData.fetch("BTC-USD", start="2020", end="2021")
>>> vbt.pprint(data) # (1)!
YFData(
wrapper=ArrayWrapper(...),
data=symbol_dict({
'BTC-USD': <pandas.core.frame.DataFrame object at 0x7f7f1fbc6cd0 with shape (366, 7)>
}),
single_symbol=True,
symbol_classes=symbol_dict(),
fetch_kwargs=symbol_dict({
'BTC-USD': dict(
start='2020',
end='2021'
)
}),
returned_kwargs=symbol_dict({
'BTC-USD': dict()
}),
last_index=symbol_dict({
'BTC-USD': Timestamp('2020-12-31 00:00:00+0000', tz='UTC')
}),
tz_localize=datetime.timezone.utc,
tz_convert='UTC',
missing_index='nan',
missing_columns='raise'
)
>>> vbt.pdir(data) # (2)!
type path
attr
align_columns classmethod vectorbtpro.data.base.Data
align_index classmethod vectorbtpro.data.base.Data
build_column_config_doc classmethod vectorbtpro.data.base.Data
... ... ...
vwap property vectorbtpro.data.base.Data
wrapper property vectorbtpro.base.wrapping.Wrapping
xs function vectorbtpro.base.indexing.PandasIndexer
>>> vbt.phelp(data.get) # (3)!
YFData.get(
columns=None,
symbols=None,
**kwargs
):
Get one or more columns of one or more symbols of data.
- Just like the Python's
printcommand to pretty-print the contents of any vectorbtpro object - Just like the Python's
dircommand to pretty-print the attributes of a class, object, or module - Just like the Python's
helpcommand to pretty-print the signature and docstring of a function
Chunking¶
- An innovative new chunking mechanism that takes a specification of how arguments should be chunked, automatically splits array-like arguments, passes each chunk to the function for execution, and merges back the results. This way, you can split large arrays and run any function in a distributed manner! Additionally, vectorbtpro implements a central registry and provides the chunking specification for all arguments of most Numba-compiled functions, including the simulation functions. Chunking can be enabled by a single command. No more out-of-memory errors!
Backtest at most 100 parameter combinations at once
>>> data = vbt.YFData.fetch(["BTC-USD", "ETH-USD"])
>>> @vbt.chunked(
... size=vbt.LenSizer(arg_query="fast_windows"), # (1)!
... arg_take_spec=dict(
... data=None, # (2)!
... fast_windows=vbt.ChunkSlicer(), # (3)!
... slow_windows=vbt.ChunkSlicer()
... ),
... merge_func=lambda x: pd.concat(x).sort_index(), # (4)!
... chunk_len=100,
... show_progress=True,
... clear_cache=True, # (5)!
... collect_garbage=True
... )
... def backtest(data, fast_windows, slow_windows):
... fast_ma = vbt.MA.run(data.close, fast_windows, short_name="fast")
... slow_ma = vbt.MA.run(data.close, slow_windows, short_name="slow")
... entries = fast_ma.ma_crossed_above(slow_ma)
... exits = fast_ma.ma_crossed_below(slow_ma)
... pf = vbt.PF.from_signals(data.close, entries, exits)
... return pf.total_return
>>> fast_windows, slow_windows = zip(*combinations(np.arange(2, 100), 2))
>>> backtest(data, fast_windows, slow_windows)
- Get the count of the passed parameter combinations
- Don't split data
- Slice both parameter arrays
- Merge the Series returned by each chunk into one Series
- Clear cache and collect garbage after processing each chunk
fast_window slow_window symbol
2 3 BTC-USD 193.124482
ETH-USD 12.247315
4 BTC-USD 159.600953
ETH-USD 15.825041
5 BTC-USD 124.703676
...
97 98 ETH-USD 3.947346
99 BTC-USD 25.551881
ETH-USD 3.442949
98 99 BTC-USD 27.943574
ETH-USD 3.540720
Name: total_return, Length: 9506, dtype: float64
Parallel Numba¶
- Most Numba-compiled functions were rewritten to process columns in parallel using automatic parallelization with
@jit, which can be enabled by a single command. Best suited for lightweight functions applied on wide arrays.
Benchmark the rolling mean without and with parallelization
>>> df = pd.DataFrame(np.random.uniform(size=(1000, 1000)))
>>> %timeit df.rolling(10).mean() # (1)!
45.6 ms ± 138 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
>>> %timeit df.vbt.rolling_mean(10) # (2)!
5.33 ms ± 302 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)
>>> %timeit df.vbt.rolling_mean(10, jitted=dict(parallel=True)) # (3)!
1.82 ms ± 5.21 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
- Using Pandas
- Using Numba without parallelization
- Using Numba with parallelization
Multithreading¶
- Integration of ThreadPoolExecutor from
concurrent.futures, ThreadPool frompathos, and Dask backend for running multiple chunks across multiple threads. Best suited for accelerating heavyweight functions that release GIL, such as Numba and C functions. Multithreading + Chunking + Numba =
Benchmark 1000 random portfolios without and with multithreading
>>> data = vbt.YFData.fetch(["BTC-USD", "ETH-USD"])
>>> %timeit vbt.PF.from_random_signals(data.close, n=[100] * 1000)
613 ms ± 37.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
>>> %timeit vbt.PF.from_random_signals(data.close, n=[100] * 1000, chunked="threadpool")
294 ms ± 8.91 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
Multiprocessing¶
- Integration of ProcessPoolExecutor from
concurrent.futures, ProcessPool and ParallelPool frompathos, and Ray backend for running multiple chunks across multiple processes. Best suited for accelerating heavyweight functions that do not release GIL, such as regular Python functions, and accept leightweight arguments that are easy to serialize. Ever wanted to test billions of hyperparameter combinations in a matter of minutes? This is now possible by scaling functions and entire applications up in the cloud using Ray clusters
Benchmark running a slow function on each column without and with multiprocessing
>>> @vbt.chunked(
... size=vbt.ArraySizer(arg_query="items", axis=1),
... arg_take_spec=dict(
... items=vbt.ArraySelector(axis=1)
... ),
... merge_func=np.column_stack
... )
... def bubble_sort(items):
... items = items.copy()
... for i in range(len(items)):
... for j in range(len(items) - 1 - i):
... if items[j] > items[j + 1]:
... items[j], items[j + 1] = items[j + 1], items[j]
... return items
>>> items = np.random.uniform(size=(1000, 3))
>>> %timeit bubble_sort(items)
456 ms ± 1.36 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
>>> %timeit bubble_sort(items, _execute_kwargs=dict(engine="pathos"))
165 ms ± 1.51 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
Jitting¶
- Jitting means just-in-time compiling. In the vectorbtpro universe though, jitting simply means accelerating. Although Numba remains the primary jitter, vectorbtpro now enables implementation of custom jitter classes, such as that for vectorized NumPy and even JAX with GPU support. Every jitted function is registered globally, so you can switch between different implementations or even disable jitting entirely using a single command.
Run different implementations of the cumulative sum
>>> data = vbt.YFData.fetch("BTC-USD", start="7 days ago")
>>> log_returns = np.log1p(data.close.pct_change())
>>> log_returns.vbt.cumsum() # (1)!
Date
2023-01-31 00:00:00+00:00 0.000000
2023-02-01 00:00:00+00:00 0.024946
2023-02-02 00:00:00+00:00 0.014271
2023-02-03 00:00:00+00:00 0.013310
2023-02-04 00:00:00+00:00 0.008288
2023-02-05 00:00:00+00:00 -0.007967
2023-02-06 00:00:00+00:00 -0.010087
Freq: D, Name: Close, dtype: float64
>>> log_returns.vbt.cumsum(jitted=False) # (2)!
Date
2023-01-31 00:00:00+00:00 0.000000
2023-02-01 00:00:00+00:00 0.024946
2023-02-02 00:00:00+00:00 0.014271
2023-02-03 00:00:00+00:00 0.013310
2023-02-04 00:00:00+00:00 0.008288
2023-02-05 00:00:00+00:00 -0.007967
2023-02-06 00:00:00+00:00 -0.010087
Freq: D, Name: Close, dtype: float64
>>> @vbt.register_jitted(task_id_or_func=vbt.nb.nancumsum_nb) # (3)!
... def nancumsum_np(arr):
... return np.nancumsum(arr, axis=0)
>>> log_returns.vbt.cumsum(jitted="np") # (4)!
Date
2023-01-31 00:00:00+00:00 0.000000
2023-02-01 00:00:00+00:00 0.024946
2023-02-02 00:00:00+00:00 0.014271
2023-02-03 00:00:00+00:00 0.013310
2023-02-04 00:00:00+00:00 0.008288
2023-02-05 00:00:00+00:00 -0.007967
2023-02-06 00:00:00+00:00 -0.010087
Freq: D, Name: Close, dtype: float64
- Using the built-in Numba-compiled function
- Using the built-in function but with Numba disabled → regular Python → slow!
- Register a NumPy version for the built-in Numba function
- Using the NumPy version
Caching¶
- Caching has been reimplemented from the ground up, and now it's being managed by a central registry. This allows for tracking useful statistics of all cacheable parts of vectorbtpro, such as to display the total cached size in MB. Full control and transparency
Get the cache statistics after computing the statistics of a random portfolio
>>> data = vbt.YFData.fetch("BTC-USD")
>>> pf = vbt.PF.from_random_signals(data.close, n=5)
>>> _ = pf.stats()
>>> pf.get_ca_setup().get_status_overview(
... filter_func=lambda setup: setup.caching_enabled,
... include=["hits", "misses", "total_size"]
... )
hits misses total_size
object
portfolio:0.drawdowns 0 1 70.9 kB
portfolio:0.exit_trades 0 1 70.5 kB
portfolio:0.filled_close 6 1 24.3 kB
portfolio:0.init_cash 3 1 32 Bytes
portfolio:0.init_position 0 1 32 Bytes
portfolio:0.init_position_value 0 1 32 Bytes
portfolio:0.init_value 5 1 32 Bytes
portfolio:0.input_value 1 1 32 Bytes
portfolio:0.orders 9 1 69.7 kB
portfolio:0.total_profit 1 1 32 Bytes
portfolio:0.trades 0 1 70.5 kB
Meta methods¶
- Many methods such as rolling apply are now available in two flavors: regular (instance methods) and meta (class methods). Regular methods are bound to a single array and do not have to take metadata anymore, while meta methods are not bound to any array and act as micro-pipelines with their own broadcasting and templating logic. Here, vectorbtpro closes one of the key limitations of Pandas - the inability to apply a function on multiple arrays at once.
Compute the rolling z-score on one array and the rolling correlation coefficient on two arrays
>>> @njit
... def zscore_nb(x): # (1)!
... return (x[-1] - np.mean(x)) / np.std(x)
>>> data = vbt.YFData.fetch("BTC-USD", start="2020", end="2021")
>>> data.close.rolling(14).apply(zscore_nb, raw=True) # (2)!
Date
2020-01-01 00:00:00+00:00 NaN
...
2020-12-27 00:00:00+00:00 1.543527
2020-12-28 00:00:00+00:00 1.734715
2020-12-29 00:00:00+00:00 1.755125
2020-12-30 00:00:00+00:00 2.107147
2020-12-31 00:00:00+00:00 1.781800
Freq: D, Name: Close, Length: 366, dtype: float64
>>> data.close.vbt.rolling_apply(14, zscore_nb) # (3)!
2020-01-01 00:00:00+00:00 NaN
...
2020-12-27 00:00:00+00:00 1.543527
2020-12-28 00:00:00+00:00 1.734715
2020-12-29 00:00:00+00:00 1.755125
2020-12-30 00:00:00+00:00 2.107147
2020-12-31 00:00:00+00:00 1.781800
Freq: D, Name: Close, Length: 366, dtype: float64
>>> @njit
... def corr_meta_nb(from_i, to_i, col, a, b): # (4)!
... a_window = a[from_i:to_i, col]
... b_window = b[from_i:to_i, col]
... return np.corrcoef(a_window, b_window)[1, 0]
>>> data2 = vbt.YFData.fetch(["ETH-USD", "XRP-USD"], start="2020", end="2021")
>>> vbt.pd_acc.rolling_apply( # (5)!
... 14,
... corr_meta_nb,
... vbt.Rep("a"),
... vbt.Rep("b"),
... broadcast_named_args=dict(a=data.close, b=data2.close)
... )
symbol ETH-USD XRP-USD
Date
2020-01-01 00:00:00+00:00 NaN NaN
... ... ...
2020-12-27 00:00:00+00:00 0.636862 -0.511303
2020-12-28 00:00:00+00:00 0.674514 -0.622894
2020-12-29 00:00:00+00:00 0.712531 -0.773791
2020-12-30 00:00:00+00:00 0.839355 -0.772295
2020-12-31 00:00:00+00:00 0.878897 -0.764446
[366 rows x 2 columns]
- Access to the window only
- Using Pandas
- Using the regular method, which accepts the same function as pandas
- Access to one to multiple whole arrays
- Using the meta method, which accepts metadata and variable arguments
Robust crossovers¶
- Crossovers are now robust to NaNs.
Remove a bunch of data points and plot the crossovers
>>> data = vbt.YFData.fetch("BTC-USD", start="2022-01", end="2022-03")
>>> fast_sma = vbt.talib("SMA").run(data.close, vbt.Default(5)).real
>>> slow_sma = vbt.talib("SMA").run(data.close, vbt.Default(10)).real
>>> fast_sma.iloc[np.random.choice(np.arange(len(fast_sma)), 5)] = np.nan
>>> slow_sma.iloc[np.random.choice(np.arange(len(slow_sma)), 5)] = np.nan
>>> crossed_above = fast_sma.vbt.crossed_above(slow_sma, dropna=True)
>>> crossed_below = fast_sma.vbt.crossed_below(slow_sma, dropna=True)
>>> fig = fast_sma.rename("Fast SMA").vbt.lineplot()
>>> slow_sma.rename("Slow SMA").vbt.lineplot(fig=fig)
>>> crossed_above.vbt.signals.plot_as_entries(fast_sma, fig=fig)
>>> crossed_below.vbt.signals.plot_as_exits(fast_sma, fig=fig)
>>> fig.show()
Array expressions¶
- When combining multiple arrays, they often need to be properly aligned and broadcasted before the actual operation. Using Pandas alone won't do the trick because Pandas is too strict in this regard. Luckily, vectorbtpro has an accessor class method that can take a regular Python expression, identify all the variable names, extract the corresponding arrays from the current context, broadcast them, and only then evaluate the actual expression (also using NumExpr!)
Evaluate a multiline array expression based on a Bollinger Bands indicator
>>> data = vbt.YFData.fetch(["BTC-USD", "ETH-USD"])
>>> low = data.low
>>> high = data.high
>>> bb = vbt.talib("BBANDS").run(data.close)
>>> upperband = bb.upperband
>>> lowerband = bb.lowerband
>>> bandwidth = (bb.upperband - bb.lowerband) / bb.middleband
>>> up_th = vbt.Param([0.3, 0.4])
>>> low_th = vbt.Param([0.1, 0.2])
>>> expr = """
... narrow_bands = bandwidth < low_th
... above_upperband = high > upperband
... wide_bands = bandwidth > up_th
... below_lowerband = low < lowerband
... (narrow_bands & above_upperband) | (wide_bands & below_lowerband)
... """
>>> mask = vbt.pd_acc.eval(expr)
>>> mask.sum()
low_th up_th symbol
0.1 0.3 BTC-USD 344
ETH-USD 171
0.4 BTC-USD 334
ETH-USD 158
0.2 0.3 BTC-USD 444
ETH-USD 253
0.4 BTC-USD 434
ETH-USD 240
dtype: int64
Resource management¶
- New profiling tools to measure the execution time and memory usage of any code block
Profile getting the Sharpe ratio of a random portfolio
>>> data = vbt.YFData.fetch("BTC-USD")
>>> with (
... vbt.Timer() as timer,
... vbt.MemTracer() as mem_tracer
... ):
... print(vbt.PF.from_random_signals(data.close, n=100).sharpe_ratio)
0.33111243921865163
>>> print(timer.elapsed())
74.15 milliseconds
>>> print(mem_tracer.peak_usage())
459.7 kB
Hyperfast rolling metrics¶
- Rolling metrics based on returns were optimized for best performance - up to 1000x speedup!
Benchmark the rolling Sortino ratio
>>> import quantstats as qs
>>> index = pd.date_range("2020", periods=100000, freq="1T")
>>> returns = pd.Series(np.random.normal(0, 0.001, size=len(index)), index=index)
>>> %timeit qs.stats.rolling_sortino(returns, rolling_period=10) # (1)!
2.79 s ± 24.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
>>> %timeit returns.vbt.returns.rolling_sortino_ratio(window=10) # (2)!
8.12 ms ± 199 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)
- Using QuantStats
- Using VectorBT PRO
Local data¶
Once remote data has been fetched, you most likely want to persist it on disk. There are two new options for this: either serialize the entire data class, or save the actual data to CSV or HDF5. Each dataset can be stored in a single flat file, which makes it easier to work with than a database. Upon saving, the data can be effortlessly loaded back either by deserializing, or by using data classes that specialize in loading data from CSV and HDF5 files. These classes support a variety of features, including filtering by row and datetime ranges, updating, chunking, and even a smart dataset search that can traverse sub-directories recursively and return datasets that match a specific glob pattern or regular expression 
Fetch and save symbols separately, then load them back jointly
>>> btc_data = vbt.BinanceData.fetch("BTCUSDT")
>>> eth_data = vbt.BinanceData.fetch("ETHUSDT")
>>> btc_data.to_hdf()
>>> eth_data.to_hdf()
>>> data = vbt.BinanceData.from_hdf(start="2020", end="2021")
>>> data.close
symbol BTCUSDT ETHUSDT
Open time
2020-01-01 00:00:00+00:00 7200.85 130.77
2020-01-02 00:00:00+00:00 6965.71 127.19
2020-01-03 00:00:00+00:00 7344.96 134.35
2020-01-04 00:00:00+00:00 7354.11 134.20
2020-01-05 00:00:00+00:00 7358.75 135.37
... ... ...
2020-12-27 00:00:00+00:00 26281.66 685.11
2020-12-28 00:00:00+00:00 27079.41 730.41
2020-12-29 00:00:00+00:00 27385.00 732.00
2020-12-30 00:00:00+00:00 28875.54 752.17
2020-12-31 00:00:00+00:00 28923.63 736.42
[366 rows x 2 columns]
Cash deposits¶
- Cash can be deposited and withdrawn at any time.
DCA $10 into Bitcoin each month
>>> data = vbt.YFData.fetch("BTC-USD")
>>> cash_deposits = data.symbol_wrapper.fill(0.0)
>>> month_start_mask = ~data.index.tz_convert(None).to_period("M").duplicated()
>>> cash_deposits[month_start_mask] = 10
>>> pf = vbt.PF.from_orders(
... data.close,
... init_cash=0,
... cash_deposits=cash_deposits
... )
>>> pf.input_value # (1)!
1020.0
>>> pf.final_value
20674.328828315127
- Total invested
Cash earnings¶
- Cash can be continuously earned or spent depending on the current position.
Backtest Apple without and with dividend reinvestment
>>> data = vbt.YFData.fetch("AAPL", start="2010")
>>> pf_kept = vbt.PF.from_holding( # (1)!
... data.close,
... cash_dividends=data.get("Dividends")
... )
>>> pf_kept.cash.iloc[-1] # (2)!
93.9182408043298
>>> pf_kept.assets.iloc[-1] # (3)!
15.37212731743495
>>> pf_reinvested = vbt.PF.from_orders( # (4)!
... data.close,
... cash_dividends=data.get("Dividends")
... )
>>> pf_reinvested.cash.iloc[-1]
0.0
>>> pf_reinvested.assets.iloc[-1]
18.203284859405468
>>> fig = pf_kept.value.rename("Value (kept)").vbt.plot()
>>> pf_reinvested.value.rename("Value (reinvested)").vbt.plot(fig=fig)
>>> fig.show()
- Keep dividends as cash
- Final cash balance
- Final number of shares in the portfolio
- Reinvest dividends at the next bar
In-outputs¶
- Portfolio can now take and return any user-defined arrays filled during the simulation, such as signals. In-output arrays can broadcast together with regular arrays using templates and broadcastable named arguments. Additionally, vectorbtpro will (semi-)automatically figure out how to correctly wrap and index each array, for example, whenever you select a column from the entire portfolio.
Track the debt of a random portfolio during the simulation
>>> data = vbt.YFData.fetch(["BTC-USD", "ETH-USD"], missing_index="drop")
>>> size = data.symbol_wrapper.fill(np.nan)
>>> rand_indices = np.random.choice(np.arange(len(size)), 10)
>>> size.iloc[rand_indices[0::2]] = -np.inf
>>> size.iloc[rand_indices[1::2]] = np.inf
>>> @njit
... def post_segment_func_nb(c):
... for col in range(c.from_col, c.to_col):
... col_debt = c.last_debt[col]
... c.in_outputs.debt[c.i, col] = col_debt
... if col_debt > c.in_outputs.max_debt[col]:
... c.in_outputs.max_debt[col] = col_debt
>>> pf = vbt.PF.from_def_order_func(
... data.close,
... size=size,
... post_segment_func_nb=post_segment_func_nb,
... in_outputs=dict(
... debt=vbt.RepEval("np.empty_like(close)"),
... max_debt=vbt.RepEval("np.full(close.shape[1], 0.)")
... ) # (1)!
... )
>>> print(pf.get_in_output("debt")) # (2)!
symbol BTC-USD ETH-USD
Date
2017-11-09 00:00:00+00:00 0.000000 0.000000
2017-11-10 00:00:00+00:00 0.000000 0.000000
2017-11-11 00:00:00+00:00 0.000000 0.000000
2017-11-12 00:00:00+00:00 0.000000 0.000000
2017-11-13 00:00:00+00:00 0.000000 0.000000
... ... ...
2023-02-08 00:00:00+00:00 43.746892 25.054571
2023-02-09 00:00:00+00:00 43.746892 25.054571
2023-02-10 00:00:00+00:00 43.746892 25.054571
2023-02-11 00:00:00+00:00 43.746892 25.054571
2023-02-12 00:00:00+00:00 43.746892 25.054571
[1922 rows x 2 columns]
>>> pf.get_in_output("max_debt") # (3)!
symbol
BTC-USD 75.890464
ETH-USD 25.926328
Name: max_debt, dtype: float64
- Tell portfolio class to wait until all arrays are broadcast and create a new floating array of the final shape
- Portfolio instance knows how to properly wrap a custom NumPy array into a pandas object
- The same holds for reduced NumPy arrays
Flexible attributes¶
- Portfolio attributes can now be partly or even entirely computed from user-defined arrays. This allows great control of post-simulation analysis, for example, to override some simulation data, to test hyperparameters without having to re-simulate the entire portfolio, or to avoid repeated reconstruction when caching is disabled.
Compute the net exposure by caching its components
>>> data = vbt.YFData.fetch("BTC-USD")
>>> pf = vbt.PF.from_random_signals(data.close, n=100)
>>> value = pf.get_value() # (1)!
>>> long_exposure = vbt.PF.get_gross_exposure( # (2)!
... asset_value=pf.get_asset_value(direction="longonly"),
... value=value,
... wrapper=pf.wrapper
... )
>>> short_exposure = vbt.PF.get_gross_exposure(
... asset_value=pf.get_asset_value(direction="shortonly"),
... value=value,
... wrapper=pf.wrapper
... )
>>> del value # (3)!
>>> net_exposure = vbt.PF.get_net_exposure(
... long_exposure=long_exposure,
... short_exposure=short_exposure,
... wrapper=pf.wrapper
... )
>>> del long_exposure
>>> del short_exposure
>>> net_exposure
Date
2014-09-17 00:00:00+00:00 1.0
2014-09-18 00:00:00+00:00 1.0
2014-09-19 00:00:00+00:00 1.0
2014-09-20 00:00:00+00:00 1.0
2014-09-21 00:00:00+00:00 1.0
...
2023-02-08 00:00:00+00:00 0.0
2023-02-09 00:00:00+00:00 0.0
2023-02-10 00:00:00+00:00 0.0
2023-02-11 00:00:00+00:00 0.0
2023-02-12 00:00:00+00:00 0.0
Freq: D, Length: 3071, dtype: float64
- Call the instance method to use the data stored in the portfolio
- Call the class method to provide all the data explicitly
- Delete the object as soon as it's no needed, to release memory
Shortcut properties¶
- In-output arrays can be used to override regular portfolio attributes. Portfolio will automatically pick the pre-computed array and perform all future calculations using this array, without wasting time on its reconstruction.
Modify the returns from within the simulation
>>> data = vbt.YFData.fetch("BTC-USD")
>>> size = data.symbol_wrapper.fill(np.nan)
>>> rand_indices = np.random.choice(np.arange(len(size)), 10)
>>> size.iloc[rand_indices[0::2]] = -np.inf
>>> size.iloc[rand_indices[1::2]] = np.inf
>>> @njit
... def post_segment_func_nb(c):
... for col in range(c.from_col, c.to_col):
... return_now = c.last_return[col]
... return_now = 0.5 * return_now if return_now > 0 else return_now
... c.in_outputs.returns[c.i, col] = return_now
>>> pf = vbt.PF.from_def_order_func(
... data.close,
... size=size,
... size_type="targetpercent",
... post_segment_func_nb=post_segment_func_nb,
... in_outputs=dict(
... returns=vbt.RepEval("np.empty_like(close)")
... )
... )
>>> pf.returns # (1)!
Date
2014-09-17 00:00:00+00:00 0.000000
2014-09-18 00:00:00+00:00 0.000000
2014-09-19 00:00:00+00:00 0.000000
2014-09-20 00:00:00+00:00 0.000000
2014-09-21 00:00:00+00:00 0.000000
...
2023-02-08 00:00:00+00:00 -0.015227
2023-02-09 00:00:00+00:00 -0.053320
2023-02-10 00:00:00+00:00 -0.008439
2023-02-11 00:00:00+00:00 0.005569 << modified
2023-02-12 00:00:00+00:00 0.001849 << modified
Freq: D, Length: 3071, dtype: float64
>>> pf.get_returns() # (2)!
Date
2014-09-17 00:00:00+00:00 0.000000
2014-09-18 00:00:00+00:00 0.000000
2014-09-19 00:00:00+00:00 0.000000
2014-09-20 00:00:00+00:00 0.000000
2014-09-21 00:00:00+00:00 0.000000
...
2023-02-08 00:00:00+00:00 -0.015227
2023-02-09 00:00:00+00:00 -0.053320
2023-02-10 00:00:00+00:00 -0.008439
2023-02-11 00:00:00+00:00 0.011138
2023-02-12 00:00:00+00:00 0.003697
Freq: D, Length: 3071, dtype: float64
- Pre-computed returns
- Actual returns
Templates¶
It's super-easy to extend classes, but vectorbtpro revolves around functions, so how do we enhance them or change their workflow? The easiest way is to introduce a tiny function (i.e., callback) that can be provided by the user and called by the main fucntion at some point in time. But this would require the main function to know which arguments to pass to the callback and what to do with the outputs. Here's a better idea: allow most arguments of the main function to become callbacks and then execute them to reveal the actual values. Such arguments are called "templates" and such a process is called "substitution". Templates are especially useful when some arguments (such as arrays) should be constructed only once all the required information is available, for example, once other arrays have been broadcast. Also, each such substitution opportunity has its own identifier such that you can control when a template should be substituted. In vectorbtpro, templates are first-class citizens and are integrated into most functions for an unmatched flexibility! 
Design a template-enhanced resampling functionality
>>> def resample_apply(index, by, apply_func, *args, template_context={}, **kwargs):
... grouper = index.vbt.get_grouper(by) # (1)!
... results = {}
... with vbt.get_pbar() as pbar:
... for group, group_idxs in grouper: # (2)!
... group_index = index[group_idxs]
... context = {"group": group, "group_index": group_index, **template_context} # (3)!
... final_apply_func = vbt.substitute_templates(apply_func, context, sub_id="apply_func") # (4)!
... final_args = vbt.substitute_templates(args, context, sub_id="args")
... final_kwargs = vbt.substitute_templates(kwargs, context, sub_id="kwargs")
... results[group] = final_apply_func(*final_args, **final_kwargs)
... pbar.update(1)
... return pd.Series(results)
>>> data = vbt.YFData.fetch(["BTC-USD", "ETH-USD"], missing_index="drop")
>>> resample_apply(
... data.index, "Y",
... lambda x, y: x.corr(y), # (5)!
... vbt.RepEval("btc_close[group_index]"), # (6)!
... vbt.RepEval("eth_close[group_index]"),
... template_context=dict(
... btc_close=data.get("Close", "BTC-USD"), # (7)!
... eth_close=data.get("Close", "ETH-USD")
... )
... )
- Build a grouper. Accepts both group-by and resample instructions.
- Iterate over groups in the grouper. Each group consists of the label (such as
2017-01-01 00:00:00+00:00) and row indices corresponding to this label. - Populate a new context with the information on the current group and user-provided external information
- Substitute the function and arguments using the newly-populated context
- Simple function to compute the correlation coefficient of two arrays
- Define both arguments as expression templates where we select the data corresponding to each group. All variables in these expressions will be automatically recognized and replaced by the current context. Once evaluated, the templates will be substituted by their outputs.
- Here we can specify additional information our templates depend upon
2017 0.808930
2018 0.897112
2019 0.753659
2020 0.940741
2021 0.553255
2022 0.975911
2023 0.974914
Freq: A-DEC, dtype: float64
And many more...¶
- Expect more killer features to be added on a weekly basis!