# Strategy optimization (/features/optimization/strategy-optimization)

## Paramables \[#paramables]

New in v2024.4.1

✅ Each analyzable VBT object (such as data, indicator, or portfolio) can now be split into
items, which are multiple objects of the same type, each containing only one column or group. This
makes it possible to use VBT objects as standalone parameters and process only a subset of
information at a time, such as a symbol in a data instance or a parameter combination in an
indicator.

```python title="Combine outputs of a SMA indicator combinatorially"
>>> @vbt.parameterized(merge_func="column_stack")
... def get_signals(fast_sma, slow_sma):  # (1)
...     entries = fast_sma.crossed_above(slow_sma)
...     exits = fast_sma.crossed_below(slow_sma)
...     return entries, exits

>>> data = vbt.YFData.pull(["BTC-USD", "ETH-USD"])
>>> sma = data.run("talib:sma", timeperiod=range(20, 50, 2))  # (2)
>>> fast_sma = sma.rename_levels({"sma_timeperiod": "fast"})  # (3)
>>> slow_sma = sma.rename_levels({"sma_timeperiod": "slow"})
>>> entries, exits = get_signals(
...     vbt.Param(fast_sma, condition="__fast__ < __slow__"),  # (4)
...     vbt.Param(slow_sma)
... )
>>> entries.columns
MultiIndex([(20, 22, 'BTC-USD'),
            (20, 22, 'ETH-USD'),
            (20, 24, 'BTC-USD'),
            (20, 24, 'ETH-USD'),
            (20, 26, 'BTC-USD'),
            (20, 26, 'ETH-USD'),
            ...
            (44, 46, 'BTC-USD'),
            (44, 46, 'ETH-USD'),
            (44, 48, 'BTC-USD'),
            (44, 48, 'ETH-USD'),
            (46, 48, 'BTC-USD'),
            (46, 48, 'ETH-USD')],
           names=['fast', 'slow', 'symbol'], length=210)
```

1.  Regular function that takes two indicators and returns signals.
2.  Run an SMA indicator once on all time periods.
3.  Copy the indicator and rename the parameter level to get a distinct indicator instance.
4.  Pass both indicator instances as parameters. This splits each instance into smaller
    instances with only one column. Also, remove all columns where the fast window is greater than or
    equal to the slow window.

## Lazy parameter grids \[#lazy-parameter-grids]

New in v2023.12.23

✅ The [parameterized decorator](#parameterized-decorator) no longer needs to materialize
parameter grids if you are only interested in a subset of all parameter combinations. This change
enables the generation of random parameter combinations almost instantly, no matter how large the
total number of possible combinations is.

```python title="Test a random subset of a huge number of parameter combinations"
>>> @vbt.parameterized(merge_func="concat")
... def test_combination(data, n, sl_stop, tsl_stop, tp_stop):
...     return data.run(
...         "from_random_signals",
...         n=n,
...         sl_stop=sl_stop,
...         tsl_stop=tsl_stop,
...         tp_stop=tp_stop,
...         seed=42,
...     ).total_return

>>> n = np.arange(10, 100)
>>> sl_stop = np.arange(1, 1000) / 1000
>>> tsl_stop = np.arange(1, 1000) / 1000
>>> tp_stop = np.arange(1, 1000) / 1000
>>> len(n) * len(sl_stop) * len(tsl_stop) * len(tp_stop)
89730269910

>>> test_combination(
...     vbt.YFData.pull("BTC-USD"),
...     n=vbt.Param(n),
...     sl_stop=vbt.Param(sl_stop),
...     tsl_stop=vbt.Param(tsl_stop),
...     tp_stop=vbt.Param(tp_stop),
...     _random_subset=10,
...     _seed=42
... )
n   sl_stop  tsl_stop  tp_stop
18  0.476    0.485     0.862       2.233997
21  0.530    0.697     0.017       0.091041
49  0.499    0.560     0.954      15.987541
50  0.535    0.200     0.477       2.182477
72  0.763    0.360     0.129      -0.040304
78  0.503    0.071     0.533      10.851910
79  0.656    0.388     0.930      11.854461
80  0.746    0.042     0.644       0.702603
87  0.274    0.539     0.434       2.072847
97  0.806    0.206     0.426       7.136395
dtype: float64
```

## Mono-chunks \[#mono-chunks]

New in 1.13.0

✅ The [parameterized decorator](#parameterized-decorator) now supports splitting parameter
combinations into "mono-chunks," merging the parameter values within each chunk into a single value,
and running the entire chunk with a single function call. This means you are no longer limited to
processing only one parameter combination at a time 🌪️ Keep in mind that your function
must be adapted to handle multiple parameter values, and you should modify the merging function as
needed.

```python title="Test 100 combinations of SL and TP values per thread"
>>> @vbt.parameterized(
...     merge_func="concat",
...     mono_chunk_len=100,  # (1)
...     chunk_len="auto",  # (2)
...     engine="threadpool",  # (3)
...     warmup=True  # (4)
... )
... @njit(nogil=True)
... def test_stops_nb(close, entries, exits, sl_stop, tp_stop):
...     sim_out = vbt.pf_nb.from_signals_nb(
...         target_shape=(close.shape[0], sl_stop.shape[1]),
...         group_lens=np.full(sl_stop.shape[1], 1),
...         close=close,
...         long_entries=entries,
...         short_entries=exits,
...         sl_stop=sl_stop,
...         tp_stop=tp_stop,
...         save_returns=True
...     )
...     return vbt.ret_nb.total_return_nb(sim_out.in_outputs.returns)

>>> data = vbt.YFData.pull("BTC-USD", start="2020")  # (5)
>>> entries, exits = data.run("randnx", n=10, seed=42, hide_params=True, unpack=True)  # (6)
>>> sharpe_ratios = test_stops_nb(
...     vbt.to_2d_array(data.close),
...     vbt.to_2d_array(entries),
...     vbt.to_2d_array(exits),
...     sl_stop=vbt.Param(np.arange(0.01, 1.0, 0.01), mono_merge_func=np.column_stack),  # (7)
...     tp_stop=vbt.Param(np.arange(0.01, 1.0, 0.01), mono_merge_func=np.column_stack)
... )
>>> sharpe_ratios.vbt.heatmap().show()
```

1.  100 values are combined into one array, forming a single mono-chunk.
2.  Execute N mono-chunks in parallel, where N is the number of cores.
3.  Use multithreading.
4.  Execute one mono-chunk to compile the function before distributing other chunks.
5.  The function above operates with only one symbol.
6.  Pick 10 entries and exits randomly.
7.  For each mono-chunk, stack all values into a two-dimensional array.

Heatmap of BTC-USD total returns across stop-loss and take-profit combinations. [Figure data (JSON)](/assets/figures/features/optimization/chunked-parameters.faa469abe97f.json)

## Conditional parameters \[#conditional-parameters]

New in 1.8.1

✅ Parameters can depend on each other. For example, when testing a crossover of moving averages,
it makes no sense to test a fast window that is longer than the slow window. By filtering
out such cases, you only need to evaluate about half as many parameter combinations.

```python title="Test slow windows being longer than fast windows by at least 5"
>>> @vbt.parameterized(merge_func="column_stack")
... def ma_crossover_signals(data, fast_window, slow_window):
...     fast_sma = data.run("sma", fast_window, short_name="fast_sma")
...     slow_sma = data.run("sma", slow_window, short_name="slow_sma")
...     entries = fast_sma.real_crossed_above(slow_sma.real)
...     exits = fast_sma.real_crossed_below(slow_sma.real)
...     return entries, exits

>>> entries, exits = ma_crossover_signals(
...     vbt.YFData.pull("BTC-USD", start="one year ago UTC"),
...     vbt.Param(np.arange(5, 50), condition="slow_window - fast_window >= 5"),
...     vbt.Param(np.arange(5, 50))
... )
>>> entries.columns
MultiIndex([( 5, 10),
            ( 5, 11),
            ( 5, 12),
            ( 5, 13),
            ( 5, 14),
            ...
            (42, 48),
            (42, 49),
            (43, 48),
            (43, 49),
            (44, 49)],
           names=['fast_window', 'slow_window'], length=820)
```

## Random search \[#random-search]

New in 1.7.0

✅ While grid search tests every possible combination of hyperparameters, random search selects
and tests random combinations of hyperparameters. This is especially useful when there is a huge number of
parameter combinations. Random search has also been shown to find equal or better values than
grid search with fewer function evaluations. The indicator factory, parameterized decorator,
and any method that performs broadcasting now support random search out of the box.

```python title="Test a random subset of SL, TSL, and TP combinations"
>>> data = vbt.YFData.pull("BTC-USD", start="2020")
>>> stop_values = np.arange(1, 100) / 100  # (1)
>>> pf = vbt.PF.from_random_signals(
...     data,
...     n=100,
...     seed=42,
...     sl_stop=vbt.Param(stop_values),
...     tsl_stop=vbt.Param(stop_values),
...     tp_stop=vbt.Param(stop_values),
...     broadcast_kwargs=dict(random_subset=1000, seed=42)  # (2)
... )
>>> pf.total_return.sort_values(ascending=False)
sl_stop  tsl_stop  tp_stop
0.02     0.22      0.55       1.093685
0.93     0.60      0.99       1.091450
0.88     0.68      0.99       1.091450
0.79     0.69      0.99       1.091450
         0.62      0.99       1.091450
                                   ...
0.08     0.84      0.14      -0.466277
0.14     0.09      0.12      -0.502329
0.80     0.11      0.11      -0.509395
0.19     0.11      0.09      -0.529528
0.29     0.11      0.06      -0.563992
Name: total_return, Length: 1000, dtype: float64
```

1.  100 combinations of each parameter = 100 ^ 3 = 1,000,000 combinations.
2.  The indicator factory and parameterized decorator accept this argument directly.

## Parameterized decorator \[#parameterized-decorator]

New in 1.7.0

✅ There is a special decorator that allows any Python function to accept multiple
parameter combinations, even if the function itself supports only one. The decorator wraps the function,
gains access to its arguments, identifies all arguments acting as parameters,
builds a grid from them, and calls the underlying function on each parameter combination from that grid.
The execution can be easily parallelized. Once all outputs are ready, it merges them into
a single object. Use cases are endless: from running indicators that cannot be wrapped with the indicator
factory, to parameterizing entire pipelines! 🪄

=== "Example 1: Basic SMA indicator"
    ```python title="Parameterize a basic SMA indicator without Indicator Factory"
    >>> @vbt.parameterized(merge_func="column_stack")  # (1)
    ... def sma(close, window):
    ...     return close.rolling(window).mean()

    >>> data = vbt.YFData.pull("BTC-USD")
    >>> sma(data.close, vbt.Param(range(20, 50)))
    ```

    1.  Use `column_stack` to merge time series in the form of DataFrames and
        complex VBT objects such as portfolios.

    Combination 30/30: 100%

    ```text title="Output"
    window                               20            21            22  \
    Date
    2014-09-17 00:00:00+00:00           NaN           NaN           NaN
    2014-09-18 00:00:00+00:00           NaN           NaN           NaN
    2014-09-19 00:00:00+00:00           NaN           NaN           NaN
    ...                                 ...           ...           ...
    2024-03-07 00:00:00+00:00  57657.135156  57395.376488  57147.339134
    2024-03-08 00:00:00+00:00  58488.990039  58163.942708  57891.045455
    2024-03-09 00:00:00+00:00  59297.836523  58956.156064  58624.648793

    ...

    window                               48            49
    Date
    2014-09-17 00:00:00+00:00           NaN           NaN
    2014-09-18 00:00:00+00:00           NaN           NaN
    2014-09-19 00:00:00+00:00           NaN           NaN
    ...                                 ...           ...
    2024-03-07 00:00:00+00:00  49928.186686  49758.599330
    2024-03-08 00:00:00+00:00  50483.072266  50303.123565
    2024-03-09 00:00:00+00:00  51040.440837  50846.672353

    [3462 rows x 30 columns]
    ```

=== "Example 2: Bollinger Bands pipeline"
    ```python title="Parameterize an entire Bollinger Bands pipeline"
    >>> @vbt.parameterized(merge_func="concat")  # (1)
    ... def bbands_sharpe(data, timeperiod=14, nbdevup=2, nbdevdn=2, thup=0.3, thdn=0.1):
    ...     bb = data.run(
    ...         "talib_bbands",
    ...         timeperiod=timeperiod,
    ...         nbdevup=nbdevup,
    ...         nbdevdn=nbdevdn
    ...     )
    ...     bandwidth = (bb.upperband - bb.lowerband) / bb.middleband
    ...     cond1 = data.low < bb.lowerband
    ...     cond2 = bandwidth > thup
    ...     cond3 = data.high > bb.upperband
    ...     cond4 = bandwidth < thdn
    ...     entries = (cond1 & cond2) | (cond3 & cond4)
    ...     exits = (cond1 & cond4) | (cond3 & cond2)
    ...     pf = vbt.PF.from_signals(data, entries, exits)
    ...     return pf.sharpe_ratio

    >>> bbands_sharpe(
    ...     vbt.YFData.pull("BTC-USD"),
    ...     nbdevup=vbt.Param([1, 2]),  # (2)
    ...     nbdevdn=vbt.Param([1, 2]),
    ...     thup=vbt.Param([0.4, 0.5]),
    ...     thdn=vbt.Param([0.1, 0.2])
    ... )
    ```

    1.  Use `concat` to merge metrics in the form of scalars and Series.
    2.  Builds the Cartesian product of 4 parameters.

    Combination 16/16: 100%

    ```text title="Output"
    nbdevup  nbdevdn  thup  thdn
    1        1        0.4   0.1     1.681532
                            0.2     1.617400
                      0.5   0.1     1.424175
                            0.2     1.563520
             2        0.4   0.1     1.218554
                            0.2     1.520852
                      0.5   0.1     1.242523
                            0.2     1.317883
    2        1        0.4   0.1     1.174562
                            0.2     1.469828
                      0.5   0.1     1.427940
                            0.2     1.460635
             2        0.4   0.1     1.000210
                            0.2     1.378108
                      0.5   0.1     1.196087
                            0.2     1.782502
    dtype: float64
    ```

## Array-like parameters \[#array-like-parameters]

New in 1.5.0

✅ The broadcasting mechanism has been completely refactored and now supports parameters. Many parameters
in VBT, such as SL and TP, are array-like and can be provided per row, per column, or even per element.
Internally, even a scalar is treated as a regular time series and is broadcast along with other proper
time series. Previously, to test multiple parameter combinations, you had to tile other time series so that
all shapes matched perfectly. With this feature, the tiling procedure is performed automatically!

```python title="Write a steep slope indicator without indicator factory"
>>> def steep_slope(close, up_th):
...     r = vbt.broadcast(dict(close=close, up_th=up_th))
...     return r["close"].pct_change() >= r["up_th"]

>>> data = vbt.YFData.pull("BTC-USD", start="2020", end="2022")
>>> fig = data.plot(plot_volume=False)
>>> sma = vbt.talib("SMA").run(data.close, timeperiod=50).real
>>> sma.rename("SMA").vbt.plot(fig=fig)
>>> mask = steep_slope(sma, vbt.Param([0.005, 0.01, 0.015]))  # (1)

>>> def plot_mask_ranges(column, color):
...     mask.vbt.ranges.plot_shapes(
...         column=column,
...         plot_close=False,
...         add_shape_kwargs=dict(fillcolor=color),
...         fig=fig
...     )
>>> plot_mask_ranges(0.005, "orangered")
>>> plot_mask_ranges(0.010, "orange")
>>> plot_mask_ranges(0.015, "yellow")
>>> fig.update_xaxes(showgrid=False)
>>> fig.update_yaxes(showgrid=False)
>>> fig.show()
```

1.  Tests three parameters and generates a mask with three columns, one for each parameter.

BTC-USD OHLC and 50-day SMA with steep-slope ranges at three thresholds from 2020 through 2021. [Figure data (JSON)](/assets/figures/features/optimization/steep-slope.27dbe5be055b.json)

## Parameters \[#parameters]

New in 1.5.0

✅ There is a new module for working with parameters.

```python title="Generate 10,000 random parameter combinations for MACD"
>>> from itertools import combinations

>>> window_space = np.arange(100)
>>> fastk_windows, slowk_windows = list(zip(*combinations(window_space, 2)))  # (1)
>>> window_type_space = list(vbt.enums.WType)
>>> param_product = vbt.combine_params(
...     dict(
...         fast_window=vbt.Param(fastk_windows, level=0),  # (2)
...         slow_window=vbt.Param(slowk_windows, level=0),
...         signal_window=vbt.Param(window_space, level=1),
...         macd_wtype=vbt.Param(window_type_space, level=2),  # (3)
...         signal_wtype=vbt.Param(window_type_space, level=2),
...     ),
...     random_subset=10_000,
...     seed=42,
...     build_index=False
... )
>>> pd.DataFrame(param_product)
      fast_window  slow_window  signal_window  macd_wtype  signal_wtype
0               0            1              0           2             2
1               0            3              5           1             1
2               0            3             55           4             4
3               0            3             80           2             2
4               0            4             17           2             2
...           ...          ...            ...         ...           ...
9995           97           99             14           0             0
9996           97           99             35           2             2
9997           98           99              6           2             2
9998           98           99             44           4             4
9999           98           99             78           4             4

[10000 rows x 5 columns]
```

1.  Fast windows should be shorter than slow windows.
2.  Fast and slow windows were already combined, so they share the same product level.
3.  Window types do not need to be combined, so they share the same product level.