# Parallel execution and caching (/features/performance/parallel-execution-and-caching)

## Chunk caching \[#chunk-caching]

New in v2024.5.15

✅ Most workflows that use the VBT's execution framework—such as data pulling, chunking,
parameterization, splitting, and optimization—can now offload intermediate results to disk and
reload them if the workflow crashes and restarts. You can confidently test billions of parameter
combinations on cloud instances without worrying about losing your data.

```python title="Execute a basic range with random fallouts while caching successful attempts"
>>> np.random.seed(42)

>>> @vbt.parameterized(cache_chunks=True, chunk_len=1)  # (1)
... def basic_iterator(i):
...     print("i:", i)
...     rand_number = np.random.uniform()
...     if rand_number < 0.2:
...         print("failed ⛔")
...         raise ValueError
...     return i

>>> attempt = 0
>>> while True:
...     attempt += 1
...     print("attempt", attempt)
...     try:
...         basic_iterator(vbt.Param(np.arange(10)))
...         print("completed 🎉")
...         break
...     except ValueError:
...         pass
attempt 1
i: 0
i: 1
i: 2
i: 3
i: 4
failed ⛔
attempt 2
i: 4
failed ⛔
attempt 3
i: 4
failed ⛔
attempt 4
i: 4
i: 5
i: 6
i: 7
failed ⛔
attempt 5
i: 7
i: 8
i: 9
completed 🎉
```

1.  Cache each function call.

## Chunking \[#chunking]

New in 1.0.0

✅ A new, innovative chunking mechanism lets you specify how arguments should be chunked. It
automatically splits array-like arguments, passes each chunk to the function for execution, and then
merges the results. This enables you to split large arrays and run any function in a distributed
manner. VBT also features a central registry and provides chunking specifications for all arguments
of most Numba-compiled functions, including simulation functions. Chunking can be enabled with a
single command. You no longer have to worry about out-of-memory errors! 🎉

```python title="Backtest at most 100 parameter combinations at once"
>>> @vbt.chunked(
...     chunk_len=100,
...     merge_func="concat",  # (1)
...     execute_kwargs=dict(  # (2)
...         clear_cache=True,
...         collect_garbage=True
...     )
... )
... def backtest(data, fast_windows, slow_windows):  # (3)
...     fast_ma = vbt.MA.run(data.close, fast_windows, short_name="fast")
...     slow_ma = vbt.MA.run(data.close, slow_windows, short_name="slow")
...     entries = fast_ma.ma_crossed_above(slow_ma)
...     exits = fast_ma.ma_crossed_below(slow_ma)
...     pf = vbt.PF.from_signals(data.close, entries, exits)
...     return pf.total_return

>>> param_product = vbt.combine_params(  # (4)
...     dict(
...         fast_window=vbt.Param(range(2, 100), condition="fast_window < slow_window"),
...         slow_window=vbt.Param(range(2, 100)),
...     ),
...     build_index=False
... )
>>> backtest(
...     vbt.YFData.pull(["BTC-USD", "ETH-USD"]),  # (5)
...     vbt.Chunked(param_product["fast_window"]),  # (6)
...     vbt.Chunked(param_product["slow_window"])
... )
```

1.  Concatenate the Series returned by each chunk into one Series.
2.  Show a progress bar, and also clear cache and collect garbage after processing each chunk.
3.  This function takes a data instance, as well as two parameter arrays: fast and slow window lengths.
    Both arrays will have the same number of values; for example, the first combination matches the
    first value in `fast_windows` with the first value in `slow_windows`.
4.  Generate conditional parameter combinations.
5.  Do not split the data into chunks.
6.  Split both parameter arrays into chunks.

Chunk 48/48: 100%

```text title="Output"
fast_window  slow_window  symbol
2            3            BTC-USD    193.124482
                          ETH-USD     12.247315
             4            BTC-USD    159.600953
                          ETH-USD     15.825041
             5            BTC-USD    124.703676
                                            ...
97           98           ETH-USD      3.947346
             99           BTC-USD     25.551881
                          ETH-USD      3.442949
98           99           BTC-USD     27.943574
                          ETH-USD      3.540720
Name: total_return, Length: 9506, dtype: float64
```

## Multithreading \[#multithreading]

New in 1.0.0

✅ Integration of [ThreadPoolExecutor](https://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor)
from `concurrent.futures`, [ThreadPool](https://pathos.readthedocs.io/en/latest/pathos.html#pathos.pools.ThreadPool)
from `pathos`, and [Dask](https://dask.org/) backend for running multiple chunks across multiple
threads. This is best for speeding up heavyweight functions that release the GIL, such as Numba and C
functions. Multithreading + Chunking + Numba = 💪

```python title="Benchmark 1000 random portfolios without and with multithreading"
>>> data = vbt.YFData.pull(["BTC-USD", "ETH-USD"])

>>> %timeit vbt.PF.from_random_signals(data.close, n=[100] * 1000, seed=42)
613 ms ± 37.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

>>> %timeit vbt.PF.from_random_signals(data.close, n=[100] * 1000, seed=42, chunked="threadpool")
294 ms ± 8.91 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
```

## Multiprocessing \[#multiprocessing]

New in 1.0.0

✅ Integration of [ProcessPoolExecutor](https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor)
from `concurrent.futures`, [ProcessPool](https://pathos.readthedocs.io/en/latest/pathos.html#pathos.pools.ProcessPool)
and [ParallelPool](https://pathos.readthedocs.io/en/latest/pathos.html#pathos.pools.ParallelPool) from
`pathos`, [WorkerPool](https://sybrenjansen.github.io/mpire/usage/workerpool/index.html) from
`mpire`, and [Ray](https://www.ray.io/) backend for running multiple chunks across multiple processes.
This is best for speeding up heavyweight functions that do not release the GIL, such as regular Python
functions, as well as lightweight arguments that are easy to serialize. Ever wanted to test billions of
hyperparameter combinations in just a few minutes? Now you can by scaling functions and entire
applications in the cloud using [Ray clusters](https://docs.ray.io/en/latest/cluster/getting-started.html)
👀

```python title="Benchmark running a slow function on each column without and with multiprocessing"
>>> @vbt.chunked(
...     size=vbt.ArraySizer(arg_query="items", axis=1),
...     arg_take_spec=dict(
...         items=vbt.ArraySelector(axis=1)
...     ),
...     merge_func=np.column_stack
... )
... def bubble_sort(items):
...     items = items.copy()
...     for i in range(len(items)):
...         for j in range(len(items) - 1 - i):
...             if items[j] > items[j + 1]:
...                 items[j], items[j + 1] = items[j + 1], items[j]
...     return items

>>> np.random.seed(42)
>>> items = np.random.uniform(size=(1000, 3))

>>> %timeit bubble_sort(items)
456 ms ± 1.36 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

>>> %timeit bubble_sort(items, _execute_kwargs=dict(engine="pathos"))
165 ms ± 1.51 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
```

## Caching \[#caching]

New in 1.0.0

✅ Caching has been completely reimplemented and is now managed by a central registry.
This enables tracking useful statistics for all cacheable parts of VBT, such as showing the total
cached size in MB. You get full control and transparency 🪟

```python title="Get the cache statistics after computing the statistics of a random portfolio"
>>> data = vbt.YFData.pull("BTC-USD")
>>> pf = vbt.PF.from_random_signals(data.close, n=5, seed=42)
>>> _ = pf.stats()

>>> pf.get_ca_setup().get_status_overview(
...     filter_func=lambda setup: setup.caching_enabled,
...     include=["hits", "misses", "total_size"]
... )
                                 hits  misses total_size
object
portfolio:0.drawdowns               0       1    70.9 kB
portfolio:0.exit_trades             0       1    70.5 kB
portfolio:0.filled_close            6       1    24.3 kB
portfolio:0.init_cash               3       1   32 Bytes
portfolio:0.init_position           0       1   32 Bytes
portfolio:0.init_position_value     0       1   32 Bytes
portfolio:0.init_value              5       1   32 Bytes
portfolio:0.input_value             1       1   32 Bytes
portfolio:0.orders                  9       1    69.7 kB
portfolio:0.total_profit            1       1   32 Bytes
portfolio:0.trades                  0       1    70.5 kB
```