Walk-forward and cross-validation

Does an optimized strategy hold up on data it has never seen? VBT runs walk-forward optimization and time-series cross-validation in Python: split history into training and test periods, pick parameters on each training period, and judge them only on the period that follows.

from itertools import product

data = vbt.YFData.pull("BTC-USD", start="2020-01-01", end="2025-01-01")
splitter = vbt.Splitter.from_rolling(  
    data.index,
    length=455,
    split=365,
    offset=90,
    offset_anchor="prev_start",
    set_labels=["train", "test"],
)
splitter.n_splits

16

pairs = [(f, s) for f, s in product(range(10, 55, 5), range(60, 220, 20))]
fast_windows, slow_windows = zip(*pairs)
fast = vbt.MA.run(data.close, window=list(fast_windows), short_name="fast")
slow = vbt.MA.run(data.close, window=list(slow_windows), short_name="slow")
entries = fast.ma_above(slow)  
exits = fast.ma_below(slow)

bounds = splitter.get_bounds(index_bounds=True)
rows, oos_returns = [], []
for split in range(splitter.n_splits):
    train_start, train_end = bounds.loc[(split, "train")]
    test_start, test_end = bounds.loc[(split, "test")]
    train = slice(train_start, train_end - pd.Timedelta(days=1))
    test = slice(test_start, test_end - pd.Timedelta(days=1))
    train_pf = vbt.PF.from_signals(
        data.loc[train], entries.loc[train], exits.loc[train], fees=0.001
    )
    best = train_pf.sharpe_ratio.idxmax()  
    test_pf = vbt.PF.from_signals(
        data.loc[test], entries.loc[test, best], exits.loc[test, best], fees=0.001
    )
    rows.append(dict(
        test_start=test_start.date(),
        fast=best[0],
        slow=best[1],
        train_sharpe=train_pf.sharpe_ratio[best],
        test_sharpe=test_pf.sharpe_ratio,
    ))
    oos_returns.append(test_pf.returns)

summary = pd.DataFrame(rows)
summary.head(6).round(2)

   test_start  fast  slow  train_sharpe  test_sharpe
0  2020-12-31    10   200          3.19         3.56
1  2021-03-31    10   200          3.42        -2.00
2  2021-06-29    20    60          2.83         0.98
3  2021-09-27    20    60          2.97         0.12
4  2021-12-26    20    80          1.86        -2.01
5  2022-03-26    20    80          0.80        -1.74

summary[["train_sharpe", "test_sharpe"]].mean().round(2)

train_sharpe    1.92
test_sharpe     0.24
dtype: float64

oos = pd.concat(oos_returns)  
hold = data.close.loc[oos.index] / data.close.loc[oos.index].iloc[0]
print(round((1 + oos).prod(), 3), round(hold.iloc[-1], 3))

2.744 3.36

The parameters that looked best in each training year had an average Sharpe ratio of 1.92 there, and 0.24 in the quarter that followed. Stitched together, the out-of-sample strategy grew 2.7 times while holding Bitcoin grew 3.4 times. That gap between in-sample and out-of-sample results is what walk-forward testing exists to reveal.

Walk-forward optimization

Walk-forward testing repeats one cycle: optimize on a training window, apply the result to the next window, and move forward. Join the test returns to evaluate the repeated selection process. Each test window above is simulated separately and can enter on its first bar when the fast average is above the slow one. For strategies that must be flat at each boundary, add an exit on the last bar of every test window, and for strategies whose state must carry over, continue the portfolio from one window to the next as described on the Live simulation page. Test-period portfolios join along time with vbt.PF.row_stack, which gives one out-of-sample portfolio with every metric and chart.

You choose how much history each decision uses. Rolling windows keep a fixed amount of recent data, such as the latest year. Expanding windows retain the earlier history as new observations arrive. Calendar-based splits let you match a research schedule, such as selecting parameters monthly or retraining a model each quarter.

Splitters

vbt.Splitter builds the splits:

Each split can hold several sets, such as training and test, sized in bars, fractions, or time periods, with gaps between them. splitter.plot() shows every split on a timeline, and get_coverage and get_overlap_matrix measure how much of the data each set covers and how much splits overlap, as the Splitter highlight below shows.

Leave a gap between training and testing

An expanding window can grow its training history while reserving a fixed test period and a gap between the two. Here, every test covers three days, with two days left out before it starts. The table shows the first and last included dates of each set.

index = pd.date_range("2025-01-01", periods=18, freq="D")
splitter = vbt.Splitter.from_expanding(
    index,
    min_length=12,
    offset=3,
    split=(1.0, vbt.RelRange(length=2, is_gap=True), 3),
    split_range_kwargs=dict(backwards=True),
    set_labels=["train", "test"],
)
splitter.get_bounds(index_bounds=True, right_inclusive=True)

bound            start        end
split set
0     train 2025-01-01 2025-01-07
      test  2025-01-10 2025-01-12
1     train 2025-01-01 2025-01-10
      test  2025-01-13 2025-01-15
2     train 2025-01-01 2025-01-13
      test  2025-01-16 2025-01-18

is_gap=True leaves those dates out of both sets. Splitting backwards reserves the test period and gap first, then gives the remaining history to training. Use the same splitter to take matching slices from your prices, features, and labels.

Applying splits

splitter.take(obj) cuts data, indicators, or portfolios into the splits, and splitter.apply runs a function on each one. The @vbt.split decorator adds splitting to any function, and @vbt.cv_split combines splitting with a parameter search: it optimizes on each training set, picks the best combination, and evaluates it on the test set in one call, as the CV decorator and Split decorator highlights below show. Results come back labeled by split and set, ready to compare. By default, cv_split keeps the combination with the highest result, so the function should return one number per combination, such as a Sharpe ratio. With selection="min" it keeps the lowest, and a template over the grid results can apply any other rule, for example a score built from several metrics.

Inspect full parameter grids

Keep the training grid with return_grid=True, or evaluate the full grid on each set with return_grid="all". This lets you compare the surrounding parameter values and see whether a strong training result belongs to a broad region or one isolated combination. The selection still comes from the training results. Use the extra test results to study robustness.

Validate across assets and timeframes

You can run the validation workflow separately for each asset by placing a parameterized function around cv_split. The outer search selects the asset, while the inner search selects strategy parameters within each training window. Results retain labels so you can compare which settings work across symbols and which depend on one market.

Split boundaries can also be defined on one time index and applied to data at another frequency. For example, define daily training and test boundaries, then slice the minute bars used by your backtest.

Keep indicator history available

A test window can use earlier prices to warm up a moving average without counting those earlier bars in its performance. Precompute indicators that only use present and past prices, as in the opening example, or give the function a history buffer and trim it before simulation. This is useful when a long lookback would otherwise leave much of a short test window without signals.

Purged and combinatorial cross-validation

When labels look into the future, as many machine learning targets do, a training sample near a test period can contain information from that period. Purging removes those samples, and an embargo leaves a gap after each test period, following López de Prado. PurgedWalkForwardCV purges walk-forward folds, and PurgedKFoldCV runs combinatorial purged cross-validation: every combination of a chosen number of folds becomes a test set, so one history yields many out-of-sample tests instead of one, as the Purged CV highlight below shows.

For labels whose outcomes become known at different times, supply prediction and evaluation times so purging can account for each sample's information period. Use purged walk-forward splits for chronological retraining, or combinatorial purged splits to compare many train/test arrangements. The ten-fold, two-test-fold example below produces 45 combinations.

scikit-learn

Splitters work with scikit-learn in both directions. vbt.SplitterCV turns a VBT splitter into a scikit-learn cross-validator for GridSearchCV or cross_val_score, and vbt.Splitter.from_sklearn turns a scikit-learn splitter into a VBT one for plotting and analysis.

This also gives an existing scikit-learn model a route into the workflow. Split features and labels together, fit the model on training data, then score predictions on the matching test set. You can take the same slices into your own Python pipeline when fitting a model, generating signals, and backtesting predictions require separate steps.

✅ Added support for walk-forward cross-validation (CV) with purging, as well as combinatorial CV with purging and embargoing, based on Marcos Lopez de Prado's Advances in Financial Machine Learning.

splitter = vbt.Splitter.from_purged_kfold(
    vbt.date_range("2024", "2025"),
    n_folds=10,
    n_test_folds=2,
    purge_td="3 days",
    embargo_td="3 days"
)
splitter.plots().show()

✅ Most cross-validation tasks involve testing a grid of parameter combinations on the training data, selecting the best parameter combination, and validating it on the test data. This process must be repeated for each split. The cross-validation decorator combines the parameterized and split decorators to automate this task.

@vbt.cv_split(
    splitter="from_rolling",
    splitter_kwargs=dict(length=365, split=0.5, set_labels=["train", "test"]),
    takeable_args=["data"],
    parameterized_kwargs=dict(random_subset=100, seed=42),
    merge_func="concat"
)
def sma_crossover_cv(data, fast_period, slow_period, metric):
    fast_sma = data.run("sma", fast_period, hide_params=True)
    slow_sma = data.run("sma", slow_period, hide_params=True)
    entries = fast_sma.real_crossed_above(slow_sma)
    exits = fast_sma.real_crossed_below(slow_sma)
    pf = vbt.PF.from_signals(data, entries, exits, direction="both")
    return pf.deep_getattr(metric)

sma_crossover_cv(
    vbt.YFData.pull("BTC-USD", start="4 years ago"),
    vbt.Param(np.arange(20, 50), condition="x < slow_period"),
    vbt.Param(np.arange(20, 50)),
    "trades.expectancy"
)

split  set    fast_period  slow_period
0      train  20           25               8.015725
       test   20           23               0.573465
1      train  40           48              -4.356317
       test   39           40               5.666271
2      train  24           45              18.253340
       test   22           36             111.202831
3      train  20           31              54.626024
       test   20           25              -1.596945
4      train  25           48              41.328588
       test   25           30               6.620254
5      train  26           32               7.178085
       test   24           29               4.087456
6      train  22           23              -0.581255
       test   22           31              -2.494519
dtype: float64

✅ Normally, to run a function on each split, you need to build a splitter specifically targeted at the input data provided to the function. This means that each time the input data changes, you must recreate the splitter. The split decorator automates this process by wrapping the function, giving it access to all arguments so it can make splitting decisions as needed. Essentially, it can "infect" any Python function with splitting functionality 🦠

@vbt.split(
    splitter="from_grouper",
    splitter_kwargs=dict(by="Q"),
    takeable_args=["data"],
    merge_func="concat"
)
def get_quarter_return(data):
    return data.returns.vbt.returns.total()

data = vbt.YFData.pull("BTC-USD")
get_quarter_return(data.loc["2021"])

Date
2021Q1    1.005805
2021Q2   -0.407050
2021Q3    0.304383
2021Q4   -0.037627
Freq: Q-DEC, dtype: float64

get_quarter_return(data.loc["2022"])

Date
2022Q1   -0.045047
2022Q2   -0.572515
2022Q3    0.008429
2022Q4   -0.143154
Freq: Q-DEC, dtype: float64

✅ Splitters in scikit-learn are not ideal for validating ML-based and rule-based trading strategies. VBT provides a juggernaut class that supports many splitting schemes that are safe for backtesting, including rolling windows, expanding windows, time-anchored windows, random windows for block bootstraps, and even Pandas-native groupby and resample instructions such as "M" for monthly frequency. As a bonus, the produced splits can be easily analyzed and visualized! For example, you can detect any split or set overlaps, convert all splits into a single boolean mask for custom analysis, group splits and sets, and analyze their distribution relative to each other. This class contains more lines of code than the entire backtesting.py package, so do not underestimate the new king in town! 🦏

data = vbt.YFData.pull("BTC-USD", start="4 years ago")
splitter = vbt.Splitter.from_rolling(
    data.index,
    length="360 days",
    split=0.5,
    set_labels=["train", "test"],
    freq="daily"
)
splitter.plots().show()