Pairs trading

A pairs trading strategy is a statistical arbitrage and convergence strategy based on the historical correlation between two instruments. It involves taking a long position in one instrument and a short position in the other. These offsetting positions create a hedging strategy that aims to profit from both positive and negative trends. A high positive correlation, typically at least 0.8, between the two instruments is the main source of the strategy's profits. When the correlation deviates, we look to buy the underperforming instrument and sell short the outperforming one. If the securities revert to their historical correlation, which is the outcome we aim for, a profit is generated from the convergence of their prices. As a result, pairs trading can be used to seek profits regardless of market conditions, whether the market is trending up, down, or moving sideways.

Selection

When designing a pairs trading strategy, it is more important to select pairs based on cointegration rather than just correlation. Correlated instruments usually move in a similar way, but over time, the price ratio (or spread) between them may diverge significantly. Cointegrated instruments, however, do not always move together in the same direction: the spread between them can widen on some days, but their prices usually "pull back together" to the mean, providing optimal conditions for pairs arbitrage trading.

The two main methods for identifying cointegration are the Engle-Granger test and the Johansen test. We will use the Engle-Granger test, as its augmented version is available in statsmodels. The concept behind the Engle-Granger test is straightforward. We perform a linear regression between the two asset prices and check if the residual is stationary using the Augmented Dickey-Fuller (ADF) test. If the residual is stationary, then the two asset prices are cointegrated.

First, let's create a universe of instruments from which to select our pairs. We will search for all available USDT symbols on Binance and download their daily history. Instead of downloading all of them at once, we will fetch each symbol individually and append it to an HDF file. We take this approach because most symbols have limited history, and we want to avoid using extra RAM by extending datasets with NaNs. We will also skip this entire process if the file already exists.

from vectorbtpro import *

SYMBOLS = vbt.BinanceData.list_symbols("*USDT")  
POOL_FILE = "temp/data_pool.h5"
START = "2018"
END = "2023"

# vbt.remove_dir("temp", with_contents=True, missing_ok=True)
vbt.make_dir("temp")  

if not vbt.file_exists(POOL_FILE):
    with vbt.ProgressBar(total=len(SYMBOLS)) as pbar:  
        collected = 0
        for symbol in SYMBOLS:
            try:
                data = vbt.BinanceData.pull(
                    symbol,
                    start=START,
                    end=END,
                    show_progress=False,
                    silence_warnings=True
                )
                data.to_hdf(POOL_FILE)  
                collected += 1
            except Exception:
                pass
            pbar.set_prefix(f"{symbol} ({collected})")  
            pbar.update()

Although this process takes some time, we now have a file containing data for each symbol under its own key. One major advantage of using HDF files (and VBT in particular) is that we can load the entire file and join all contained keys with a single command.

We still have one more decision to make: which period should we analyze to select the optimal pair? It is important not to use the same date range for both pair selection and strategy backtesting, as this could introduce survivorship bias. Therefore, let's reserve a more recent period for backtesting.

SELECT_START = "2020"
SELECT_END = "2021"

data = vbt.HDFData.pull(
    POOL_FILE,
    start=SELECT_START,
    end=SELECT_END,
    silence_warnings=True
)

print(len(data.symbols))

179

We have imported 179 datasets, but some may be incomplete. To ensure smooth analysis, we should remove the incomplete datasets.

data = data.select([
    k
    for k, v in data.data.items()
    if not v.isnull().any().any()
])

print(len(data.symbols))

62

We have removed a large portion of the incomplete data.

The next step is to find the pairs that pass our cointegration test. There are several approaches to finding viable pairs. On one hand, we may have prior knowledge and specifically test a particular pair for cointegration. On the other hand, we can search through hundreds of instruments to find any viable pairs according to the test results. In this exhaustive search scenario, we might encounter a multiple comparisons bias, which is an increased chance of incorrectly identifying a significant p-value when performing many tests. For example, if we run 100 tests on random data, we would expect about 5 p-values below 0.05 just by chance. In practice, we should include a second verification step when identifying pairs in this way, which we will do later.