Features

Data storage and databases

Cache, query, save, and update market data across local files and databases

Data caching

✅ Data caching is now available for all data classes, allowing you to cache data on disk when pulling from a (remote) data source. Caching is performed using LMDB, which provides efficient key-value storage. The cache key is generated based on the hash of the class, symbol, and other fetch parameters, ensuring that different data pulls are cached separately.

Pull AAPL minute data from TradingView or cache if available
data = vbt.TVData.pull(
    "AAPL",
    exchange="NASDAQ",
    timeframe="1 minute",
    tz="America/New_York",
    cache=True  
)

ArcticDB

ArcticDB is a serverless DataFrame database engine designed for the Python Data Science ecosystem. It can store, retrieve and process Series and DataFrames at scale, backed by commodity object storage (S3-compatible storages and Azure Blob Storage). ArcticDB requires zero additional infrastructure beyond a running Python environment and access to object storage and can be installed in seconds.

Save minute data to a ArcticDB database, and read one day
data = vbt.TVData.pull(
    "AAPL",
    exchange="NASDAQ",
    timeframe="1 minute",
    tz="America/New_York"
)

data.to_arcticdb()

day_data = vbt.ArcticDBData.pull(
    "AAPL",
    start="2026-02-27 09:30:00",
    end="2026-02-27 16:00:00",
    tz="America/New_York",
)
day_data.get()
                             Open    High     Low    Close    Volume
datetime
2026-02-27 09:30:00-05:00  272.90  272.90  269.80  269.960   49256.0
2026-02-27 09:31:00-05:00  269.97  270.12  269.19  269.230   43715.0
2026-02-27 09:32:00-05:00  269.18  270.05  269.18  269.890   33913.0
2026-02-27 09:33:00-05:00  269.85  270.01  269.60  269.770   27569.0
2026-02-27 09:34:00-05:00  269.74  269.98  269.40  269.500   22963.0
...                           ...     ...     ...      ...       ...
2026-02-27 15:55:00-05:00  264.41  264.51  263.15  263.490  162153.0
2026-02-27 15:56:00-05:00  263.51  263.78  263.23  263.670  114741.0
2026-02-27 15:57:00-05:00  263.67  263.78  262.89  263.655  113641.0
2026-02-27 15:58:00-05:00  263.61  263.92  263.25  263.620  158462.0
2026-02-27 15:59:00-05:00  263.68  264.28  263.65  264.200  107619.0

[390 rows x 5 columns]

SQL queries

✅ Thanks to DuckDB, you can now run SQL queries directly on data instances.

Run a rolling average of 14 days on minute data using SQL
data = vbt.TVData.pull(
    "AAPL",
    exchange="NASDAQ",
    timeframe="1 minute",
    tz="America/New_York"
)

data.sql("""
    SELECT datetime, AVG(Close) OVER(
        ORDER BY "datetime" ASC
        RANGE BETWEEN INTERVAL 14 DAYS PRECEDING AND CURRENT ROW
    ) AS "Moving Average"
    FROM "AAPL";
""")
datetime
2023-09-11 09:30:00-04:00    180.080000
2023-09-11 09:31:00-04:00    179.965000
2023-09-11 09:32:00-04:00    180.000000
2023-09-11 09:33:00-04:00    180.022500
2023-09-11 09:34:00-04:00    179.984000
                                    ...
2023-10-20 15:55:00-04:00    177.786669
2023-10-20 15:56:00-04:00    177.785492
2023-10-20 15:57:00-04:00    177.784322
2023-10-20 15:58:00-04:00    177.783166
2023-10-20 15:59:00-04:00    177.781986
Name: Moving Average, Length: 11700, dtype: float64

DuckDB

DuckDB is a high-performance analytical database system that offers a robust SQL dialect for interacting with various data stores. Not only can it run analytical queries on local data, even if the data does not fit into memory and without needing a distributed framework, but it can also query CSV, Parquet, and JSON files directly.

Save minute data to a DuckDB database, and read one day
data = vbt.TVData.pull(
    "AAPL",
    exchange="NASDAQ",
    timeframe="1 minute",
    tz="America/New_York"
)

URL = "database.duckdb"
data.to_duckdb(connection=URL)

day_data = vbt.DuckDBData.pull(
    "AAPL",
    start="2023-10-02 09:30:00",
    end="2023-10-02 16:00:00",
    tz="America/New_York",
    connection=URL
)
day_data.get()
                              Open    High     Low   Close    Volume
datetime
2023-10-02 09:30:00-04:00  171.260  171.34  170.93  171.10   61654.0
2023-10-02 09:31:00-04:00  171.130  172.37  171.13  172.30   53481.0
2023-10-02 09:32:00-04:00  172.310  172.64  172.16  172.64   44750.0
2023-10-02 09:33:00-04:00  172.640  172.97  172.54  172.78   53195.0
2023-10-02 09:34:00-04:00  172.780  173.07  172.75  173.00   47416.0
...                            ...     ...     ...     ...       ...
2023-10-02 15:55:00-04:00  173.300  173.51  173.26  173.51   61619.0
2023-10-02 15:56:00-04:00  173.525  173.59  173.42  173.43   45066.0
2023-10-02 15:57:00-04:00  173.430  173.55  173.42  173.50   45220.0
2023-10-02 15:58:00-04:00  173.510  173.60  173.46  173.56   47371.0
2023-10-02 15:59:00-04:00  173.560  173.78  173.56  173.75  161253.0

[390 rows x 5 columns]

SQLAlchemy

SQLAlchemy provides a standard interface that allows developers to create database-agnostic code for communicating with a wide range of SQL database engines. With its help, you can now easily store data in SQL databases and read from them as well.

Save minute data to a PostgreSQL database, and read one day
data = vbt.TVData.pull(
    "AAPL",
    exchange="NASDAQ",
    timeframe="1 minute",
    tz="America/New_York"
)

URL = "postgresql://postgres:postgres@localhost:5432"
data.to_sql(engine=URL)

day_data = vbt.SQLData.pull(
    "AAPL",
    start="2023-10-02 09:30:00",
    end="2023-10-02 16:00:00",
    tz="America/New_York",
    engine=URL
)
day_data.get()
                              Open    High     Low   Close    Volume
datetime
2023-10-02 09:30:00-04:00  171.260  171.34  170.93  171.10   61654.0
2023-10-02 09:31:00-04:00  171.130  172.37  171.13  172.30   53481.0
2023-10-02 09:32:00-04:00  172.310  172.64  172.16  172.64   44750.0
2023-10-02 09:33:00-04:00  172.640  172.97  172.54  172.78   53195.0
2023-10-02 09:34:00-04:00  172.780  173.07  172.75  173.00   47416.0
...                            ...     ...     ...     ...       ...
2023-10-02 15:55:00-04:00  173.300  173.51  173.26  173.51   61619.0
2023-10-02 15:56:00-04:00  173.525  173.59  173.42  173.43   45066.0
2023-10-02 15:57:00-04:00  173.430  173.55  173.42  173.50   45220.0
2023-10-02 15:58:00-04:00  173.510  173.60  173.46  173.56   47371.0
2023-10-02 15:59:00-04:00  173.560  173.78  173.56  173.75  161253.0

[390 rows x 5 columns]

PyArrow & FastParquet

✅ Data can be written to Feather with PyArrow and to Parquet using PyArrow or FastParquet. Parquet performs especially well in write-once, read-many scenarios, providing highly efficient data compression and decompression, making it a great choice for storing time series data.

Save minute data to a Parquet dataset partitioned by day, and read one day
data = vbt.TVData.pull(
    "AAPL",
    exchange="NASDAQ",
    timeframe="1 minute",
    tz="America/New_York"
)

data.to_parquet(partition_by="day")  

day_data = vbt.ParquetData.pull("AAPL", filters=[("group", "==", "2023-10-02")])
day_data.get()
                              Open    High     Low   Close    Volume
datetime
2023-10-02 09:30:00-04:00  171.260  171.34  170.93  171.10   61654.0
2023-10-02 09:31:00-04:00  171.130  172.37  171.13  172.30   53481.0
2023-10-02 09:32:00-04:00  172.310  172.64  172.16  172.64   44750.0
2023-10-02 09:33:00-04:00  172.640  172.97  172.54  172.78   53195.0
2023-10-02 09:34:00-04:00  172.780  173.07  172.75  173.00   47416.0
...                            ...     ...     ...     ...       ...
2023-10-02 15:55:00-04:00  173.300  173.51  173.26  173.51   61619.0
2023-10-02 15:56:00-04:00  173.525  173.59  173.42  173.43   45066.0
2023-10-02 15:57:00-04:00  173.430  173.55  173.42  173.50   45220.0
2023-10-02 15:58:00-04:00  173.510  173.60  173.46  173.56   47371.0
2023-10-02 15:59:00-04:00  173.560  173.78  173.56  173.75  161253.0

[390 rows x 5 columns]

Data saver

✅ Imagine a script that can periodically pull the latest data from an exchange and save it to disk, all without your intervention. VBT implements two classes that can do exactly this: one that saves to CSV and another that saves to HDF.

BTCUSDT_1m_saver.py
from vectorbtpro import *

import logging
logging.basicConfig(level=logging.INFO)

if __name__ == "__main__":
    if vbt.CSVDataSaver.file_exists():
        csv_saver = vbt.CSVDataSaver.load()
        csv_saver.update()
        init_save = False
    else:
        data = vbt.BinanceData.pull(
            "BTCUSDT",
            start="10 minutes ago UTC",
            timeframe="1 minute"
        )
        csv_saver = vbt.CSVDataSaver(data)
        init_save = True
    csv_saver.update_every(1, "minute", init_save=init_save)
    csv_saver.save()  
Run in console and then interrupt
$ python BTCUSDT_1m_saver.py
2023-02-01 12:26:36.744000+00:00 - 2023-02-01 12:36:00+00:00: : 1it [00:01,  1.22s/it]
INFO:vectorbtpro.data.saver:Saved initial 10 rows from 2023-02-01 12:27:00+00:00 to 2023-02-01 12:36:00+00:00
INFO:vectorbtpro.utils.schedule_:Starting schedule manager with jobs [Every 1 minute do update(save_kwargs=None) (last run: [never], next run: 2023-02-01 13:37:38)]
INFO:vectorbtpro.data.saver:Saved 2 rows from 2023-02-01 12:36:00+00:00 to 2023-02-01 12:37:00+00:00
INFO:vectorbtpro.data.saver:Saved 2 rows from 2023-02-01 12:37:00+00:00 to 2023-02-01 12:38:00+00:00
INFO:vectorbtpro.utils.schedule_:Stopping schedule manager
Run in console again to continue
$ python BTCUSDT_1m_saver.py
INFO:vectorbtpro.utils.schedule_:Starting schedule manager with jobs [Every 1 minute do update(save_kwargs=None) (last run: [never], next run: 2023-02-01 13:42:08)]
INFO:vectorbtpro.data.saver:Saved 5 rows from 2023-02-01 12:38:00+00:00 to 2023-02-01 12:42:00+00:00
INFO:vectorbtpro.utils.schedule_:Stopping schedule manager

Local data

✅ Once you have fetched remote data, you will most likely want to save it to disk. There are two new options for this: you can either serialize the entire data class, or save the actual data to CSV or HDF5. Each dataset can be stored in a single flat file, which makes handling the data easier than using a database. After saving, you can easily load the data back either by deserializing or by using data classes that specialize in loading from CSV and HDF5 files. These classes support a range of features, including filtering by row and datetime ranges, updating, chunking, and even a smart dataset search that can recursively walk through sub-directories and return datasets that match a specific glob pattern or regular expression 🧲

Fetch and save symbols separately, then load them back jointly
btc_data = vbt.BinanceData.pull("BTCUSDT")
eth_data = vbt.BinanceData.pull("ETHUSDT")

btc_data.to_hdf()
eth_data.to_hdf()

data = vbt.BinanceData.from_hdf(start="2020", end="2021")
Key 2/2
data.close
symbol                      BTCUSDT  ETHUSDT
Open time
2020-01-01 00:00:00+00:00   7200.85   130.77
2020-01-02 00:00:00+00:00   6965.71   127.19
2020-01-03 00:00:00+00:00   7344.96   134.35
2020-01-04 00:00:00+00:00   7354.11   134.20
2020-01-05 00:00:00+00:00   7358.75   135.37
...                             ...      ...
2020-12-27 00:00:00+00:00  26281.66   685.11
2020-12-28 00:00:00+00:00  27079.41   730.41
2020-12-29 00:00:00+00:00  27385.00   732.00
2020-12-30 00:00:00+00:00  28875.54   752.17
2020-12-31 00:00:00+00:00  28923.63   736.42

[366 rows x 2 columns]

Copyright © 20212026 Oleg Polakow. All rights reserved.

Site content and documentation are provided for using and evaluating VectorBT PRO and for educational purposes. Any other use, including building or supporting competing products or services, requires prior written consent.