quantlab.dataset.massive.trade_bars.MassiveTradeBarDataset

class quantlab.dataset.massive.trade_bars.MassiveTradeBarDataset(dataset_config)

Bases: StockDataset

Dense Trade bar panel on the Massive ticker axis; see the module docstring.

Parameters:

dataset_configMassiveTradeBarsDatasetConfig

The raw tier, the store, the date range, the bar size, the session window and the roster.

Attributes:

last_statsdict or None

The statistics sidecar content after the last day converted.

Examples

>>> ds = MassiveTradeBarDataset(config).from_raw_data_chunked(granularity="day")
>>> panel = ds.panel("2024-11-29", "2024-11-30")
>>> list(panel.data_vars)[:5]
['open', 'high', 'low', 'close', 'volume']

DATA_TYPE = 'trades'

The data type the panel is built from.

DEFAULT_RESAMPLE_HOW = {'buy_volume': 'sum', 'close': 'last', 'close_auction_price': 'last', 'close_auction_volume': 'sum', 'dollar_volume': 'sum', 'high': 'max', 'low': 'min', 'n_trades': 'sum', 'oddlot_volume': 'sum', 'offexchange_volume': 'sum', 'open': 'first', 'open_auction_price': 'first', 'open_auction_volume': 'sum', 'sell_volume': 'sum', 'volume': 'sum'}

resample(freq) aggregates each variable this way unless told otherwise.

NEW_LISTING_STRATEGIES = ('refuse', 'widen')

No "rebuild": a ticker a day brings has no trades before it, so a store is only refused or extended (and rebuilding years of the whole market from raw is a job for the conversion script, into a new store).

add_converted_day(converted)

Use a day convert_day converted, possibly in another process, instead of reading it again.

The day must have been converted under this instance’s settings (interval, session window, roster); the caller vouches for that.

Parameters:

convertedConvertedTradeDay

The converted day.

Raises:

TypeError

If converted is not a ConvertedTradeDay.

Examples

>>> appender.add_converted_day(MassiveTradeBarDataset(config).convert_day(day))

config_cls

The config class used to rebuild the dataset from a saved config.json.

alias of MassiveTradeBarsDatasetConfig

convert_day(day)

Read and resample one trading day, once per instance; the store is not touched.

Parameters:

daydate

A trading day with a raw trade file.

Returns:

ConvertedTradeDay

The day’s bars and statistics, vendor check included.

Examples

In a worker process, then appended by another instance:

converted = MassiveTradeBarDataset(config).convert_day(date(2024, 11, 29))
appender.add_converted_day(converted)
appender.update(granularity="day")

converted_through()

Return the last day the store holds, from its chunk ledger, or None without one.

Days are appended in date order, so every day up to this one that has a raw trade file is in the store. The statistics sidecar is not consulted: it is written before a day’s bars, so a crash between the two leaves a day it records and the store lacks.

Examples

After appending 2024-11-29, this returns date(2024, 11, 29):

MassiveTradeBarDataset(config).update(granularity="day").converted_through()

has_raw_data()

Return whether the raw tier holds at least one trade file.

Examples

>>> MassiveTradeBarDataset(config).has_raw_data()
True

resample_labels(timestamps, freq)

Return the bar each Trade bar belongs to, cut by XNYS session.

As TradeBarResampler cuts bars: right-closed from the session’s open, labelled at their end, the last cut at the close; "1d" is one bar per session, labelled with its date. The sessions are the store’s recorded window, not the reading config’s.

Examples

>>> bars = pd.to_datetime(["2024-11-27 20:31", "2024-11-27 21:00"])
>>> ds.resample_labels(bars.values, "1h").astype("datetime64[m]").tolist()
[2024-11-27T21:00, 2024-11-27T21:00]

property stats_path

Return the path of the per-day statistics sidecar beside the store.

Examples

>>> ds.stats_path.name
'massive_trade_bars_1m.zarr.massive_stats.json'