quantlab.dataset.massive.mapped_bars.MappedPermatickerDataset

class quantlab.dataset.massive.mapped_bars.MappedPermatickerDataset(config)

Bases: MarketDataset

A permaticker view’s panel, read from a store the view writes; see the module docstring.

A subclass names the view it keeps (SOURCE_CLS) and how many days a write appends (BATCH_DAYS).

Parameters:

configMappedTradeBarsConfig or MappedDayBarsConfig

The view (source) and the store (zarr_file_path).

Raises:

ValueError

If source is not a SOURCE_CLS of its own bars.

Examples

With bars a permaticker view built beforehand (PermatickerTradeBarDataset):

mapped = MappedTradeBarDataset(MappedTradeBarsConfig(source=bars, zarr_file_path="mapped.zarr"))
mapped.update().panel("2024-11-27", "2024-11-30")["symbol"].values.tolist()
# [101, 202, 303, 404, 505]

BATCH_DAYS

Session days mapped and appended per write.

SOURCE_CLS

The permaticker view the store is written from.

copy()

Return a dataset over a copy of the view and the same store.

Examples

With mapped the mapped store of the class example:

other = mapped.copy()
other == mapped, other.source is mapped.source
# (True, False)

resample(freq, how=None)

Return this dataset on freq bars, aggregated per session as the Trade bars are.

Examples

With mapped the mapped store of the class example:

mapped.resample("1h").panel("2024-11-27", "2024-11-30").sizes["timestamp"]
# 11

resample_labels(timestamps, freq)

Return the bars the source store’s dataset cuts timestamps into.

property sidecar_path

<store>.permaticker_map.json, the map digest of every stored day.

Examples

With mapped the mapped store of the class example:

mapped.sidecar_path.name
# 'mapped.zarr.permaticker_map.json'

property source

The permaticker view the store is written from.

Examples

With mapped the mapped store of the class example:

mapped.source is bars
# True

property store_path

The store of the mapped bars.

Examples

With mapped the mapped store of the class example:

mapped.store_path
# 'mapped.zarr'

stored_days()

Return the session days the store holds, oldest first.

Examples

With mapped the mapped store of the class example:

mapped.stored_days()[-1]
# datetime.date(2024, 11, 29)

stored_symbols()

Return the store’s permatickers, in stored order.

Examples

With mapped the mapped store of the class example:

mapped.stored_symbols()
# [101, 202, 303, 404, 505]

unplaced()

Return {day: count} of the stored tickers the map could not place, per stored day.

Recorded when the day was mapped; a day on which every ticker was placed is left out. Their bars are in no column of the store.

Examples

With mapped the mapped store of the class example:

mapped.unplaced()   # ZZZ, a test symbol no security ever held
# {datetime.date(2024, 11, 27): 1, datetime.date(2024, 11, 29): 1}

update(end=None, workers=1)

Bring the store up to date with the source store and the Ticker map.

First every stored day whose map rows changed since it was written is mapped again and written over its own bars; then the session days after the store’s last that both the source store and the Ticker map hold, up to end, are mapped and appended, BATCH_DAYS per write. Without a store this builds it from the source store’s first day. The sidecar is written after every write, so an interrupted update resumes where it stopped.

Parameters:

endstr, datetime.date or pd.Timestamp, optional

The last day to append; None for the last day both sources hold.

workersint, default 1

Processes mapping batches of days ahead of the writes.

Returns:

Self

self, for chaining.

Raises:

ValueError

If the dataset is resampled.

Examples

With mapped the mapped store of the class example:

mapped.update("2024-11-27").stored_days()[-1]
# datetime.date(2024, 11, 27)
mapped.update().stored_days()[-1]          # the last day both hold
# datetime.date(2024, 11, 29)