quantlab.dataset.massive.ticker_map.TickerMap#

class quantlab.dataset.massive.ticker_map.TickerMap(config)[source]#

Bases: Component

The Ticker map of one directory: append a day, rebuild all, read rows.

See the module docstring for the rules and the layout. Sharadar’s holders and splits are read once per instance; its closes once per batch of days mapped.

Parameters:

configTickerMapConfig: The map’s directory and the two raw tiers it reads.

Examples

Needs the raw tiers:

ticker_map = TickerMap(config)
ticker_map.append(date(2020, 3, 2))
ticker_map.days()  # [datetime.date(2020, 3, 2)]

append(day)[source]#

Map day and add its rows to the map.

Parameters:

daystr or date: A day whose Massive day aggregates are in the raw tier.

Returns:

pl.DataFrame: The day’s rows.

Raises:

ValueError: If the map already holds day; rebuild maps it again.
FileNotFoundError: If the raw tier has no day aggregates of day.

Examples

>>> ticker_map.append("2020-03-02").height
8954

config_cls#: alias of TickerMapConfig

static day_file(root, day)[source]#

Return the file of day’s rows under the map directory root.

Examples

>>> TickerMap.day_file("/m", date(2020, 3, 2))
PosixPath('/m/2020/2020-03-02.parquet')

days()[source]#

Return the mapped days, in order.

Examples

>>> ticker_map.days()
[datetime.date(2020, 3, 2), datetime.date(2020, 3, 3)]

map_days(days)[source]#

Return the rows of days without writing them.

Parameters:

dayslist of date: Days whose day aggregates are in the raw tier.

Returns:

pl.DataFrame: Rows of ROW_SCHEMA, one per day and ticker of the day aggregates.

Raises:

FileNotFoundError: If a day’s aggregates are missing.

Examples

>>> ticker_map.map_days([date(2020, 3, 2)]).filter(pl.col("ticker") == "AAPL")["evidence"].item()
'price'

property path#

The map’s directory.

Examples

>>> ticker_map.path.name
'massive_ticker_map'

read(start=None, end=None)[source]#

Return the rows of the mapped days from start to end, both inclusive.

Parameters:

start, endstr or date, optional: The first and last day; None leaves that side open.

Returns:

pl.DataFrame: Rows of ROW_SCHEMA, by date and ticker.

Examples

>>> ticker_map.read("2020-03-02", "2020-03-02").columns
['date', 'ticker', 'permaticker', 'evidence', 'reason', 'massive_close', 'sharadar_close', 'rules_version']

read_days(days, tickers=None)[source]#

Return the rows of some mapped days, of some tickers only.

Reads only the day files asked for, and with tickers only their rows: a placement of a few hundred thousand REST rows over ten years reads the map’s files once, not as a whole.

Parameters:

daysiterable of str or date: The days; one the map does not hold is skipped.
tickersiterable of str, optional: The Massive tickers wanted; None for every one.

Returns:

pl.DataFrame: Rows of ROW_SCHEMA, by date and ticker.

Examples

AAPL’s permaticker on one day:

ticker_map.read_days(["2020-03-02"], tickers=["AAPL"])["permaticker"].to_list()

rebuild(days=None)[source]#

Map every day again and replace the map with the result.

The days are mapped into a sidecar directory beside the map, which then replaces it by two renames: a run that fails before them leaves the map as it was, and one killed between them leaves it at <path>.superseded.tmp, put back by the next call. Do not append while a rebuild runs: the swap would drop the appended day.

Parameters:

daysiterable of str or date, optional: The days to map; None maps every day whose day aggregates are in the raw tier.

Returns:

list of date: The days mapped.

Raises:

ValueError: If there is no day to map: an empty map would replace this one.

Examples

>>> ticker_map.rebuild()
[datetime.date(2020, 3, 2), datetime.date(2020, 3, 3)]