quantlab.acquisition.sharadar.raw_tier#

The writers of Sharadar’s raw tier: watermarks, bulk-pull records and the TICKERS history.

SharadarClient (quantlab.acquisition.sharadar.client) pulls tables into <download-dir>/sharadar/<table>/ and records what it pulled with these writers. The layout, and every reader of it (raw_through, scan_raw_table, pull_time, tickers_history, snapshot_for), stay in quantlab.dataset.sharadar.tables: the dataset layer reads the raw tier, only the acquisition layer writes it.

A table’s watermark is a dated watermark (quantlab.utils.watermark) labelled here: <sharadar>/<table>/_watermark.json holding table and through. Unlike Massive’s watermarks it may move back: a date-window pull records the window’s last day, whatever the watermark was.

Examples#

>>> import tempfile
>>> from datetime import date
>>> from quantlab.dataset.sharadar.tables import read_watermark
>>> root = tempfile.mkdtemp()
>>> write_table_watermark(root, "sep", date(2024, 1, 11))
>>> read_watermark(root, "sep")
datetime.date(2024, 1, 11)

Functions

`fold_tickers_history`	Fold every TICKERS snapshot into the TICKERS history, and return its path.
`prune_tickers_snapshots`	Delete the TICKERS snapshots no raw file maps with, and return them.
`write_bulk_pull`	Record when a table's bulk file was pulled (`BULK_PULL_FILE`), in UTC.
`write_table_watermark`	Record that a table's raw tier is complete through `through`.