quantlab.dataset.sharadar.stores#

The Sharadar stores: the one list of them, built and kept up to date from the raw tier.

Every store the Sharadar raw tier feeds is one SharadarStore in STORES: its name, its folder under the data root, what it holds (the first line of its README.md) and how its dataset is built. Each store sits in its own folder, <data-dir>/<folder>/<name>/<name>.zarr, beside its README.md; the price, fundamental, valuation and panel stores under market/sharadar/, the S&P 500 membership under universe/sharadar/.

Three entry points act on them:

build_stores builds or extends them after a first download (scripts/sharadar/download.py);
update_stores brings them up to date after a raw-tier refresh (scripts/sharadar/update.py, the daily update’s sharadar_stores step): each store appends its new bars from the first day it already holds; a panel store not built yet is built from the SEP store’s first day; a store refusing because its stored symbols are no longer the raw tier’s axis (ResumeRefused) is rebuilt from the raw tier when asked, the old store kept beside it; one store failing does not stop the others;

both return a StoreUpdate, one StoreResult per store (a StoreState and the conversion result), and log what they did; the scripts print StoreUpdate.lines();
write_ticker_sidecars writes the ticker sidecar of the price stores (scripts/sharadar/ticker_sidecar.py).

Examples#

>>> from pathlib import Path
>>> [s.name for s in STORES][:4]
['sharadar_sep_1d', 'sharadar_sfp_1d', 'sharadar_sp500_1d', 'sharadar_spy_1d']
>>> store("sharadar_sp500_membership").path(Path("/data/quantlab")).as_posix()
'/data/quantlab/universe/sharadar/sharadar_sp500_membership/sharadar_sp500_membership.zarr'

Classes

`SharadarStore`	One Sharadar store: where it goes and how its dataset is built.
`StoreResult`	What `build_stores` or `update_stores` did to one store.
`StoreState`	What happened to one store in `build_stores` or `update_stores`.
`StoreUpdate`	What `build_stores` or `update_stores` did, one `StoreResult` per store.

Functions

`build_stores`	Build or extend the stores from the raw tier, each beside a README.md and a component.json.
`missing_stores`	The stores `build_stores` makes that are not under `data_dir`, panels left out.
`store`	The store named `name`.
`update_stores`	Bring the stores up to date from the raw tier, one store at a time.
`write_ticker_sidecars`	Write `<store>.sharadar_tickers.json` beside each existing price store.