quantlab.acquisition.registry#

Registry of every data source quantlab can download from.

A data source is one vendor, such as Sharadar, Massive or FRED. Each vendor is described once by a SourceDescriptor: the names of the environment variables that hold its credentials, the Capability rows it actually serves, and how its raw tier is downloaded. A capability is one (market, frequency, data_type) combination, for example US equities at daily frequency.

Downloading and converting are two separate steps. A download writes a raw tier: the vendor’s data as files on disk, plus small watermark files that record how far each part has been downloaded so that a later run can resume. A dataset class then converts the raw tier into a Zarr store holding a panel, an xarray.Dataset indexed by timestamp and symbol; convert() does that without the caller naming any vendor class.

There are two ways to download, by what the caller knows (ADR 0049):

refresh_raw_tier() keeps a vendor’s raw tier current from its watermarks up to today. It is the one way to keep a raw tier current, and what the daily update calls. It runs the function the vendor registered as SourceDescriptor.raw_tier_refresh (Sharadar, Massive, FRED).
download_symbols() downloads an explicit roster over an explicit window (an AcquisitionConfig) through the vendor’s SymbolBatchAcquisition class (Tiingo, Alpaca, FRED, the retired WRDS CRSP and TAQ downloads).

Descriptors register themselves by calling register_source (quantlab.acquisition.base) in the vendor’s module, which adds them to DataSourceRegistry. The vendor modules are imported at the bottom of this file and never import it, so import quantlab.acquisition.registry alone makes every source available.

Descriptors hold environment variable names only. No function here returns, logs or embeds a credential value, not even a masked one, and no descriptor holds a base URL or host.

Examples#

>>> import quantlab.acquisition.registry
>>> from quantlab.acquisition.base import DataSourceRegistry
>>> [d.vendor for d in DataSourceRegistry.all()]
['alpaca', 'fred', 'massive', 'sharadar', 'tiingo', 'wrds']

Functions

`convert`	Convert an already downloaded raw tier into a Zarr store, in-process.
`credential_status`	Return `{variable_name: is_set}` for each credential the source names.
`download_symbols`	Download a roster of symbols over a window, in the current process.
`is_configured`	Return whether every credential variable the source names is set.
`refresh_raw_tier`	Keep a vendor's raw tier current from its watermarks, in the current process.