quantlab.acquisition.registry.download_symbols#

quantlab.acquisition.registry.download_symbols(descriptor, config, *, refresh=False, reporter=None, cancel=None)[source]#

Download a roster of symbols over a window, in the current process.

This is the symbol-batched download: config names the symbols and the window, and the vendor’s SymbolBatchAcquisition class fetches them in batches. The class is looked up from the descriptor’s capability for (config.market, config.frequency, config.kwargs["data_type"]), so the caller never names a vendor class. Constructing that class is the first point at which a credential is required. The download writes raw parquet files (shards) and watermark files and stops there; converting them to Zarr is the separate convert() step.

Keeping a vendor’s whole raw tier current from its watermarks, with no roster or window, is refresh_raw_tier() instead (ADR 0049).

Parameters:

descriptorSourceDescriptor: The source to download from.
configAcquisitionConfig: What to download; its market, frequency and optional kwargs["data_type"] pick the capability.
refreshbool, default False: If True, call SymbolBatchAcquisition.refresh(), which downloads the covered window again. Otherwise call download(), which fills only what is missing.
reporterProgressReporter or None, default None: Receives a ProgressEvent per batch. None keeps the default progress bar on stderr.
cancelCancelToken or None, default None: Setting the token stops the download at the next batch boundary. Completed batches stay on disk, and a later call resumes from them.

Returns:

AcquisitionResult: The result the acquisition stored on its last_result attribute. Its failures are the ones this call met. The failure manifest on disk (a JSON file of failed symbols) accumulates across calls and may name more symbols; read it through SourceInspector.failures.

Raises:

ValueError: If the source has no symbol-batched download (Sharadar, Massive): its raw tier is kept current by refresh_raw_tier().

Examples

Needs the vendor’s credential in the environment and network access:

from quantlab.acquisition.config import AcquisitionConfig
from quantlab.utils.progress import CancelToken
from quantlab.acquisition.base import DataSourceRegistry
from quantlab.acquisition.registry import download_symbols

source = DataSourceRegistry.get("tiingo")
config = AcquisitionConfig(
    market="us_equity", frequency="1d", vendor="tiingo",
    raw_data_dir_path="data/us_equity/1d/nasdaq_data/tiingo",
    watermark_path="data/us_equity/1d/nasdaq_data/_watermarks/tiingo",
    symbols=("AAPL", "MSFT"),
    start_date="2024-01-01", end_date="2024-05-31",
)
token = CancelToken()
result = download_symbols(source, config, cancel=token)
result.failures  # {symbol: reason} for this call only

A vendor without a symbol-batched download is refused before anything is touched:

>>> from quantlab.acquisition.base import DataSourceRegistry
>>> from quantlab.acquisition.config import AcquisitionConfig
>>> config = AcquisitionConfig(
...     market="us_equity", frequency="1d", vendor="sharadar",
...     raw_data_dir_path="downloads/sharadar", watermark_path="downloads/_w",
...     symbols=("AAPL",), start_date="2024-01-01",
... )
>>> download_symbols(DataSourceRegistry.get("sharadar"), config)
Traceback (most recent call last):
...
ValueError: Sharadar ... has no symbol-batched download; keep its raw tier current ...