quantlab.acquisition.sharadar.client#
Sharadar API client: pull a table as a bulk zip and keep it as raw parquet.
Sharadar (api.sharadar.com/v1.0) is the primary US-equity vendor (ADR
0023). It is reached through its own API, not Nasdaq Data Link: the
purchased key answers only there. The key is read from the
SHARADAR_API_KEY environment variable and nowhere else, and it travels in
the x-api-key header, so it never appears in a URL a log could print.
A bulk pull asks GET /data/<table>?years=full; Sharadar answers with an
HTTP 302 to a short-lived pre-signed URL of a zipped CSV. The client follows
the redirect itself, without the key, downloads the zip to disk in parallel
byte ranges (one stream when the storage ignores Range), checks that
the CSV header is the table’s schema verbatim, and writes the rows as one
parquet file under <download-dir>/sharadar/<code>/ (see
quantlab.dataset.sharadar.tables). The raw tier is the vendor’s exact
rows, so every Zarr store can be rebuilt from it without the network.
A date-window pull (window_table) refreshes a date-keyed table every
morning: it asks REST for every calendar day from a trailing number of
trading days before the table’s watermark through today, one request per
day and page, and writes the rows as one window file, a complete copy of
the table over those dates that supersedes earlier rows of them. An
updated pull (updated_table) refreshes a table keyed by more than its
date (SF1): it asks REST for every row whose lastupdated is on or after
the table’s watermark, and each row replaces the earlier row with the same
primary key. Each pull then records the table’s watermark, so an
interrupted refresh restarts from the last one that finished.
Every HTTP request goes through one transport function, the client’s only
seam: tests replace it and run offline. A 401 or 403, or a missing key,
raises SharadarEntitlementError naming the table. A 429 (rate limited) or
a 5xx is retried after a back-off: Retry-After when the response gives
one, otherwise a wait that doubles on each attempt.
Examples#
Pull SEP and TICKERS (needs SHARADAR_API_KEY and the network); each lands
at /data/quantlab/downloads/sharadar/<code>/<code>.parquet:
client = SharadarClient()
client.bulk_table("sep", "/data/quantlab/downloads")
client.bulk_table("tickers", "/data/quantlab/downloads")
Classes
Pull Sharadar tables into the raw tier through one transport function. |
Functions
Send one GET without following redirects; the client's default transport. |