quantlab.acquisition.refresh#
The raw-tier refresh: one call per vendor that keeps its raw tier current.
A vendor whose raw tier the daily update keeps current registers one
function on its SourceDescriptor (raw_tier_refresh): given the
download root, it brings every part of the raw tier the vendor keeps (a
Sharadar table, a Massive REST endpoint, a FRED series) from its watermark up
to end, and says what it did in a RawTierRefreshResult.
quantlab.acquisition.registry.refresh_raw_tier finds the function through
the registry, so a caller names a vendor, never a client.
The function is a RawTierRefresh: a protocol rather than a root class,
because the three vendors share no implementation (whole tables, REST pages,
symbol batches) and each refresh is a plain function over its own client
(ADR 0017). What every refresh shares is the call, the result and how it
fails.
The failure rule is the same for every vendor. A part that fails (a table, an
endpoint, a series) is recorded as RefreshedPart.failure(name, error) and
the other parts still run. “Fails” means the vendor or the transport failed
it: the vendor’s own error types (a refusal of the key for that part, an HTTP
error status, a TransientError of
quantlab.acquisition._support.request_policy past its retries, a
requests exception, a vendor file the client cannot read), each vendor
naming the types it takes for one (TABLE_FAILURES in
quantlab.acquisition.sharadar.refresh, ENDPOINT_FAILURES in
quantlab.acquisition.massive.refresh; FRED’s symbol-batched download keeps
a series’ failure itself). Any other error, a bare ValueError,
TypeError or KeyError of our own code, is raised: it is a bug, not a
part the vendor failed. Only an error that stops the whole refresh before any
part can run raises by design: a missing credential, checked before the first
request.
Examples#
>>> from datetime import date
>>> result = RawTierRefreshResult("example", (
... RefreshedPart("prices", "40 rows through 2024-01-05"),
... RefreshedPart.failure("news", "HTTP 503"),
... ))
>>> result.failed
('news',)
>>> result.summary()
'prices: 40 rows through 2024-01-05; news: failed: HTTP 503'
Classes
Keep a vendor's raw tier current from its watermarks up to |
|
What one raw-tier refresh did, part by part. |
|
What a refresh did for one part of a raw tier: refreshed, with a summary, or failed, with an error. |