quantlab.acquisition.massive.refresh.refresh_raw_tier

quantlab.acquisition.massive.refresh.refresh_raw_tier(download_dir, *, end=None, endpoints=None, client=None, today=None)

Pull the REST endpoints into the raw tier under <download_dir>/massive/rest/.

Parameters:

download_dirstr or Path

The download root.

enddate or None, default None

The last day asked for (a backfill in steps); no bound when omitted.

endpointsiterable of str or None, default None

The endpoints to pull, by name; every one of ENDPOINTS when omitted, in its order.

clientMassiveClient or None, default None

The client to pull through; one reading the credentials from the environment when omitted.

todaydate or None, default None

Today in New York, passed to MassiveClient.pull; the system clock’s when omitted.

Returns:

RawTierRefreshResult

One part per endpoint: the rows and pages pulled and the watermark after, or why it failed (one of ENDPOINT_FAILURES: a refusal, a transport error past the retries, a page the client cannot read).

Raises:

ValueError

If an endpoint is not in ENDPOINTS; nothing is requested.

MassiveCredentialError

If client is omitted and a credential variable is unset; nothing is requested.

Exception

Any error not in ENDPOINT_FAILURES (a bug of ours), from the endpoint it happened on; the endpoints before it are pulled.

Examples

Needs the credentials and the network:

refresh_raw_tier("/data/quantlab/downloads", endpoints=("dividends",), end=date(2010, 12, 31))