quantlab.utils.atomic#
Atomic JSON and parquet file writes.
Every JSON sidecar quantlab persists (resume ledgers, acquisition watermarks,
failure manifests) goes through write_json_atomically, and every parquet
file written whole (raw vendor pulls, the Ticker map’s days) through
write_parquet_atomically. The payload is
written to a temporary file in the destination’s own directory and then
renamed over the destination, so an interrupted write leaves either the
previous complete file or the new one, never a truncated file that a resumed
run would fail to parse.
No lock is taken: two concurrent writers still race, but the loser sees a complete file rather than a fragment. This module imports only the standard library so any layer can use it without creating an import cycle.
Functions
Serialise |
|
Write |