quantlab.dataset.sharadar.permatickers#
Map the tickers of Sharadar’s raw files to permatickers, as of each file’s pull.
Sharadar keys every table but TICKERS by ticker, and renames a security’s whole history when its ticker changes (LESL became LESLQ: every LESL row reads LESLQ from then on). Each raw file therefore names its securities by the tickers of its own pull: a bulk file pulled before a change holds the old ticker, a window pulled after it the new one. TICKERS, pulled whole every run, holds only each permaticker’s current ticker, so the old ticker of a file pulled before a change is not in it.
PermatickerResolver maps one file at a time (annotate, the hook of
scan_raw_table), each ticker in this order:
the TICKERS snapshot of the file’s own run (
tables.snapshot_for), which knows exactly the tickers of that pull. A ticker the snapshot gives to two permatickers is refused, as TICKERS always was.otherwise the ticker’s holders, the securities known to have used it: the permaticker current TICKERS gives it to (a ticker current TICKERS gives to two permatickers is refused); the securities the ACTIONS
tickerchangefromchain says used it, each from its change into the ticker (or its first price) until its change away (ticker_changes); the security TICKERS renamed to the ticker plus a number when another took the ticker over (NSTR became NSTR1), often with no ACTIONS row: itsrelatedtickerslists the old ticker, held from its first to its last price (related_spans); and, for a store with one, the securities its ticker sidecar (<store>.sharadar_tickers.json) names by it, the sidecar having been written from an earlier TICKERS. A ticker with a single holder maps to it, whatever the dates (so a vendor slow to rename a security still maps). Several holders (the ticker was reused) are told apart by the file’s pull date: the holder whose span covers it, the latest-starting one if several do, or the latest to have used it if none does. A change dated on the pull’s own day counts from the next day, as the vendor applies it in its evening update.The TICKERS history (
quantlab.acquisition.sharadar.raw_tier.fold_tickers_history) adds holders too: each security a folded snapshot showed under the ticker, until the day before the history first shows it renamed or the ticker reused (a rename TICKERS made before ACTIONS has a row of it). A holder the history shows under another ticker on or after the file’s pull, and under this one only later, is not one of the file’s, whatever date ACTIONS gives its change: the vendor renames rows when TICKERS shows the new ticker.A holder is not taken when another security, under any table, held the ticker at the file’s pull: the old ticker of a stock that a fund (SFP) has taken over names the fund in a file of that pull (ACTIONS and SF3A hold funds; ACTIONS names the FB fund’s dividends FB, not Meta’s, #235). Who held it is the file’s own TICKERS snapshot, or without one current TICKERS, each security counted from the day after it took the ticker (its last ACTIONS ticker change, else its first price). A security first priced after the file’s latest row of the ticker does not count: it owns none of them (a fund listed under a dead stock’s old ticker after the file’s rows).
A row whose ticker none of them maps is left out rather than refused:
left_out drops it, logs a warning and writes the report
<store>.unmapped.json (UNMAPPED_SUFFIX) naming each ticker’s first
and last date, row count and raw files, so a daily update completes. So does
a ticker whose holders tie (two securities starting on the same day).
One security can then have rows under two tickers for one key: SF3A, pulled
whole, keeps some quarters of a security under its current ticker and under
a former one. settle keeps one row per security and key (the security’s
own ticker as of the file’s pull, else the ticker it used on the row’s date,
else one of rows identical but for their ticker) and leaves out, as
unmapped, the rows it cannot choose between.
Examples#
>>> resolver = PermatickerResolver("/data/downloads/sharadar", "sep")
>>> prices = scan_raw_table("/data/downloads/sharadar", "sep", annotate=resolver.annotate).collect()
>>> prices = resolver.left_out(prices, owner="demo", report_path="/data/market/sharadar/sep/sep.zarr.unmapped.json")
Classes
One security's use of one ticker: from |
|
Map each raw file's tickers to permatickers as of the file's pull. |
|
One security under one ticker in the TICKERS history ( |
Functions
Read one raw table with each row's permaticker, leaving out and reporting the unmapped rows. |
|
Return the former tickers TICKERS' |
|
Return each permaticker's ticker changes, |