quantlab.dataset.sharadar.permatickers.PermatickerResolver#

class quantlab.dataset.sharadar.permatickers.PermatickerResolver(vendor_root, code, *, sidecar_path=None)[source]#

Bases: object

Map each raw file’s tickers to permatickers as of the file’s pull.

See the module docstring for the order. One resolver serves one mapping table (code: the TICKERS rows of tables.SharadarTable.mapping_labels), and may annotate the files of several raw tables mapped through it (a price table and ACTIONS).

Parameters:

vendor_rootstr or Path: <download-dir>/sharadar.
codestr: The table whose TICKERS rows map the tickers ("sep", "sf1").
sidecar_pathstr or Path, optional: The store’s ticker sidecar (<store>.sharadar_tickers.json): the securities it names keep their tickers as of when it was written, including the former ones it carries. Missing or unreadable, it is ignored.

Examples

>>> resolver = PermatickerResolver("/data/downloads/sharadar", "sep")
>>> frame = scan_raw_table("/data/downloads/sharadar", "sep", annotate=resolver.annotate)
>>> "permaticker" in frame.collect_schema().names()
True

ambiguous#: Tickers whose rows settle left out on some key, and why.

annotate(path, frame)[source]#

Return one raw file’s scan with a permaticker column, null where none maps.

The hook scan_raw_table(..., annotate=resolver.annotate) calls for every file; it reads the file’s distinct tickers and, for a table with a date column (SF3A’s is text), each one’s latest date.

left_out(frame, *, owner, report_path=None, date_column='date', quiet=False, report_unmapped=True)[source]#

Drop the rows no permaticker was found for, report them, and return the rest.

Parameters:

framepl.DataFrame: Annotated rows (annotate), with ticker and date_column.
ownerstr: Named in the warning.
report_pathstr or Path, optional: Where to write the report (<store>.unmapped.json), atomically: {"table", "owner", "unmapped": [{"ticker", "first_date", "last_date", "rows", "raw_files", "reason"}, ...]}. An existing report is deleted when nothing is left out, so its presence means bars are missing. None writes nothing.
date_columnstr, default “date”: The column whose first and last values the report gives.
quietbool, default False: Log at info instead of warning, for a table where rows of securities outside the mapping are expected (SF3A holds funds).
report_unmappedbool, default True: List the rows of tickers that map to nothing in the report. False lists only the rows settle could not choose between, for such a table.

Returns:

pl.DataFrame: frame without the rows whose permaticker is null.

Examples

>>> kept = resolver.left_out(frame, owner="demo", report_path="sep.zarr.unmapped.json")

own#: {ticker: permatickers} mapped as the security’s own ticker of a pull: by the file’s TICKERS snapshot, or by current TICKERS for a file without one.

resolve(path, tickers, *, latest=None)[source]#

Return the permaticker of each of tickers as named in the raw file path.

Tickers left unmapped are recorded in unresolved under path; those mapped as the security’s own ticker of the pull in own.

Parameters:

pathstr or Path: The raw file; its pull time decides the TICKERS snapshot and the holders (see the module docstring).
tickersiterable of str: The tickers the file names.
latestdict of str to date, optional: Each ticker’s latest row date in the file. A security first priced after it owns none of the ticker’s rows, so it does not stop them mapping to an earlier holder. annotate passes it for a table with a date column.

Returns:

dict of str to int: {ticker: permaticker} of the tickers mapped.

Raises:

ValueError: If the file’s TICKERS snapshot, or current TICKERS, gives one of the tickers to several permatickers.

Examples

>>> resolver.resolve("/data/downloads/sharadar/sep/sep.parquet", ["LESL"])
{'LESL': 632479}

resolve_on(day, tickers)[source]#

Return the permaticker of each of tickers as traded on day, and why the rest are not mapped.

For a vendor that names each security by the ticker it traded under that day (Massive’s trade files), rather than by the tickers of a later pull. A ticker change dated day is in force on day, so the tickers are mapped as a file pulled at the start of the next day would name them; the order is the module docstring’s.

Returns:

mappeddict of str to int: {ticker: permaticker}.
unresolveddict of str to str: {ticker: reason} for the tickers left unmapped. Nothing is recorded in unresolved, which is keyed by raw file.

Raises:

ValueError: If TICKERS (or the TICKERS snapshot of that pull) gives one of the tickers to several permatickers.

Examples

>>> resolver.resolve_on(date(2016, 11, 25), ["AAPL", "ZVZZT"])
({'AAPL': 199059}, {'ZVZZT': 'no permaticker'})

settle(frame, key, *, date_column='date')[source]#

Keep one row per security and key where several of its tickers give one.

Rows of one permaticker sharing key under different tickers (several rows of one ticker are the caller’s to judge) are settled, in order:

the row of the security’s own ticker as of a pull (own);
else the row of the ticker the security used on the row’s date_column (its holder span, see the module docstring);
else, rows equal in every column but ticker and name keep one.

The other rows of the key are dropped (another name for the row kept). Rows none of these chooses between keep no permaticker: left_out then leaves them out and reports them, with the reason recorded in ambiguous.

Parameters:

framepl.DataFrame: Annotated rows (annotate) with ticker, permaticker, key and date_column.
keyiterable of str: The columns that, with the permaticker, name one row (("date",) for a price table).
date_columnstr, default “date”: The date the row is of, checked against the tickers’ spans.

Returns:

pl.DataFrame: frame with the settled rows dropped and the ambiguous ones without a permaticker, in its order.

Examples

>>> rows = resolver.settle(rows, ("date",))
>>> rows = resolver.left_out(rows, owner="demo")

several_permatickers(path, tickers)[source]#

Return those of tickers that resolve would refuse for path, each with its permatickers.

A ticker is refused when the TICKERS snapshot of the file’s pull, or current TICKERS for a ticker the snapshot does not hold, gives it to several permatickers. A caller that can leave such a ticker unmapped (an ACTIONS counterparty) asks this first and resolves the rest.

Returns:

dict of str to list of int: {ticker: [permaticker, ...]}, sorted.

Examples

Needs the Sharadar raw tier:

resolver.several_permatickers("/data/downloads/sharadar/actions/actions.parquet", ["DUP", "AAPL"])
# {'DUP': [501, 502]}

ticker_holders()[source]#

Return each ticker’s holders, one span per permaticker, built once.

Every security known to have used the ticker is a holder, whatever its dates; resolve then chooses among them, and the Massive Ticker map (quantlab.dataset.massive.ticker_map) checks each against the day’s close instead.

A span’s start is the security’s change into the ticker (ACTIONS), else the sidecar’s start, else its first price date (TICKERS); its end the change away (ACTIONS, an effective date: kept to the TICKERS history’s last sighting of the pair when that is later, as the vendor renames rows only when it publishes the change), else none for its current ticker, else the day before the TICKERS history first shows it renamed or the ticker reused (_history_spans), else the sidecar’s next span, else the day after its last price for a relatedtickers former ticker (related_spans).

Returns:

dict: {ticker: [HolderSpan, ...]}.

Examples

Needs the Sharadar raw tier:

spans = PermatickerResolver("/data/downloads/sharadar", "sep").ticker_holders()["TWO"]
[(span.permaticker, span.covers(date(2020, 3, 2))) for span in spans]

unresolved#: Per raw file, the tickers it names that were not mapped, and why.