quantlab.dataset.nbbo.resample.NbboResampler#

class quantlab.dataset.nbbo.resample.NbboResampler(bar_interval, policy=None)[source]#

Bases: object

Resample NBBO records onto a regular right-closed bar grid.

Records are sorted by (symbol, date, timestamp, wrds_row_ord) with a stable sort, so the input row order never changes the output. wrds_row_ord is the position of the record in the order it was downloaded; it is the only tie-breaker for tables without a nanosecond field. All records at one instant count as updates, but only the last one in that order is kept as the quote: it is the quote in force after that instant, and the ones before it lasted zero time.

Data from 2018 on is unique on (symbol, time_m, time_m_nano), so the sort key fully decides the order. Earlier tables only have microseconds, and a few percent of rows share a microsecond with a record holding different values, where download order is the only clue to the true order. n_ambiguous_ties counts, per bar, the records that share their timestamp with a record holding different values (identical duplicates count 0), so a user can see which bars depend on that order.

Parameters:

bar_intervalstr: A key of BAR_INTERVAL_SECONDS such as "1m".
policyNbboFilterPolicy, optional: The record filter; NbboFilterPolicy() when None.

Attributes:

bar_intervalstr: The bar size name.
secondsint: The bar length in seconds.
policyNbboFilterPolicy: The record filter in use.

Raises:

ValueError: If bar_interval is not a known bar size.

Examples

A three-minute session, one symbol, four records (the third is crossed and dropped by the default policy):

>>> sessions = pl.DataFrame({
...     "date": [date(2024, 1, 24)],
...     "open": [datetime(2024, 1, 24, 14, 30)],
...     "close": [datetime(2024, 1, 24, 14, 33)],
... })
>>> records = pl.DataFrame({
...     "symbol": ["AAPL"] * 4,
...     "date": [date(2024, 1, 24)] * 4,
...     "timestamp": [
...         datetime(2024, 1, 24, 14, 29, 50),
...         datetime(2024, 1, 24, 14, 30, 30),
...         datetime(2024, 1, 24, 14, 31, 0),
...         datetime(2024, 1, 24, 14, 32, 10),
...     ],
...     "wrds_row_ord": [1, 2, 3, 4],
...     "best_bid": [100.0, 100.1, 100.3, 100.2],
...     "best_bidsizeshares": [200.0, 300.0, 100.0, 100.0],
...     "best_ask": [100.2, 100.3, 100.2, 100.4],
...     "best_asksizeshares": [100.0, 100.0, 100.0, 300.0],
...     "qu_cond": ["R"] * 4,
... })
>>> panel = NbboResampler("1m").resample(records, sessions)
>>> panel.select("timestamp", "bid", "ask", "mid", "n_updates")
shape: (3, 5)
┌─────────────────────┬───────┬───────┬───────┬───────────┐
│ timestamp           ┆ bid   ┆ ask   ┆ mid   ┆ n_updates │
│ ---                 ┆ ---   ┆ ---   ┆ ---   ┆ ---       │
│ datetime[ns]        ┆ f64   ┆ f64   ┆ f64   ┆ f64       │
╞═════════════════════╪═══════╪═══════╪═══════╪═══════════╡
│ 2024-01-24 14:31:00 ┆ 100.1 ┆ 100.3 ┆ 100.2 ┆ 1.0       │
│ 2024-01-24 14:32:00 ┆ 100.1 ┆ 100.3 ┆ 100.2 ┆ 0.0       │
│ 2024-01-24 14:33:00 ┆ 100.2 ┆ 100.4 ┆ 100.3 ┆ 1.0       │
└─────────────────────┴───────┴───────┴───────┴───────────┘

labels(sessions)[source]#

Return (date, timestamp) for every bar label of every session.

Parameters:

sessionspl.DataFrame: A frame with date, open and close columns, the last two in naive UTC.

Returns:

pl.DataFrame: One row per bar label, in ascending order. The open itself is not a label.

Examples

>>> NbboResampler("1m").labels(sessions)
shape: (3, 2)
┌────────────┬─────────────────────┐
│ date       ┆ timestamp           │
│ ---        ┆ ---                 │
│ date       ┆ datetime[ns]        │
╞════════════╪═════════════════════╡
│ 2024-01-24 ┆ 2024-01-24 14:31:00 │
│ 2024-01-24 ┆ 2024-01-24 14:32:00 │
│ 2024-01-24 ┆ 2024-01-24 14:33:00 │
└────────────┴─────────────────────┘

resample(records, sessions)[source]#

Return only the bar panel of resample_with_stats.

Parameters:

recordspl.DataFrame: NBBO records, as for resample_with_stats.
sessionspl.DataFrame: Session bounds, as for resample_with_stats.

Returns:

pl.DataFrame: The bar panel.

Examples

>>> NbboResampler("1m").resample(records, sessions).shape
(3, 16)

resample_with_stats(records, sessions)[source]#

Resample records and return the bar panel with the filter stats.

Parameters:

recordspl.DataFrame: A frame with symbol, date (session date), timestamp (naive UTC), wrds_row_ord, best_bid, best_bidsizeshares, best_ask, best_asksizeshares and optionally qu_cond.
sessionspl.DataFrame: A frame with date, open and close columns.

Returns:

panelpl.DataFrame: Columns symbol, date, timestamp and the PANEL_VARIABLES as float64, with one row per bar label for every (symbol, date) in the input. A pair whose records were all filtered out still gets its rows, all NaN.
statspl.DataFrame: One row per input (date, symbol) with the FILTER_STATS_COUNTS columns.

Examples

>>> panel, stats = NbboResampler("1m").resample_with_stats(
...     records, sessions
... )
>>> stats.select("symbol", "records_in", "dropped_crossed")
shape: (1, 3)
┌────────┬────────────┬─────────────────┐
│ symbol ┆ records_in ┆ dropped_crossed │
│ ---    ┆ ---        ┆ ---             │
│ str    ┆ i64        ┆ i64             │
╞════════╪════════════╪═════════════════╡
│ AAPL   ┆ 4          ┆ 1               │
└────────┴────────────┴─────────────────┘