quantlab.portfolio.decision_inputs.DecisionInputs

class quantlab.portfolio.decision_inputs.DecisionInputs(dataset, constructor, *, fill_column, valuation_column, rebalance_periods, anchor, execution=None, end=None)

Bases: object

Assemble a rule’s decision inputs from a price dataset and decide with it.

The one place a PortfolioContext is built. weights decides a whole prediction panel, replaying the holdings between rebalances, for the research backtester; context builds one bar’s context with the holdings an executor injects, and rebalances tells it which bars to decide. On every rebalance bar both give the same context. A replay building a context per bar reads its window once with preloaded.

Parameters:

datasetMarketDataset

The price dataset: its panel, calendar, tradable_bars and delisting_bars.

constructorPortfolioConstructor

The rule, already bound to the predictor’s labels.

fill_column, valuation_columnstr

The price orders fill at and the price the portfolio is valued at.

rebalance_periodsint

Rebalance every this many bars of the dataset’s calendar.

anchorpd.Timestamp or datetime-like

The bar the schedule counts from: the first bar of the prediction panel a run decides on.

executionExecutionSettings, optional

The sizing basis, fees and slippage weights replays the holdings with. Default: sizing at the fill price, no costs.

endpd.Timestamp or datetime-like, optional

The last bar of a replay, on which rebalances is False (an order decided there has no next bar to fill on); None for an open-ended run. weights never rebalances its panel’s last bar.

Raises:

TypeError

If constructor is not a PortfolioConstructor.

ValueError

If rebalance_periods is smaller than 1.

Examples

>>> import numpy as np, pandas as pd, xarray as xr
>>> from quantlab.portfolio.config import TopNConfig
>>> from quantlab.dataset.memory import FrameDataset
>>> from quantlab.portfolio.predefined.top_n import TopNConstructor
>>> ts = pd.bdate_range("2024-01-01", periods=4)
>>> symbols = ["AAA", "BBB", "CCC"]
>>> close = xr.DataArray(
...     [[10.0, 20.0, 30.0], [11.0, 20.0, 31.0], [12.0, np.nan, 30.0], [12.0, 21.0, 29.0]],
...     dims=("timestamp", "symbol"), coords={"timestamp": ts, "symbol": symbols},
... )
>>> prices = FrameDataset(xr.Dataset({"open": close, "close": close}))
>>> inputs = DecisionInputs(
...     prices,
...     TopNConstructor(TopNConfig(direction="long_only", top_n=1)),
...     fill_column="open",
...     valuation_column="close",
...     rebalance_periods=2,
...     anchor=ts[0],
... )
>>> scores = xr.Dataset(
...     {"ret": (("timestamp", "symbol"), [[0.1, 0.9, 0.2], [0.0, 0.0, 0.0], [0.9, 0.5, 0.1], [0.0, 0.0, 0.0]])},
...     coords={"timestamp": ts, "symbol": symbols},
... )
>>> inputs.weights(scores)["weight"].values
array([[ 0.,  1.,  0.],
       [nan, nan, nan],
       [ 0.,  1.,  0.],
       [nan, nan, nan]])

BBB is halted at the third bar, so it stays locked. An executor holding all of BBB there gets the same context:

>>> [inputs.rebalances(t) for t in ts]
[True, False, True, False]
>>> held = xr.DataArray([1.0], dims="symbol", coords={"symbol": ["BBB"]})
>>> bar = inputs.context(ts[2], scores["ret"].isel(timestamp=2).to_dataset(), held)
>>> bar.tradable.values, bar.locked.values
(array([ True, False,  True]), array([False,  True, False]))

context(t, predictions, current_weights)

Build the context of one bar from the dataset and the holdings given.

The bar’s symbols are the predicted ones followed by any held symbol without a prediction (its predictions NaN, so it is not selectable and stays locked where it cannot trade). The tradability, the return window and staleness of the last history_bars valuation prices up to t, and the rule’s factor values at t (each computed up to t with its own warm-up) are read from the dataset, and its risk model’s exposures and estimate row at t from the model (the row on every call, so a store extended since is seen); with the holdings the panel entry replayed, the context equals the one weights built at t. No warning is given for a short history. Inside preloaded the same values are sliced from memory.

Parameters:

tpd.Timestamp or datetime-like

The bar decided on, a bar of the dataset’s calendar.

predictionsxr.Dataset

Every label’s prediction at the bar, one variable per label on symbol. A scalar timestamp coordinate is dropped.

current_weightsxr.DataArray

The weights held, valued at the bar’s valuation price, on symbol: finite, 0.0 where nothing is held. A symbol it lacks is not held.

Returns:

PortfolioContext

Raises:

ValueError

If an input is not on symbol alone or repeats a symbol, a current weight is not finite, t is not a bar of the dataset, the risk model’s exposures store does not cover t (under "read"), or, inside preloaded, t is outside the preloaded window.

Examples

BBB has no price at the bar, and top-n reads a window of one bar:

>>> bar.current_weights.values, bar.staleness.values
(array([0., 1., 0.]), array([ 0., nan,  0.]))

decision_symbols(predictions)

Return which symbols of a prediction panel a run decides on.

A symbol with a finite prediction of any label on some bar of the panel, the only symbols a rule can buy (and so hold), or with a finite value of one of the rule’s axis_factors on some bar from the panel’s first bar to its last (those factors computed once over the range, with their own warm-up). A run drops the other symbols from its axis: the rule never holds them, and none of its inputs read them. A rule that keeps every symbol (keeps_every_symbol: a covariance estimated from the context’s symbols together, such as LedoitWolfEstimator’s shrinkage) keeps them all.

Parameters:

predictionsxr.Dataset

One variable per label on (timestamp, symbol).

Returns:

np.ndarray

Booleans on the panel’s symbol.

Raises:

ValueError

If an axis factor is not an output of a declared factor.

Examples

>>> inputs.decision_symbols(scores.where(scores.symbol != "CCC"))
array([ True,  True, False])

classmethod from_run(run_dir, *, end=None)

Rebuild the decision inputs of a recorded backtest run.

Read through BacktestRun: the rule (rebuild("constructor")), the price dataset (rebuild("price_dataset"), an in-memory one reading the copy under the run directory), the market columns, the execution settings and the rebalance periods; from the run’s prediction panel, the label specs the rule is bound to and the anchor (the panel’s first bar). Without end the schedule is open-ended, as a live run’s is: it keeps counting past the run’s last bar. Neither the model nor the factor, label or backtest layers are imported (a rule declaring factors imports its factors’ layer). weights on the run’s predictions reproduces the run’s weights.

Parameters:

run_dirstr or os.PathLike

A run directory written by run() or run_cv().

endpd.Timestamp or datetime-like, optional

The last bar of a replay (the run’s own last bar to replay it whole), on which rebalances is False; None for an open-ended run.

Returns:

DecisionInputs

Raises:

FileNotFoundError

If the run has no prediction panel (a run_weights() run has no model and so no panel).

ValueError

If the run is not a backtest run, records no constructor, or the rule refuses the specs.

Examples

With run_dir the directory of a run() with a top-2 rule:

>>> inputs = DecisionInputs.from_run(run_dir)
>>> inputs.constructor
TopNConstructor(direction='long_only', top_n=2, score_label=None)
>>> predictions = BacktestRun.open(run_dir).predictions().predictions
>>> weights = inputs.weights(predictions)  # the run's weights

history_prices(first, last)

Return the fill and valuation prices the bars from first to last read, warm-up included.

From history_bars - 1 bars before first (a warning names the shortfall when the dataset holds fewer, as weights warns) to last, on every symbol, in one request: what panel and panel_reads take as prices for any block of those bars.

Parameters:

first, lastpd.Timestamp or datetime-like

The first and last bar.

Returns:

xr.Dataset

Examples

>>> inputs.history_prices(ts[1], ts[3]).sizes["timestamp"]  # top-n reads no warm-up
3

panel(predictions, *, prices=None, reads=None)

Return the decision inputs of a block of decision bars at once, for the approximate backtest.

The panel counterpart of context: each bar’s predictions, its tradability (tradable_bars, judged from nothing later than the bar) and the raw valuation prices its return window and staleness are read from (the rule’s history_bars up to the bar), and the values of the rule’s declared factors (one Factor.compute each from the first to the last bar, with its own warm-up), as the per-bar context reads them; DecisionPanel.return_windows and DecisionPanel.staleness give each bar’s window and staleness as its context holds them. With a declared factor risk model, its exposures at the bars and its estimate store’s rows there, as panel_reads reads them. No holdings: an approximate rule’s decide is handed those.

Parameters:

predictionsxr.Dataset

One variable per label on (timestamp, symbol), at the decision bars (bars of the dataset’s calendar, in time order).

pricesxr.Dataset, optional

The fill and valuation prices on (timestamp, symbol) from the first bar’s warm-up (history_prices) to the last bar or beyond, already read (an approximate backtest reads its window once); read from the dataset when omitted.

readsPanelReads, optional

The tradability, declared factors and risk model’s exposures and estimate rows covering the bars, already read (panel_reads over a longer span, read once for many blocks); read over the bars when omitted.

Returns:

DecisionPanel

Raises:

ValueError

If a bar is not a bar of the prices, or prices starts after the first bar’s warm-up.

Examples

>>> block = inputs.panel(scores.isel(timestamp=[0, 2]))
>>> block.tradable
array([[ True,  True,  True],
       [ True, False,  True]])
>>> block.staleness()
array([[ 0.,  0.,  0.],
       [ 0., nan,  0.]])

panel_reads(first, last, *, prices=None)

Read what panel takes for any block of the bars from first to last, once.

Each in one request, as weights reads them over its panel: the tradability of every bar of the range (tradable_bars, judged from nothing later than the bar), the rule’s declared factors (one Factor.compute each over the range, with its own warm-up), and a declared factor risk model’s exposures (FactorRiskModel.exposures, read or computed per its exposure_data_strategy) and its estimate store’s rows over the part of the range it covers. All on every symbol.

Parameters:

first, lastpd.Timestamp or datetime-like

The first and last bar.

pricesxr.Dataset, optional

The fill and valuation prices covering the range, already read (history_prices); read from the dataset when omitted.

Returns:

PanelReads

Raises:

ValueError

If the exposures store does not cover the range (under "read").

Examples

>>> reads = inputs.panel_reads(ts[1], ts[3])  # top-n declares no factor or risk model
>>> reads.tradable.values
array([[ True,  True,  True],
       [ True, False,  True],
       [ True,  True,  True]])
>>> reads.factors is None, reads.risk_exposures is None
(True, True)

panel_scheduled(timestamps)

Return which bars of a panel are scheduled rebalance bars: scheduled, the panel’s last bar excepted.

A weight decided at a bar fills on the next one, so the panel’s last bar, with no bar after it, is never decided. Both backtests take their schedule from here (weights and the approximate backtest).

Parameters:

timestampsarray_like of datetime-like

The panel’s bars, in order.

Returns:

np.ndarray

One boolean per bar, the last one False.

Examples

>>> inputs.panel_scheduled(ts)
array([ True, False,  True, False])

preloaded(start, end)

Read what the contexts of the bars from start to end need once; slice them from memory inside.

For a replay that builds a context per bar (an event-driven executor’s closed loop, a recheck of its decisions), which otherwise reads its sources again on every bar. On entering, over the dataset’s bars from start to end (the window) it reads, each in one request through the usual read seams, so an open quantlab.runs.record.DataRecorder records each as one request over the window: the fill and valuation prices from the history_bars - 1 bars before the window’s first bar (or the dataset’s first bar) to its last, on every symbol; their tradability (tradable_bars); the rule’s declared factors (Factor.compute over the window, as weights computes them over its panel); its declared risk model’s exposures; and the model’s estimate rows over the part of the window its estimate store covers.

Inside, context at a bar of the window slices the bar’s inputs from what was read and returns the context it builds without the preload, value for value: the price window and its warm-up are counted on the dataset’s calendar as bar_before counts them, so a bar near the window’s start has the same history, and nothing after a bar enters its context. context at a bar outside the window raises ValueError; weights and rebalances are unaffected. On leaving, everything read is dropped. A factor whose value at a bar depends on where its computation starts, beyond its warmup_bars, would differ, as it differs between weights and context already.

Parameters:

start, endpd.Timestamp or datetime-like

The window: the first and last bar whose context is built. A date-only end includes every bar of that day.

Yields:

DecisionInputs

self, preloaded.

Raises:

ValueError

If the dataset has no bar from start to end, self is already preloaded, or a read refuses its range (an exposures store not covering the window, under "read").

Examples

>>> with inputs.preloaded(ts[1], ts[3]):
...     again = inputs.context(ts[2], scores["ret"].isel(timestamp=2).to_dataset(), held)
>>> again.tradable.values, again.staleness.values
(array([ True, False,  True]), array([ 0., nan,  0.]))
>>> with inputs.preloaded(ts[2], ts[3]):
...     inputs.context(ts[1], scores["ret"].isel(timestamp=1).to_dataset(), held)
Traceback (most recent call last):
ValueError: 2024-01-02 00:00:00 is outside the preloaded window 2024-01-03 00:00:00 to 2024-01-04 00:00:00; ...

rebalances(t)

Return whether bar t is a scheduled rebalance bar.

Every rebalance_periods-th bar of the dataset’s calendar from the anchor rebalances, except end. A bar before the anchor, at or after end or off the calendar never does. The calendar’s timestamps (never its prices) are read once up to the dataset’s last bar, and again only for a bar past it. weights decides the bars of its panel this schedule names, its last bar excepted.

Parameters:

tpd.Timestamp or datetime-like

The bar.

Returns:

bool

Examples

>>> inputs.rebalances(ts[2]), inputs.rebalances(ts[3])
(True, False)

scheduled(timestamps)

Return which of timestamps are scheduled rebalance bars: rebalances for many bars.

Parameters:

timestampsarray_like of datetime-like

The bars.

Returns:

np.ndarray

One boolean per bar.

Examples

>>> inputs.scheduled(ts)
array([ True, False,  True, False])

weights(predictions, *, delisted=None)

Decide every rebalance bar of a prediction panel and return the target weights.

Reads the dataset’s fill and valuation prices from the history_bars - 1 bars before the panel’s first bar to its last (a warning names the shortfall when the dataset holds fewer, and the first windows are short), the tradability (tradable_bars) of the panel’s bars, the rule’s declared factors over the panel, each with its own warm-up, and the exposures of its declared risk model over the panel and its estimate store’s rows from the first to the last rebalance bar, in one request. It then loops decide over the rebalance bars in time order. Each bar’s context holds its predictions, its tradability, the return window and staleness of its last history_bars valuation prices, its factor values, its risk exposures and estimate row, and the weights currently held: the earlier targets replayed by the Execution module under the execution settings, exactly as the simulation trades them (filled at the next bar’s fill price, rejected without a raw price, a holding marked in delisted settled at its last valuation on the next bar, sells before buys, fees and slippage), valued at the bar’s forward-filled valuation price; all 0.0 before the first rebalance.

A bar that returns all NaN holds, and so does one whose construct raises PortfolioConstructionError (with a warning), listed in attrs["failed_bars"]. Events a row reports in attrs["events"] are gathered by name into the result’s attrs["events"], one record per bar: {"bar", "symbols"} for an event naming its symbols, {"bar", "count"} for one counting them. A bar that does not rebalance gets an all-NaN row.

The result records the rebalances: scheduled marks the bars the schedule named (rebalances, the panel’s last bar excepted) and decided those of them the rule decided, so a held bar (a failure or an all-NaN row) is scheduled and not decided. decided is exactly the rows with a finite weight; readers take it from the record rather than testing the rows.

Parameters:

predictionsxr.Dataset

One variable per label on (timestamp, symbol); its bars are consecutive bars of the dataset’s calendar, the first at or after the anchor.

delistedxr.DataArray, optional

Booleans marking each delisted symbol’s last priced bar, on the panel’s labels or a part of them: the marks the simulation settles with. Default: the dataset’s delisting_bars of the panel’s bars.

Returns:

xr.Dataset

weight on the predictions’ labels, scheduled and decided (booleans on timestamp), attrs["failed_bars"] (ISO timestamps) and attrs["events"].

Raises:

ValueError

If the panel starts before the anchor, a prediction bar is not a price bar, the risk model’s exposures store does not cover the panel (under "read"), or construct returns a row breaking the weights contract.

Examples

>>> out = inputs.weights(scores)
>>> out.attrs["failed_bars"], out.attrs["events"]
([], {})
>>> out["scheduled"].values, out["decided"].values
(array([ True, False,  True, False]), array([ True, False,  True, False]))