quantlab.backtest.base.BaseBacktester#

class quantlab.backtest.base.BaseBacktester(config)[source]#

Bases: Component, ABC

Abstract base of every backtester: the public entry points and shared steps.

The engine varies by inheritance and the market and selection logic by composition. The hierarchy is BaseBacktester (this class), then an engine layer such as VectorBtBacktester that implements _simulate, _simulate_benchmark and _engine_stats and sets fill_delay_bars, then a named concrete class that composes a MarketSpec (the MARKET class attribute) and a signal generator (_generate_signals) onto that engine. Concrete classes also set config_cls, the config class the config setter accepts.

The public entry points run(), run_cv() and run_weights() are defined here and never overridden. run() and run_cv() run the same fixed sequence of steps and call the hooks above along the way: prepare the model, request the factor panels for the window and predict, generate signals, simulate, compute metrics, then write the run directory. run_weights() starts from given target weights, so it skips the model and _generate_signals and needs no model on the config.

Parameters:

configBacktestConfig: The backtest configuration, an instance of config_cls. It is validated and normalized on assignment; see the config setter.

Attributes:

MARKETMarketSpec or None: The market conventions. None on abstract classes; a concrete class must set it.
fill_delay_barsint: Bars between the bar a weight forms on and the bar it fills on, set by the engine layer. run() and run_cv() refuse a model whose label delay differs from it, since the label would then measure a return the engine never trades.
expected_fingerprintdict or None: The data fingerprint of a previous run of the same config (the stitched pass of a run_cv run). When set, each run compares the data it reads against it, by digest, and warns on a difference. It is filled in when a run is rebuilt with BacktestRun.rebuild_backtester.
expected_fold_fingerprintsdict or None: The data fingerprint of each fold of a previous run_cv run, by fold index; each fold of run_cv compares with its own and the warning names the fold. Filled in by rebuild_backtester too.
expected_training_fingerprint, expected_training_codedict or None: The training data record and the code record of the trained unit a previous train-mode run used. When set, a train-mode run() compares the unit it trains with them and warns on a difference. Filled in by rebuild_backtester for a train-mode run; load mode trains nothing and compares neither.

Examples

A concrete class over the vectorbt engine needs only three members:

class EqualWeightBacktester(VectorBtBacktester):
    config_cls = BacktestConfig
    MARKET = MarketSpec("open", "close", 252, 390)

    def _generate_signals(self, predictions, prices, delisted):
        ...  # return a ``weight`` panel on (timestamp, symbol)

>>> backtester = EqualWeightBacktester(config)
>>> result = backtester.run()

ATTRIBUTION_GROUPS = 10#: Score groups of the attribution (quantlab.runs.backtest_attribution.ATTRIBUTION_GROUPS).

account(simulation)[source]#

Return the account record of simulation, as the run’s metrics read it.

The simulation’s value, returns, orders, round trips, settlements, rebalance record, rejected orders, largest target deviation and holdings, with config.init_cash and the market’s year (MARKET.year_freq) over the simulation’s bar interval. quantlab.runs.run_metrics.run_metrics of it and the engine’s own whole-window rows are the run’s metric blocks (ADR 0034).

Parameters:

simulationSimulationResult: A simulation this backtester measured (result.simulation), with its rebalance record.

Returns:

AccountRecord: The record.

Examples

>>> result = backtester.run_weights(weights)
>>> account = backtester.account(result.simulation)
>>> account.init_cash, account.year_freq / pd.Timedelta(account.bar_interval)
(1000000.0, 252.0)

property class_name#

The class’s short name, used as the prefix of every message it logs.

Examples

>>> backtester.class_name
MyBacktester

property config#

The validated config this backtester was built with.

Examples

>>> backtester.config.start_date, backtester.config.rebalance_periods
('2024-02-12', 5)

abstract property config_cls#

The config class this backtester accepts.

Concrete classes satisfy it with a plain class attribute; the config setter checks isinstance(config, config_cls) first.

Examples

>>> class MyBacktester(VectorBtBacktester):
...     config_cls = BacktestConfig
...     MARKET = MarketSpec("open", "close", 252, 390)
...     def _generate_signals(self, predictions, prices, delisted): ...

property data_fingerprint#

The data fingerprint of the last run; empty before a run.

What the run’s DataRecorder recorded: each dataset or factor store the run read, keyed by its component path in the backtester (price_dataset, model.factors.0.dataset), with one entry per distinct request. For run_cv it is the stitched pass; each fold’s is in its fold record. A run directory records the same mapping (quantlab.runs.backtest_run.BacktestRun.data_fingerprint); it is computed for a run kept in memory too.

Examples

>>> _ = backtester.run_weights(weights)  # output_dir=None
>>> sorted(backtester.data_fingerprint)
['price_dataset']
>>> backtester.data_fingerprint["price_dataset"][0]["variables"]
['adjClose', 'adjOpen']

classmethod from_config(config, run_dir=None)[source]#

Rebuild a backtester from its get_config(), a run’s recipe.

Every config field must be present: a missing one is refused rather than filled from today’s dataclass default, which may differ from the value the run used. The fields are then rebuilt by the component rule, with run_dir passed to every dataset at any depth. A run directory is rebuilt through quantlab.runs.backtest_run.BacktestRun.rebuild_backtester, which also sets the run’s data fingerprint as expected_fingerprint.

Parameters:

configdict: The dict get_config() returned.
run_dirstr or os.PathLike, optional: The run directory config was read from. Required when the config names stores relative to it.

Returns:

BaseBacktester: A backtester ready to run.

Raises:

ValueError: If a config field is missing, a key is unknown, or the config names stores relative to a run directory and run_dir is not given.

Examples

>>> rebuilt = type(backtester).from_config(backtester.get_config())
>>> rebuilt.get_config() == backtester.get_config()
True

report_figure(result)[source]#

Return the chart of report.html for result as a plotly figure.

The same figure a run directory’s report embeds (equity, drawdown and monthly returns, the in-sample range shaded, the deepest drawdown marked, the benchmark rows when a benchmark ran), built from the result in memory, so a run kept with output_dir=None can be looked at too. result must come from run() or run_weights() of a backtester with this config, which is checked without reading any data: its bars lie inside the configured window, its curve starts at init_cash, and it has a benchmark curve exactly when a benchmark is configured. A run_cv() result is refused: its stitched curve is drawn in the report.html of its run directory.

Parameters:

resultBacktestResult: The result to draw.

Returns:

plotly.graph_objects.Figure: The figure, not yet shown or written.

Raises:

TypeError: If result is a CVBacktestResult or not a BacktestResult.
ValueError: If result does not match this config (see above), naming what differs.

Examples

>>> import numpy as np
>>> import pandas as pd
>>> import xarray as xr
>>> from quantlab.backtest.predefined.weights import WeightsVectorBt
>>> from quantlab.backtest.config import WeightsBacktestConfig
>>> from quantlab.dataset.memory import FrameDataset
>>> bars = pd.bdate_range("2024-01-01", periods=5)
>>> prices = FrameDataset(pd.DataFrame({
...     "timestamp": np.repeat(bars, 2), "symbol": ["AAA", "BBB"] * 5,
...     "open": np.linspace(10.0, 14.0, 10), "close": np.linspace(10.5, 14.5, 10),
... }))
>>> backtester = WeightsVectorBt(WeightsBacktestConfig(
...     price_dataset=prices, start_date="2024-01-01", end_date="2024-01-05",
...     output_dir=None, rebalance_periods=1,
...     fill_price_column="open", valuation_price_column="close",
...     trading_days_per_year=252, session_minutes_per_day=390,
... ))
>>> weights = xr.DataArray(
...     [[0.5, 0.5]] + [[np.nan, np.nan]] * 4, dims=("timestamp", "symbol"),
...     coords={"timestamp": bars, "symbol": ["AAA", "BBB"]},
... )
>>> result = backtester.run_weights(weights)
>>> figure = backtester.report_figure(result)
>>> [trace.name for trace in figure.data][:2]
['equity', 'drawdown']

run()[source]#

Backtest one model over the configured window and write a run directory.

Subclasses do not override this method. In train mode the model is trained on its own dates first; in load mode the checkpoint is restored and the training dates recorded beside it define the in-sample split (a warning is logged if they select different bars than config.model’s dates). Each factor is computed (or read) for the window, warming itself up by its own warmup_bars; no dataset, factor or label config is changed. The model predicts the window, the concrete class turns the predictions into target weights, the engine simulates them, metrics are computed for the whole window and for the in-sample and out-of-sample parts, and everything is written to a new directory under config.output_dir. The data the window reads is recorded (see data_fingerprint) and compared with expected_fingerprint when one is set, also on the failure path; training reads are the trained unit’s, not the backtest’s.

Returns:

BacktestResult: A BacktestResult with the run directory, the predictions and weights on the window bars, the simulation and the metrics.

Raises:

ValueError: If config.model is None, model_mode="load" comes without config.checkpoint, a label’s delay differs from fill_delay_bars, or the window has no price bars.

Examples

>>> backtester = MyBacktester(
...     BacktestConfig(
...         price_dataset=prices,
...         model=model,
...         model_mode="load",
...         checkpoint="models/head.joblib",
...         start_date="2024-02-12",
...         end_date="2024-03-11",
...         output_dir="runs",
...         rebalance_periods=5,
...     )
... )
>>> result = backtester.run()
>>> result.weights["weight"].dims, result.simulation.value.sizes
(('timestamp', 'symbol'), Frozen({'timestamp': 21}))
>>> result.metrics["out_of_sample_ranges"]
[('2024-02-12', '2024-03-11')]

run_cv()[source]#

Replay a train_cv run fold by fold and simulate the stitched weights.

Subclasses do not override this method. It opens the walk-forward unit config.cv_project_dir with quantlab.runs.trained_run.TrainedRun, keeps the folds whose test segment meets the backtest window, cutting a fold that straddles start_date or end_date to the window (so the account can start from cash on any bar, #301), and checks on the price calendar that those test segments are contiguous and non-overlapping before any model is loaded (a stitched curve with a gap or an overlap corresponds to no real trading path). Each fold’s checkpoint then predicts its own test segment, and the fold’s in-sample split uses that fold’s training dates. A label looks a few bars ahead (its lookahead), so the training labels of a fold already saw the bars after train_end. train_cv purges those bars from every training window and each fold records its fitted window, so a test bar counts as in-sample only if a label reads further than the purge removed.

The fold predictions are then concatenated and turned into weights in one pass of _generate_signals over the prices from the first test_start to the last test_end, so the holdings a rule is handed, locked positions included, carry across fold boundaries and the rebalance schedule runs on from the first bar; the weights are simulated once, with capital carried across as well, and the stitched metrics record the pass’s portfolio_construction. A fold is that account over its test segment, never a second account started from cash (ADR 0033): its metrics are the stitched curve’s over its bars, its first bar’s return included, so the folds’ returns chain to the stitched total. Each fold records the data its prediction reads, compared with that fold’s expected fingerprint (the warning names the fold); the run’s own fingerprint is what the stitched pass reads.

Returns:

CVBacktestResult: A CVBacktestResult with the run directory, the per-fold records, and the stitched weights, simulation and metrics.

Raises:

ValueError: If config.model is None, config.cv_project_dir is unset, model_mode is not "load", a label’s delay differs from fill_delay_bars, cv_project_dir is not a walk-forward unit TrainedRun can open or has no fold, no fold’s test segment meets the window, or the fold test segments are not contiguous.
FileNotFoundError: If cv_project_dir or a fold checkpoint is missing.

Examples

>>> backtester = MyBacktester(
...     BacktestConfig(
...         price_dataset=prices,
...         model=model,
...         model_mode="load",
...         cv_project_dir="models/head_trial_20260925",
...         start_date="2024-02-12",
...         end_date="2024-04-17",
...         output_dir="runs",
...         rebalance_periods=2,
...     )
... )
>>> cv = backtester.run_cv()
>>> len(cv.folds), cv.weights.sizes
(8, Frozen({'timestamp': 48, 'symbol': 6}))
>>> cv.metrics["stitched"]["in_sample_ranges"]
[]

run_weights(weights)[source]#

Backtest a precomputed target-weight panel over the configured window.

Subclasses do not override this method. No model is involved: config.model and config.model_mode are ignored and may both be None, and _generate_signals is not called. The fill and valuation prices of config.price_dataset are read for the window config.start_date..``config.end_date``, the weights are checked against the target-weight contract on exactly those bars, and the engine simulates them (a weight formed at bar t fills at bar t+1’s fill price). The benchmark, when configured, is simulated and compared as in run(). There is no training window, so the metrics cover the whole window only: whole (plus benchmark and relative, each with a whole block, when a benchmark ran) and notes, with no in-sample or out-of-sample block. The run directory is written like run()’s when config.output_dir is set; the data fingerprints cover the prices and the benchmark.

Parameters:

weightsxarray.Dataset or xarray.DataArray: Target weights on (timestamp, symbol), in either axis order. A dataset must carry a weight variable; a data array is used whatever its name. A run’s weights, read with BacktestRun.open(run_dir).weights(), replay that run. The timestamps must be exactly the price bars of the window and the symbols exactly the price dataset’s symbols, in any order (they are aligned to the price axes). A NaN keeps the symbol’s holding and a finite value is its target; the targets of a row have a gross exposure, the sum of their absolute values, of at most 1.

Returns:

BacktestResult: The run directory (None without output_dir), the weights aligned to the price axes, the simulation, the metrics and the benchmark simulation; predictions is None.

Raises:

ValueError: If the window has no price bars, the weights are not on (timestamp, symbol), their bars or symbols differ from the prices’ (naming the first missing or extra ones), or a row breaks the contract (naming the offending bar).

Examples

With prices a price dataset and weights a weight panel on its 30 bars from 2024-02-12 to 2024-03-22 (here the weights of an earlier run()):

>>> backtester = USEquityCrossectionSelectStockVectorBt(
...     CrossSectionBacktestConfig(
...         price_dataset=prices,
...         start_date="2024-02-12",
...         end_date="2024-03-22",
...         output_dir=None,
...         rebalance_periods=5,
...         constructor=TopNConstructor(TopNConfig(direction="long_only", top_n=2)),
...     )
... )
>>> result = backtester.run_weights(weights)
>>> result.run_dir is None, result.predictions is None
(True, True)
>>> sorted(result.metrics), result.simulation.value.sizes
(['execution', 'notes', 'whole'], Frozen({'timestamp': 30}))

property ticker_lookup#

Lookup that names the price dataset’s symbol ids as of a day.

The price dataset says which lookup applies (MarketDataset.ticker_lookup()): for CRSP data (the Center for Research in Security Prices), whose symbols are PERMNOs, permanent numeric security ids, it reads the .crsp_tickers.json sidecar beside the store, which records which ticker each PERMNO traded under on each date. The engine labels its settlement and rejected-order records through it. It is asked of the dataset on first access, kept so a sidecar is read once per run, and reset whenever a new config is assigned. A dataset that names no lookup (a FrameDataset, even one read back from a run directory, or a vendor without tickers) makes the property None, and _symbol_labels shows its symbols as they are.

Examples

>>> backtester.ticker_lookup is None  # its price dataset names no lookup
True