quantlab.runs.backtest_run.BacktestRun

class quantlab.runs.backtest_run.BacktestRun(path, kind, written_at, window, market, annualization, data_fingerprint, code=None, folds=(), index=None, _trained_run=None, _benchmark_source=None, _components=<factory>, _recipe_dir=None)

Bases: object

One backtest run directory, as its run.json describes it.

Attributes:

pathPath

The run’s directory.

kindstr

"run", "run_cv", "run_weights" or "fold".

written_atstr

When run.json was written, an ISO 8601 UTC timestamp.

windowtuple of str

The first and last bar simulated, as bar labels.

marketMarket

The price columns the backtester filled and valued at.

annualizationAnnualization

How its statistics were annualized.

data_fingerprintdict

What the run read, by component path, as its DataRecorder recorded it: a run_cv run’s stitched pass, what a fold’s prediction read.

codedict or None

The code record of the backtester’s tree when the run was written (git, modules, libraries); a fold’s is its run’s.

foldstuple of BacktestRun

A run_cv run’s folds, in fold order; empty otherwise.

indexint or None

A fold’s index in the walk; None for other kinds.

Examples

>>> cv = BacktestRun.open(cv_result.run_dir)
>>> cv.kind, [fold.index for fold in cv.folds][:3]
('run_cv', [0, 1, 2])
>>> cv.trained_run().kind
'walk_forward'

property backtester_class

The import path of the backtester class that wrote the run.

Examples

>>> BacktestRun.open(run_dir).backtester_class.rsplit(".", 1)[-1]
'USEquityCrossectionSelectStockVectorBt'

property benchmark_source

Where the benchmark was read from: its store, or the dataset held in memory.

None for a run without a benchmark. Recorded when the run was written, as its report names it.

Examples

>>> BacktestRun.open(run_dir).benchmark_source
'/data/market/sharadar/sharadar_spy_1d/sharadar_spy_1d.zarr'

equity()

The equity curve: value and returns, plus the benchmark’s when one ran.

A fold’s is its run_cv run’s, cut to the fold’s window.

Examples

>>> sorted(BacktestRun.open(run_dir).equity().data_vars)
['returns', 'value']

property execution

How orders were sized and charged: sizing_basis, fees, slippage.

Examples

>>> BacktestRun.open(run_dir).execution
ExecutionSettings(sizing_basis='fill', fees=0.001, slippage=0.0)

factor_attribution()

The per-bar factor attribution, or None for a run without a risk model.

Examples

>>> BacktestRun.open(run_dir).factor_attribution()["contribution"].dims
('timestamp', 'term')

property has_predictions

Whether the run has a prediction panel (a run with a model), without reading it.

Examples

>>> BacktestRun.open(run_dir).has_predictions
True

holdings()

The simulated holdings, or None for a run written without them.

A fold’s are its run_cv run’s, cut to the fold’s window.

A holding panel on (timestamp, symbol): each symbol’s value at each bar’s close over the book’s value, cash included, on every bar of the window; 0 where nothing is held.

Examples

>>> BacktestRun.open(run_dir).holdings()["holding"].dims
('timestamp', 'symbol')

property init_cash

The cash the simulation started with.

Examples

>>> BacktestRun.open(run_dir).init_cash
1000000.0

log_report(tracking_run)

Attach the run’s HTML report to tracking_run (its log_file).

Parameters:

tracking_runquantlab.tracking.base.TrackingRun

The open tracking run of the backtest.

Examples

A backtester attaches its report to the tracking run it opened:

BacktestRun.open(result.run_dir).log_report(tracking_run)

metrics()

The run’s metrics, as the backtester returned them (NaN as None).

Examples

>>> sorted(BacktestRun.open(run_dir).metrics())[:2]
['execution', 'in_sample']

classmethod open(path)

Read the backtest run at path, with its folds.

Parameters:

pathPath or str

The run’s directory, or a file in it.

Returns:

BacktestRun

Raises:

FileNotFoundError

If path does not exist.

ValueError

If the run, or one of its folds, has no run.json, another format_version, or is not a backtest run.

Examples

>>> BacktestRun.open(result.run_dir).kind
'run'

predictions()

The predictions the rule read, or None for a run without a model or a fold.

Examples

>>> [spec.name for spec in BacktestRun.open(run_dir).predictions().labels]
['fwd_ret_1']

property rebalance_periods

The number of bars between rebalances.

Examples

>>> BacktestRun.open(run_dir).rebalance_periods
5

rebuild(name)

Rebuild the component the run’s config holds in the field name.

Datasets recorded under the run directory are read from it.

Parameters:

namestr

A config field declared to hold a component ("price_dataset", "model", "constructor", …), or several.

Returns:

object

The rebuilt component, a list of them, or None for an empty field.

Raises:

ValueError

If name is not a field declared to hold components.

Examples

>>> BacktestRun.open(run_dir).rebuild("constructor")
TopNConstructor(direction='long_only', top_n=2, score_label=None)

rebuild_backtester(**overrides)

Rebuild the backtester that wrote the run, with some fields replaced.

The recipe is rebuilt by the component rule, datasets recorded under the run directory read from it, and the run’s data fingerprint is set as the backtester’s expected_fingerprint (and each fold’s as expected_fold_fingerprints), so a re-run warns when its data differs, naming the fold. A train-mode run also sets its trained unit’s training record as expected_training_fingerprint, so the retrained unit is compared with the one the run used; not when an override replaces model or model_mode, since the unit then describes another model. The rebuilt tree’s code record is compared with the run’s code right away; a component an override replaces is left out of both comparisons, data and code, on both sides, and a train-mode run’s trained unit’s code is set as expected_training_code; differences only warn.

Parameters:

**overrides

Values keyed by config field name, used in place of the recorded ones: objects for component fields (model=None, tracker=...), plain values otherwise (output_dir=...).

Returns:

BaseBacktester

Raises:

ValueError

If an override names no config field, or the run is a fold (rebuild its run_cv run).

Examples

>>> run = BacktestRun.open(result.run_dir)
>>> weights_only = run.rebuild_backtester(model=None, model_mode=None, checkpoint=None)
>>> replay = weights_only.run_weights(run.weights())

recipe()

The run’s rebuild recipe, the backtester’s get_config() as recorded.

For presentation that takes a config mapping (the config of quantlab.runs.backtest_report.write_backtest_report, its Setup lines); a component is rebuilt with rebuild(field) and the backtester with rebuild_backtester.

Examples

>>> BacktestRun.open(run_dir).recipe()["rebalance_periods"]
5

report()

The run’s self-contained HTML report, as text.

Examples

>>> "<html" in BacktestRun.open(run_dir).report()
True

settlements()

The delisted holdings settled into cash during the simulation.

A fold’s are its run_cv run’s settled inside the fold’s window.

Examples

>>> BacktestRun.open(run_dir).settlements()
[]

trained_run()

The trained unit the backtest used, or None for a run_weights run.

Examples

>>> BacktestRun.open(run_dir).trained_run().kind
'model'

weights()

The simulated target weights, a weight panel on (timestamp, symbol), with the rebalance record.

The record is scheduled (the bars the rebalance schedule named) and decided (those the rule decided; a held bar is scheduled and not decided), booleans on timestamp. A fold’s are its run_cv run’s, cut to the fold’s window.

Examples

>>> BacktestRun.open(run_dir).weights()["weight"].dims
('timestamp', 'symbol')
>>> sorted(BacktestRun.open(run_dir).weights().data_vars)
['decided', 'scheduled', 'weight']