quantlab.model.base.BaseModel

class quantlab.model.base.BaseModel(config)

Bases: Component, ABC

Framework-agnostic base class of every model head.

A head is configured with a list of factor objects (its features) and a list of label objects (its targets). collect() pulls both into one panel indexed by (timestamp, symbol); train() fits the head on the train_* dates of its config and writes a checkpoint; load() restores one; predict() and predict_panel() run inference. Those public entry points are implemented once, here, and are not overridden by any head.

The training framework is the job of the two variants. TorchModel is the torch variant and LibraryModel the numpy variant. Both accept the one ModelConfig, named by the class attribute config_cls: the config setter checks it first, and the config loader reads it from the class before creating an instance. Each variant declares checkpoint_suffix, the checkpoint file suffix. train and train_cv use it to name files, and load() uses it to reject a file of the wrong kind before building any model.

train writes a trained unit (quantlab.runs.trained_run) under config.model_save_dir as {class}_trial_{timestamp}/: the checkpoint {class}_total{suffix}, the config.json that rebuilds the model, the evaluation files and run.json, which records the windows, the metrics and what the head was trained on.

Parameters:

configModelConfig

The model configuration.

Attributes:

modelobject or None

The fitted model (an nn.Module or a library model object), or None before train() or load().

collected_panelCollectedPanel or None

What collect() read (or the panel use_panel() handed over), None before either.

Raises:

TypeError

If config is not an instance of config_cls.

Examples

Given a head MyHead (a subclass of LibraryModel or TorchModel), a factor object exposing variables f_a and f_b, and a label object exposing ret:

>>> model = MyHead(ModelConfig(
...     factors=[factor], labels=[label],
...     model_save_dir="checkpoints",
...     factor_data_strategy="read", label_data_strategy="read",
...     train_start="2024-01-01", train_end="2024-01-30",
...     test_start="2024-01-31", test_end="2024-02-09",
... ))
>>> checkpoint = model.collect().train()
>>> checkpoint.name
'MyHead_total.joblib'

check_checkpoint(p)

Check a checkpoint’s recorded variables against the model’s own.

The feature and label names (and their order) the checkpoint was trained on are read from trained_on in its unit’s run.json. Only run.json is read, so the check can run before any data is collected; load runs it too.

Parameters:

pPath | str

Path to a checkpoint file.

Returns:

TrainedRun

The checkpoint’s trained unit.

Raises:

FileNotFoundError

If p does not exist.

ValueError

If the checkpoint is not part of a trained run TrainedRun can open, or the recorded and declared variables differ, naming the checkpoint and both variable lists.

Examples

>>> model.check_checkpoint(checkpoint).kind  # trained on the same variables
'model'
>>> other.check_checkpoint(checkpoint)
Traceback (most recent call last):
ValueError: FirstFeatureHead: checkpoint .../FirstFeatureHead_total.joblib
was trained on factor variables ['past_ret_1'] (trained_on in its
run.json), but this model declares ['past_ret_2']; loading it would
feed the model different or permuted inputs

check_hyperparameters()

Validate the reserved hyperparameters this variant reads.

Called first by collect, train and train_cv, before any data is read or any tracking run or checkpoint directory is opened. The default checks nothing.

Raises:

ValueError

If a reserved hyperparameter is invalid.

Examples

>>> model.check_hyperparameters()  # a valid setting returns None

check_reload(p, bars=5, rtol=0.0001, atol=1e-06)

Check that the model predicts its checkpoint’s first test bars as the training run did.

The first warmup_bars + bars bars of the predictions the trained run stored (test_predictions.zarr) are predicted again with predict_window, from freshly requested features, as a backtest or a live run predicts them, and compared on the symbols both hold. The training run predicted them from one panel read from its start_date; a difference means a later prediction does not see the inputs the model was scored on: a feature requested without its warm-up, a store rewritten since training, or a different checkpoint loaded. Call it on the trained or loaded model.

Parameters:

pPath | str

Path to the checkpoint file; its trained unit holds the stored predictions.

barsint

Bars compared after the warm-up bars.

rtol, atolfloat

Tolerances of numpy.isclose; NaN equals NaN.

Returns:

TrainedRun

The checkpoint’s trained unit.

Raises:

ValueError

If the unit stored no test predictions, or a label’s predictions differ on some bar, naming the bars and the largest difference.

Examples

>>> model.check_reload(checkpoint).kind
'model'

abstract property checkpoint_suffix

The checkpoint file suffix, including the leading dot.

Concrete variants satisfy it with a plain class attribute.

Examples

>>> LibraryModel.checkpoint_suffix
'.joblib'

property class_name

The head’s class name.

Examples

>>> model.class_name
MyHead

collect()

Read the features and labels into the model’s collected panel.

The hyperparameters are checked first (check_hyperparameters), so an invalid one fails before any data is read. Each factor and label is asked for its panel from start_date to end_date: read(start, end) from its store under the "read" strategy, compute(start, end) from its inputs under "cal". Factor panels start warmup_bars bars earlier (see _collect_all_features); labels do not, so they are NaN on those bars. The factor panels are merged on the union of their (timestamp, symbol) coordinates, and the labels are joined onto the features’ bars and symbols (CollectedPanel.assemble: a label’s symbol or bar without features is dropped); the panel is kept, sorted on both axes, as collected_panel. Call this before train() or train_cv().

What it reads is recorded by a DataRecorder keyed by component path within the model (factors.0.dataset), and written into the run.json of the unit train() or train_cv() writes next (see training_record).

Returns:

Self

The model itself, for chaining.

Raises:

ValueError

If a hyperparameter this variant reads is invalid.

Examples

>>> model.collect().num_times
40

property config

The model’s configuration object.

Examples

>>> model.config.model_save_dir
checkpoints

config_cls

The config class every model accepts; the config loader reads it from the class to rebuild a model from config.json.

alias of ModelConfig

evaluation_segments()

The bars the model is scored on: its purged train, validation and test segments.

A segment config.metric_splits does not name is empty.

The collected panel’s timestamps, cut as training cuts them (see _fit_segments): the training window split by val_size, then each segment followed by another losing its last L bars, L the largest lookahead_bars() of the model’s labels. Call collect() first.

Returns:

Segments

Timestamps of each segment; val is empty when val_size is 0 or the purge empties it.

Raises:

ValueError

If no training bar is left, by val_size or by the purge.

Examples

With val_size 0:

>>> segments = model.collect().evaluation_segments()
>>> [split for split, _ in segments.splits()]
['train', 'test']

property fitted_train_bounds

The training window actually fitted, (train_start, train_end), after the purge.

After train() it is the window the fit used; after load() the one the checkpoint’s run.json records.

Raises:

RuntimeError

If neither train() nor load() has been called.

Examples

>>> model.train_bounds, model.fitted_train_bounds
(('2024-01-01', '2024-02-09'), ('2024-01-01', '2024-02-07T00:00:00.000000000'))

get_factor_names()

Return the feature variable names, in factor order then variable order.

Each factor contributes its pinned config.factor_names when set, else every name its class can produce (see _variable_names).

Examples

>>> model.get_factor_names()
['f_a', 'f_b']

get_label_names()

Return the label variable names, in label order then variable order.

Examples

>>> model.get_label_names()
['ret']

head_hyperparameters(hyperparameters)

Return hyperparameters without the keys this variant reads itself.

A torch head that forwards its hyperparameters to a network passes them through this first; LibraryModel applies it before calling _init_model, so a library head receives the result. Only the variant’s own reserved_hyperparameters are dropped: a library head keeps lr, which a torch head reserves, because the library may take it. The dict given is not modified.

Examples

>>> xgb_head.head_hyperparameters({"early_stopping": True, "lr": 0.1})
{'lr': 0.1}
>>> torch_head.head_hyperparameters({"epochs": 5, "hidden": 8})
{'hidden': 8}

property label_delays

Each label’s delay in bars, in the order of labels.

Examples

>>> model.label_delays
(1,)

property label_scales

Each label name’s prediction scale: "raw" or "standardized".

"raw" means the prediction is in the label’s own units (a forward return in return units), "standardized" that it only ranks the cross-section. A model is "raw" exactly when its training target is the label itself; the model layer’s variants report "standardized" for every label once a head overrides _transform_target. A model without a training-target hook predicts its labels as they are.

Examples

>>> model.label_scales
{'fwd_ret_1': 'raw'}

property labels

The label objects the model is trained on, in config order.

Examples

>>> [label.get_factor_names() for label in model.labels]
[('fwd_ret_1',)]

load(p)

Restore the model from a checkpoint file.

The file suffix is checked first, so a .pth file is never handed to joblib and a .joblib file never to torch.load. The feature and label variables recorded in the unit’s run.json must then match this model’s declared variables, name for name and in order; neither torch nor a tree library would notice a permuted or substituted input by itself. Finally the checkpoint is loaded, and the model takes the recorded training and test windows: train_bounds, test_bounds and fitted_train_bounds then describe the checkpoint, whatever the config said before.

Parameters:

pPath | str

Path to a checkpoint file ending in checkpoint_suffix.

Returns:

Self

The model itself, for chaining.

Raises:

FileNotFoundError

If p does not exist.

ValueError

If the suffix is wrong, the checkpoint is not part of a trained run quantlab.runs.trained_run.TrainedRun can open, or the recorded variables differ from the model’s declared variables.

Examples

>>> model.load(checkpoint) is model
True

property model_save_dir

The directory train and train_cv create trial directories in.

Examples

>>> model.model_save_dir.name
'checkpoints'

property num_factors

Number of feature variables across all configured factors.

Examples

>>> model.num_factors
2

property num_labels

Number of label variables across all configured labels.

Examples

>>> model.num_labels
1

property num_null

Total number of NaN cells across every variable of the collected panel.

Examples

>>> model.num_null
0

property num_symbols

Number of symbols in the collected panel.

Examples

>>> model.num_symbols
3

property num_times

Number of timestamps in the collected panel.

Examples

>>> model.num_times
40

predict(data)

Return [T, S, L] predictions for a [T, S, F] input.

The variant’s _predict decides the accepted and returned types: the torch variant returns a tensor, the numpy variant an array.

Raises:

ValueError

If neither train() nor load() has been called.

Examples

>>> x = np.random.default_rng(0).standard_normal((5, 3, 2))
>>> model.predict(x).shape
(5, 3, 1)

predict_panel(features)

Predict from a feature panel and return a panel of the same shape.

The result has one variable per label name, on ("timestamp", "symbol"), with coordinates taken from the sorted panel that was actually fed to to_array, so values and coordinates cannot drift apart. Positions where every feature is NaN get NaN for every label: a head that fills NaN with zero would otherwise produce finite predictions for symbols that do not exist yet. Any symbol may be present, including ones the model never saw in training.

The variant hook _predict_panel_array performs the numeric prediction. A torch head predicts bar t from the bars before it in features, so pass warmup_bars bars before the first bar you need.

Parameters:

featuresxr.Dataset

A dataset containing every variable named by get_factor_names().

Raises:

ValueError

If a factor variable is missing, or if the prediction does not have shape [num_times, num_symbols, num_labels].

Examples

>>> feats = panel[["f_a", "f_b"]].isel(timestamp=slice(0, 5))
>>> out = model.predict_panel(feats)
>>> dict(out.sizes), list(out.data_vars)
({'timestamp': 5, 'symbol': 3}, ['ret'])

predict_scored()

Predict the bars of the collected panel the model is scored on, from the panel’s feature array.

The bars run from warmup_bars bars before the first bar of evaluation_segments() (its first scored segment) to the last, so a windowed head’s first scored bar has its history; each is predicted as predict_panel would. The array is the collected_panel’s, built once in the panel’s own type and shared by every caller, so evaluating after each fit does not convert the panel again.

Returns:

xr.Dataset

One variable per label on the predicted bars and the panel’s symbols.

Examples

With metric_splits=("test",) only the 10 test bars are predicted:

>>> model.collect().train()
>>> model.predict_scored().sizes["timestamp"]
10

predict_window(start, end)

Predict every bar from start to end from freshly requested features.

Each factor is asked for its panel from warmup_bars bars before start (counted as _feature_starts counts them) to end, by config.factor_data_strategy; the panel goes through predict_panel and the result is cut to start..``end``. No config is changed. The model must be trained or loaded.

Parameters:

start, endstr

First and last bar to predict, inclusive.

Returns:

xr.Dataset

One variable per label name on (timestamp, symbol).

Examples

>>> out = model.predict_window("2024-02-12", "2024-03-11")
>>> list(out.data_vars), out.sizes["timestamp"]
(['fwd_ret_1'], 21)

provenance()

What the top unit records about its data and its code.

data_fingerprint is training_record; code the code record of this model’s tree (quantlab.runs.record.code_of), taken when the unit is written.

Examples

>>> sorted(model.collect().provenance())
['code', 'data_fingerprint']

property purge_bars

L, the largest lookahead_bars() among the model’s labels.

Each segment followed by another loses its last L bars, so no fitted label reads a bar of the next segment.

Examples

>>> model.purge_bars  # one forward-return label over 1 bar, delay 1
2

reserved_hyperparameters = frozenset({})

The hyperparameters keys this variant reads itself, which head_hyperparameters leaves out.

property symbols

Symbols of the collected panel, as a plain list.

Examples

>>> model.symbols
['S0', 'S1', 'S2']

property test_bounds

The configured test window, (test_start, test_end).

Examples

>>> model.test_bounds
('2024-02-05', '2024-02-09')

to_array(data, variables, dtype=None)

Convert a panel to a [num_times, num_symbols, len(variables)] array.

Both axes are sorted, and the last axis follows the order of variables exactly (not alphabetical order), so x[..., i] is always variables[i].

Parameters:

dataxr.Dataset

A dataset indexed by (timestamp, symbol).

variableslist[str]

The data variables to stack, in the wanted order.

dtypenumpy dtype, optional

The array’s type; by default the variables’ common type.

Examples

>>> panel = model.collected_panel.data
>>> x = model.to_array(panel, ["f_b", "f_a"])
>>> x.shape
(40, 3, 2)
>>> np.allclose(x[..., 0], panel["f_b"].values)
True
>>> model.to_array(panel, ["f_b", "f_a"], dtype=np.float32).dtype
dtype('float32')

property tracker

The tracker every run of the model is opened through, config.tracker.

Examples

>>> type(model.tracker).__name__
'NullTracker'

property tracking_project

The default project of the model’s tracking runs: its class name.

A tracker’s own project replaces it.

Examples

>>> model.tracking_project
'MyHead'

train()

Train once on the config’s train_* / test_* dates and save.

A new trial directory is created under model_save_dir and is the trained unit: train_into trains into it under a tracking run {class}_total, grouped by the directory’s name, and writes the checkpoint, config.json, the evaluation files and run.json. Returning the path lets a caller record exactly which model was trained and reload it later instead of retraining; TrainedRun.open reads the rest of the run from it.

Returns:

Path

Absolute path of the checkpoint file, named {class}_total{checkpoint_suffix}.

Examples

>>> checkpoint = model.train()
>>> run = TrainedRun.open(checkpoint)
>>> checkpoint.name, run.path.name.startswith("MyHead_trial_")
('MyHead_total.joblib', True)
>>> sorted(run.metrics)[:4]
['test_ic', 'test_icir', 'test_loss', 'test_mae']
>>> sorted(p.name for p in run.path.iterdir())
['MyHead_total.joblib', 'config.json', 'ic_series.csv', 'run.json', 'test_predictions.zarr']

property train_bounds

The configured training window, (train_start, train_end), before the purge.

Examples

>>> model.train_bounds
('2024-01-01', '2024-02-02')

train_cv(train_periods, expanding=False, test_periods=None)

Run a walk-forward cross-validation and return the walk-forward unit.

Walk-forward cross-validation trains on a window of past data and tests on the period right after it, then moves the test period forward, so a test period never precedes its training data. The training window either slides with it at a fixed length (the default) or, with expanding=True, keeps the first fold’s start and grows to all history before the test period. Both modes test on the same periods, so their results compare bar for bar. Folds are laid out by quantlab.model.split.walk_forward_folds over the timestamps between config.start_date and config.end_date. Each fold’s training window loses its last L bars, L being the largest lookahead_bars() among the labels, so no fitted label reads a test-period bar.

A new trial directory {class}_trial_{timestamp}/ is created under model_save_dir and becomes a "walk_forward" unit (quantlab.runs.trained_run). Every fold trains on its own dates into its own "model" unit fold_{i}/, with the checkpoint {class}_cv_fold_{i}{suffix} and a tracking run of that name. The fold means of every train_* / val_* / test_* metric, keyed cv_mean_{key}, plus cv_n_folds are written to the summary of a separate {class}_cv_summary run. Last, the trial’s run.json records the folds and those means. A backtester’s run_cv replays the unit. The procedure is quantlab.model.walk_forward_training.train_walk_forward, the one an ensemble’s train_cv runs too. Afterwards the model keeps the dates it was configured with, not the last fold’s.

Parameters:

train_periodsint

Number of timestamps in the first fold’s training segment, and in every fold’s when sliding.

expandingbool, default False

Train every fold from the first fold’s start instead of sliding a fixed-length window. Test segments, fold count and the purge are those of the sliding mode; the validation segment stays the last val_size share of each growing window. The mode is not recorded: the fold windows carry it.

test_periodsint, optional

Number of timestamps in each fold’s test segment, and the step from one fold to the next. Defaults to train_periods // 5. Like the mode, it is carried by the fold windows.

Returns:

TrainedRun

The "walk_forward" unit: folds holds each fold’s unit (index, configured and fitted training window, test window, metrics, checkpoint) and cv_mean the fold means (empty when no fold has metrics).

Raises:

ValueError

If test_periods is below 1, or is not given and train_periods is below 5 (the test segment would be empty), no timestamps fall inside the config’s date range, or the purge leaves a fold no training bar, or a reserved hyperparameter is invalid (see check_hyperparameters).

Examples

>>> cv = model.train_cv(train_periods=20)
>>> cv.kind, len(cv.folds)
('walk_forward', 4)
>>> cv.folds[0].index, cv.folds[0].checkpoint.name
(0, 'MyHead_cv_fold_0.joblib')

An expanding run tests on the same bars, and every fold trains from the first fold’s start:

>>> expanding = model.train_cv(train_periods=20, expanding=True)
>>> len({fold.train_window[0] for fold in expanding.folds})
1
>>> [f.test_window for f in expanding.folds] == [f.test_window for f in cv.folds]
True

test_periods sets the test length instead of one fifth:

>>> len(model.train_cv(train_periods=20, test_periods=10).folds)
2

train_fold(fold, run_dir, group)

Train one walk-forward fold into run_dir, then restore the model’s own dates.

The config gets the fold’s training window before the purge and its test window (fold_config); train_into purges, trains and writes the "model" unit with the checkpoint and tracking run {class}_cv_fold_{i}. The config the model had before is put back afterwards, also when training raises.

Parameters:

foldFold

The fold to train.

run_dirPath or str

The fold’s unit directory, fold_{i}/ of the trial.

groupstr

Tracking group, the trial directory’s name.

Examples

>>> before = model.config
>>> model.collect().train_fold(fold, tmp / "fold_0", group="trial")
>>> model.config == before, TrainedRun.open(tmp / "fold_0").kind
(True, 'model')

train_into(run_dir, group, experiment_name, provenance=None)

Train, evaluate and save once into a caller-given run directory.

The random generators are reseeded from config.random_seed first, so models trained one after another in one process do not share random state. A tracking run named experiment_name is opened in the group group (see _tracking_run), and the variant’s _fit trains and writes the checkpoint {experiment_name}{checkpoint_suffix} and its config.json into run_dir, returning its {split}_loss. The trained model is then scored by _evaluate, which writes ic_series.csv and test_predictions.zarr beside the checkpoint; the losses and the evaluation metrics, merged, go once to the tracking run’s summary and, last, to run.json, which makes run_dir a trained unit (quantlab.runs.trained_run); the fitted window becomes fitted_train_bounds. No trial directory is created: train and every train_cv fold pass the directory they lay out themselves.

Parameters:

run_dirPath or str

Directory that receives the checkpoint and the evaluation files. It must not exist yet; it is created with its parents.

groupstr

Tracking group of the run, the trial directory’s name.

experiment_namestr

Tracking run name, also the checkpoint file’s stem.

provenancedict, optional

provenance(), for the top unit (train); a fold or a member records none.

Returns:

tuple[Path, dict or None]

The absolute checkpoint path and the metrics: the losses _fit returned and the _evaluate metrics, or None when _fit returned None.

Raises:

RuntimeError

If run_dir already exists (see _save_model).

Examples

>>> checkpoint, metrics = model.collect().train_into(
...     tmp / "member_0", group="trial", experiment_name="MyHead_member_0"
... )
>>> checkpoint.name, sorted(metrics)[:1]
('MyHead_member_0.joblib', ['test_ic'])

property training_record

What the last collect() read, by component path; empty before it.

train() and train_cv() write it as it is: a config changed after collect() is not re-read, so collect again after changing what is read.

Examples

>>> sorted(model.collect().training_record)
['factors.0.dataset', 'labels.0.factor.dataset']

use_panel(panel)

Take panel, collected by a model reading the same data, as this model’s collected panel.

The models then share the panel and every array built from it (each fit converts it once in all), and this model’s training_record is the panel’s. The panel must have been read for the same factor and label configs over the same start_date and end_date, with the same warm-up and read strategies; other hyperparameters and the seed may differ.

Parameters:

panelCollectedPanel

Another model’s collected_panel.

Returns:

Self

The model itself, for chaining.

Raises:

ValueError

If panel was collected for other factors, labels or dates.

Examples

>>> deep = MyHead(dataclasses.replace(config, hyperparameters={"max_depth": 6}))
>>> deep.use_panel(model.collect().collected_panel).collected_panel is model.collected_panel
True

walk_forward_bars()

The collected bars between config.start_date and config.end_date.

train_cv lays its folds out over them. Call collect() first.

Examples

>>> len(model.collect().walk_forward_bars())
30

property warmup_bars

Bars of features the model needs before the first bar it predicts.

0 here: a row model predicts each bar from that bar alone. TorchModel returns window_bars - 1.

Examples

>>> model.warmup_bars
0