quantlab.model.predefined.double_ensemble.DoubleEnsembleRegressor#

class quantlab.model.predefined.double_ensemble.DoubleEnsembleRegressor(config)[source]#

Bases: QlibLibraryHead

Qlib’s DoubleEnsemble of LightGBM sub-models on the rank target’s rows.

num_models LightGBM sub-models are trained one after the other, each for epochs rounds. After each one but the last, the current ensemble’s squared error on every training row feeds the two modules of the paper:

sample re-weighting (enable_sr): each row’s weight for the next sub-model comes from how hard the ensemble finds it and how much the last sub-model’s loss on it fell over its trees (sample_weights);
feature selection (enable_fs): each feature’s column is permuted once across the training rows, and the mean over the rows of the ensemble’s added squared error, over its standard deviation, is its g-value; the next sub-model trains on a draw that keeps more of the features with higher g-values (select_features).

The prediction is the sub_weights-weighted mean of the sub-models’. The training target is Qlib’s CSRankNorm (training_target "cs_rank"). Early stopping is off, as in Qlib’s benchmark; early_stopping=True stops each sub-model after early_stopping_patience rounds without a better validation loss. Hyperparameters, with Qlib’s Alpha158 benchmark values as defaults: num_models (3), enable_sr / enable_fs (True), alpha1 / alpha2 (1.0), bins_sr (10), bins_fs (5), decay (0.5), sample_ratios ([0.8, 0.7, 0.6, 0.5, 0.4]), sub_weights ([1, 1, 1]), epochs (28, the rounds of each sub-model); every other key goes to LightGBM over SUBMODEL_ALPHA158.

Known differences from Qlib’s implementation: the permutations and draws use a generator seeded from random_seed rather than numpy’s global one, and a sub-model that does not use a permuted feature is not asked to predict again (its prediction cannot change).

Parameters:

configModelConfig: Factors, one label, dates and hyperparameters.

Raises:

ValueError: At training, if the model has more than one label, or sample_ratios does not have bins_fs entries or sub_weights does not have num_models.

Examples

>>> head = DoubleEnsembleRegressor(ModelConfig(
...     factors=[factor], labels=[label], model_save_dir="checkpoints",
...     factor_data_strategy="read", label_data_strategy="read",
...     train_start="2024-01-01", train_end="2024-05-31",
...     test_start="2024-06-01", test_end="2024-07-18",
...     hyperparameters={"num_threads": 4},
... )).collect()
>>> head.train().name
'DoubleEnsembleRegressor_total.joblib'

DEFAULTS = {'alpha1': 1.0, 'alpha2': 1.0, 'bins_fs': 5, 'bins_sr': 10, 'colsample_bytree': 0.8879, 'decay': 0.5, 'enable_fs': True, 'enable_sr': True, 'epochs': 28, 'lambda_l1': 205.6999, 'lambda_l2': 580.9768, 'learning_rate': 0.2, 'max_depth': 8, 'num_leaves': 210, 'num_models': 3, 'num_threads': 20, 'objective': 'mse', 'sample_ratios': [0.8, 0.7, 0.6, 0.5, 0.4], 'sub_weights': [1, 1, 1], 'subsample': 0.8789, 'verbosity': -1}#: The library’s parameters, Qlib’s benchmark values.

LIBRARY_DEFAULTS = {'early_stopping': False, 'early_stopping_patience': 50, 'training_target': 'cs_rank'}#: Defaults of the reserved keys this variant reads itself.