factor_agent.acceptance#
Acceptance: the accepted factors’ incremental test in the acceptance model, the one-time hold-out, and their export.
validate trains the acceptance model (AcceptanceConfig.model, a
template that factor_agent._acceptance_model fills in) twice per seed: on
its baseline (its own factors, or the reference set) and on that plus a
selection of the factors the campaign accepted (every one by default, else
named as export names them), all read through the campaign’s
Scoring (on the members of day t, or on every symbol without a
membership). It trains on the training segment and predicts the validation
segment, both purged by Segments. Each accepted factor enters the model
as the Evaluator scored it: its graph ranked across the whole market and
neutralised when the scoring neutralises (a store written under the run
directory), read as the scoring reads. A factor of an intraday family is
computed in its stages, its session columns from the intraday dataset
(GraphFactor.staged), as the Evaluator computes it.
With acceptance.contribution, validate also measures each factor of
the selection’s contribution, with the same seeds: drop-one trains the model
on the baseline plus every factor of the selection but it (the whole
selection against that), add-one on the baseline plus only it (against the
baseline). A feature set equal to another (a one-factor selection’s drop-one
is the baseline) is trained once.
open_holdout makes the same comparison on the hold-out for a selection
that validate reported on, trained on the training and validation
segments. It first records the open in the lock
(Segments.open_holdout): a second open is refused unless forced, and a
forced one is stated in its report, and it never measures contribution. Reports of every accepted factor are
acceptance/<segment>.json|md, of a selection of fewer
acceptance/<segment>_<digest>.json|md.
Every trained model’s result is cached (factor_agent._model_cache), so a
re-run trains only the models not finished; each run first prints how many
it will train and how many are cached.
The comparison, on the members of each day of the predicted segment (every symbol without a membership) and against the label the model trains on, per model and averaged over seeds:
rank_ic,ic: the mean daily rank and Pearson IC of the prediction;decile_returns,monotonicity,long_short: the mean label of each tenth of the daily cross section (lowest predictions first), their Spearman correlation with the decile number and the top decile’s minus the bottom one’s;top_k_return,top_k_excess: the mean label of the plain TopK book (thetop_kmembers with the highest prediction, equal weights, no smoothing) and its excess over all members, fee-free, over the label’s horizon;top_k_annualisedcompoundstop_k_returnto a year of 252 bars. These read the label as a raw forward return.
export writes each named accepted factor as a module of
AcceptanceConfig.package holding a GraphFactor subclass, and its
whole-market store, the ranked graph scored as above (neutralised or not), under
factor_root/<name>/ with README.md and component.json. An
intraday factor’s class holds its daily graph, the intraday graph of each
session column it reads and the intraday dataset’s config, so it recomputes
from the intraday bars wherever the module is imported.
Classes
Validate, open the hold-out of, and export a campaign's accepted factors; see the module docstring. |
|
The baseline against the baseline plus the accepted factors, seed by seed and on average. |
|
One accepted factor's contribution in one mode: the models with it against the same models without it. |
|
One model's statistics on the predicted segment; see the module docstring. |