factor_agent.acceptance.Acceptance.open_holdout#
- Acceptance.open_holdout(selection, force=False)[source]#
Open the hold-out, once, and compare on it with a selection, trained on the training and validation segments.
- Parameters:
- selectionlist of str
The accepted factors to add, named as in
validate;validatemust have written the report of exactly this selection first.- forcebool, default False
Open a hold-out that was opened before.
- The open is recorded in ``holdout.lock`` before any hold-out data is
- read, so an open that fails midway still counts (the hold-out may
- have been seen) and running it again needs ``force``; the lock counts
- opens of any selection. Writes ``acceptance/holdout.json|md``, or
- ``acceptance/holdout_<digest>.json|md`` for a selection of fewer than
- every accepted factor.
- Raises:
- HoldoutLocked
If the hold-out was opened before and
forceis false.- KeyError
If a name names no accepted factor, or a description several.
- ValueError
If there is no validation or hold-out segment, no accepted factor, the selection is empty or names a factor twice, or there is no validation report of the selection.