The campaign config#

One YAML file holds every setting of a campaign. CampaignConfig.from_file(path) reads it, resolving the field, operator and template paths relative to the file, and refuses unknown sections, malformed entries and credentials before any data loads. An unknown key is refused rather than ignored: an option an LLM role’s backend does not take (timeout for timeout_seconds, say) is refused naming the role and listing the options that backend takes, and a segment entry takes only start and end. Campaign(config).run() runs it.

Sections#

Section	Required	Holds
`run_dir`	yes	the directory the run writes to
`market`	yes	the data as the prompts describe it, one line
`segments`	yes	`training`, `confirmation`, `validation`, `holdout` (`{start, end}` each, in that order) and `purge_bars`; `confirmation` is optional, see admission
`data`	yes	`prices`, `label` and optionally `membership` and `intraday`
`scoring`	yes	`neutralize`: the exposures to neutralise against, or `null`
`fields`, `operators`	yes	the vocabulary files; `operator_modules` optionally
`reference`	no	the reference set; empty, candidates are only compared with each other
`known`	no	the known factors
`directions`	no	seed research directions for the planner, one string each
`mechanisms`, `prompts`	no	mechanism types and replaced templates; see prompts
`evaluator`, `gate`, `admission`, `round`, `context`, `trial_graph`	no	settings with defaults (below)
`llm`	yes	one entry per role; see LLM backends
`acceptance`	yes	the acceptance model and seeds; see acceptance

Data#

data, reference and known hold quantlab components. An entry is one of:

a component’s get_config() dict, written inline;
the path of a store folder, whose component.json describes the factor that owns it;
{store: <folder>, factor_names: [...]}: that store’s factor, pinned to some of its outputs.

data:
  prices: {...}                                   # the whole-market price dataset
  label: /path/to/quantlab/labels/market/ret_5       # a forward label store
  membership: {...}                               # the members of each day, or omitted
  intraday: {...}                                 # the whole-market intraday bars, for an intraday family
scoring:
  neutralize:
    exposures: {...}
    size_column: marketcap                        # its logarithm is regressed on
    industry_column: industry                     # an integer industry code
reference:
  - /path/to/quantlab/factors/market/alpha158_neutral
  - {store: /path/to/quantlab/factors/market/barra_style, factor_names: [size, beta]}
known:
  - /path/to/quantlab/factors/market/alpha101_neutral

Factors are computed on the whole market, then masked to the day’s members. A candidate is ranked across the whole market, neutralised when scoring.neutralize is given, and masked; a reference or known entry given as a store is read as stored and masked, one given as a component without a store is scored as a candidate is. scoring.neutralize must be written, as null to score ranked values, so no campaign is neutralised by default.

An output of a known factor that the reference set already holds is taken from the reference set only, so whole classes can be listed in known.

intraday is the dataset an intraday family’s fields read (one-minute trade bars, with configs/fields/massive_minute_bars.yaml as their field file, in configs/us3000_intraday_ret5.yaml): a campaign that opens an intraday family must name it, and the run keeps it with its snapshot, so a resume with another intraday dataset is refused as a resume with other field files is. A candidate’s session values are matched to the price dataset by date and by symbol label, so permatickers of the intraday bars find the same labels of the daily bars.

Every stored reference or baseline factor is read from its store’s first bar, so a store that starts after the training segment (an intraday store from 2016, say) is empty there.

The segments must lie inside the data of every family the field files open. The Evaluator reads the fields over the training segment, and acceptance over the validation and hold-out segments; a field whose data starts later or ends earlier is missing there, and so is every candidate that reads it. A campaign that opens a high-frequency family starts its training segment no earlier than the family’s statistics: configs/us3000_hf_ret5.yaml trains from 2016-10-11, the first session of its minute data, and the intraday example configs/us3000_intraday_ret5.yaml from 2016-10-10, the first session of its minute bars. A day before an intraday family’s bars begin has no session values and counts against a candidate’s coverage. The warm-up bars read before the training segment may lie outside the data; a long-window candidate is then missing on the segment’s first bars, and its coverage is measured over the dates that have values.

The Evaluator keeps the known factors only on one scored training date in five, as normal scores in a float32 panel of its own: 4 bytes per known output per sampled member-day. For us3000 (training 2000 to 2014, about 750 sampled dates of about 3,000 members) that is about 9 MB per known output, about 2.7 GB for 300 outputs. It is built once per start, one known factor at a time.

Settings with defaults#

Key	Default	Meaning
`evaluator.warmup_bars`	260	bars read before the segment so every window is filled
`evaluator.njobs`	51	KunQuant executor threads
`gate.timeout_seconds`	10	seconds the sandboxed graph build may take, counted from when the sandbox is ready
`gate.memory_mb`	1024	memory the sandboxed build may use
`gate.max_nodes`	200	nodes a candidate’s graph may have
`gate.max_free_parameter_ratio`	0.5	hand-set constants (other than 0 and ±1) per operator
`gate.max_base_fields`	5	distinct input fields a candidate may read
`gate.max_common_subtree_nodes`	5	the largest subtree a candidate may share with a zoo factor
`admission.t_threshold`	3.0	partial-IC t the first evaluated trial needs
`admission.trial_exponent`	1.0	power of the trial count the threshold’s correction divides by; below 1 it rises more slowly
`admission.max_correlation`	0.7	largest absolute correlation with a reference, accepted or known factor
`admission.min_monotonicity`	0.6	smallest Spearman correlation of the deciles with the decile number
`admission.min_coverage`	0.5	smallest mean daily share of the members with a value
`round.candidates`	20	candidates per round
`round.min_per_type`	3	slots of every mechanism type per round
`round.retries`	2	times a refused candidate goes back to the proposer
`round.groups`	1	proposal groups per round (the repository’s configs use 4)
`round.max_rounds`	30	rounds at most
`round.max_trials`	600	trials at most; a round that would pass it is not started
`round.patience`	5	rounds in a row without an acceptance before the run stops
`round.memory_every`	5	rounds between two distillations
`round.seed`	0	seed of the Thompson sampling
`round.max_children`	null	trials built on one trial at which no direction may build on it any more (see below); needs `context.feedback`
`round.picked_parents`	0	slots a round whose parent the code chooses, the Picked parents (see below); needs `context.theme_correlation`
`context.recent_rounds`	5	rounds whose trials the planner reads one line each
`context.feedback`	true	whether the planner and the proposer read the earlier trials and a memory is distilled; off, every direction is new (a control run)
`context.theme_correlation`	null	the absolute correlation at which an evaluated trial is grouped with an earlier one; set, the planner reads the groups (see below)
`context.send_back_repeats`	false	whether a new direction’s candidate in a group is rewritten once before it is evaluated; needs `context.theme_correlation`
`context.lineage`	false	whether the planner’s accepted factors and the proposer’s parent carry each trial’s Lineage (see below); needs `context.feedback`
`context.fertile_children`	3	best children’s outcomes a trial’s fertility is the mean of (see below)
`trial_graph.gamma`	0.75	share of each utility the Picked parents take from the related trials, in [0, 1) (see below)
`trial_graph.iterations`	100	propagation steps at most
`trial_graph.tolerance`	1e-8	the largest move of a utility at which propagation stops
`trial_graph.weights`	1/3 each	each relation’s share, `generation`, `structure` and `correlation`, summing to 1
`trial_graph.min_jaccard`	0.20	smallest share of subtrees two trials have in common that joins them by structure
`trial_graph.structure_neighbours`	30	structure neighbours each trial keeps, the closest
`trial_graph.min_correlation`	0.10	smallest absolute correlation that joins two trials by correlation
`trial_graph.correlation_neighbours`	30	correlation neighbours each trial keeps, the closest
`trial_graph.nearest_accepted`	10	accepted factors a trial’s novelty is measured against, the nearest
`trial_graph.max_expansions`	5	trials built on one trial at which it is no longer picked

Checking a config#

Campaign(config).check() finds what run() would refuse before its first LLM call, without starting the run: it writes nothing (a run directory that does not exist yet is not made, one that does is left byte for byte as it was) and computes no factor, so it takes seconds where a start would first score the reference set. It checks each part on its own and reports every problem at once:

each data component loads and the stores it reads exist, and the price dataset’s columns can be read;
the scoring’s exposures load;
each role’s backend can be built: an unset API key variable is a problem;
the reference and known factors load;
on a resume, the field, operator and template files and the recorded settings are the run snapshot’s (a difference is a problem, as it would refuse the resume), and the code versions are the run’s last (a difference is a warning, as a resume only warns of it);
the vocabulary builds against the price dataset’s columns, and the templates with it.

The vocabulary needs the price dataset (and the intraday one, when the config names it), and the templates need the vocabulary, the label and the scoring; when one of those fails, the check that needs it is reported as skipped, with the reason, so a clean report after a fix is not mistaken for a complete one.

From the shell, factor-agent check prints the report and exits with status 1 when it holds a problem, so a script or CI can gate a launch on it; a config that cannot be read (bad YAML, an unknown section or key) is printed as its first problem instead of a traceback:

factor-agent check campaign.yaml           # 0: the run can start, 1: a problem
factor-agent check campaign.yaml --ping    # also send each backend one short request

ok       data.prices
ok       data.label
...
problem  llm.reviewer
         - the environment variable DEEPSEEK_API_KEY holding the API key is not set
ok       run snapshot
ok       vocabulary
ok       templates
1 problem(s), 0 warning(s), 0 check(s) skipped; the run would be refused

In Python, check() returns the report:

from factor_agent.campaign import Campaign
from factor_agent.config import CampaignConfig

report = Campaign(CampaignConfig.from_file("campaign.yaml")).check()
print(report.text())    # one line per check: ok, problem, warning or skipped
if not report.ok:       # report.problems lists them as "<check>: <problem>"
    raise SystemExit(1)

Without a ping the check sends no request to a provider, so a wrong model name or base URL is found only at the first call. With --ping (check(ping=True)) each distinct backend (roles with the same settings share one) gets one short request after the other checks, outside the LLM log; its check, named after its roles (ping planner, proposer), gives how long the answer took, or the provider’s error as a problem. Roles on the replay or scripted backend need no provider and are not pinged, nor is a role whose backend could not be built; each is listed as skipped with the reason. A ping costs one short call per backend.

Stopping and resuming#

Before each round the run checks its stop conditions: max_rounds, max_trials and patience. The first that holds ends the run, and state.json records which one. When a provider refuses a call because the balance or quota is exhausted, the run drops the round in progress, records the stop provider_quota with the provider’s message and returns normally; the library counts no money, so spending limits belong to the provider.

run() on a run directory that has a state resumes it: the rounds written are kept, a round a crash left half written is dropped and run again, and the zoo, the pool and the graphs the Gate compares with are rebuilt from the ledger. The calls of a dropped round move to llm_log/dropped/, so the log holds exactly the calls the ledger was written from. The accepted factors’ values, and with themes the evaluated trials’, are read back from columns/ rather than computed again; a resume only checks that their graphs still compile. The stored values are kept across a change of code, so a resume after a fix judges its new trials against the values the earlier ones were judged against; a change of the data, the scoring, the warm-up or a segment empties a segment’s store, and its factors are computed once more. Deleting columns/ costs one such computation.

The first start copies the field, operator and replaced template files into snapshot/ with their hashes, and the run searches with those copies. A resume whose files differ fails: a new search space needs a new run directory. A resume on other commits of factor_agent or quantlab only warns, and state.json lists the commits of every start and resume. round.groups may change on a resume; context.recent_rounds, context.feedback, context.theme_correlation, context.send_back_repeats, context.lineage, context.fertile_children, round.max_children, round.picked_parents and, with Picked parents, the trial_graph values may not. The reference set and the known factors are read from the config on every start. Every start and resume also keeps the config it was given as config/open-<k>.yaml (the first start is open-1.yaml), so a run says which settings it ran with even after the config file has changed.

The run directory#

Path	Holds
`ledger.jsonl`	one line per candidate: round, candidate, status, stage, reasons, the Gate’s statistics, the graph, the metrics, refused attempts
`state.json`	rounds written, the stop, the memory’s round, the code versions of every start
`llm_log/`	every LLM call in full: prompt, parameters, reply, token usage, the raw reply
`events.jsonl`	what the campaign did, one event per line as it happened (see The events file)
`memory.md`, `memory/round_NN.md`	the distilled memory and every version of it
`snapshot/`, `snapshot.json`	the search files of the first start and their hashes
`config/open-<k>.yaml`	the config of the k-th start or resume, as it was given
`columns/<segment>/`	the scored values of every factor in a segment’s pool and, with themes, of every evaluated trial, one file per graph, for a resume to read instead of computing them; about 45 MB per accepted factor and 2.3 MB per evaluated trial on 3000 members over 15 years
`ic/round_NN.json`	each evaluated trial’s daily rank IC and partial IC in round NN, by slot, for the frontend’s cumulative IC
`trial_graph/round_NN.json`	with `round.picked_parents`, the Trial graph round NN picked its parents on: every evaluated trial’s own and propagated utilities and whether it could be picked (and why not), and the edges of each relation with their weights
`zoo.json`	the reference and known factors the Gate covers, and those without a graph it cannot
`acceptance/`	validation and hold-out reports and the model cache
`holdout.lock`	written when the hold-out is opened

A trial’s status is accepted, rejected (by the reviewer, or by admission after evaluation) or failed (refused by the parser or the Gate after every retry, or its graph did not compile). Its stage is where it ended: parse, gate, review, evaluate or admission.

The events file#

The ledger and the state are written at a round’s end. To show the round in progress, the campaign also appends to events.jsonl as it goes: one JSON object per line, never rewritten. The campaign itself never reads the file, so it takes no part in a resume, a replay or the run’s snapshot; deleting it loses the record of the run’s progress and changes no result. A resume that drops a round a crash left half written does not remove that round’s events: it writes a resume event naming the dropped round, and the round’s events from the new start follow it.

Every line has schema (the format’s version, now 1), type, time (UTC, ISO 8601) and the event’s fields. A slot is a candidate’s place in its round, from 0, in the order of the round’s research directions and of the round’s lines in the ledger. A value the campaign holds as NaN is written as null.

`type`	When	Fields
`start`, `resume`	the campaign opens the run directory, before its first call	`pid`, `host`, `versions` (the code versions), `rounds` (written so far), `candidates`, `max_rounds`, `max_trials`, `patience`, `t_threshold`, `trial_exponent`, `max_correlation`, `min_coverage`, `min_monotonicity`; a `resume` also has `dropped`, the round it drops or `null`
`plan`	the planner’s directions are set	`round`, `directions` (in slot order: `mechanism_type`, `action`, `parent`, `direction`), `quota`, `issues`
`call_start`	an LLM call is sent	`call` (a number unique since the latest `start` or `resume`), `round`, `role`, `model`, `group` (the proposal group, `null` for the planner and the distiller), `slots`
`call_end`	an LLM call comes back or fails	the fields of its `call_start`, and `file` (the call file in `llm_log/`, `null` when the call failed), `seconds`, `input_tokens`, `output_tokens`, `cache_hits`, `reasoning`, `truncated` (cut off at the output limit), `empty`, `error`
`gate`	the parser and the Gate check one version of a slot’s candidate	`round`, `slot`, `step` (`proposed`, `fixed` after a refusal, or `rewritten` after the reviewer), `item` (the version), `passed`, `stage` (`parse` or `gate` when refused), `reasons`, `hints`
`review`	the reviewer judges a slot’s candidate	`round`, `slot`, `consistent`, `reason`, `rewrite` (whether it judged a rewrite)
`evaluation`	the Evaluator starts on the round	`round`, `slots` (those the reviewer found consistent)
`admission`	a slot’s outcome after evaluation	`round`, `slot`, `status`, `stage` (`admission`, or `evaluate` when its graph did not compile), `reasons`, `threshold` (the partial-IC t in force), `metrics`
`round_end`	the round’s trials are written to the ledger and the state	`round`, `trials`, `accepted`
`stop`	the run stops	`by`, `reason`, as `state.json` records them

A token count is null when the provider’s reply does not report it: cache_hits are input tokens read from the provider’s cache and reasoning output tokens the model spent reasoning. A crash writes nothing: a run whose process has gone without a stop crashed. factor_agent.events.read reads the file back into the event classes.

Memory and context#

The planner and the proposer read the same context: the seed directions, the reference factor families and then the known ones (marked as known), the memory, the last round’s training statistics and rejection reasons, and the accepted factors with their weight and residual IC in the accepted pool. The planner also reads the trials of the last context.recent_rounds rounds one line each, and the older trials counted by mechanism type and outcome. Every round.memory_every rounds the distiller folds the trials since the last memory into memory.md; an empty or unparsable reply keeps the previous memory. It also reads every factor accepted so far, each with its mechanism type, hypothesis and direction, and the memory’s working patterns are about those factors only: a factor a refusal names is one of the reference set, a known one or one accepted earlier.

Every trial is named in the prompts by its id, T<round>-<slot> (T3-07 is round 3’s seventh). A research direction may build on any evaluated trial, accepted or refused: a refinement measures the parent’s mechanism another way, to remove what refused it or to strengthen an accepted factor; a mutation follows another channel the parent’s result points to. The direction names the parent by its id, and the proposer is given the parent’s code, statistics and reasons. A direction whose parent is not an evaluated trial (or is one round.max_children has capped, see below) is kept as a new one, and the plan event records the issue.

With context.theme_correlation set (0.5, say), the Evaluator keeps every evaluated trial, as the normal scores of its factor on every 20th scored date, and the trials are grouped by what their factors measure: each joins the group of the earlier trial it is most correlated with, when that correlation reaches the setting in absolute value, and opens a group of its own otherwise. The planner reads the groups, the largest first, each with the trial that opened it, its trials, how many were accepted, its largest residual t, the last round it was tried in and its refusals by kind. On a resume the evaluated trials are computed again from the ledger.

With context.send_back_repeats: true as well, before a round is scored, a new direction’s candidate correlated that much with an earlier trial is sent back to the proposer once, with that trial named and no statistic but the correlation; the version sent back is not evaluated and is not a trial. Its rewrite is checked by the Gate again, and fixed up to round.retries times as a proposed candidate is, then by the reviewer, and evaluated with the rest, whatever it measures. A refinement or a mutation is not checked, and a slot is rewritten once in all: a candidate the reviewer sent back is not checked again.

With context.lineage: true, each accepted factor in the planner’s context, and the parent trial the proposer is given for a refinement or a mutation, carries the trial’s Lineage, one line on the trials whose direction names it as their parent (its direct children):

lineage: 6 trials built on it (4 refinements, 2 mutations), 2 stopped before evaluation, 1 accepted (T2-03); best evaluated child T2-03, residual t in its direction +5.00 against this trial's +3.50

A child that stopped before evaluation (refused by the parser, the Gate or the reviewer, or not compiled) counts, so a parent whose children all failed does not look untried. The residual t is the partial IC’s Newey-West t in each trial’s own direction. The Lineage is computed from the ledger before each round; off, no prompt holds it.

With round.max_children: n, a trial that n trials have built on (its Lineage’s count, those that stopped before evaluation included) is no longer one a refinement or a mutation may name: a direction naming it is kept as a new one, and the plan event records the issue, as for a parent that is not an evaluated trial. The cap needs no context.lineage; the planner just sees no Lineage line without it. Unset, every evaluated trial stays a parent.

With round.picked_parents: k, the code chooses the parent of k slots each round, the Picked parents, after the quota is drawn. It judges every evaluated trial, accepted or not, by three utilities: its quality, the residual t in its direction; its fertility, the mean of its context.fertile_children (3) best children’s outcomes in its Lineage (how far each child’s residual t cleared the threshold it was judged against, 0 for a child that stopped before evaluation); and its novelty, its mean distance, in structure (the subtrees of its graph) and in correlation, from the trial_graph.nearest_accepted (10) nearest factors accepted so far. The trials form the Trial graph, joined by three relations: generation (parent and child), structure (the share of subtrees two trials have in common, from trial_graph.min_jaccard) and correlation (from trial_graph.min_correlation), each trial keeping its closest structure_neighbours and correlation_neighbours (30). Each trial then shares its utilities with the trials related to it, the relations weighted by trial_graph.weights, so a trial whose relatives did well is judged by that evidence too: its utility after propagation lies trial_graph.gamma (0.75) of the way from its own to its neighbours’ average. The trials are ranked by Pareto front on the three, not by a weighted sum, within a front by how far each lies from the others, and a trial built on trial_graph.max_expansions (5) times is not picked. In that order, each takes one slot of its own mechanism type while that type has a slot not yet taken, until k are taken; a trial round.max_children has capped is passed over.

The trial_graph section’s defaults are the values of the CosmosAlpha paper (arXiv:2611.04555, Tables 6 and 7), so a config without it runs the method as published:

trial_graph:
  gamma: 0.75
  iterations: 100
  tolerance: 1.0e-8
  weights: {generation: 0.3333333333, structure: 0.3333333333, correlation: 0.3333333334}
  min_jaccard: 0.20
  structure_neighbours: 30
  min_correlation: 0.10
  correlation_neighbours: 30
  nearest_accepted: 10
  max_expansions: 5

A value out of its range (gamma outside [0, 1), weights that are negative or do not sum to 1, a count below 1, a threshold outside [0, 1]) is refused when the config is read, and factor-agent check reports it. With Picked parents on, the run snapshot records every trial_graph value at the first start, so a resume that changes one is refused and a replay picks with the recorded run’s values whatever its own config says; without Picked parents the section is not recorded. A run with Picked parents started before the section existed ran with its defaults: it resumes while the config keeps them. context.fertile_children is recorded when it is not 3, and fixed alike.

The planner is told the picked slots in its prompt, each with its type and its parent, and writes each as a refinement or a mutation of that parent, choosing the action and the idea: the first direction of the type that names the parent fills the slot. A picked slot the planner leaves out is planned as a refinement of its parent, and the plan event records the issue. The plan event in events.jsonl lists the picked parents with their utilities and Pareto fronts (empty with round.picked_parents: 0). The correlations come from the evaluated trials that context.theme_correlation keeps, so picked parents need it. A replaced plan template must hold the placeholder {picked}, where the picked slots go, when picked parents are on; without it the run is refused at its start, and the check reports it.

Each round that picks parents writes the graph it picked them on to trial_graph/round_NN.json once the round is planned: γ, every evaluated trial with its own utilities, its utilities after propagation, the average it took from its neighbours and, when it could not be picked, why (expansion_cap, built on trial_graph.max_expansions times, or max_children), and the edges of each relation (generation, structure, correlation) with the weights propagation used. It holds no ranks; factor_agent._trial_graph.read(run_dir, round) reads it back, and its pick_order() gives the order the round’s parents were claimed in. A resume that plans a round again writes its file anew.

A candidate refused for its coverage is told where its graph loses its values: the first step whose coverage falls below admission.min_coverage while its inputs’ does not, with its inputs’ coverage and the share of the member days on which a step is infinite (a division by zero, say, on a day without a range), or the data column that has too few values.

With context.feedback: false the planner and the proposer read only the data, the seed directions and the factor families, every direction is new and no memory is distilled: a run that cannot learn from its trials, to compare a run that can with.