factor_agent.campaign#
The Campaign: the research loop, from one config to accepted and rejected candidates.
One round:
the quota: every mechanism type gets
min_per_typeslots and the rest go by Thompson sampling on each type’s admission rate (_scheduler);the planner plans one direction per slot: a new mechanism, an orthogonal mutation of an accepted factor, or a refinement of one;
the proposer writes one candidate per direction; the Gate checks each against the reference set, the known factors and the accepted factors (the
Zoo) and the refused ones go back to the proposer to be fixed;the reviewer, in its own call, judges each gated candidate’s hypothesis, description and code for consistency, with the data section’s fields; the inconsistent ones go back to the proposer once, all in one fix request with the reviewer’s reasons; each rewrite passes the Gate once (a refusal fails it) and the reviewer judges the rewrites in one more call: one still inconsistent is rejected without being evaluated; with
round.groupsabove 1, steps 3 and 4 make these calls per proposal group, the groups’ calls of each stage side by side (Researcher);the Evaluator scores the rest on the training segment in one compile, admission decides (
_admission) in proposal order, and an accepted candidate joins the zoo and the reference pool before the next is admitted; a refusal for coverage says at which step of the graph the values are lost (Evaluator.trace_coverage);the round’s records go to the ledger and the state to
state.json.
Before anything else run checks the replaced prompt templates for
their placeholders and builds the Vocabulary, checking every field’s
column against the price dataset; every problem is reported at once,
before any LLM call and before anything is written, so a run refused for
its files starts cleanly once they are fixed. It then opens the run’s
RunSnapshot: a first start copies the config’s field, operator and
template files into the run directory, a resume fails if they changed
since and warns if the commits did; every commit is listed in
state.json. The run searches with the snapshot’s copies. check
makes these checks and the others a start makes before its first LLM call
(the data and the stores it reads, the roles’ backends, the reference and
known factors, the snapshot’s differences) part by part, writing
nothing, and reports every problem at once in a preflight.CheckReport. A replay (roles on the replay backend) searches with the
snapshot of the run whose LLM log it replays, whatever the config’s files
say now.
The prompts (factor_agent.prompts.Prompts) describe the config’s market, the
label’s horizon (its span_bars()), the scoring, the vocabulary and the
config’s mechanism types, through the snapshot’s templates.
The planner and the proposer read the context of _feedback, a pure
function of the ledger and the config: the seed directions, the reference
factor families and the known ones (marked as known), the memory, the last
round’s feedback and the current best mechanisms; the planner also reads the recent history (the trials of the
last context.recent_rounds rounds, one line each) and the earlier
summary (the older trials counted by mechanism type and outcome). The
snapshot records recent_rounds at the first start and a resume may not
change it. Every memory_every rounds the distiller folds the trials
since the last memory into it; an empty or unparsable reply keeps the
previous memory. memory.md holds the current memory, which every later
prompt carries, and memory/round_NN.md every version, by the round it
was written before. Every trial is named by its id (_feedback.trial_id);
a direction may build on any evaluated trial by its id, and the proposer
is given that trial (_feedback.parent). With context.feedback off
the planner and the proposer read only the seed directions and the factor
families, every direction is new and no memory is distilled. With
context.theme_correlation every evaluated trial joins the Evaluator’s
tried trials and the planner also reads them grouped by what their factors
measure (_feedback.themes); a resume computes the tried trials again
from the ledger. With context.lineage each round computes every trial’s
Lineage (_lineage) once, and the planner’s accepted factors and the
proposer’s parent block carry it; round.max_children takes every trial
built on that often off the trials a direction may build on. With
round.picked_parents (which needs the tried trials) each round, after
the quota, builds the Trial graph (_trial_graph) and gives each trial in
its pick order a slot of its mechanism type while that type has one unclaimed
(_claim); the planner writes those slots as refinements or mutations of
their parents, and the plan event records them. The round’s graph is kept
in trial_graph/round_NN.json (_trial_graph.write) after the plan,
and written anew when a resume plans the round again. With
context.send_back_repeats too, before a round is scored, a new
direction’s candidate that correlates that much with an earlier trial is
rewritten once (_renew); a slot is rewritten once in
all, so a candidate the reviewer sent back is not checked again.
Before each round the stop conditions are checked: the round limit, the
trial limit and patience rounds in a row without an acceptance; the
first that holds ends the run and is recorded. The library counts no
money: spending limits belong to the provider. When the provider refuses a
call because the balance or quota is exhausted (ProviderQuotaExhausted),
the round in progress is dropped, the stop is recorded as
provider_quota with the provider’s message, and the run returns; a
resume, once topped up, runs that round again. Every call goes
to the run’s LLM log (llm.log), which the replay backend serves
back to reproduce the run without a provider; on a resume the calls of the
dropped round are moved to llm_log/dropped/, so the log holds exactly
the calls the ledger was written from. run resumes
a run directory from its state: the rounds written are kept, a round a
crash left half written is dropped and run again, and the zoo, the pool and
the graphs the Gate compares duplicates with are restored from the ledger.
The reference set and the known factors are read from the config on every
start, a resume included; neither is snapshotted.
As it goes, the campaign appends what it does to the run’s events file
(factor_agent.events): every start and resume, plan, LLM call, Gate
check, verdict, evaluation, admission, round end and stop. It never reads
the file, so a resume, a replay and the snapshot do not depend on it.
Classes
Run a mining campaign from its config. |
|
Every trial record of the run, why it stopped, and where the ledger is. |
|
Why the campaign stopped: |