What it is

A training run needs perhaps a dozen decisions before it starts: how to split the data, what batch size fits the card, which learning rate, whether to weight the classes, when to stop. Most of them arrive as defaults nobody chose — values typed into a graph months ago, against a different cohort.

Tick Smart Training on the Training page and those decisions are made from the run in front of you instead: the cohort you actually selected, the architecture you actually wired, the GPU you actually have. It shows you all of them before anything runs, lets you overrule any of them, and puts the whole list — including what you overruled — into the run record and the exported Methods section.

                       SMART TRAINING
                           │
          ┌────────────────┼────────────────┐
          ▼                ▼                ▼
       DATASET           MODEL            DEVICE
       ANALYSIS          ANALYSIS         ANALYSIS
          │                │                │
          └────────────────┼────────────────┘
                           ▼
                    TRAINING PLANNER
          split · batch size · learning rate · optimiser
          regularisation · stopping rule · recovery
                           │
                           ▼
                     BASELINE + RUN
                           │
                  ┌────────┴────────┐
                  ▼                 ▼
             Diagnostics       Search / tuning
                  └────────┬────────┘
                           ▼
              VERDICT · REPAIR · NEXT EXPERIMENT

The claim it makes

It does not claim to make your model better. That cannot be supported at the moment you tick a checkbox, and a feature implying it would be believed on the runs where it was false.

The two claims it does make are both checkable from the run record:

1. Provenance. Every configuration decision this run required was made deliberately, derived from this cohort and this device, and written down with its reason — instead of inherited from a default nobody chose.
2. Effort, bounded and stated. Within the compute budget you declared, the configurations tried, the ones discarded and the rule that discarded them are all in the record — and the number that reaches a report came from folds no selection ever saw.

The one-line contract, cited from every module that implements it: Smart Training may optimise, but never hide.

What “smart” means here

Stripped of marketing, this is a rule engine over facts about your run, plus a record of what it decided. It reads cohort facts (12 subjects, 3.6:1 imbalance), device facts (6.2 GB free), model facts (4.1 M parameters), applies deterministic rules, and writes the result down. Same inputs, same plan, every time.

“Smart” is the accurate word for that, in its oldest software sense. Smart quotes apply a rule about apostrophes; a smart playlist applies rules over metadata; a smart thermostat applies rules over a schedule. None of them learn, and all of them are called smart correctly. This is the same kind of thing — cohort-derived configuration, resource-aware sizing, budgeted search with early elimination, stopping rules floored by sampling noise, and diagnosis against a measured baseline. Genuinely more than defaults, and nothing more than rules.

The Four Phases

Fourteen decisions, each with a stable id (SD-1 … SD-14) that appears in the plan, in the run record and in the exported methods. An id is permanent, so a decision cited in an old run is always the same decision.

A — Understand the experiment

12 subjects detected
Cross-subject evaluation recommended
  → leave_one_subject_out
  → grouped by subject
  → random split rejected: same-subject trials would sit on both sides

B — Configure the training

C — Optimise within a budget

Researcher and up. You give it one number — minutes — instead of typing a value list per parameter. It seeds a search space from your model family, then eliminates configurations that are behind after a fraction of their epochs, so the same clock explores several times what a grid would.

60 configurations
       ↓  10% of the epoch budget
keep the best 20
       ↓  30%
keep the best 6
       ↓  70%
keep the best 2
       ↓  full training

• SD-10 — a wall-clock budget, not a grid.
• SD-11 — successive halving, stopping early when the leader has clearly separated. The record states the budget granted and the budget used: “60 granted, 23 evaluated, stopped on separation” is a different methods statement from “60 evaluated”, and both can be true of the same clock.
• SD-12 — the baseline competition: an LDA fitted on the same split, so the question stops being “what did the model score?” and becomes “did the complexity buy anything?”

D — Understand the result

SD-13 promotes the fit diagnosis from a paragraph under the metrics to a banner beside the number it qualifies — and adds the half a verdict without a repair is missing.

OVERFIT
Train/held-out gap 28 pp (SE 4 pp).
The learning curve is still rising on the subject axis — more
participants may close it.
This graph has no dropout layer, so Smart Training could not
regularise with one; adding a Dropout node is a change only you can make.

SD-14 then offers up to three ranked next moves, each with an estimated cost relative to the run just finished. A clean run offers none — deliberately.

The Plan Preview

Ticking the box opens one step before Start. It replaces the sequence of confirmation dialogs the app used to show at submit — and it is a step rather than a toast on purpose: a plan that is right 95% of the time teaches people to stop reading it, so reading it is how you get to the run.

┌──────────────────────────────────────────────────────┐
│ Smart Training                                 │
│ 12 subjects · 3 sessions · 1,842 trials · 3 classes  │
│ Chance 0.34 empirical (0.33 theoretical)             │
│                                                      │
│ SD-2  split_strategy            derived              │
│   leave_one_subject_out  (was stratified)            │
│   12 subjects across 3 sessions. A random split puts │
│   trials from the same person on both sides.         │
│   [Accept] [Use mine] [Reject]        [Always…]      │
│                                                      │
│ SD-4  batch_size                derived              │
│   32  (was 128)                                      │
│   Estimated peak 22.4 GB of VRAM at batch 128, and   │
│   6.2 GB is free. Batch 32 fits.                     │
│                                                      │
│ 6 derived · 4 heuristic · ⚠ 1 fallback               │
│              [Cancel]  [Apply plan & train]          │
└──────────────────────────────────────────────────────┘

Note the (was …) on every row. The plan is a diff, not a settings dump — which is what makes “what did it change?” answerable at a glance and every change reversible.

How much it asks you

The ladder splits on measurement versus configuration, not on a general risk scale. A batch size cannot make a result wrong; an evaluation strategy can. That is why the default can be both safe and quiet — and why there is no “skip everything” rung above Auto-apply.

Accept, edit, reject

Rejecting is a first-class action, and it is allowed to make the run worse. Reject the grouped split and you get your ungrouped split; it is your study. What you do not get is a run that looks like the other kind:

• the publication-grade flag reflects the policy actually used, not the one proposed;
• the protocol lint re-runs against the edited graph and can block again;
• the decision stays in the record with what the planner proposed, what your graph said, and the fact that you declined — three values, not one.

Override freely; silently never.

Standing rules

After the third time you accept the same decision, press Always…. The rule is saved per project and stops that decision asking again.

A rule never removes a decision from the panel. It removes its ability to stop you. The summary line always says how many decisions a standing rule applied, and names them.

Derived, Heuristic, Fallback

Every decision carries one of three labels, and the distinction is the most important thing on the panel.

Phase A is mostly derived. Phase B is mostly heuristic, and the panel counts them separately on the summary line so you can see it without opening anything. The failure this prevents is specific: someone writing “the app derived a learning rate of 3e-4 for this dataset” in a methods section, when what happened was a lookup table. The exported methods paragraph says “set by convention … not measured on this dataset” for exactly that reason.

What No Mode Can Do

The mode ladder controls what you are asked about. It does not control what is recorded, and it does not reach the guard rails. No mode, standing rule, plan or setting can:

1. Turn off the record. Every decision, its disposition and the mode in force are in the manifest on every run, Auto-apply included.
2. Auto-confirm a blocking protocol finding. It stops every mode identically.
3. Report a search flat. See below.
4. Let the test partition be touched for anything but scoring it once.
5. Suppress the fallback sentence or the “not measured” clause in the report.
6. Mark an overridden run publication-grade when the policy it used is not.

The selection-bias rule

The moment a hyperparameter is chosen by looking at held-out scores, that held-out number stops answering the question it appears to answer. It is the best of k attempts on the same rows, biased upwards by an amount nobody can state.

Untreated, this would be worse with Smart Training than without it, and worse in the direction that would discredit the whole feature: a plan searching 60 configurations would report systematically higher numbers than one running 1, for no reason except that it searched more. That is not “the paid tier finds better models”. That is the paid tier producing inflated numbers.

So the search is handed the training rows only. The hold-out that produces your reported score is never passed to it, and the search’s own estimate of how good its winner is never appears anywhere — that estimate is precisely the biased quantity.

Measured, not assumed

A control that is not measured to bind is a control that is assumed to bind. So the rule above is checked by running the same graph three ways on 30 cohorts whose true effect is known — including a deliberately wrong version built only so the bias can be seen.

Read the bottom-right cell first. On labels carrying no information at all, the forbidden version reports 0.557 — it finds signal in pure noise, every time. The shipping path on the same cohorts sits at 0.506, indistinguishable from chance (p = 0.62) and indistinguishable from not searching.

What Gets Recorded

Every run manifest carries the plan: each decision with

• the value the planner proposed, what your graph said before, and what actually ran;
• the reason, and the numbers the reason was computed from;
• its confidence label;
• how it was approved — automatically, accepted, edited, rejected, or by a standing rule;
• the autonomy mode in force, and any standing rule that fired or expired.

A rejected decision is recorded, not erased. That is the half of the override system that makes the other half safe: you can overrule anything precisely because overruling is never invisible.

The methods paragraph

Export Methods on the Training page writes the plan out as prose you can paste into a manuscript:

Smart Training v1. Cohort: 12 subjects, 3 sessions, 1,842 trials.
Chance level 0.34 empirical (0.33 theoretical). Automatic configuration:
split_strategy stratified → leave_one_subject_out; batch_size 128 → 32;
class_weight none → balanced; lr 0.001 → 0.0003. lr was set by convention
from the model family and cohort size, not measured on this dataset.
Operator overrides: the planner proposed split_strategy =
leave_one_subject_out and the authored stratified was kept at the
operator's direction. Plan review: review measurement mode — 6 decisions,
4 applied automatically, 1 accepted on review, 1 rejected.

Every clause is a fact the run recorded; none is a claim about quality. The counts are reported flatly and never as a grade — “4 applied automatically” is a fact about a workflow, not a criticism of one.

The Run Ledger

A tab on the Training page listing every recorded run, newest first, with its verdict and its delta against the current best for that dataset and target.

What Each Plan Gets

The rule behind that table: a plan tier changes what gets tried; it never changes how anything is measured. Metric computation, split policy, chance level, confidence intervals, permutation tests, lint and every verdict are identical on every plan. Fit the same model on the same split on Free and on Lab and the numbers match to the last digit, or it is a bug — there is a test that fails the build if a plan ever reaches a measurement path.

Correctness is free because gating it would be the wrong product and the wrong risk. What Free cannot do is spend: search and the range test are compute, and metering compute is defensible in a way that metering rigour is not.

Common Questions

My score went down after ticking the box. Is it broken?

Almost certainly not. The most common cause is SD-2 replacing a random or stratified split with a grouped one. The earlier number was measuring how well the model recognised people it had already seen; the new one measures what a cross-subject claim actually means. Check the split named in the plan against what your graph had before.

Why is the run slower?

SD-12 fits an LDA baseline on the same split — a second fit, which on a large pooled cohort is real time. It buys the floor that turns your score into a comparison. Reject SD-12 in the preview if you do not want it for a particular run.

I do not see the search budget field.

It is Researcher and up, and it is hidden rather than greyed out on Free. There is no locked row with an upsell over it — that would contradict the free promise rather than sell against it.

Can it change my graph without telling me?

No. Every change is a property edit shown in the preview with its previous value, and applied only after you approve the plan. In Plan only mode nothing is applied at all. Decisions you reject are still listed in the record, so the run history shows what was proposed as well as what happened.

Does it need internet, or send my data anywhere?

No. Everything described here runs locally against your own recordings. Nothing about your cohort, your graph or your results leaves the machine.

What if it cannot work something out?

It marks the decision fallback, keeps the authored value, and names the gap in the report as a limitation. Where the cohort cannot support a defensible plan at all — a class too small to appear in every fold — it refuses to plan and tells you why, rather than producing folds that silently omit a class.