Three readers, one report

The same document gets read three ways, and each reader is checking something different. A supervisor or reviewer is checking whether the claim survives scrutiny. A student is checking whether it will be accepted. A company is checking whether it can still be relied on in a year. A report can be structurally perfect — IMRaD, declarations, an integrity appendix — and still leave all three asking a question the document should have answered.

None of this is a new place to type things. Every number below is computed from what your study record already holds — the score distribution the training runs wrote, the confusion matrix the analysis saved, the comparison the Results Validation tab ran. The report was throwing that information away at the point it drew a picture.

The order it reads in

A report can hold every right section and still be unreadable, because a reader goes through it once, top to bottom. If the model pipeline appears before the preprocessing that fed it, or a number arrives before the reader knows how much of the study it came from, they have to jump backwards — and a document you have to jump around in is one people stop trusting, because they cannot tell what ran on what.

Methods, in the order the work happens

2.1  Participants
2.2  Data
2.3  Signal pipeline (NeuralFlow)
2.4  Preprocessing
2.5  Analysis                      ← the Analysis Suite, step by step
2.6  Workflow and model pipelines
2.7  Model training and evaluation
2.8  Statistical analysis
2.9  Software and reproducibility

Each canvas gets its own subsection at the point in the study where it runs: the NeuralFlow signal pipeline before preprocessing, the model pipelines and the workflow after the analysis they consume. The Doc Editor’s From report opens a document in this same shape, so moving between the generated report and the written one never means re-learning where things are.

What each stage actually touched

Between “data were recorded from 20 participants” and the result, a great deal happens that a report normally never mentions. Filtering might run on all 20, epoching on 20, and the workflow that produced the headline on 4 of them. Every one of those is a legitimate research decision — and none of them used to be visible, so a 4-participant result sat under a heading that had already said 20.

Now every stage states its own coverage, as a funnel:

#  Step                  Participants   Sessions   Recordings
1  Filtering             20 of 20       2          40
2  Epoching              20 of 20       2          40
3  Channel selection      4 of 20       2           8
4  Feature extraction     4 of 20       2           8

The workflow ran on 4 of 20 participants, 2 sessions.

A workflow whose scores were recorded pooled across participants has no per-participant coverage to report, and says so rather than claiming one participant because the pooled id happens to be one string. Participant identifiers are printed as identifiers, too — a path in a manuscript is noise at best, and on a shared study it publishes a directory layout and somebody’s username with it.

Results at three grain sizes

3.1  Summary of results          the study
3.2  Results by participant      who
3.3  Results by session          when
3.4  each experiment
     Comparison across experiments
     Statistical comparison

Each level answers the previous one’s obvious question. The per-participant table is the one that shows whether a result is carried by the whole cohort or by three members of it — a cohort mean sits at the foot of it, so the spread above can be read against something without doing arithmetic. Both breakdowns are built from the same scores the Results Validation tab tests, so the report and the test cannot disagree about what a participant scored.

A number that can be judged

Accuracy 98.8% is a number a reader can believe or disbelieve. It is not one they can evaluate. Reports now quote the estimate with what it rests on:

before   Accuracy 98.8%

after    Accuracy 84.2%  (95% CI 79.1–89.3%, n = 20 scores, chance 50.0%)
                          └─ interval ─┘  └─ what n counts ─┘  └ baseline ┘

Two intervals, never mixed

There are two honest intervals for a classification study and they answer different questions. BCILattice computes both, labels which is which, and never blends them.

Three rules follow, and each exists because the alternative misleads:

• One score gets no interval. Not a zero-width one, not one borrowed from elsewhere — the column reads “—”, and the check tells you the record cannot support one. An interval invented from a single observation is a fabricated claim about spread.
• A rate interval stays on the scale. Accuracy intervals are clamped to 0–100%. An upper bound reading 103% is arithmetically correct and visibly wrong, and it costs the reader their trust in the rest of the table.
• Near the ceiling, Wilson rather than the textbook approximation. At 98.8% over 240 trials the normal approximation runs past 1.0 and claims an impossible accuracy is plausible. Wilson stays inside the scale and is asymmetric, which is the true shape of the interval near a boundary.

Chance level

An accuracy with no baseline cannot be read at all. Reports state two, because the reader who is going to object is objecting about the second one:

Both are printed whenever they differ, and the report says the accuracy should be read against the higher of the two. A 70% accuracy on a set that is 68% one class is not the result it looks like, and that is a fact the document now states about itself rather than leaving to a reviewer to notice.

Per-class performance

The confusion matrix was already in the report — as a picture. A reviewer cannot quote a picture, and a picture hides the finding that matters most: an accuracy of 0.94 with every class near 0.94, and the same 0.94 with one class at recall 0.31, look almost identical as heat maps and are entirely different results.

So there is now a table: precision, recall, F1 and support per class, plus two averages. The macro average counts every class once and is the one that notices a failing minority class; the weighted average counts each class by its support and tracks overall accuracy. A gap between them is the finding.

It appears automatically in the generated report for any experiment with a confusion matrix recorded, and can be inserted into the Doc Editor as a linked block that rebuilds itself on refresh. An experiment with no matrix simply has no such section — nothing is estimated in its place.

The statistical procedure

When a comparison has been run in Results Validation and added to the report, the Methods section now names it in full: the test, the design, the metric compared, the grouping, the observation unit, the number of independent units, alpha, the effect size with its own confidence interval, the multiple-comparison correction, the post-hoc procedure and the APA line.

Effect size and correction are the two things a reviewer asks for by name, and they were the two the Methods table was silently omitting. Nothing here is generated when no comparison was run — the section says so instead.

Why that many participants

The study protocol has always held a target participant count. It now also holds the reason for that number — a power analysis, the sample size of the study being replicated, or the site’s recruitment ceiling — and the report prints it under Methods ▸ Participants ▸ Sample size. A study with a target and no stated justification gets a paragraph saying exactly that, which is better coming from your own report than from a reviewer.

Check report

Review ▸ Check report in the Doc Editor, and automatically before every export from either editor. One pass over the document, listing what a reader would see before they see it.

What it looks for

Every finding that belongs to a place in the document says where, and double-clicking it takes you there. A list of problems you then have to go and find is a list people read once.

It never blocks and never edits

Length against a limit

Review ▸ Word count reports words per section against a limit you set once and that is stored with the document. One number for a whole document is the number you already know is too big and cannot act on; a thesis chapter and a journal both constrain length section by section. Going over the limit is a Problem in the check.

A document that says what it is

Confidentiality marking

An internal draft and a document cleared to leave the organisation used to be indistinguishable — same file, same running head, nothing anywhere saying which was which. A document is now marked Unmarked, Draft, Internal, Confidential, Restricted or Released, and that marking reaches the reader four ways:

• in the running head, leading whatever you wrote there;
• as a diagonal page watermark, behind the text, on every sheet;
• in the Word and LaTeX running heads of the exports, so it survives the file leaving the app;
• in the exported file’s name — a Confidential PDF is recognisable in a downloads folder before anyone opens it.

Released versions

Everything in a BCILattice report re-resolves from the study record on refresh and at the top of every export. That is what makes it trustworthy while you are writing it — and it is exactly what makes it indefensible afterwards. Retrain a model, reopen the report, and the PDF your sponsor is holding no longer matches the one the app produces. Nothing said which version went out.

A release is that document frozen: the whole thing, with the results digest it was frozen against, the marking it carried, its sign-off and its contributors. It lives in the study beside the working document, so it travels with the study rather than as a file someone has to remember to keep.

Review ▸ Releases

  v3 · 2026-09-08 14:22 · R. Naeem · Confidential
     the study's results have changed since
     Author: R. Naeem   Approved by: Dr Khan
     "sent to the sponsor"

  v2 · 2026-08-30 09:10 · R. Naeem · Internal
     results unchanged since

Each release carries the digest of every number in the study at the moment it was frozen, so the app can tell you whether it still matches. That answer is three-valued on purpose: “unchanged”, “the results have moved since”, and “cannot be compared” for a release frozen before digests existed. “We do not know” and “they match” must never look the same.

Sign-off

Review ▸ Sign-off records an Author, a Reviewer and an Approver, each with a name and the date they signed, stamped with the account signed in at the time. It is held on the document and frozen into every release.

One signature per role — three approvers tells a reader nothing about which to believe — and any of them can be withdrawn, because a sign-off that cannot be withdrawn is one nobody gives. A superseded approval belongs to a superseded version, which is what a release is for.

Several people, one report

On a shared study the written report used not to sync at all. Two people on the same study each had a report the other could not see, which made a co-authored report impossible on a platform whose whole premise is that a study travels with its evidence.

The report, its template, its released versions and its saved statistical comparison are now part of the study’s Reports & exports sync section, so they push and merge like every other part of a shared study — see Lab plan & team sharing. Three things come with that:

What it does not do

• It does not check your prose. A sentence you wrote containing a number is your writing and your responsibility. The check reads the document’s structure and its linked values, not your claims.
• It does not compute a power analysis. The sample-size field records your justification; it does not produce one.
• A marking is a label, not an access control. Marking a PDF Confidential says what it is; it does not encrypt it or stop anyone forwarding it.
• Sign-off is a recorded claim, not a cryptographic signature. It records who signed, when, and from which account. It is an audit record, not a legal e-signature product.
• Nothing here is a regulatory claim. See Research & Clinical Scope.

Report integrity is part of the Reports page and applies to both editors. The Doc Editor itself is a paid feature — see Doc Editor for plans and storage.