Your results, your analysis

However many charts a tool draws, it will not draw the one your reviewer asks for. The Results page answers what did this study measure, and Results Validation answers does the difference survive a test. This page is about the third question, the one no fixed feature can answer in advance: the analysis we did not build.

There are two answers to it, and you can use either or both. Take the whole result set out as tables and work on it in whatever you already use. Or stay in the app and write the analysis against the results directly, with everything already loaded.

Export and notebook

What each plan gets

Export is free on every plan, including Free. These are your measurements; a tool that recorded them and then charged you to read them elsewhere would not deserve them.

Running a notebook inside the app is part of Researcher and above. It is the same boundary the ML Suite’s Custom Code block already uses: a Free plan gets the whole catalogue, and what is paid for is the escape hatch out of it.

Exporting Results

Where to start it

Two places, one dialog — they cannot drift apart:

The dialog lists all nine tables with a checkbox each, the three formats, and a switch for the starter notebook. The export itself runs on a background thread — a study with thousands of scores spends real time flattening every configuration into the parameters table, and the window stays usable while it does.

The nine tables

How they join

Every row of every experiment-level table carries experiment_id as its first column. Open the folder in anything — pandas, R, Excel — and the tables rejoin without the app:

import pandas as pd

obs    = pd.read_csv("observations.csv")
params = pd.read_csv("parameters.csv")
runs   = pd.read_csv("runs.csv")

# every score, next to the settings that produced it
settings = params[params.scope == "experiment"].pivot_table(
    index="experiment_id", columns="parameter", values="value", aggfunc="first")
obs.merge(settings, on="experiment_id", how="left")

The observations table is the one most analyses start from:

experiment_id  experiment      run     pipeline   subject   session     metric    value
exp-1          Motor Imagery   Run 1   EEGNet     sub-01    Session 1   accuracy  0.7318
exp-1          Motor Imagery   Run 1   CSP+LDA    sub-01    Session 1   accuracy  0.6741
exp-1          Motor Imagery   Run 1   EEGNet     sub-01    Session 2   accuracy  0.7204
   ⋮

These are the same rows the Results Validation tab tests, produced by the same reader. An export whose numbers disagree with the screen they came from would be worse than no export, and the only way to guarantee they agree is for there to be one source.

The parameters table

The table that makes the rest of the export usable as evidence. A set of scores whose settings are written down nowhere cannot be reproduced, and “we used the defaults” is not a record of what the defaults were.

experiment_id  experiment      scope       run     section        parameter        value    type
exp-1          Motor Imagery   experiment          preprocessing  bandpass.low     8.0      float
exp-1          Motor Imagery   experiment          preprocessing  bandpass.high    30.0     float
exp-1          Motor Imagery   experiment          preprocessing  channels         C3; Cz; C4  list
exp-1          Motor Imagery   experiment          ml_training    folds            5        int
exp-1          Motor Imagery   run         Run 1   hyperparameters lr              0.001    float
exp-1          Motor Imagery   run         Run 1   hyperparameters epochs          50       int

Nested settings are flattened to a dotted path (bandpass.low). A list of plain values stays one parameter — C3; Cz; C4 — because splitting it into three rows turns one setting into three that no longer read as one.

Formats

The starter notebook

Every export can write an analysis.ipynb beside the tables. It is not a format description — it is a file that already runs. Open it in Jupyter or drop the folder into Colab and it loads the CSVs, summarises each metric by pipeline, draws a bar chart with error bars, runs a paired t-test across subjects, and pivots the parameters table.

jupyter lab analysis.ipynb

It uses nothing but pandas, matplotlib and scipy, so it opens anywhere. It is the answer to “can I do my own analysis on this?” that does not require reading a schema first.

The Notebook

A third tab on the Results page, beside Results and Results Validation. Notebooks on the left, cells in the middle. It is a notebook in the ordinary sense — cells of Python, run one at a time, output underneath, and state carried from one cell to the next.

What makes it worth having inside the app rather than beside it is two things.

What is already loaded

The study is not something you load; it is already there. Every name below exists in every cell from the first line you type:

That validation module is the second reason this belongs inside the app. A “custom” analysis written against a re-implementation of the study’s statistics can quietly disagree with the tab beside it. Written against the same module, it cannot.

The Data button

You do not have to remember any of the table above. Data on the toolbar opens a reference onto your own study and answers the three questions that otherwise cost the first ten minutes:

Example values are read from your data rather than invented — “what does this actually contain” is the question being asked, and a made-up "sub-01" answers a different one. Copy name puts the selected name on the clipboard ready to paste into a cell (a table comes across as sheets["parameters"], the form that works).

Running cells

Cells run on a background thread, never the window’s. The app stays responsive while a cell works, and Interrupt is always reachable.

Outputs

Each output lands under its cell in the form that suits it:

Figures and the right-click menu

A chart you write is a chart the app drew. Figures come back as live matplotlib objects and go onto the same canvas every other chart in BCILattice uses, which means yours gets the same right-click menu without asking for it:

• Save image — PNG at 300 or 600 dpi, SVG, or PDF
• Copy to clipboard
• Zoom and pan, in data space, with reset
• Transpose, grid, legend and X-label rotation

That is why the notebook runs in the app rather than in a separate process: a picture of a chart cannot be re-exported at 600 dpi or saved as vector, and a figure is the thing most likely to end up in a paper.

Reading and writing files

Each notebook has a folder of its own. workdir() returns the path, save_csv(frame, "name.csv") and save_figure(fig, "name.png", dpi=300) write into it and print where they landed, and a relative open("out.csv", "w") resolves there too.

save_csv(results, "scores.csv")
save_figure(fig, "accuracy_by_pipeline.png", dpi=300)
print(workdir())

Reading is unrestricted — point pandas at any file on your machine. Writing outside the notebook’s folder is refused, with a message that says where the folder is. It is a guardrail against a stray path, not a security boundary, and it is documented as one in the code.

Saving, and what is stored

Notebooks are saved inside the study, not as loose files, so they travel with it — export the study, share it with your team, sync it, and the notebooks go along.

Templates

New notebook offers a blank one and four templates. Each is a whole small notebook that runs unchanged against any study — none of them hard-codes a metric or a pipeline name, because a template that raises on its first cell teaches you the feature is broken.

Worked Examples

results and df are the same table — use whichever name you prefer; these examples use results.

A chart the app does not draw

Average within an observation unit before plotting — fourteen subjects × three pipelines is 42 numbers but not 42 independent ones — and show the spread, because a bar with no error bar is a claim with no evidence.

metric = results["metric"].value_counts().index[0]
subset = results[results["metric"] == metric]

per_subject = subset.groupby(["pipeline", "subject"])["value"].mean().reset_index()
summary = per_subject.groupby("pipeline")["value"].agg(["mean", "std", "count"])

fig, ax = plt.subplots(figsize=(7, 4.2))
ax.bar(summary.index, summary["mean"], yerr=summary["std"].fillna(0),
       capsize=5, color="#14b8a6", edgecolor="none")
ax.set_ylabel(metric)
ax.set_title(f"{metric} by pipeline (mean ± SD over {summary['count'].iloc[0]} subjects)")
ax.spines[["top", "right"]].set_visible(False)
fig.tight_layout()
fig

A validation of your own

The Validation tab’s full analysis, driven from a cell — design selection, assumption checks, omnibus test, cross-check, corrected pairwise family and an APA line:

observations = validation.collect_observations(experiments)
metric = validation.available_metrics(observations)[0]

result = validation.validate(
    observations,
    metric=metric,
    group_factors=["pipeline"],    # what is being compared
    unit_factors=["subject"],      # what an independent observation is
    design="auto",                 # paired when the design supports it
    alpha=0.05,
)
print(validation.report_text(result))

Or the test written out by hand, when you want one the tab does not offer:

from math import sqrt

wide = (results[results["metric"] == metric]
        .groupby(["subject", "pipeline"])["value"].mean()
        .unstack("pipeline").dropna())

a, b = wide.columns[:2]
diff = wide[a] - wide[b]
n = len(diff)
t, p = stats.ttest_rel(wide[a], wide[b])
half = stats.t.ppf(0.975, n - 1) * diff.std(ddof=1) / sqrt(n)

print(f"{a} vs {b}")
print(f"  mean difference {diff.mean():+.4f}  95% CI [{diff.mean() - half:+.4f}, {diff.mean() + half:+.4f}]")
print(f"  t({n - 1}) = {t:.2f}, p = {validation.format_p(p)}, n = {n}")

Reshape and re-export

# one row per subject, one column per pipeline
wide = (results[results["metric"] == metric]
        .groupby(["subject", "pipeline"])["value"].mean()
        .unstack("pipeline").round(4))

# a publication table
summary = (results[results["metric"] == metric]
           .groupby(["pipeline", "subject"])["value"].mean().reset_index()
           .groupby("pipeline")["value"].agg(["mean", "std", "count"]))
table = summary.apply(
    lambda r: f"{r['mean']:.3f} ± {r['std']:.3f} (n={int(r['count'])})", axis=1)

save_csv(wide.reset_index(), f"{metric}_wide.csv")

Jupyter, Colab and .ipynb

The notebook format here is the Jupyter format — .ipynb v4, nothing invented. A notebook written in BCILattice opens in Jupyter, Colab, VS Code, nbviewer and GitHub; a notebook written in any of those opens here.

Every results export also ships an analysis.ipynb for the same reason: nothing about this feature should make your work harder to take somewhere else.

Safety and Limits

A notebook runs real Python with the same access as the rest of the app. Four things follow from that, and each is handled rather than assumed away:

Other limits worth knowing:

• One cell runs at a time; Run all queues the rest.
• A cell’s printed output is capped at 200,000 characters, and a displayed table at the first 200 rows and 60 columns — a runaway loop is a normal mistake, and turning it into hundreds of megabytes of window is not a normal consequence.
• Restarting or switching study discards every variable. Notebook code is never lost by either.

Troubleshooting