How review works

What a review submission is

Submitting creates a review submission: a workflow instance that sits alongside your study rather than on top of it. It has its own status and its own version counter, which is why your study can be in submitted while the review submission is on version 3 of an argument about your cross-validation split.

Pipelines and stages

A review pipeline is a reusable, ordered template — for example a technical stage followed by a methodological one. Pipelines are configured by platform administrators, not by authors. Each stage carries four things that decide how it behaves:

Required reviewers

The quorum. A stage only advances once that many DISTINCT reviewers have completed their participation, and a reviewer carrying an uncleared conflict of interest does not count toward it. New stages require two by default: two independent reviewers is what "peer review" means. A stage may be configured to need one, and studies reviewed that way are labelled editorial review rather than peer review.

Reviewer pool

Optional scoping. If a stage targets a pool, only members of that pool may join it; otherwise any reviewer may.

SLA days

How long a reviewer has before their participation is flagged overdue and both they and administrators are nudged.

Blind mode

Set on the pipeline, not the stage. When on, you never see reviewer identities — see below.

When you submit, a review submission is created on the default pipeline and enters its first stage automatically. If the stage targets a pool, its members are notified; otherwise the submission simply appears in the available pool that eligible reviewers browse. You never have to nominate a reviewer yourself.

The flow, end to end

1. 1

   Reviewers join the stage

   Eligible reviewers opt in. Joining is what puts a submission in someone's queue with a due date.

2. 2

   One of them claims the decision

   Exactly one reviewer per stage is the decision owner — the person who records the outcome. Claiming is race-safe: if two reviewers click at the same moment, one gets it and the other is told so.

3. 3

   Comments accumulate

   Reviewers leave structured comments, some internal to the review team and some shared with you. Working through disagreement in private and sending you a considered position is a feature, not something being hidden.

4. 4

   The decision owner records an outcome

   One of approved, changes_requested, or rejected, with a note. Every decision is written to a permanent decision log you can read.

5. 5

   Quorum decides what happens next

   An approval only moves things along once the stage has enough completed reviewers. Short of quorum, the submission holds on the stage and waits for the others.

6. 6

   Advance, or finish

   With quorum met, the submission advances to the next stage and its reviewers are notified. If it was the last stage, the submission is approved and the study is published.

approved + quorum not met      → hold on this stage
approved + quorum + next stage → advance to the next stage
approved + quorum + last stage → approve submission, publish the study

What the badge on a published study means

A published study does not just say “reviewed”. It states how many independent reviewers actually completed a review of it, because those are different claims and conflating them would be the platform spending trust it has not earned:

Reviewers are counted as people, not rows — one reviewer working three stages is one reviewer — and an author never counts toward their own study.

Conflicts of interest

Declaring a conflict is still expected of you, but it is no longer the only check. Before anyone is seated on your submission the platform cross-checks what it can actually see:

A declared conflict is not cosmetic: until a manager clears it, that reviewer cannot own the stage decision and does not count toward quorum. They can still read and comment.

Appealing a rejection

A rejection is no longer the end of the road. You may appeal once, with a written reason, which reopens the submission at its current stage and notifies the platform administrators. The reviewer who rejected it cannot be the one to rule on the appeal — the point is a second opinion, not the same opinion again. The original decision stays on the permanent record either way.

Withdrawing your submission

Submitting is not a one-way door. Until a study is published you can withdraw it at any point — from My Submissions on the web or in the desktop app — and you do not need anyone’s permission. Review stops, the reviewers seated on it are told and their assignments are released, the study returns to draft, and it leaves the moderation queue. Nothing you have already received is deleted: the decisions, the shared comments and the version history all stay, and you can submit again when you are ready.

After publication the equivalent move is a retraction, which needs a reason and a public notice.

If nobody picks up your submission

Reviewers opt in rather than being assigned, so a thin pool can leave a submission waiting. After three days without quorum every eligible reviewer is notified that your study is waiting; after seven, the platform administrators are told to assign someone directly.

What you see, and what you do not

Visibility is decided on the server before anything is sent to your browser. The interface is not filtering a full payload it was given — it never receives the parts you are not entitled to.

Blind mode

On a blind pipeline every reviewer identity you would otherwise see is replaced, before it leaves the server, with a stable label of the form Reviewer 2 — Technical Review. The same reviewer keeps the same number across comments, participant lists, and the contributor credits, so you can follow one person's argument without learning who they are. The "viewing now" indicator shows you a count only.

Comment types

Comments carry a type — general, minor issue, major issue, approval, or rewrite request — along with an optional severity, a scope, and an anchor pointing at the specific part of the work being discussed. Reviewers can share an internal comment with you later, or pull a shared one back to internal; both actions are audited.

Responding and resubmitting

• You may reply to any comment on your own study, including as a thread on a reviewer's point.
• Resubmission is only available from changes_requested.
• A resubmission bumps the review version and re-enters the same stage — it does not send you back to stage one.
• Everyone reviewing the submission is notified that a new version is ready.
• The version log records each resubmission with your note, so the trail of what changed and why stays attached to the work.

Timing and nudges

Each stage sets an SLA. Overdue participations are swept periodically: the reviewer is notified, and administrators get a summary of what is aging. There is no automatic escalation that decides on your behalf — a human still records every outcome.

Credit for reviewers

When a submission publishes, reviewers who completed a stage are added to its contributor list with their role and stage. Under a blind pipeline they appear under their anonymised labels. Review work is visible work here rather than an invisible favour.

Related

Track your own submissions from My Submissions in the dashboard.