Reviewer handbook

Becoming a reviewer

Reviewer is a capability added to an ordinary account, so you keep publishing your own work while you review. Applications are read by a human — there is no automatic approval.

1. 1

   Sign in, then apply

   Applications are tied to your account, so you need to be signed in. Open the reviewer application.

2. 2

   Tell us what you actually work on

   Pick your research areas from the list — EEG, fNIRS, EMG, MEG, motor imagery, P300, SSVEP, neurofeedback, deep learning, signal processing, feature extraction, clinical BCI, rehabilitation, cognitive neuroscience. These become your expertise tags, and they are what reviewer matching reads.

3. 3

   Write the motivation properly

   A short statement of why you want to review and what you can judge competently. It is the part the decision actually turns on. Optional links — publications, GitHub, ORCID — help.

4. 4

   Accept the reviewer agreement

   Confidentiality, conflicts of interest, and conduct. You must accept it to submit, and it binds you for every review you take.

5. 5

   Wait for the decision

   Applications are reviewed manually and answered by email, usually within a few days. One pending application at a time.

What you are eligible for

Doing a review

1. 1

   Join the stage

   Joining is what puts the submission in your queue and starts the clock. It is idempotent — joining twice does nothing extra. A due date is set from the stage SLA. A study under review is private: before you join you see its title, abstract, modality and task; joining is what opens the study itself to you.

2. 2

   Read the study as readers will

   From the review room, open the Summary and then the full study — the same Summary and app-style reader every reader gets once it is published, pinned to the version submitted. Only the author, the reviewers on the study and administrators can open it until it is published.

3. 3

   Decide whether to claim the decision

   Each stage has exactly one decision owner. Claim it if you intend to carry the outcome; leave it if you are contributing an opinion. Claiming is race-safe, so a lost race is reported cleanly rather than producing two owners.

4. 4

   Declare any conflict

   Shared supervisor, same lab, direct competitor, prior collaboration — declare it. Declarations are recorded against the review, and clearing one later is recorded too.

5. 5

   Work through the checklist

   The guidance checklist is per reviewer and per version. It never decides anything on its own; it exists so nothing obvious gets skipped, and the decision owner can see the checklists of everyone on the stage before deciding.

6. 6

   Comment

   Comment where you read: every Summary section, figure, table, Analysis page, canvas and run has its own comment button, and the Comments panel takes comments on the study as a whole and lists every comment with a link back to where it was made. Mark each as must fix, suggestion or question. Every reviewer on the study sees every comment, so several reviewers can work on it at once; comments start internal to the review team, so share the ones the author needs. The author replies under them, and marks a point addressed when a new version deals with it — you then resolve it (Undo is there for a slip) or reopen it. Choose Ask for a filewhen you need something the study does not show; the request is always shared and the author adds the file under it. Select any block on a canvas to open its Property Editor, read-only: every field, the author's value or the default.

7. 7

   Record the decision

   Decision owners only. One of approved, changes_requested, or rejected, with a note. Recording it completes your participation and writes to the permanent decision log.

What your approval triggers

• Below the stage quorum, the submission holds where it is and waits for the other reviewers.
• At quorum with a later stage, it advances and that stage's reviewers are notified.
• At quorum on the last stage, the submission is approved and the study is published.
• changes_requested hands it back to the author, who resubmits into this same stage.
• rejected closes the submission.

A study sent with its data

When an author publishes a study with its raw recordings, the dataset is linked to the study and waits on the study's review. It does not enter the dataset queue, and you can open it while you review. The approval that publishes the study (the last stage, or a moderator's approval) also decides what happens to the data:

• Accept with the data: the dataset is published with the study.
• Data goes to a separate review: the study is published now. The dataset joins the dataset queue and is published when that review passes.
• Accept the study without the data: the dataset is rejected with your reason, which the author sees. The author can fix it and submit it on its own.

Requesting changes or rejecting the study leaves the data waiting, so a revision brings it back with the study.

Signing off parts and passing a review on

Every reviewer on a study sees who holds its review. You do not have to review all of it yourself. Check the parts you know best, sign them off, and pass the rest to a reviewer who knows those parts. Open the study and choose Parts in the review bar.

Sign off a part

Dataset, Preprocessing, Analysis, Pipelines, Models, Results or Write-up: "I checked this and it is fine", with a note on what you checked. Only the reviewer holding the review signs parts off.

A sign-off is locked

The next reviewer reads it, with your comments and notes, but cannot change it. You can withdraw your own sign-off only while you still hold the review.

Disagreeing with a sign-off

Ask the author to change that part. This sends a must-fix comment shared with the author, and tells the reviewer who signed it off. The sign-off stands until the author revises.

Pass to a reviewer

Name a reviewer and leave a note on where you stopped. The review stays yours until they accept. If they decline, or do not answer within 3 days, it stays with you.

Step off

The review goes back to the pool. The next reviewer picks it up where you left it: comments, files, sign-offs and your note.

Who can receive it

Any reviewer except an author of the study: its owner, its submitter, every listed co-author (accepted or not) and its own team members. A blocking conflict of interest also rules them out. A disclosable conflict is shown to them before they accept.

What stays with your name

• If you pass a review on with at least one sign-off, your name stays on the study and lists the parts you signed off. You can still read the review, but you cannot comment or change anything in it.
• If you pass it on with no sign-off, nothing carries your name and you lose access to the private study.
• Credit for a part-review counts toward your record only once the review has ended. It never counts toward a stage's quorum or the study's peer-review tier, because you did not review the whole study.
• When the author revises, each sign-off is measured against the part as it was when signed off. If the part is unchanged, the sign-off stays. If ≤70% of it changed, the sign-off still counts, but the part is reopened so the current reviewer can check it again. If more than 70% changed, the sign-off is undone. If that leaves you with no sign-off on the study, your name comes off it and you are notified.

Conduct and confidentiality

Everything under review is confidential

Do not circulate, reuse, or act on unpublished material you see as a reviewer, including data, code, and results.

Review the work, not the author

Comments should be specific and actionable. "The split policy is not stated, so leakage cannot be ruled out" is useful; "this is sloppy" is not.

Declare conflicts early

Before commenting, not after a decision is questioned.

Internal is for deliberation, not concealment

Anything the author needs in order to fix the work should be shared. Sharing and unsharing are both audited.

Blind means blind both ways

On a blind pipeline the author sees you as a numbered label. Do not defeat it by identifying yourself in a shared comment.

Load, deadlines, and metrics

• Your queue shows every active participation with its due date and how long it has been open.
• An overdue sweep notifies you and reports aging reviews to administrators. Nothing is decided on your behalf.
• Your reviewer metrics — volume, turnaround, decisions — are visible to you, and inform pool membership.
• If your workload changes, ask an administrator to adjust your maximum active load rather than silently going quiet.
• Review history can be exported as JSON or CSV, which is what you want when writing a service statement for a promotion file.

Team admins are not global reviewers

A team administrator can comment on work authored by their own team members, but those comments are always internal and they cannot approve, reject, or publish anything. Sharing comments with an author and moving a study through review are reserved for reviewers and administrators.

Credit

Completing a stage on a submission that goes on to publish adds you to its contributor list with your role and stage — under your anonymised label if the pipeline was blind. Reviewers also appear in the public reviewer directory and on the community leaderboard.

Related