In short. A conference paper review works when four things are settled before the first paper arrives: what "double-blind" is allowed to hide, how each paper is matched to reviewers who have no conflict with it, how disagreement is resolved, and how decisions become letters and then a schedule. Do those in order, write each rule down, and the programme committee spends its time on the papers that are genuinely close rather than on process.
Programme chairs usually inherit a review workflow from last year's chair, along with a shared folder, a spreadsheet of reviewer bids and a set of email templates. It runs until the submission count doubles, a reviewer turns out to be a co-author of the paper they were given, or an author's name appears in a PDF footer. This guide sets out the whole path, from the call for papers to the public programme, in the order a chair actually meets it.
What Double-Blind Review Means in Practice
Double-blind means two things at once. Reviewers do not know who wrote the paper, and authors do not know who reviewed it. The second half is easy, because reviewers are simply never named in decision letters. The first half is where most conferences leak.
Anonymity has to be checked at three places:
- The submission form. Author names, affiliations, ORCID identifiers and email addresses are entered as data, so the system can hold them back from reviewers. They should never be typed into the title or abstract fields.
- The paper itself. The PDF must not carry names in the header, the acknowledgements, a funding line or the file properties. Self-citations written as "our earlier work [4]" are fine; "as we showed in Sharma and Rao (2025)" is a hint. Ask authors to cite themselves in the third person.
- Written answers. Cover notes, "novelty over prior work" boxes and responses to questions often contain a name or an institution. A blind process has to look there too.
A short author checklist reduces the work later. Before submitting, authors confirm that the PDF has no names or affiliations, the acknowledgements section is removed or empty, the file properties are cleaned, links point to anonymised repositories, and self-citations are in the third person. Put this checklist on the submission page and ask for a tick against each line.
Decide too how strict you will be. Desk rejection for a visible name is the harshest option and the least popular. A fairer practice is a short window after the deadline in which the chair's office asks the author to resubmit an anonymised file, with a single reminder and a clear cut-off.
Conflicts of Interest
A conflict is any relationship that could make a reviewer's judgement of a paper doubtful, or look doubtful. Common ones are a current or recent co-author, the same institution, a current or former supervisor or student, a close family or financial tie, and a collaboration on a funded project. Most conferences use a window of three to five years for co-authorship and shared affiliation. Pick one and state it in the call.
There are two sources of conflict data, and a sound process uses both:
- Declared by the reviewer. Each reviewer lists their own conflicts, or confirms each paper offered to them. Declarations are the reviewer's responsibility, and a paper assigned in spite of a declared conflict is a process failure, not a reviewer failure.
- Detected from the data. Authors list institutions and, where they choose, collaborators. Matching these against reviewer affiliations catches the conflicts that no one remembered to declare.
Treat a late-found conflict as routine. A reviewer who notices it halfway through a paper says so, the paper is reassigned, and the chair records why. The point is a record, not a reprimand.
Matching Papers to Reviewers
Reviewer bidding, where reviewers rank papers by interest, produces good expertise matches and poor workload balance. Chairs who rely on bidding alone find that popular topics are over-reviewed and unfashionable ones are covered by whoever has capacity. A workable pattern is a hybrid:
- Compute an expertise score from track, keywords and the reviewer's own stated areas.
- Fix a load cap per reviewer, typically four to eight papers for a one-round conference, and enforce it.
- Require at least three reviews per paper, and add a fourth for papers from a sub-area with few reviewers.
- Remove conflicted pairs before anything else, so no ranking can reintroduce them.
- Reserve a pool of senior reviewers to take late reassignments.
Scoring, Disagreement and the Discussion Phase
A review form that asks for a single "accept or reject" gives the chair almost nothing. Ask instead for a score on a short set of criteria, a recommendation, and written comments, split into comments for the authors and comments for the committee only. Common criteria are significance, soundness, clarity, originality and relevance to the track. Add a confidence rating so that a strong view from a non-expert can be weighed accordingly.
Once reviews are in, the useful number is not the average. It is the spread. A paper scored 7, 7, 8 needs little discussion. A paper scored 3, 8, 8 needs a conversation, because either one reviewer saw a flaw the others missed or two reviewers were generous. A simple rule works well: any paper whose scores differ by more than a set margin, or whose reviewers recommend opposite outcomes, goes to a discussion thread, and the thread must produce a short written consolidation before the chair decides.
Decisions and Letters
The decision is a recorded outcome for each paper: accept (with a format such as oral, poster or short talk), reject, or, in two-stage conferences, accept with minor or major revisions. Set a capacity first. If the programme holds 60 talks, the chair knows how many accepts are available before the ranking begins, and the borderline band is the few papers around that line.
Letters are where careless process becomes visible to authors. A good letter puts the decision in its first sentence, then the review comments written for authors, then the next dates (camera-ready, registration, presentation), and nothing that identifies a reviewer. Test every template on a sample address before it reaches the real list, and keep a copy of what was sent to whom.
Building the Schedule and a Public Programme
Accepted papers become a timetable. The mechanics are not complicated, but the order matters:
- Fix the grid. Days, rooms, slot lengths, and the keynotes and breaks that are already committed.
- Group by track. Put papers of one track in consecutive slots in one room, so attendees do not have to hop between buildings.
- Respect speaker constraints. Time zones for remote speakers, a speaker with two papers, a chair who is also presenting.
- Assign a session chair to each block, and never a chair who is an author in that block.
- Put posters in their own session, with a clear count of boards.
- Publish only when the speakers have confirmed. A programme that changes three times after publication costs more goodwill than a week's delay.
The public programme page is also the page that search engines and attendees will find. Give each session a title, a time, a room, the track and the chair, and link every paper to its abstract. Keep a stable web address across editions so that earlier programmes remain citable.
Where IdeaScore Fits
For conferences, IdeaScore keeps this path in one place. Authors submit with an anonymity check, and reviewers score double-blind: affiliation, institution, author and ORCID answers stay hidden from them. Reviewers declare conflicts of interest, and conflict-aware assignment draws its plan with those conflicts kept apart. The chair's Decisions page shows how many papers have been reviewed, how many are accepted so far, where reviewers disagree and which conflicts are still open, with author names hidden while the decision is made.
Letters can be tested with "Send a test to me" before they go out, and the Decision Trail keeps a record of what was decided. When the accepted list is settled, chairs build the schedule, manage speakers and publish a public programme page.
None of this removes the chair's judgement. It removes the clerical work around it.
A Chair's Calendar
For a conference with a spring event, a realistic sequence looks like this:
- Four months out: call for papers published, with the anonymity rules and conflict window stated.
- Deadline: submissions close; the chair's office runs the anonymity check and asks for fixes within a week.
- Two weeks after: conflicts and expertise matched; assignments sent with a review deadline of three weeks.
- Review deadline: reminders at one week, three days and the day itself; reassign anything still empty.
- Discussion: one week for papers with a wide spread.
- Decisions: the committee meets, outcomes are recorded and checked, and letters go out together on the same day.
- Camera-ready and schedule: schedule drafted, speakers confirmed, programme published.
Questions Chairs Ask
How many reviewers should each paper have? Three is the working minimum, and four where the sub-area is thin or the paper is borderline.
Should authors see reviewer scores? Most conferences share the comments and the recommendation, and many share the scores. Decide before the call, not after.
What if a reviewer breaks anonymity? Log it, reassign the paper, and tell the author only what they need to know.
Do we need a rebuttal phase? If the calendar has room for it, a short author response (a few hundred words, a week) improves decisions on the borderline band and costs little.
