Reviewer Guidelines

Thank you for agreeing to serve as a reviewer for IEEE Spoken Language Technology Workshop (SLT 2022)! Your time and expertise directly contribute to the quality of the workshop as well as the speech and language research community.

Main Tasks

  1. Before the paper assignments: Please make sure that you have completed your CMT user profile. This is important for having a good match between the submitted papers and the reviewers.
  2. Check the paper assignments: As soon as you receive the notification for the paper assignments, you are expected to log in to CMT to check for conflicts and to check that papers fall within your area of expertise. If you are not at all confident about reviewing an assigned paper, please let the TPC know immediately.
  3. Prepare and enter your reviews: Please prepare a review that you would appreciate seeing as an author. Information about the review form and some specific suggestions for writing a good review are given below.

Information about the review form

  1. Reviewer's confidence: Please indicate your level of confidence. If you have serious doubts about your ability to assess the paper, please inform the TPC Chairs.
  2. Technical Correctness: Please indicate your opinion about the technical correctness of the approach including the assumptions, mathematical formulation, algorithms, experimental design and whether the results are trustworthy.
  3. Novelty: Please indicate your opinion about the novelty of the paper. Note that novelty may be in the formulation, algorithm, and application. In addition, new findings about a known approach that would be of interest to the community could be also considered novel.
  4. Reproducibility: Please indicate your opinion about the reproducibility of the approach presented in the paper. While providing an open source implementation and evaluation on freely accessible data would be ideal for reproducibility, the approach should be considered reproducible as long as enough details are given to reproduce the results.
  5. Clarity of Presentation: Please indicate your opinion about the clarity of the presentation including the organization of the paper, the writing style and use of language, the quality of figures and tables.
  6. Overall Recommendation: Please indicate your overall opinion about whether you would like to see the paper at the workshop.
  7. Brief Summary of the Paper: This part tells the committee what the major contributions are, what the authors did, how they did it, and what the results were. It also helps authors to verify that the reviewer understood their approach and interpretation of the results.
  8. Key Strengths of the Paper: Please list the key strengths of the paper.
  9. Main Weakness of the Paper: Please list the main weakness of the paper.
  10. Detailed Comments for Authors: Please supply detailed comments to back up your rankings. The comments will help the committee decide the outcome of the paper, and will help justify this decision for the authors. If the paper is accepted, the comments should guide the authors in making revisions for a final manuscript. Hence, the more detailed you make your comments, the more useful your review will be - both for the committee and for the authors.
  11. Confidential Comments for Committee (Unseen by the authors): Please supply any comments that could reveal your identity or you would not want the authors to see.

Suggestions for a good review

The reviews for the workshop serve two main purposes: (i) the TPC makes the decision to accept or reject the submission based on the information provided in the review; (ii) the authors improve their submission based on the feedback provided by the reviewers. Therefore, a good review should include enough information for the TPC to make a healthy decision and for the authors to benefit. Here are a few suggestions for writing a good review.

  • Be fair. Do not let personal feelings affect your review.
  • Be polite. There is no need to include any comments that may be interpreted as rude.
  • Be specific: Vague statements are difficult for authors to understand and address.
  • Justify your assessment of the paper. Comments without justification are not as helpful.
  • Evaluate the submission as is. Unlike a journal submission, the authors cannot be expected to make major revisions.

Paper-length

The paper length of SLT 2022 is limited to a maximum of 6 pages for main contents + extra 2 pages for references. However, the authors are suggested to limit the paper length to 5 pages for main contents + 1 extra page for references if they think of converting their workshop paper as a basis for the IEEE journals and transactions. Therefore, paper length should not be a criterion impacting your review.

Supplementary Material

Note that submitted papers must be entirely self-contained without the supplementary material. Reviewers are encouraged but not obliged to look at the provided supplementary material. The review should be based on the submitted paper, not the supplementary material.

Double-blind policy

The review procedure for SLT 2022 is double-blind. This means that the reviewers will be unknown to the authors, and the authors will be unknown to the reviewers. The authors are not allowed to provide the information that would identify the authors (such as an acknowledgment footnote). Anonymity should be kept in mind, during the paper submission and review process.

Please keep the following in mind during the review process:

  1. Reviewers should also keep their identity invisible to the authors.
  2. Reviewers should not ask authors to cite their papers unless it is essential (e.g., the author is expanding on the reviewer's previous work or is using their dataset); this is unprofessional and also compromises the reviewer's anonymity.
  3. If you accidentally discover the identity of the authors of a paper, make every effort to treat the paper fairly. It is NOT acceptable to accept or reject a paper based on the prior bias a reviewer might have about its authors.
  4. Please report any potential breach of the double-blinf policy in your assigned reviews.

ArXiv Papers

With the increase in popularity of publishing technical reports and arXiv papers, sometimes the reviewer may know the authors of a paper.

  1. Reviewers should not attempt to identify authors based on arXiv submissions or other publicly available technical reports.
  2. If the reviewer accidentally uncovers the authors' identity via arXiv, they should not allow this information to influence their review.
  3. ArXiv papers, which are not published in peer-reviewed conference, journals, etc., are not considered prior work since they have not been peer-reviewed. Therefore, citations to these papers are not required and reviewers should not penalize a paper that fails to cite an arXiv submission.
  4. If a reference points to an arXiv paper which has also been published at a conference proceedings or a journal, please provide enough information for the authors to correct the reference.