CS91.3 Lab 12: Five Peer Review

Due Tuesday, April 19th, by midnight (23:59, EST)

Goals

The goals for this lab assignment are:

  • Get comfortable with using Reviewer Guide from CS conferences

  • Get comfortable with the Double-blind Peer Review Process

  • Provide helpful suggestions to your peers to improve the abstracts

  • Think about your abstract, how can you improve

  • Prepare to write your Author Response

  • (Back to Course Index Page)

1. Reviewer Guide

For this week, please read the Entire Page of the Reviewer Guide. Pay special attention to the sections of:

  • Reviewer best practices

  • Reviewer Instructions

2. Review Examples

Please review the three examples below:

8-8-10, Best paper award Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Research By Bernard Koch, Emily Denton, Alex Hanna, and Jacob Gates Foster.

8-9-9, Best paper award ATOM3D: Tasks on Molecules in Three Dimensions By Raphael John Lamarre Townshend, Martin Vögele, Patricia Adriana Suriana, Alexander Derry, Alexander Powers, Yianni Laloudakis, Sidhika Balachandar, Bowen Jing, Brandon M. Anderson, Stephan Eismann, Risi Kondor, Russ Altman, and Ron O. Dror.

7-7-10, ML and BCI EEGEyeNet: a Simultaneous Electroencephalography and Eye-tracking Dataset and Benchmark for Eye Movement Prediction By Ard Kastrati and Martyna Beata Plomecka and Damian Pascual and Lukas Wolf and Victor Gillioz and Roger Wattenhofer and Nicolas Langer.

3. Write Your Own Review

  • Review the FIVE 800-word Abstract for HCII you are assigned in Course folder

  • You will get a reply email to your lab 11 submission about which five abstracts you are required to review.

  • Follow the instructions about Double-blind reviewing on the Reviewer Guidelines page.

  • Write your review based on the Reviewer Guide and Review Examples above. You should include:

    1. Summary and contributions: # At least five sentences, including its innovation, connection, and impact.

    2. Strengths: # At least five sentences, with details from the abstract to support your opinion.

    3. Weaknesses: # At least eight sentences, with details from the abstract to support your opinion. 'Your comments should be detailed, specific, and polite. Please avoid vague, subjective complaints. ' 'Always be constructive and help the authors understand your viewpoint, without being dismissive or using inappropriate language. '

    4. Correctness: # At least one sentence.

    5. Clarity: # At least three sentences, examples are required 'Give examples of what parts of the paper need revision to improve clarity.'

    6. Relation to prior work: # At least three sentences, 'The related work section should not just list prior work, but explain how the proposed work differs from prior work appeared in the literature.'

    7. Reproducibility: # At least five sentences, example required 'Mark whether the work is reasonably reproducible.'

    8. Additional feedback: # Optional.

    9. Overall score: # At least one sentence, I expect 6 out of 10 or lower, compared to the EEGEyeNet (7-7-10). '6: Marginally above the acceptance threshold.' The rubric focuses on how you support your peers to improve their papers. This overall score is purely for improving your peers' abstracts, NOT for the lab scores.

    10. Confidence score: # At least one sentence.

    11. Broader impact: # At least two sentences.

    12. Ethical concerns: # At least one sentence.

  • The '#' parts above are the comments for each section, similar to comments in Python.

4. Submission Guide

  • Each team only submits one file, lab_12_lastname1_lastname2.zip, including

    1. FIVE PDF files for each of the five abstracts you reviewed, in the Double-Blind way, for example, Submission_3rd_Reviewer_yF7c.PDF

5. Notes

  • Each team only needs to submit one ZIP file with both names on it. The file size should be less than 10 M.

  • Email 'xqu1@swarthmore.edu' your lab 11 files as lab_12_lastname1_lastname2.zip.

  • The team members from the same team may get the same score.

  • Lab assignments will typically be released on Wednesday and will be due by midnight on the following Tuesday.

  • This lab was released on 04/13 and will be due by midnight on 04/19.