The role of the reviewer is to assist the Program Committee in assessing the quality of proposals submitted. Reviews do not wholly determine which proposals will be accepted or rejected, but rather provide expert information which the Program Committee uses in making its decisions. The second major role for reviewers is to provide helpful, constructive feedback to authors, which can strengthen both the quality of the conference and the collegiality and intellectual rigor of the digital humanities.

A good review will suggest concrete ways in which the proposal may be strengthened. This feedback is important whether you are recommending that the paper or poster be rejected or accepted. If the former, it will enable the author to submit a stronger proposal next year (and may encourage and benefit a new member or a young scholar); if the latter, it will result in a stronger paper being presented. In either case, constructive criticism projects collegiality and an interest in others' work.

Whatever you may think privately of the proposal or project, and whatever you may know (or think you know) about the author, it is essential that you be uncompromisingly professional and courteous in reviewing all submissions. The rudeness of any sort is destructive to the morale of the community and is absolutely unacceptable in a review. Comments which are purely negative should be addressed solely to the program committee.

To a certain extent, given the small size of the DH conference community, the identity of paper authors has often been evident to an astute reviewer, and discretion has always been required in handling that knowledge appropriately. Since ADHO has discontinued the anonymization of papers during the review process (moving to single-blind review), this discretion has become even more necessary.

The identity of an author will necessarily be a factor in evaluating the proposal, but it may operate in complex ways. A famous name should not be taken as de facto evidence of a strong proposal, and a weak proposal by such an author should be given the same critique as a weak proposal by an unknown author. However, there may also be cases where an author's known expertise may strengthen the value of the proposal: for instance, a representative of a standards workgroup might have more credibility in discussing the standard than someone with no involvement in the effort. Reviewers should feel free to be candid in their comments to the Program Committee in cases where they feel the identity of the author plays a significant role in the assessment of the paper.

You will be allowed to read comments by others who have been assigned to assess the same submission. These peer reviews will only be visible after you have submitted your own review, and all will remain anonymous. Reviewers are expected to assess each contribution independently, on its own merits, and are asked not to address other reviewers’ comments directly. If, however, you are prompted to think more deeply and augment the text of your own review after reading peer assessments, you are welcome to do so up to the review deadline, and are expected to justify any scoring change in the text of your review.

Submissions will be evaluated based on:

• Overall organization and clarity of proposed submission (25%)

• Explicit engagement with relevant scholarship, with references and justifications displaying knowledge of the current state of appropriate fields (30%)

• Clear theoretical, methodological, or pedagogical framework and explicit statement of purpose (25%)

• Applicability, significance, and value of the theoretical, methodological, and/or practical contribution to the digital humanities generally (20%)

Reviewing Submissions by Type

Please review the submission types before completing your reviews. Reviewers are welcome to suggest alternative presentation formats (for instance, considering long paper proposals for short paper presentations, etc).

Posters

Posters are an appropriate venue for project descriptions and updates. Poster proposals should be evaluated based on the clarity of the information being conveyed.

Short presentations

Short papers are appropriate for reporting on works in progress, limited scholarly interventions, or for describing a singular tool or project. Reviewers should also assess whether the submission can be accomplished in the time allotted (10 minutes).

Long presentations

Proposals for long papers should deal with substantial completed research, report the development of new methodologies; or present rigorous theoretical, speculative, or critical discussions. Long-paper submissions seek substantive feedback and discussion of the submission’s relationship to other scholarship in the field. Reviewers should also assess whether the submission can be accomplished in the time allotted (20 minutes).

Panels

Panels should focus on a single theme and be inherently coherent in presenting a substantial body of research or a research question. A panel should be conceived as a 90-minute session of four to six speakers. Reviewers are advised to consider whether the constitution of the panel reflects the constitution of the field and/or research topic that is being addressed and ADHO’s expressed commitment to diversity or to explicitly address problems in those areas. If not, suggestions to submitters are welcomed.

Pre-conference workshops and tutorials

Workshops and tutorials should be evaluated based on their syllabus and/or planned workshop materials. Workshops and tutorials may take the form of hands-on experiences, skills development, as well as group discussions around shared readings. Reviewers are encouraged to assess how, based on their CV(s), the proposer(s) are suited to lead the particular session. Reviewers should also assess whether the submission can be accomplished in the time allotted (either 2 hours or 4 hours).

Ten Tips for Conference Reviewing

1. Be aware of the Review Criteria

2. Offer constructive feedback that gives submitters direct suggestions for improvement

3. Use positive language and affirmative statements rather than negative statements (e.g. “This submission could be improved by considering the following:…” rather than “You are missing X.”)

4. Include full citations or links to information that you feel would benefit the submission

5. Use specific examples when evaluating writing style: feel free to quote from the submission.

6. If you disagree with the entire premise of the submission, be considerate of the work completed by the submitter when outlining your rationale. It is not appropriate to disagree without providing evidence to support one’s position.

7. If you find a proposal wholly compelling, please still give one or two sentences complimenting its strengths

8. Reviewing is a dialogue between the submitter, reviewers, and the program committee. Be aware that you may be asked to revise or update your review with additional information.

9. Reviewing is a service to the digital humanities. Submitting and reviewing deserves to be recognized: list your contribution(s) on your CV or share your accomplishments.

10. Follow the golden rule of reviewing: review others as you would want to be reviewed: that is, fairly and constructively.