Writing a Rebuttal. Hints & Tips

Yesterday, I was pleased to learn that I was accepted to ECOOP 2019. The submission process was double blind, and included a rebuttal period.

Matt Might’s Advice

Writing a rebuttal is a non-trivial process, and Matt Might has written excellent advice on the subject. He advises that you should be:

• Polite.
• Conciliatory.
• Thorough.

These are good points you should adhere to. I would advise reading his post for more intuition.

Further, he advises a six stage process:

1. Indent the reviews into blockquotes.
2. Reply to all points, inline, as you would an email.
3. Sort the points and identify the most important and least important points.
4. Cut your response down to respect any word length.
5. Polish the review. Place most accurate reviewer summary at the top, say thank you, and revise the comments!
6. Relax & Revise the paper.

This is a very good process to follow, and I cannot recommend it more! However, we can improve upon Matt’s advice in several ways.

Extra Material

Alistair Donaldson of Imperial provided some sage advice for word constrained rebuttals.

• Ensure that the rebuttal itself is within the word limit, and provide well marked appendices for extra material. The reviewer can then choose to ignore or take into consideration the extra material.

I use this to add BibTeX entries for references included within the review.

Formatting.

ECOOP (and many other conferences in my field) use HotCRP¹ to handle submissions. A nice feature of HotCRP is that the renderer for reviews & rebuttals is CommonMark² and LaTeX Math aware³! We can use this to our own benefit⁴! Further, Matt Might advises using Markdown’s blockquote syntax to isolate and reply to reviewer comments, and as such this advice ties in nicely.

With CommonMark, we can use headers for grouping points together, and to distinguish between extra material.

More importantly, all my points where placed in an enumerated list. Allowing rebuttal points to be referenced without impacting too much on word count.

Generally speaking, the use of CommonMark syntax does impact upon the word count, but not significantly enough to not justify its use.

Don’t provide full citations! DBLP & DOIs are your friend.

Providing references in your rebuttal impacts on your word count. Rather than provide citations & references, we can provide the citation alone.

Some extensions for CommonMark allow one to provide links using brackets. For instance, footnotes are provides as: [^1. Pandoc uses this syntax to incorporate citations in their tooling, and we can do this to add citations. The problem is which format should we use for the key!

I often use the DBLP citation-key, but a DOI based key would be as convenient and still make the rebuttal self-contained. I also add the BibTex entries in the rebuttal’s appendix With this approach there will be no spaces in your citation and hopefully the word count won’t be affected too much.

For example:

De Muijnck-Hughes describes how cool dependent types are in their PhD thesis [DBLP:phd/ethos/MuijnckHughes16]

Although one should provide a more authoritative link such as:

De Muijnck-Hughes describes how cool dependent types are [in their PhD thesis](https://dblp.uni-trier.de/rec/bibtex/phd/ethos/MuijnckHughes16)

The former is that little but more succinct and readable. However, depending on how CommonMark aware your submission system is, then the latter might be more useful.

Grouping comments.

One of the regrets I had with my ECOOP rebuttal is that I followed Matt Might’s advice to the letter. Ordering of points allows one to place important rebuttals to the top. I also grouped rebuttal points under themes, that is the common points made by each reviewer. This allowed me to provide some sense of structure for the rebuttal, and maybe show the reviewers commonality.

However, what I should have done was kept the rebuttals organised by reviewer. You want to make the reviewer’s job easy here. By mixing the comments together you will make it harder for a reviewer to see rebuttals to their points.

Wim Vanderbauwhede suggested we could annotate the blockquotes with the Reviwer ID. However, I prefer the advice given by Edwin Brady:

Do Matt Might’s advice but keep the reviewer comments in their own section.

Thus my generic outline for a future rebuttal would be:

1. Best Summary of Paper
2. Response to Reviewer 1.
3. Response to Reviewer 2.
4. Response to Reviewer 3.
5. Appendicies/Bibliography.

Then in side we respond as per Matt’s advice. Group by theme might not work so well with the new structure and reviewers might skip other reviewers comments because of that, but we shall see.

Word Counting

One final bit is on word counting. Here is a small Bash script that uses wc and texcount to get an approximate count for words. We use grep to remove the reviewers comments. I am not aware of any CommonMark friendly word count program. One option would be to pass the rebuttal through pandoc to generate a LaTeX snippet that texcount could handle, or write a Pandoc aware program that counts all the words in titles and text blocks.

#!/bin/bash

echo "## WC"
grep -vE -e "^>" $1 | uniq | wc -w
echo ""
echo "## TeX Count"
grep -vE -e "^>" $1 | uniq | texcount -1 -

Coda

Hopefully, these hints & tricks may be of use to you. Regardless, if you do take away one thing from this then it should be Matt Might’s advice of :

• Be Polite.
• Be Conciliatory.
• Be Thorough.