Getting rejected or How to deal with Peer Review

No matter how many years of experience a researcher has, the peer-review process has something unpredictable, almost mysterious to it. Sometimes you receive a very unexpected letter of acceptance for a manuscript with a scientific contribution you considered yourself as only somewhat significant to the community. And then other times the best manuscript you have ever written gets rejected repeatedly. Being rejected is of course seldomly an occasion for celebration. Experienced researchers, however, have learned to be disappointed, shrug and then move on with their projects. They know that a rejection of a paper is not a rejection of them as a person or their scientific work as a whole. Often it is not even a rejection of this specific work, but they were merely unlucky with a specific reviewer they got assigned. 


If you are in the beginning of your scientific career, having a paper rejected can be more than a disappointment. II still remember the first set of reviews I received for a conference paper. The points they made were valid (and compared to some rejections I received later their tone was rather professional), but I still felt crushed. Why would I try to re-submit this work if it is obviously so bad? And was there even a point in continuing this project as a whole? I was lucky enough that, when I came to the office the next morning, I found a note by one of my supervisors on my desk saying: “Disappointment is an okay feeling, but don’t be discouraged.” This note is still in my office and whenever I receive a rejection, I take a good look at it to remind myself of its message. 


Over the years and especially thanks to a recent course in academic reviewing, publishing and writing, I got better with handling rejections. In the following, I have summarized some of the lessons learned over the years in the hope they can help others facing discouragement due to rejections as well.


Everyone gets rejected! To me, this is still the most important point, because rejections are still in a veil of silence. But no matter at what stage of their scientific career a researcher is, chances are high they still get more papers rejected than accepted. If you submit to a conference or journal, have a look at their acceptance rate – something around 20% isn’t unusual. That means that 4 out of 5 papers will be rejected! I’m sure many of them have put as much effort and thought into their paper as you did. Campanario (2009) presented the stories of 19 Nobel Laureates and how the very work that would earn them the Novel Price was initially rejected by (multiple) journals. What can you learn from them? Just because your work is not accepted in the first, second or even third trial doesn’t mean it will not have a high impact later. Speaking of high impact…


Review scores have less impact than one might think. The reviewers assigned to your paper hopefully have an expertise relevant to your paper, but there is still a large pool of people to pick from. So does it matter which one you end up with? Unfortunately yes. Price (2014), for example, found that most papers that were accepted at the very prestigious NIPS conference would be rejected if they were assigned to another set of reviewers. But even if your paper does get published, a high review score is no guarantee for success, as Bartneck (2016) discussed. According to him, it is not the review score that correlates with number of citations, but the number of available online versions.So even if you don’t get the best paper award, upload your paper to your personal website or ResearchGate if you’re allowed!


No reviewer is ever wrong. This advice by Starbuck (2003) seems counter-intuitive after what I have just written. However, he is actually making a good point: “Every editor and every reviewer is a sample from the population of potential readers.” If they misunderstand the point you are trying to make or don’t see the significance of your findings, chances are high there are others out there who read your paper exactly like they do. Since you already got that feedback, make use of it to reduce the pool of people who don’t get your contribution. Even if the pool is small, it always helps to be as clear as possible in what you are describing. 


Don’t take it personal. I know this is easier said than done, but reviewers likely don’t know you or the quality of your work beyond this paper. One technique that helped me over the years is to only look at the decision when you hear back from a conference or journal. If it’s accepted, be happy, close the email and do something else. If it’s rejected, be disappointed, close the email and do something else. Don’t read the reviews just yet. Let it sink in and the emotions settle a bit before reading the reviews. This way, you will be able to look at them from a more objective perspective. 
You don’t have to put up with everything. Some people out there are just outright inappropriate in their reviews and there is a limit to what you need to accept. If someone becomes personally rude or unprofessional, reach out to the editors or program chairs. Often they are quite happy you point this out to them – who knows how many other researchers have been dealing with that person before and may never submit to that journal or conference again because of their experience? Unfortunately editors and meta-reviewers do not always have the time to thoroughly read all comments before sending the final decision to you. That doesn’t mean, though, you can’t send an email to draw their attention to the matter.

This blog post was written by Maike Paetzel from the Uppsala Social Robotics Lab.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.