Friday, August 10, 2012

Bad Reviews and Trolling

"You weren't smart enough two capitalize Christian
The cosmos exist, and they have an obvious design.
God exists."
-Random Comment Found in YouTube

I recently tweeted that "Bad Reviews are like Trolling", and I kept wondering that perhaps such a thought deserved a bit of explaining, and we might as well learn a bit on how to do a good review.When someone asks you to review a paper for a journal or a conference, they expect you'll devote an appropriate amount of time to read, analyze and review the paper. The thing is, people might get 5 or 6 of these every month, and it may increase in months when there is a conference looming. I have to say that I've receive most bad reviews than good ones, I'm not saying I got rejected, I'm saying reviews were lazy, ill written and obviously rushed. Like a Youtube comment, there are so many videos that demand your attention that you cannot bother on writing good comprehensive reviews for each of them. So a lot of people do what is commonly known as trolling, that is, they just give a negative comment without any suggestion or space to discuss.

Trolling usually is characterized by 3 things:

  1. Bad grammar, trolling is obviously done without care, so there is no care in writing well either.
  2. Usually a negative comment without any suggestion or room for discussion. 
  3. If they disagree it is based on a deep personal belief rather than a well informed and researched decision.

If you see these three points, a bad review of a paper usually has these same characteristics. Most reviewers won't even tell you what is wrong if they rejected it.

What is worse, most times, a reviewer is the pipeline worker and final judge on whether a paper is accepted or rejected for publication or a conference.It seems almost unfair that months of work get to be evaluated in a short burst by someone who might be unprepared or not willing to do the job.

But not everything is lost, I've seen great reviews, with constructive criticism, and always a chance to reply the comments. Not only that, they also are written in pristine and clear English, so the review itself is not confusing to the authors.Some suggestions you might like to follow when doing a review:

  • If you attack the author's spelling, try to give concrete examples of what you think is a mistake, perhaps the authors do not consider it that way.
  • If you attack the author's methodology, try suggesting a better one, and point out the errors or points you would change in what they are doing so far.
  • If you attack the idea in general: Please do not do it, if at the end it was a good idea, you'll look foolish, ideas are too personal and a paper should never be rejected on the basis "I did not like your idea"
  • If you attack the organization of the paper: Perhaps you could suggest a better way to organize it, and give a suggestion or two on why the paper should be organized in the way you are suggesting.
  • If you have issues with the theory behind the paper, be clear to point out why aren't you convinced and point out references or proofs that the theory is wrong.

Remember that you are not in a review committee as an almighty god but rather as a humble quality control manager, your job is to see that the work is not "plagiarized", that the work makes scientific sense and that the work is readable to most of the audience. You are not an editor, so you do not get to impose your style of writing and you are not a scientific adviser, so you are not to impose your scientific ideas.