Suppose you're an assistant professor, striving for tenure at your university. You're diligently conducting your research, scrambling to put together your publications and tending to your teaching duties. Then one day your dean phones you up with an ominous request: “I'd like to talk to you about the negative ratings you've received on RateMyProfessors.com.”

In the United States, websites such as RateMyProfessors.com, which allow students to post anonymous reviews of their university professors, are growing in popularity. The comments range from the positive (“The definition of a perfect professor”), to the disparaging (“He is horrible”), to the slightly tawdry (“Quite possibly the hottest prof you'll ever find”). Perhaps unsurprisingly, these sites are proving less of a hit with US professors, many of whom are uneasy about the unvetted comments that are allowed.

But in early August, an analysis of the effectiveness of RateMyProfessors.com had some cautious praise for the site and its approach (J. Otto, D. A. Sanford and D. N. Ross Assess. Eval. High. Educ. 33, 355–368; 2008). The authors found that the feedback and ratings seemed surprisingly free of the universally high or low ratings that might be expected, given that the site is likely to attract students predisposed to wanting to praise or damn their professors. The site, suggest the authors, could potentially be seen as a valid measure of teaching effectiveness. Although the possibility of individual bias remains, they note that if further research backs their initial assessment, sites such as RateMyProfessors.com could be used to help inform decisions on hiring and promoting faculty members.

That would be a radical shift — although it is hard to contest the idea that honest feedback is a valid metric for teaching performance. But for the ratings sites to gain true credibility, some degree of policing is needed. The raters' background and study course should be revealed, and only students who actually attended the relevant class should be allowed to post a comment on it. As for the relevance of how 'hot' a professor is, this will just have to be left up to the individual discerning student.