This morning at MiT6 Kathleen Fitzpatrick presented a chapter from her upcoming book (linked here, that examines the role of peer review in new networked publishing, and argues that it may be getting in the way. While it may be getting in the way, I worry that the perception that online publishing, and particularly open publishing, is not peer reviewed has already slowed its adoption. When I suggested adding to our tenure guidelines that we give special preference to open access materials, the first thing voiced by a colleague was “but that wouldn’t be peer reviewed!” The idea is a bit silly–and I was lucky enough to be able to name a small number of open journals that were considered of high quality *and* were peer reviewed–but it is also pervasive. So, while I agree that reform is desirable, to say the least, I think it is a necessary evil if we are to make any progress on opening access, which I think is a first gateway. Once that beachhead is established, then we can make more progress on peer review. (That’s not to say we shouldn’t be experimenting now, just that I think we can get more traction by focusing on open access with peer review at this stage.)
But I was particularly struck by one of her points, reiterated after her presentation. She suggested that tenure review committees would have to actually read materials, rather than rely on publication in high-impact journals, and that letters of support from others in the field would carry more weight.
In an ideal world, I would love for that to happen. In practice, I’m in a program that has traditionally been a film program. The “interactive” piece is litterally tagged on the end (“Film, Video, and Interactive Media”). If anything, this near equal billing poorly reflects our influence, at least for now, on the undergraduate curriculum, for example. But more importantly, I don’t trust my own expertise enough to determine whether a colleague’s documentary film is of high quality. I can say whether I think it is good, but that opinion would have to be taken with a huge chunk of salt.
Now, that it was chosen to be broadcast on public television suggests that it is probably pretty good. That it won an Emmy suggests the same. That it appears in a juried show does as well. These are not “traditional” article peer review, but they are modeled on it, and were written into the tenure guidelines for that reason. Likewise, although I am happy when my film colleagues read my stuff, it isn’t really in their field, and I would be appalled if a journal reviewer picked them to review an article.
Now, maybe this says something about the nature of my own Frankendepartment(s), rather than something about tenure committees’ relience on peer review and impact factors as a proxy for quality. (Impact factors can have pernicious effects, but I think they are an improvement over simply “count up the peer reviewed articles.”) In any case, I suspect the case of my department is far from unique. And it may not be a good thing for colleagues to be the primary judges.
In the long term, this would be the narrowing of focus of departments. At Buffalo, there was an effort–and perhaps still is–to value the “mathematical” study of communication, as some of the faculty called it. That is, not just pushing toward empiricism, but toward quantitative and pure modeling (e.g., game theory). Under those conditions, there would be no way I would get tenure–and perhaps that is a good thing! That sort of focus, however, would mean that you hired more of your own graduates, since they were the ones who knew what they were doing. Group think in the department.
Again, there are real advantages to this. I would love a department that was made up of people who were immersed in social media. And frankly, departments that strategically hire in narrow areas can use this to their advantage in many cases. But systematically, I wonder what it does to ideas of fields and disciplines, and whether that is a good or a bad thing.