How many non-peer-reviewed publications should a scientist produce?

Peer-reviewed writing moves science forwards; non-peer-reviewed writing moves science sideways.  

That’s my publication philosophy in one sentence.  In other words, when scientists write research papers and book chapters that are peer-reviewed, the underlying rationale is that we are adding to the sum total of human knowledge, providing insights into a topic, and moving a field forwards. When we write non-peer-reviewed articles we are generally writing about science for a broader audience, with little original content (though perhaps with some original ideas).  This moves concepts out of a narrow subject area and into the purview of wider society, which can be other scientists in different fields, or government agencies or policy makers, or the general public.

There can be exceptions to the rule, such as the IPBES pollinators and pollination report that I’ve been discussing this year. The report was widely peer-reviewed but is intended for a much broader audience than just scientists.  Conversely, non-peer-reviewed critiques and responses to published papers can clarify specific issues or challenge findings, which will certainly move science forward (or backwards into muddier waters, depending on how you view it).  However, in general, the principle stated above holds true.

This raises the (admittedly clunky) question I’ve posed in the title of this post: just how much non-peer-reviewed publication should a scientist who is an active researcher actually do?  How much time should they spend writing for that wider audience?

It’s a question that I’ve given some thought to over the 30 years1 that I’ve been writing and publishing articles and papers.  But a couple of posts on other blogs during the past week have crystalised these thoughts and inspired this post.  The first was Meghan Duffy’s piece on Formatting a CV for a faculty job application over at the Dynamic Ecology blog. There was some discussion about how to present different types of publications in the publication list, and notions of “sorting the wheat from the chaff” in that list, which seemed to refer to peer-reviewed versus non-peer-reviewed publications.

One of the problems that I and others see is that the distinction is not so clear cut and it’s possible to publish non-peer-reviewed articles in peer-reviewed journals.  For example the “commentary” and “news and views” type pieces in NatureScience, Current Biology, and other journals are generally not peer reviewed.  But I’d certainly not consider these to be “chaff”.  To reiterate my comment on Meghan’s post, all scientific communication is important.  As I’ve discussed in a few places on my blog (see here for example) and plenty of others have also talked about, scientists must write across a range of published formats if they are going to communicate their ideas effectively to a wider audience than just the scientists who are specifically interested in their topic.

Peer-reviewed publication is seen as the gold standard of science communication and it is clearly important (though historically it’s a relatively recent invention and scientific publications were not peer reviewed for most of the history of science).  So why, you may be asking, would scientists want to write for that wider audience?  One reason is the “Impact Agenda” on which, in Britain at least, there’s been a huge focus from the Research Excellence Framework (REF) and the Research Councils. Grant awarding bodies and university recruitment panels will want to see that scientists are actively promoting their work beyond academia. That can be done in different ways (including blogging!) but articles in “popular” magazines certainly count.  I should stress though that this wider, societal impact (as opposed to academic impact, e.g. measures such as the h-index) is not about publishing popular articles, or blogging, or tweeting. Those activities can be part of the strategy towards impact but are not in themselves impactful – the REF would describe this as “Reach”2.

The second recent blog post that relates to the question of peer-reviewed versus non-peer-reviewed publications is Steve Heard’s piece at Scientistseessquirrel on why he thinks it’s still important to consider journal titles when deciding what to read.  He makes some important points about how the place of publication says a lot about the type of paper that one can expect to read based just on the title.  But the focus of Steve’s post is purely on peer-reviewed journals and (as I said above) it’s possible to publish non-peer-reviewed articles in those.  I think that it’s also worth noting that there are many opportunities for scientists to publish articles in non-peer-reviewed journals that have real value.  Deciding whether or not to do so, however, is a very personal decision.

Of the 96 publications on my publication list, 65 are peer-reviewed and 31 are not, which is a 68% rate of publishing peer-reviewed papers and book chapters.  Some of the peer-reviewed papers are fairly light weight and made no real (academic) impact following publication, and (conversely) some of the non-peer-reviewed articles have had much more influence. The non-peer-reviewed element includes those commentary-type pieces for Nature and Science that I mentioned, as well as book reviews, articles in specialist popular magazines such as New Scientist, Asklepios and The Plantsman, pieces for local and industry newsletters, and a couple of contributions to literary journal Dark Mountain that combine essay with poetry.  This is probably a more diverse mix than most scientists produce, but I’m proud of all of them and stand by them.

So back to my original question: is 68% a low rate of peer-reviewed publication?  Or reasonable?  I’m sure there are scientists out there with a 100% rate, who only ever publish peer-reviewed outputs.  Why is that?  Do they really attach no importance to non-peer-reviewed publications? I have no specific answer to the question in the title, but I’d be really interested in the comments of other scientists (and non-scientists) on this question.

I had to double check that, because it seems inconceivable, but yes, it’s 30 years this year. Gulp.

Impact is how society changes as a result of the research undertaken.  So, for ecologists, it could be how their research has been translated into active, on-the-ground changes (e.g. to management of nature reserves, or rare or exploited species), or how it’s been picked up by national and international policy documents and then influenced policies on specific issues (invasive species, pollinator conservation, etc.)

15 thoughts on “How many non-peer-reviewed publications should a scientist produce?

  1. ScientistSeesSquirrel

    Great post. Which raises an obvious self-referential question. Do you count your blog posts as non-peer-reviewed publication? Doing so would skew your numbers (as they would mine!). But not doing so seems to disregard what’s a substantial body of work with a great deal of reach (and likely impact). So, why did you decide not to count blog posts? Or did you count your whole blog as one?

    1. jeffollerton Post author

      Thanks Steve. I didn’t count my blog as I don’t see it as a “publication” in the same way as those on the list, i.e. something that has been through some kind of editorial process independently of me as a writer. But in a sense that’s an arbitrary distinction because if I was to produce a self-published book I would probably add it to the list!

      I think you raise a good point and it’s indicative of how much the whole landscape of academic publishing is changing.

  2. Manu Saunders

    Great post and a really good question! In answer, I don’t think there should be a ‘should’ rate – I think it’s a personal decision, as you say. And my personal opinion is that non-peer-reviewed writing does move science forwards, but for a different audience. 

    But I’m early career and my mentors over the years have often brought up the ‘opinion pieces don’t really count’ philosophy. I always struggled with this advice, as I think a scientist’s role is partly about being a philosopher. So I’ve always wondered why ‘shouldn’t’ opinion pieces contribute to science as much as ‘proper’ research papers?

    1. jeffollerton Post author

      Thanks Manu. I certainly take your point that “non-peer-reviewed writing does move science forwards, but for a different audience.”

      The idea that opinion pieces have no value is a curious one and I think a relatively new one that wouldn’t be recognised pre-1960s. The advice of mentors is there to be ignored, of course 🙂

  3. David Waltner-Toews

    Not sure the distinctions are useful, at least as framed. I have about 100 peer reviewed and at least an equal number of non-peer reviewed (which I lost count of over the past few decades) papers. Some of the books I’ve edited & book chapters I’ve written (and which are not included in either category) received much more intensive review from peers than any journal papers. Peer reviewed science tends to be more conservative (as it should be) and hence doesn’t “advance” science so much as consolidate and verify. Non-peer reviewed is more innovative and hypotheses-generating (as Popper would classify it) and hence pushes scientists to explore new frontiers.

    1. jeffollerton Post author

      Thanks for your comment David. Just to clarify, I was including book chapters that have been peer-reviewed in my “peer-reviewed” category; as you say, the level of review that they receive can be just as high as for journals.

      With regard to your last point, I guess it depends on the outlet for the non-peer-reviewed science: articles in popular science or horticulture magazines, for instance, would not tend to be especially innovative, though I’m sure there are exceptions.

  4. David Waltner-Toews

    I had been thinking of the first reports I read about links between toxoplasmosis and schizophrenia (my specialty is epidemiology of zoonoses). Although this is “generally accepted” now at least as a working hypothesis, I thought at the time that this was pure speculation. Now that I look back however, I see that the idea was floating around for a while in a variety of peer-reviewed journals I didn’t normally read. Even journals with names like “Speculations in Science and Technology” are peer-reviewed. Not sure what that means. I guess I am wary as there are so many journals that declare themselves to be peer-reviewed. Still, I’m not sure we have anything better.

    1. jeffollerton Post author

      That’s a fascinating subject, David, and I have a minor interest in this broad area, and specifically the “hygiene hypothesis”, having published a short paper with William Parker at Duke a few years ago:

      Parker, W. & Ollerton, J. (2013) Immunology enlightened by evolutionary biology and anthropology: an approach necessary for public health. Evolution, Medicine, and Public Health 2013(1): 89-103 doi:10.1093/emph/eot008

      It’s also the subject of one of my favourite personal anecdotes: before my wife and I decided to merge homes and live together (this was in 2011) I had always been very anti-cat, couldn’t see their attraction, was much more of a dog person, and actually allergic to felines. But my wife came with two cats that she adored. I agreed to put up with them and within weeks I was absolutely smitten; now I just can’t imagine not living with cats.

      I assumed that it was just a psychological effect of getting used to them. Then I watched a documentary about how toxoplasmosis affects human behaviour and now I wonder whether my change was actually due to infection. Very speculative, of course 🙂 But my allergy disappeared too.

  5. Gregor

    Hey Jeff,

    thanks a lot for this nice blogpost.

    I have just published my first (non-peer-reviewed) book review (in a peer-reviewed, society-run journal) and I am really asking myself if anybody is reading these pieces at all (apart from the book review editor). The feedback on Twitter and Researchgate, where I share ALL my publications was absolutely zero compared to the two peer-reviewed articles that I also just published during the last couple of weeks which got many reads on RG and many shares and likes on Twitter (maybe this is partially explained as the two peer-reviewed articles were published in higher impact factor journals than the book review). Hence my very personal, very anecdotal, and very recent experience is that non-peer reviewed publications are not worth the time investment.

    But I will probably give it another try sooner or later.

    “The advice of mentors is there to be ignored, of course”
    I love this one 🙂

    1. jeffollerton Post author

      Thanks Gregor. Book reviews are an interesting category of non-peer-reviewed publication and people treat them in different ways (see Simon Leather’s comment about how he doesn’t list them on his CV, for instance). You can be fairly sure that the author(s) of the book will have read it though!

      I’m not sure that social media is a particularly good way to determine whether or not a publication is read; it would be worth asking the journal how many time it’s been downloaded from the site. Also send the review to people who you think might want to read it – I don’t know if you’ve seen this post of mine from last year but it may be of interest:

      Don’t give up, though, and try other types of non-peer-reviewed writing!

  6. sleather2012

    Very interesting post Jeff – to add to your numbers I have 270 listed scientific publications, 198 of which are peer-reviewed – I don’t list my book reviews but have done at least twenty, so assuming that, my ratio is similar to yours 68%. I am not sure that I agree with your point that non-peer reviewed papers only move science sideways (although I see where you are coming from). Surely if they stimulate research by others then they have moved things forward?

    PS I have always had cats and never liked dogs 🙂

      1. sleather2012

        I don’t know why I don’t list them, I guess I originally never thought of them as being a publication, and as far as Imperial College were concerned there wasn’t a section for them on the annual promotion round cv, so they tended to fall beside the wayside 🙂

  7. Pingback: Elsevier successfully patents a common peer review process | Jeff Ollerton's Biodiversity Blog

  8. David Waltner-Toews

    I always emphasized to my students in courses on epidemiology of foodborne diseases & zoonoses that part of the mandate for many (not all) who work in health related sciences is to actively engage with the public in language that is generally understood. Sometimes this is done by working with “knowledge translators,” but I think the effort needs to be made. I would sometimes ask PhD candidates at their defence, “You have done a great piece of research, but who cares?” It’s not a scientific question, but unless we can respond to it, we will lose public support (not just financial). Elected politicians will be the ones making the important decisions on issues of food safety, epidemics, environment, GMOs, vaccines, etc. and we can all see the serious negative consequences of the kinds of arrogance (we have the Truth you dummies), obfuscation (using specialized dialects), protectionism (not wishing to share data or methods) and confusions between variations of what “is/ is possible” (sciences) and what is desirable (something other than sciences) that some scientists have carried into the public realm. Like all distinctions & categorizations, those between peer-reviewed and non-peer-reviewed are useful, but not absolute.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s