NCEAS Project 9241

A quantitative exploration of the role of publication-related biases in ecology

  • Christopher J. Lortie
  • Lonnie W. Aarssen
  • Julia Koricheva
  • Tom Tregenza

ActivityDatesFurther Information
Working Group23rd—31st May 2005Participant List  
Working Group1st—4th May 2006Participant List  
Working Group27th May—4th June 2008Participant List  

Abstract
Progress in a scientific discipline is normally achieved through publication and dissemination of knowledge. Number of publications and their citation frequency are also widely used for academic evaluation of individual researchers, departments, and universities. Therefore, any bias in publication and dissemination of scientific content may potentially affect the development of a field in terms of what kind of information is available for synthesis, who is successfully employed, and where funding is allocated . We will specifically focus on publication bias in ecology in this working group using meta-analysis techniques (and other standard statistics) on several sizeable collections of published papers and related online resources such as citation frequencies and impact factors. We have loosely identified three levels of attributes of the publication and dissemination process in ecology: characteristics of the study (number of hypotheses, effect size, support for main hypothesis), attributes of the publication itself (merit, length, number and gender of authors), and attributes of the journal (reputation, impact factor, circulation). General publication biases identified in medicine and ecology include the file drawer problem, overinterpretation bias, dissemination bias, status bias, visibility bias, and gender bias. Few synthetic studies however have quantitatively tested the importance of these proposed biases nor related these biases to specific attributes of the publication process. Furthermore, there has been no quantitative evaluation of the relative importance and potential interactions between these factors.

TypeProducts of NCEAS Research
Journal Article Aarssen, Lonnie W.; Tregenza, Tom; Budden, Amber E.; Lortie, Christopher J.; Koricheva, Julia; Leimu, Roosa. 2008. Bang for your buck: Rejection rates and impact factors in ecological journals. Open Ecology Journal. Vol: 1. Pages 14-19.
Journal Article Borsuk, Robyn; Budden, Amber E.; Leimu, Roosa; Aarssen, Lonnie W.; Lortie, Christopher J. 2009. The influence of author gender, national language, and number of authors on citation frequency in ecology. Open Ecology Journal. Vol: 2. Pages 25-28.
Journal Article Borsuk, Robyn; Aarssen, Lonnie W.; Budden, Amber E.; Koricheva, Julia; Leimu, Roosa; Tregenza, Tom; Lortie, Christopher J. 2009. To name or not to name: The effect of changing author gender on peer review. BioScience. Vol: 59. Pages 985–989.
Journal Article Budden, Amber E.; Aarssen, Lonnie W.; Koricheva, Julia; Leimu, Roosa; Lortie, Christopher J.; Tregenza, Tom. 2008. Does double-blind review favor female authors? Reply. Frontiers in Ecology and the Environment. Vol: 6(7). Pages 354-355.
Journal Article Budden, Amber E.; Tregenza, Tom; Aarssen, Lonnie W.; Koricheva, Julia; Leimu, Roosa; Lortie, Christopher J. 2008. Double-blind review favours increased representation of female authors. Trends in Ecology & Evolution. Vol: 23(1). Pages 4-6.
Journal Article Budden, Amber E.; Lortie, Christopher J.; Tregenza, Tom; Aarssen, Lonnie W.; Koricheva, Julia; Leimu, Roosa. 2008. Response to Webb et al.: Double-blind review: accept with minor revisions. Trends in Ecology & Evolution. Vol: 23(7). Pages 353-354.
Journal Article Budden, Amber E.; Aarssen, Lonnie W.; Koricheva, Julia; Leimu, Roosa; Lortie, Christopher J.; Tregenza, Tom. 2008. Response to Whittaker: Challenges in testing for gender bias. Trends in Ecology & Evolution. Vol: 23(9). Pages 480-481.
Journal Article Grod, Olyana; Budden, Amber E.; Tregenza, Tom; Koricheva, Julia; Leimu, Roosa; Aarssen, Lonnie W.; Lortie, Christopher J. 2008. Systematic variation in reviewer practice according to country and gender in the field of ecology and evolution. PLoS ONE. Vol: 3(9). Pages e3202.
Journal Article Grod, Olyana; Lortie, Christopher J.; Budden, Amber E. 2010. Behind the shroud: A survey of editors in ecology and evolution. Frontiers in Ecology and the Environment. Vol: 8(4). Pages 187-192.
Journal Article Leimu, Roosa; Lortie, Christopher J.; Aarssen, Lonnie W.; Budden, Amber E.; Koricheva, Julia; Tregenza, Tom. 2008. Does it pay to have a 'bigwig' as a co-author?. Frontiers in Ecology and the Environment. Vol: 6. Pages 410-411.
Journal Article Leimu, Roosa; Lortie, Christopher J.; Tregenza, Tom; Koricheva, Julia; Budden, Amber E.; Aarssen, Lonnie W. 2008. How big are bigwigs?: A reply to Havens. Frontiers in Ecology and the Environment. Vol: 6. Pages 523.
Journal Article Lortie, Christopher J.; Aarssen, Lonnie W.; Budden, Amber E.; Koricheva, Julia; Leimu, Roosa; Tregenza, Tom. 2007. Publication bias and merit in ecology. Oikos. Vol: 116. Pages 1247-1253.
Journal Article Lortie, Christopher J.; Aarssen, Lonnie W.; Budden, Amber E.; Leimu, Roosa. 2013. Do citations and impact factors relate to the real numbers in publications? A case study of citation rates, impact, and effect sizes in ecology and evolutionary biology. Scientometrics. Vol: 94(2). Pages 675-682. (Online version)