Log in

Secrets of scientific publishing - revealed! - In the Autoclave [entries|archive|friends|userinfo]
What's in the Autoclave? A Blog about Science

[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Secrets of scientific publishing - revealed! [Apr. 5th, 2008|11:24 pm]
What's in the Autoclave? A Blog about Science
Earlier this week, Yale's office of postdoctoral affairs sponsored a guest seminar by one of the editors of the scientific journal Nature, entitled "How to Publish in Nature." For those not in science careers: Nature, along with Science, is considered one of the most prestigious journals in which to have your research published. Both Science and Nature are interdisciplinary journals, covering topics ranging from molecular biology to atomic physics to environmental sciences and more. They are considered "high profile" journals because of their interdisciplinary nature, and publishing in these journals is a good way to be sure your findings will be noticed. But because they're "high profile" journals, they receive a large number of submissions -- and only a small fraction of them will actually see print, after going through a process called peer review and after being evaluated for the potential impact of the work.

I thought it would be interesting to pass along some of the insights that the editor revealed to us, in terms of how the "process" of scientific publishing works. The scientists reading this might be interested in the nuts and bolts of the process for their own work, but I thought it might also be interesting to non-scientists. A similar process occurs in all scientific journals, although the bar is set somewhat higher for a journal like Nature. But the peer review process is central to how modern science works: it is one of the major "checkpoints" to evaluate research. This isn't to say that all peer reviewed science is good science, or that all the conclusions of every published paper are correct -- but getting your work published is one of the most important places your work gets evaluated by other researchers: the process involves a step in which the authors receive feedback and get a chance to improve their work, and truly sloppy work will usually be unable to pass this checkpoint.

The following information is drawn heavily from the notes that I took during the talk. Unfortunately, the powerpoint projector was not working, so I didn't catch the Nature editor's name, and it is entirely possible that I misheard some of the statistics she quoted on (for instance) number of papers received by the journal monthly. I can probably find out who the source is if anyone's that bothered about it... sorry for being sloppy about citing my sources!

The purpose of a scientific journal, and what it takes to get into a journal like Nature

Journals exist to validate (through peer review), disseminate, archive and publicize the results of scientific studies. Two of the most high-profile journals in the sciences today are Nature and Science. Nature was founded in 1869 by Charles Darwin in England, while Science was founded in America by Alexander Graham Bell. (Science is published by the American Association for the Advancement of Science; Nature is part of the Nature Publishing Group and is, I believe, a private company. I'm not 100% sure of this, though.) Although founded on opposite sides of the Atlantic, both of these journals are international in scope and have similar formats or goals. Both are broad, interdisciplinary journals which aim to publish studies which (according to the editor) meet the following criteria:

1) They are conceptually novel
2) They are technically elegant
3) They address an outstanding question in their field
4) They open a new direction for further research
5) They have interdisciplinary interest

That is to say, papers in these journals are meant to be groundbreaking work rather than an incremental advance in a field, they should be scientifically sound, and they should be of broad interest. For example, Nature has published results such as Watson and Crick's description of the double-helical structure of DNA, and Science published one of the earliest papers describing the BRCA2 breast-cancer susceptibility gene. This isn't to say that the editors of these journals haven't had some notable misses. The Nature editor told us that the journal rejected -- without review (i.e. without even sending the papers to peer reviewers) -- the original paper describing the Krebs cycle (one of the most important biochemical pathways in metabolism) and Fermi's paper describing beta decay (a form of radiation) which went on to win the Nobel. (At this point, the audience laughed.)

As an aside, there is a bit of a hierarchy in journal publishing. Nature and Science are the top tier, but below that there are various degrees. The second tier would be more specialized journals; in my field, Immunity and Nature Immunology are considered very high-quality journals. Below that would be journals like Journal of Experimental Medicine and Journal of Immunology. There are dozens of smaller journals below these, some with a quite narrow focus (for instance, Arthritis and Rheumatism. Different journals have different types of research that they'll focus on... for instance, I've noticed that the Journal of Experimental Medicine is heavy on cellular immunology; it's rare to see a JEM paper that doesn't include lots of flow cytometry data. The Journal of Immunology publishes a lot of work in a lot of subfields of immunology, but is still considered a "good" journal and most immunologists will at least skim the table of contents of each issue. This hierarchy really has more to do with how many people will see the work than anything else -- at the end of the day, good research is good research, wherever it is published, and it will be noticed (eventually); and journals like Science and Nature do at times (despite the best efforts of the editors and peer reviewers) publish things which turn out to be wrongly-interpreted.

How a Bill Becomes a Law How a Manuscript Gets to Print

So, imagine you're a scientist with some really interesting data that you think tells an interesting story about some biological (or physical, or chemical) problem. You write up your results and your conclusions, and are ready to submit your manuscript for publication. You've followed the journal's submission guidelines and FedEx'ed the manuscript (or, more often these days, uploaded the PDF.) Where does it go from there?

At Nature, an incoming manuscript is assigned to a primary editor based on the manuscript's subject and that editor's areas of expertise. Nature has (if I heard correctly) 16 editors in the biological sciences and 8 in the physical/chemical sciences, each with a different base of knowledge. The editors are PhD scientists themselves, who work full-time for the journal. The editor who is assigned the paper will read it and decide if it meets the above criteria; at Nature as many as 75% of papers are rejected at this stage. The remaining papers are assigned peer reviewers (or "referees.")

How are reviewers chosen? Experienced editors will have travelled to scientific conferences and generally have a working knowledge of which other researchers are qualified to evaluate a piece of work. They may ask other editors for suggestions as well. Importantly, an effort is made to be aware of all the interpersonal conflicts that too often exist in science; a reviewer will try to avoid assigning review of a paper to someone known to be overtly antagonistic to the author of a manuscript and will also not assign a paper to someone with whom the authors have collaborated directly in the past, to avoid conflicts of interest and bias (positive or negative.) Additionally, in the cover letter for a manuscript submission, the authors are allowed to make suggestions of who might be qualified to evaluate the work (which the editors may or may not use) and -- most of all -- who should not review a work. Authors are allowed to explicitly ask that up to three or four other investigators not be assigned to review their paper, and the editors will honor that type of request with no questions asked. The goal here is to find referees who do not have positive or negative bias toward the author of a work, who can evaluate the quality of the work on its own scientific merit. It's an imperfect system, to be sure, but it works for the most part -- and while there is currently some debate on the merits of other methods of review, other systems have serious weaknesses as well (discussed below.)

In any case, the editor chooses three or more reviewers -- a truly interdisciplinary paper may require more -- who read and evaluate the manuscript. This process is supposed to occur within a short time frame (about a week) but the reviewers are working scientists themselves and it often can take longer for a paper to go through preliminary review. The reviewers return to the editor a written evaluation of the paper, summarizing its relevance, the quality of the work, and whether or not the conclusions are supported by the data presented. The reviewers may recommend that the manuscript be accepted for publication by the journal, that it be rejected by the journal, or (perhaps most common) that it be accepted with revisions. That is to say, the reviewers may request that additional (specific) experiments be performed to address any scientific weaknesses in the study. The editor then considers the reviews -- in the case of a split decision by the referees, sometimes the editor may even swap reviews among the referees, asking the other referees (for example) if a bad review has merit (i.e. whether the "good" reviewers may have missed something.) In any case, some manuscripts will fail at this stage, and the authors will then have to submit to a different journal. Many manuscripts will require additional experiments; if so, the authors generally will try to do the requested experiments in a short time frame or provide a written rebuttal of the review (sometimes rewriting part of the manuscript to clear up any misinterpretations.) On resubmission, the editor and the reviewers decide whether the changes that were made were sufficient to address the reviewers' concerns.

In recent years, the peer review system has been debated in the scientific community. Currently, reviewers are anonymous to the authors, but the authors are known to the reviewers. Some have argued that reviews should not be anonymous (and believe me, after your first bad review, you have a burning desire to know who that reviewer was and why they didn't see the brilliance of your work!) while others argue that the system should be double-blinded (to prevent reviewers, who often know the authors whose work they are reviewing, from being unfairly influenced by the authors' reputations.) The Nature editor briefly addressed these alternate systems, and argued that non-anonymous review does not work well because it encourages "bland, timid reviews." She said that Nature did a trial of an online "open-access" online review format in which reviewers were required to give their names, and they found that they got a greater percentage of positive reviews than when the same papers were reviewed in the traditional anonymous system. Further, she argued that the anonymous system "corrects for power imbalances" such as situations in which a younger, less established investigator must evaluate the work of a "big name" in the field. (Just because an Einstein writes a paper doesn't mean it should be accepted blindly, right?) On the other end, she argued that a double-blind system also does not work well on a very practical level: generally speaking, when you're writing a paper you are building on your own previous work, and so you cite your previous work. It would be ridiculously easy for a referee to figure out whose paper they are reviewing, even if the name of the authors are formally left off. While the current system of peer review may have its weaknesses, it is probably better than the alternatives that have so far been proposed.

So, you got rejected by Nature

The editor ended with a brief description of what not to do when your paper gets rejected by Nature. This section really applies to any manuscript submission, scientific or otherwise, and it ought to be common sense...

1) Wait. Under no circumstances contact the editor until after you've cooled down and slept on it.
2) Wait more.
3) Then contact the editors, but be patient -- they get a lot of correspondence, and it may take them time to get back to you. It is considered acceptable to argue the merits of your work on scientific grounds, and it is also acceptable to supply additional data -- particularly if it addresses the concerns of the reviewers. It is also acceptable to politely point out factual errors made by reviewers. Odds are you aren't going to change the editor's mind, but if you must make a fuss, this is the only acceptable way to do it.

At this point the editor read some actual correspondence she'd received from disappointed manuscript authors. Suffice it to say, many of them did not help their case.

But at the end of the day, not getting a paper into Nature really is not the end of the world. Read the reviews of your manuscript, address the reviewers concerns, and submit to a different journal. The most important trick to getting published in any journal is to be doing good science. Design your experiments well, carry them out with care, and draw logical conclusions from the results -- and the rest will sort itself out. Getting a paper in Nature is a huge boost to a budding career, but if will only happen if you're doing good work in the first place.

[User Picture]From: celli
2008-04-06 03:47 am (UTC)
I agree, wait and then wait more are what EVERYONE should do. :)
(Reply) (Thread)