# Academics, what comes after peer review? **URL:** \ **Category:** Factual Questions\ **Created:** [January 22, 2017, 10:47pm UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888 "2017-01-22T22:47:25Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![md2000](https://avatars.discourse-cdn.com/v4/letter/m/73ab20/32.png) [@md2000](https://boards.straightdope.com/u/md2000)\ **Post date:** [January 23, 2017, 10:24pm UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/21 "2017-01-23T22:24:10Z")
> [@Quartz](#): > > As I understand it, peer review simply means that the paper isn’t obviously hogwash and is of actual interest. Reviewers don’t try to replicate results etc. This may also be different in sciences like mathematics. So what are the next steps in the academic process? > > Fight my ignorance here! In case it wasn’t obvious - from what I’ve read: you submit your paper to some journal (or conference). Peer review is arranged by the publisher/conference organizer, who gets a trusted group of people in the field to read the paper, ensure that it looks correct, not obvious flaws or misinterpretation of data, math errors, etc. If it passes, it gets published in the journal/presented and published in the conference proceedings. It’s not the peers’ job to verify the paper, simply to say “I can’t see any flaws in what they’ve done”) there are occasional accusations or mutterings of poor reviews due to bias or rivalry, delays so the reviewer in the same field can pre-publish his similar paper, etc. Peer review is not intended to fully validate the findings, but to ensure the publication meets the standards of the field with no obvious flaws. Another point is that some publications are valued (for hiring and promotions) based on the number of citations the paper receives. One thing I read joked that it was better to publish a controversial or slightly wrong paper, as so many people would cite that paper in attempting to refute it. (A publication references previous related work is relevant or applies to the topic at hand, in citations at the end and in footnotes) Another joke, one of my in-laws did work in CERN, and mentioned that anyone who had a finger in the pie gets listed as author. So CERN papers, with dozens fo groups participating, will have a hundred or more “authors”. They are typically listed in alphabetical order, so having a surname starting with "Ab’ helps, since for brevit, the cite would be "_by A. Abalone et al"._ ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [January 23, 2017, 11:24pm UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/22 "2017-01-23T23:24:00Z")
IIRC, one of the recent big papers from CERN (the Higgs Boson one, maybe?) was “Aarne et al.”. So even “Abalone” might not be a good enough name. ---
**Author:** ![bonzer](https://avatars.discourse-cdn.com/v4/letter/b/45deac/32.png) [@bonzer](https://boards.straightdope.com/u/bonzer)\ **Post date:** [January 23, 2017, 11:30pm UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/23 "2017-01-23T23:30:34Z")
> [@md2000](#): > > So CERN papers, with dozens fo groups participating, will have a hundred or more “authors”. They are typically listed in alphabetical order, so having a surname starting with "Ab’ helps, since for brevit, the cite would be "_by A. Abalone et al"._ Trivially small these days; the Higgs discovery paper had [5,154 co-authors](http://www.nature.com/news/physics-paper-sets-record-with-more-than-5-000-authors-1.17567). Even back in the day when such papers merely had dozens of authors, everybody knew that who happened to be first author alphabetically was meaningless. Yes, the first authors did have a certain celebrity amongst those reading the papers, but purely on the basis that it was obviously the spurious distinction. ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [January 24, 2017, 12:06am UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/24 "2017-01-24T00:06:04Z")
Even in smaller groups, first author might or might not be the most prestigious position. In some disciplines, the guy who actually did most of the work is more likely to be the second author, or the last. I know that in relativity, at least, a two-author paper is likely to be a grad student (first) and his advisor (second), and decide for yourself which of those two actually has a better claim to authorship. ---
**Author:** ![LSLGuy](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lslguy/32/5813_2.png) [@LSLGuy](https://boards.straightdope.com/u/LSLGuy)\ **Post date:** [January 24, 2017, 1:16am UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/25 "2017-01-24T01:16:51Z")
The grad student? Only slightly kidding. Who had the idea for a question and who did the work to create/prove the answer are both worthwhile questions. ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [January 24, 2017, 3:35am UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/26 "2017-01-24T03:35:53Z")
Which is why I didn’t try to claim either side. ---
**Author:** ![glowacks](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/glowacks/32/5548_2.png) [@glowacks](https://boards.straightdope.com/u/glowacks)\ **Post date:** [January 24, 2017, 5:01am UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/27 "2017-01-24T05:01:17Z")
> [@Oukile](#): > > What happens to your research after publication is largely a question of how interesting it is. Some research articles are indeed of little value to anyone and will end up entirely forgotten. Some will be of direct relevance of many other labs and/or will have practical applications and the results will be reproduced many time. Some papers that established novel and important findings will become classics and will go on being read and cited for decades. > > People often wonders if scientists can publish wrong results and get away with it because no one tries to reproduce them (or no one will say something if they can’t reproduce them). The answer is: “only if no one cares about these results”. > **[Sharkovskii's theorem](https://en.wikipedia.org/wiki/Sharkovskii's_theorem)** > > In mathematics, Sharkovskii's theorem (also spelled Sharkovsky's theorem, Sharkovskiy's theorem, Šarkovskii's theorem or Sarkovskii's theorem), named after Oleksandr Mykolayovych Sharkovsky, who published it in 1964, is a result about discrete dynamical systems. One of the implications of the theorem is that if a discrete dynamical system on the real line has a periodic point of period 3, then it must have periodic points of every other period. > For some interval > > > > I > ... The story as I learned it as an undergrad: Sharkovskii publishes his theorem in the Ukrainian Journal of Mathematics. No one outside the area reads it. Some years later, a prominent American researcher publishes a paper proving a very small special case of Sharkovskii’s Theorem (that period 3 points imply points of all periods), and its succinctness and applicability are such that he wins some award. Part of the moral was that it is much more interesting to have a small easily understandable result, and get published in leading journals, than to provide a huge amount of insight that leads to a very complicated, but thorough, statement of the nature of reality, that only gets published in a local journal. Looking it up in Wikipedia, apparently the latter paper made a second claim beyond the first that was the special case of S’s Theorem, on a topic that wasn’t considered by Sharkovskii, namely that in the case of a period 3 point the map will exhibit properties they termed “chaos”. This latter point was probably why they got so much attention, as it started more research in chaos theory. But still, one might say that Sharkovskii proved far far more, but no one was interested in what seemed like just an ordering of integers giving you implications of periodic points. ---
**Author:** ![chorpler](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chorpler/32/2888_2.png) [@chorpler](https://boards.straightdope.com/u/chorpler)\ **Post date:** [January 24, 2017, 6:03am UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/28 "2017-01-24T06:03:21Z")
> [@Chronos](#): > > Even in smaller groups, first author might or might not be the most prestigious position. In some disciplines, the guy who actually did most of the work is more likely to be the second author, or the last. I know that in relativity, at least, a two-author paper is likely to be a grad student (first) and his advisor (second), and decide for yourself which of those two actually has a better claim to authorship. Huh. With all of the bioinformatics papers I wrote or was involved with, the first author would be the student or research assistant who did most of the actual work (and was responsible for actually writing at least the draft of the paper), then the rest of the people who worked on the project/application/whatever would be listed, and the advisor / professor would always be the last author. So I got into the habit of basically only looking at the first and last author names… ---
**Author:** ![Francis\_Vaughan](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/francis_vaughan/32/3093_2.png) [@Francis\_Vaughan](https://boards.straightdope.com/u/Francis_Vaughan)\ **Post date:** [January 24, 2017, 9:05am UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/29 "2017-01-24T09:05:27Z")
Yeah, it really does depend. We used to give the person that did the work actually writing the paper the first spot. Usually this was the person that did the real research work anyway. If there was more than one additional author the remainder of the list was alphabetical. Papers that were true multi-author papers, we ran the author list in only alphabetical order. It is hard to come up with fair rules, and attempts to do fine grained ordering are guaranteed to end in tears. When it comes to grad-student papers you can have just about the full spectrum of contribution. The brilliant student that the supervisor merely has to guide the work of, right through to the useless student where nearly every idea and every sentence has to be gone over multiple times with the student to beat some sense and readability into it (and the student.) ---
**Author:** ![Colibri](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/colibri/32/1841_2.png) [@Colibri](https://boards.straightdope.com/u/Colibri)\ **Post date:** [January 24, 2017, 4:56pm UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/30 "2017-01-24T16:56:30Z")
> [@md2000](#): > > Another point is that some publications are valued (for hiring and promotions) based on the number of citations the paper receives. One thing I read joked that it was better to publish a controversial or slightly wrong paper, as so many people would cite that paper in attempting to refute it. (A publication references previous related work is relevant or applies to the topic at hand, in citations at the end and in footnotes) Right. It isn’t just a numbers game based on the number of publications you have. Hiring and tenure decisions are based on your [citation index](http://libguides.utoledo.edu/citation/wos), the number of other articles that have cited you. (Citing yourself doesn’t count in the totals, and there is also an issue that citation indexes may not include book chapters and other non-journal publications.) Publishing a controversial or wrong paper can be a good strategy to get cited a lot. But the champions are papers describing methods that get cited by everyone using that method. The [all time champ](http://www.nature.com/news/the-top-100-papers-1.16224) is a 1951 paper by Lowry et al on a method of measuring proteins that now has more than 300,000 citations. Another factor taken into account is the [Journal Impact Factor](http://www.scijournal.org/?gclid=Cj0KEQiAk5zEBRD9lfno2dek0tsBEiQAWVKyuCNa50Pow-soiptZycOTLpqMxUb0TwYAJh_kvrtJnO0aAi-z8P8HAQ) of the journal where it was published. Publishing in a prestigious journal like Science or Nature is worth far more than publishing in a small local journal. ---
**Author:** ![md2000](https://avatars.discourse-cdn.com/v4/letter/m/73ab20/32.png) [@md2000](https://boards.straightdope.com/u/md2000)\ **Post date:** [January 24, 2017, 5:37pm UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/31 "2017-01-24T17:37:09Z")
> [@Chronos](#): > > Even in smaller groups, first author might or might not be the most prestigious position. In some disciplines, the guy who actually did most of the work is more likely to be the second author, or the last. I know that in relativity, at least, a two-author paper is likely to be a grad student (first) and his advisor (second), and decide for yourself which of those two actually has a better claim to authorship. > [@LSLGuy](#): > > The grad student? Only slightly kidding. Who had the idea for a question and who did the work to create/prove the answer are both worthwhile questions. As opposed to the guy who got a Nobel prize for being the one who grudgingly provided lab space. > [@](#): > > In the spring of 1921, Banting travelled to Toronto to visit J.J.R. Macleod, professor of physiology at the University of Toronto, and asked Macleod if he could use his laboratory to isolate pancreatic extracts from dogs. Macleod was initially sceptical, but eventually agreed before leaving on holiday for the summer. Before leaving for Scotland he supplied Banting with ten dogs for experiment and two medical students, Charles Best and Clark Noble, as lab assistants. > … > Banting realized that this procedure would destroy the trypsin-secreting cells but not the insulin. Once the trypsin-secreting cells had died, insulin could be extracted from the islets of Langerhans. Banting discussed this approach with J. J. R. Macleod, Professor of Physiology at the University of Toronto. Macleod provided experimental facilities and the assistance of one of his students, Dr. Charles Best. Banting and Best, with the assistance of biochemist James Collip, began the production of insulin by this means > … > Since Banting required only one assistant, Best and Noble flipped a coin to see which would assist Banting first. Best won and took the first shift. > … > Although all the team members were listed as co-authors of their publications, Banting still felt overlooked, because Macleod took over the coordination of clinical trials and the acquisition of larger amounts of extract. Macleod’s presentation at a meeting of the Association of American Physicians in Washington, D.C., on 3 May 1922 received a standing ovation, but Banting and Best refused to participate in protest. > … > Banting … felt sidelined. By the winter of 1922, he was certain that all Macleod’s colleagues were conspiring against him. > … > In 1923, the Nobel Prize Committee honoured Banting and J. J. R. Macleod with the Nobel Prize in Medicine for the discovery of insulin, ignoring Best and Collip. ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [January 24, 2017, 9:41pm UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/32 "2017-01-24T21:41:38Z")
Of course, sometimes authorship is listed [_strictly_ in alphabetical order](https://en.wikipedia.org/wiki/Alpher%E2%80%93Bethe%E2%80%93Gamow_paper). ---
**Author:** ![ftg](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/ftg/32/2801_2.png) [@ftg](https://boards.straightdope.com/u/ftg)\ **Post date:** [January 24, 2017, 9:42pm UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/33 "2017-01-24T21:42:09Z")
> [@Colibri](#): > > Right. It isn’t just a numbers game based on the number of publications you have. Hiring and tenure decisions are based on your [URL=“[Web of Science (formerly ISI Citation Indexes) - Citation Analysis - LibGuides at University of Toledo Libraries](http://libguides.utoledo.edu/citation/wos)”] I’ve never experienced this. I tracked down my citation counts for my top papers and no one thought anything about them at all. One paper alone had more citations than most people in my department had for all their papers. It was (officially) number of journal papers and what the outside letter writers thought. Unofficially it was always just total amount of grant money. (Which is against clearly written policy but …) One compromise I’ve heard about (but apparently incredibly rare) is the candidate lists their 5 best papers and they are judged on those alone. People with a 100 crappy papers lose out, people with a few gems get a boost. Quality over quantity, who ever thought that that might work? ---
**Author:** ![Colibri](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/colibri/32/1841_2.png) [@Colibri](https://boards.straightdope.com/u/Colibri)\ **Post date:** [January 24, 2017, 11:35pm UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/34 "2017-01-24T23:35:03Z")
> [@ftg](#): > > I’ve never experienced this. I tracked down my citation counts for my top papers and no one thought anything about them at all. One paper alone had more citations than most people in my department had for all their papers. > > It was (officially) number of journal papers and what the outside letter writers thought. Unofficially it was always just total amount of grant money. (Which is against clearly written policy but …) > > One compromise I’ve heard about (but apparently incredibly rare) is the candidate lists their 5 best papers and they are judged on those alone. People with a 100 crappy papers lose out, people with a few gems get a boost. > > Quality over quantity, who ever thought that that might work? I don’t actually sit on the committees that make these decisions around here, but people certainly talk about this all the time. Post-docs looking for a job are especially worried about it. ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [January 25, 2017, 1:14am UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/35 "2017-01-25T01:14:32Z")
I’ve also seen researchers measured by a blended metric, usually called “impact factor” (though that can have other meanings, too): An author’s impact factor is the largest number _n_ such that they have at least _n_ papers which have each gotten at least _n_ citations. So, for instance, if an author’s papers have gotten 10, 6, 3, 2, 1, 1, 1, 0 citations, then that author has an impact factor of 3, since they have 3 papers with 3 or more citations. If their next paper turns out to be a blockbuster and gets 73 citations, then their impact factor will still only be 3 (it can’t be 4 since their 4th-best paper still only has 3 citations). This way, you won’t be unduly rewarded just because you happened to have one really interesting idea, if that’s all you ever do, and you likewise won’t be rewarded for cranking out a paper each week that each only get a single citation. ---
**Author:** ![Enginerd](https://avatars.discourse-cdn.com/v4/letter/e/bcef8e/32.png) [@Enginerd](https://boards.straightdope.com/u/Enginerd)\ **Post date:** [January 25, 2017, 1:22am UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/36 "2017-01-25T01:22:27Z")
> [@Chronos](#): > > I’ve also seen researchers measured by a blended metric, usually called “impact factor” (though that can have other meanings, too): An author’s impact factor is the largest number _n_ such that they have at least _n_ papers which have each gotten at least _n_ citations. So, for instance, if an author’s papers have gotten 10, 6, 3, 2, 1, 1, 1, 0 citations, then that author has an impact factor of 3, since they have 3 papers with 3 or more citations. If their next paper turns out to be a blockbuster and gets 73 citations, then their impact factor will still only be 3 (it can’t be 4 since their 4th-best paper still only has 3 citations). > > This way, you won’t be unduly rewarded just because you happened to have one really interesting idea, if that’s all you ever do, and you likewise won’t be rewarded for cranking out a paper each week that each only get a single citation. I’ve always seen this called “h-index” - that’s also what Google Scholar calls it in their citation indices. ---
**Author:** ![IvoryTowerDenizen](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/ivorytowerdenizen/32/17937_2.png) [@IvoryTowerDenizen](https://boards.straightdope.com/u/IvoryTowerDenizen)\ **Post date:** [January 25, 2017, 1:34am UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/37 "2017-01-25T01:34:11Z")
We have to provide journal acceptance rates, h scores, and/or impact factor for tenure and promotion. ---
**Author:** ![Busy\_Scissors](https://avatars.discourse-cdn.com/v4/letter/b/d78d45/32.png) [@Busy\_Scissors](https://boards.straightdope.com/u/Busy_Scissors)\ **Post date:** [January 25, 2017, 9:56am UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/38 "2017-01-25T09:56:53Z")
The h-index is pretty important in my field. Unsurprisingly, it’s a popular metric with scientific bureaucrats - boiling a scholar’s work down to a single number, how nice. It’s less of a big deal in some fields - certainly, no self-respecting mathematician ever gave a fuck about their h-index, but in the chemical and life sciences it’s used a lot. Despite its inherent fatuousness, it’s hard to deny it as a necessary condition of being an influential scientist, because how could anyone lay claim to that if no one’s reading and citing your work? It is not a sufficient condition, though - I know a few folk with h-indices of 40-50 (v respectable level in my area) who are fairly pedestrian. Hard workers who keep busy in thriving areas, but aren’t especially creative. There are some exceptions where people have extremely low h-indices but have done amazing, Nobel prize level work. Either they died young, or more tragically, worked in industry. But you’re talking about a v small minority, esp these days. ---
**Author:** ![Francis\_Vaughan](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/francis_vaughan/32/3093_2.png) [@Francis\_Vaughan](https://boards.straightdope.com/u/Francis_Vaughan)\ **Post date:** [January 25, 2017, 10:30am UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/39 "2017-01-25T10:30:09Z")
> [@Busy\_Scissors](#): > > It’s less of a big deal in some fields - certainly, no self-respecting mathematician ever gave a fuck about their h-index No, but they do care about their [Erös number](https://en.wikipedia.org/wiki/Erd%C5%91s_number) 😃 Mine is 4 ---
**Author:** ![Wendell\_Wagner](https://avatars.discourse-cdn.com/v4/letter/w/8491ac/32.png) [@Wendell\_Wagner](https://boards.straightdope.com/u/Wendell_Wagner)\ **Post date:** [January 25, 2017, 10:49am UTC](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888/40 "2017-01-25T10:49:57Z")
Your Eros number is the number of connections it takes to get to Paul Erdős where a connection is having had sex with the person. If you had sex with him, your Eros number is one; if you had sex with someone who had sex with him, your Eros number is two; if you had sex with someone who had sex with someone who had sex with him, your Eros number is three; etc. Since he died a virgin, everyone’s number is infinite. [Previous page](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888.md?page=1) [Next page](https://boards.straightdope.com/t/academics-what-comes-after-peer-review/777888.md?page=3)