15 Million Hours Lost in Peer-Review Process
Many studies receive a rejection response after being submitted to a journal, but the reviews are often not shared among journal reviewers. Even in cases where there has been such sharing, the lack of a standard practice concerning this issue means every journal on the market requires the author to fulfill their own evaluation requests before making a decision. This situation may result in reviewers having to repeat the procedures previously performed by their colleagues for the same study. It is estimated that over 15 million hours are spent on additional evaluations each year.
What do the figures say?
There are two notable metrics for finding the amount of time unnecessarily spent in this process – the quantity metric, for number of studies reviewed and rejected, and the time metric, for how long each submission was peer reviewed. For example, the following shows the results related to the assessments including 12,000 journals.
Each year approximately 3,360,000 submissions are made to Sciences, Technology and Medical Journals published in English.
According to Thompson Reuters data, the average number of submissions per journal is 280, which means that the 280 submissions made to each of the 12,000 journals corresponds to roughly 3,360,000 submissions each year.
The number of submissions in question refers to how many times submissions are made, not to the number of studies performed. To clarify, a study which has been rejected by a journal but accepted by another within a year goes through two review stages, and these stages are considered as two separate submissions.
Each year roughly 1,344,000 (40%) submissions are accepted
Thomson Reuters reports that 37% of all submissions it receives are accepted, while Mark Ware PRC reports this rate as 50% on average. Based on Thomas Reuters data and other data, the average rate of acceptance is 40%, which means roughly 1,344,000 studies are accepted each year.
Factors Affecting Acceptance
- Achieving results supported by strong evidence
- Proposing a new idea or approach and seeking answers to new questions
- Pointing out findings that are extremely important to scientists in a particular field and appealing to researchers from different disciplines
- Having logical and clear arguments that are not self-contradictory, but rather, well-constructed with an easily understandable style and built on a good story
- Addressing an important topic and shedding light on unanswered questions
- Having been prepared with sound research methods
- Establishing an accurate connection with previous studies in the field
- Creating a new theory or developing an existing theory
- Clarifying why and how data affect the results
Each year roughly 705,600 (21%) submissions are rejected WITHOUT REVIEW, while roughly 1,310,400 (39%) submissions are rejected WITH REVIEW
MarkWare PRC reports that the rate of submissions rejected without peer review, which is also known as “desk rejection”, is 21%. In such cases, authors experience both time and opportunity loss and the incurrence of additional costs, and have to try their luck in another journal. Outside the 21% desk rejection rate and 40% acceptance rate, the remaining 39% of submissions are rejected later, which corresponds to 1,310,400 submissions per year.
Desk Rejection Process
In 50% of desk rejections, editors inform authors within one week. However, 17% of the journals inform authors in such cases after a period longer than four weeks. Sometimes this period can even exceed three months, prompting authors to sometimes withdraw their studies.
The desk rejection period of one-third of the journals exceeds two weeks, while that of one-sixth of the journals exceeds four weeks. The average desk rejection period is 10 days for medical journals, 11-12 days for natural sciences, health sciences and engineering journals, and 15-17 days for psychology, social sciences, math, and computer sciences journals.
In some cases, the editor may cite the unsuitability of the study for the journal as the reason for desk rejection, and therefore slow down the procedures such as preparing reports etc. Weakness in the professional working approach of journals can also affect the lengths of time for desk rejection. Accordingly, the journals with the shortest first response times are medical journals, which have the most professional approaches. In addition to the desk rejection period, the first review process and the total review period are also shorter in journals with high impact factors. Thus, the overall review process is more efficient. A more professional organizational structure also has an effect on this process. If the desk rejection period exceeds one month, the evaluation process of the reviewer reports may also be prolonged.
According to an analysis of an evaluation of 3500 manuscripts, 572 (16.3%) manuscripts were rejected without being sent to the reviewer, 693 (19.8%) manuscripts were rejected after the first revision, 2128 manuscripts (60.8%) were accepted after one or more revisions, 43 (1.2%) manuscripts were immediately accepted without peer-review process, and 64 (1.8%) manuscripts were withdrawn by the author.
The peer-review process typically begins with the editor’s acceptance or direct rejection of the study. Studies that pass this first stage are sent to reviewers and decisions are made on whether to accept or reject the study or to provide the author with the opportunity to make revisions based on the evaluation report of the reviewer.
Authors’ Approaches
In a previous study conducted on this subject, it was reported that 72% of 480 participants felt that a long or short peer-review process was not an indicator of acceptance or rejection of a manuscript. According to the same study, participants considered that it was necessary to keep the rejection time short to allow time to prepare the manuscript for submission to another journal.
In another study, it was found that in the case of an unpublished manuscript, authors tended to accuse reviewers of being incompetent or editors of being indifferent, whereas in the case of the publication of the study, authors tended to see it as their own achievement. Furthermore, native English-speaking authors tend to have more negative reactions when their manuscripts are rejected or subject to multiple revisions, which can be an indication that they have higher expectations.
Factors Affecting Rejection
- Being technically inadequate (plagiarism, being simultaneously evaluated in another journal, incomplete/missing sections, language errors, missing and incomprehensible figures, non-compliance with the publication principles of the relevant journal, and missing or outdated references
- Having been submitted to an inappropriate journal, and incompliance with the Aims and Scope of the relevant journal
- Not yet complete and including mostly observations
- Being incomprehensible and inadequate in terms of language quality
- Failing to have an interesting topic
- Giving the impression that it is the extension of another study of the author and not contributing to the improvement/development of the relevant field
- Having results that disregard the literature, data that do not support the results, and invalid arguments
- Having misleading methods or analyses, being statistically invalid and failing to comply with the standards of the relevant field
- Manuscript is too long or too short
- Non-compliance with ethical values
- Being weak in terms of presentation
- Includes theoretical methods within a weak framework
- Being weak in terms of context
- Making no new contribution to the field
- Non-compliance with article criteria
- Non-compliance with the standards of the relevant journal and with academic writing
Reviewers spend an average of 11.5 hours on each submission
According to the Mark Ware STM report, an average of 5 hours is spent for each review, and according to the Mark Ware PRC report, an average of 2.3 reviewers are assigned for each submission. This means that an average of 11.5 hours in total is spent for each submission. This period refers to the amount of time the reviewers spend on each submission. During the peer-review period, the time spent by the journal or the publishing house (e.g., appointing reviewers, editor checks, relevant software costs, etc.) and the time spent for handling the relevant studies (e.g., editor assessments, technical checks, other operational processes, etc.) are not included in the period in question.
Every year, 15,069,600 hours are wasted on peer review
Considering that 11.5 hours are spent for each submission, and that 1,310,400 submissions are reviewed and rejected, it is concluded that an average of 15 million hours is wasted every year.
Given that one year corresponds to roughly 8760 hours and assuming that a single reviewer works for 24 hours, this period corresponds to roughly 1,720 years.
