Peer review is the standard used by scientific journals to select articles for publication. However, there is no procedure to ensure that certain papers of sufficient quality are not rejected, or that some studies with methodological deficiencies are not accepted.1

High impact factor (IF) journals reject many of the originals they receive. This is due both to the quality demands of the editorial committees and to the need to present a limited number of papers in order to keep the IF high, since this is associated to the prestige of the journal.2

This rejection rate in turn gives rise to a wave of submissions to other journals, usually of lower IF, which again review the article using the same system. This phenomenon has been evaluated in many previous studies.3–20 In this evaluation circuit, the original text may gain quality thanks to the recommendations of the editorial teams and reviewers,17 or alternatively the text may become obsolete upon being rejected on several occasions. A high publication rate following a rejection may indicate either low quality of the review process or the need to limit the publications in the original journal in order to maintain its IF.3 A low publication rate after rejection may reflect the poor quality of the submitted originals, indicating limited interest on the part of the original journal.

Another issue is referred to the IF of the journals that posteriorly publish the rejected originals. It is to be expected that authors who see their paper being rejected will seek to publish it in journals with a lower IF.3,4,6–9,11–13,15,17 This is not always the case, however, and the IF of the alternative journal may actually be higher than that of the original journal, thus indicating inadequate assessment of the quality of the document by the latter.

The citations that arise from secondary publications are also indicators of the quality of the editorial process, though they often appear in journals belonging to other categories.20 The non-detection of articles that generate many citations could affect the IF of a journal as a loss of opportunity, and should be regarded by the editorial team as a sentinel event of the process if publication is made in a journal of the same category.

The aim of the present study was to analyze the publication in other journals and the generation of citations of articles rejected by the editorial board of Medicina Intensiva (MI), investigating the variables associated to such publication.

A retrospective study was made of the originals and scientific letters rejected by the editorial board of MI in the period 2015–2017, and which were published in other scientific journals between 2015–2019. The year 2015 was selected as the starting point, coinciding with the change in editorial team.

Use was made of a database of rejected articles provided by the Publisher Elsevier containing all the documents rejected in the mentioned period and which included the title, authors, submitting person, key words, abstract, submission date and rejection notification date.

The PubMed and Google Scholar databases were used to check publication after the rejection, employing the name of the first author and of the submitting person if different from the first author, the title of the paper or a combination of author and article key words. In order to consider the article to correspond to the same text as the rejected article, it was required to include the same first authors and a similar title and abstract, and had to be published after the date of rejection. Publications in university repositories or local bulletins were not considered. In cases of doubt, consensus was reached among three investigators (JGM, FGV, JLGG). The IF of the journal was analyzed in the Journal of Citations Report (JCR) corresponding to the year in which the article was published, calculating the difference with respect to the IF of MI that year. The IF was considered to be similar if the difference was ± 0.3 points (considering 20% of the mean of the IF of the three years), higher if >0.3 points, and lower if <0.3 points.

The number of citations generated by the articles published after rejection was analyzed using the Scopus database and Google Scholar. Mentions arising from articles that were duplicates since they appeared in two languages were not regarded as citations.

The following were analyzed as variables possibly related to publication: the gender of the submitting person, the gender of the first author, coincidence of the submitting person and the first author, the country of origin of the manuscript, the language in which the paper was submitted, the type of manuscript (original/scientific letter), evaluation by reviewers and the processing time (in days).

Following publication, we analyzed the publication time calculated as the difference between publication and the notification of rejection; whether the journal appeared in PubMed; whether the journal had an IF in JCR; and whether it corresponded to the specialty of intensive care. We also examined whether it was contained in the Critical Care Medicine category in JCR, which is the only one in which MI appears; the IF in the year of publication; the number of citations generated; and the amount valid for calculating the IF.

In the period 2015–2017, a total of 360 originals and 278 scientific letters were received for evaluation by the editorial board of MI. Based on the signing authors, a total of 16 originals and 15 letters were removed, thus leaving 344 originals and 263 scientific letters for analysis (Fig. 1). The overall rejection rate was 69.2%, distributed over the three years as follows: 69.1% (2015), 67.4% (2016) and 70.8% (2017) (p=0.69).

Figure 1.

Flowchart of articles rejected by Medicina Intensiva and subsequently published elsewhere.

(0.3MB).

Of the 420 rejected articles, there were more originals (n=241; 57%) than scientific letters (n=179; 43%). The proportion of originals rejected was 70.1%, versus 68.1% in the case of the scientific letters (p=nonsignificant [NS]). A total of 262 documents, of which 98 were originals (37%) and 164 scientific letters (63%), were rejected without being evaluated by the reviewers, with a mean processing time of 9.3 days (SD 6.2). A total of 258 rejected texts were submitted by males (61.4%), though the first author was a male in 57.4% of the cases. The first author and the submitting person were the same individual in 325 cases (77.4%). No submitting person gender difference was observed between the accepted or rejected articles (60.2% versus 61.4%; p=0.75).

Of the 420 rejected articles, 205 (48.8%) were subsequently published in other journals (122 originals and 83 scientific letters). The percentage of texts published after rejection did not vary significantly between the different years analyzed: 2015 (44.7%), 2016 (53.1%) and 2017 (48.8%) (p=0.53). Table 1 shows the main journals in which the rejected articles appeared.

Of the 205 documents published after rejection, 132 (64.4%) appeared in journals indexed by PubMed, 83 (40.5%) in 48 journals with IF in JCR, and 41 (20%) in 26 critical care journals – though only three appeared in two publications of the same category in JCR (Intensive Care Medicine and MI itself).

Most of the articles were published in the year following rejection: 2015 (54%), 2016 (57%) and 2017 (64%). The median (IQR) time to publication after rejection was 410 (360) days in 2015, 366 (285) days in 2016, and 305 (267) days in 2017 (p=0.177). The median (IQR) processing time in MI of the published documents was 22 (11) days in the overall period.

A total of 83 (40.5%) articles were published in journals with IF in JCR. The IF of these journals was higher in 17 (8.3%), similar in 15 (7.3%) and lower in 173 (84.4%) texts, with 122 (59.5%) being published in journals without IF. Two of the articles were subsequently published in MI, following the introduction of substantial changes.

A total of 66 articles (15.7% of the rejected papers) were cited on up to 180 occasions from other journals. Among these articles there were 51 originals and 15 scientific letters. Of the total of 180 citations, 104 (57.8%) were valid for calculation of the IF. The mean number of citations generated was 0.80 per published document and 2.7 per cited article, with 2.31 valid for IF; the maximum was 11 for one study that generated 8 citations valid for the calculation of IF. We detected 21 articles (10.2% of those published, 5% of those rejected) that provided a number of citations valid for an IF higher than that of MI in that year.

Table 2 shows the univariate analyses of the variables associated to publication after rejection, according to different factors. No significant differences in the studied variables were recorded for publication in any journal, except as regards non-coincidence of the submitting person and first author. In the case of publication in journals indexed in PubMed, differences were observed if the first author was a female, if the article came from Europe or North America, if it was written in English, and if it was evaluated by the reviewers. There were also variations in the publication rate if the first author and the submitting person were not the same. On analyzing the differences according to publication in a journal of the specialty, a lesser origin from Spain, Europe or North America was observed, with more texts written in English, and more originals versus scientific letters. On analyzing the variables related to publication in journals with IF, only an origin in Europe or North America was associated to a higher publication rate.

Multivariate analyses were made of the variables independently associated to publication after rejection in a journal indexed in PubMed (Table 3). These were found to be a female first author OR 1.58 (95%CI 1.03–2.44; p=0.037), an origin in Europe or North America OR 2.11 (95%CI 1.29–3.46; p=0.003), papers written in English OR 2.38 (95%CI 1.41–4.04; p=0.001), and papers evaluated by the reviewers OR 1.71 (95%CI 1.10–2.66; p=0.017). The only variable found to be related to publication in a journal of the specialty was an origin from a country other than Spain OR 3.04 (95%CI 1.55–6.00; p=0.001). The variables associated to publication in journals with IF were an origin in Europe or North America OR 2.27 (95%CI 1.28–4.03; p=0.005), and papers written in English OR 1.91 (95%CI 1.06–3.42; p=0.031).

The present study shows that the publication rate following editorial rejection by MI is comparable to that of other journals, and that such publication mainly takes place in journals with a lower IF or belonging to categories different from intensive care. Furthermore, the publication rate varies according to the geographical origin of the article, the language in which it is written, and the gender of the author, as well as evaluation by the reviewers.

The overall rejection rate of manuscripts submitted to medical journals varies greatly between 17–89%.3,6,9,10,12,17–19,21–27 Within this range, the rejection rate of MI is consistent with the average for other journals, and has not experienced significant changes in the three years of the study period. The causes underlying the variability of these rates are varied, and in most cases are adjusted by the journals themselves, with a view to maintaining a balance among publication demand, quality, publication delay and maintenance or improvement of the IF.2,3

The internal management of the editorial boards of medical journals has been studied in relation to the acceptance or rejection of the articles received. The lack of concordance among reviewers,23 rejection as first assessment,28 the nature of the article as an original paper,24 a large number of reviewers,23,29 or reviewer suggestion by the submitting authors23 have been associated to a higher rejection rate.

The statistical analysis made, or the interpretation of the results, is a very relevant cause of rejection.2,30 Some of the issues in this regard are: incomplete introduction,31 inappropriate research question,31–33 poorly defined target population,33 poorly defined variables,33 inadequate methodology,21,33–36 inappropriate statistical analysis,21,34 poor management of covariables,21 problems with the control group,21 lack of data quality,31–34 poorly structured results,33,35 poor interpretation of results33–35 and deficient conclusions.21,31,33 In some cases the editorial boards focus on the improvement of methodological quality as a means to raise the IF of the journal.2,33,37 Other issues associated to rejection are a lack of message,34 a lack of originality,34,36 irrelevance for the journal in question,31,34,35 incorrect style,31,32,34,35 and non-scientific31,35 or inadequate conduct.32

The proportion of articles that are rejected and which subsequently appear published in other journals has been analyzed by different authors.3–20 This proportion varies greatly from 18 to 77%. The publication rate of the articles rejected by MI is in the average range (48.8%). Table 4 compares the main studies on publication after rejection, where this variability can be seen. This situation may be explained in part by different follow-up periods, the analysis of different types of articles, and because a high IF attracts more studies and of higher quality. No linear correlation has been found between the rejection rate and the publication rate.

The analysis of the variables associated to the publication of articles that have been rejected has evidenced a relationship with evaluation by reviewers,12 a variable link to certain geographical origins11,34 and an association to the fact that those authors that modify the content according to the recommendations of the reviewers are published in journals with a higher IF.17 In our study we identified no variables of interest related to publication in any journal. However, on analyzing those indexed in PubMed, we found female gender of the submitting person to be associated to a greater probability of ulterior publication. Other determinant elements were the geographical origin and evaluation by the reviewers. Although the associations of gender to publication rate and article rejection have been studied before,38,39 the data indicate a lesser publication rate if the submitting person was female.11 An explanation for our findings could be that women are more constant in the quest for publication, or that women are more implicated in the successive manuscript revision processes – though we lack data corroborating this. During the analyzed period, there were no gender differences in the rates of acceptance and rejection of originals and scientific letters on the part of the editorial board of MI.

The publication rate in journals of the same category versus others of a different category has been studied elsewhere,20 though without analysis of the associated variables as in our study. The only observation has been a greater delay in publication in journals of a different category; we did not evidence this in our study, however. The only variable independently associated to publication in journals of the same category was an origin outside Spain.

The great majority of articles published after rejection appear in journals with a lower IF or no IF. In fact, the mean citations generated by the published articles is lower than the IF of the journal in the three years of the study, and even lower when only considering those citations valid for the calculation of IF. Only 5% of the rejected manuscripts generated more citations valid for the IF than the IF of MI corresponding to that same year. The mean proportion of articles published in journals with a lower IF in other studies is 81% (range 43–98%),3,4,6–9,11–13,15,17 with MI being located in the average zone (84.4%). All this validates the manuscript acceptance and rejection editorial process.

The present study has some limitations. The databases used might not have been sufficiently exhaustive in obtaining publications, though the results comparable to those of other series reinforce the data obtained. Another possible limitation is the fact that other types of manuscripts were not included, though we considered it preferable to focus on these two categories that have more possibilities of being published and cited.

The articles rejected in MI show a stable subsequent publication rate, similar to that of other journals. Most of these texts are published in journals with a lower IF and with a number of citations below the IF of MI. A European or North American origin, papers in English, a woman as submitting person, and evaluation by reviewers are variables associated to a higher rate of posterior publication in journals indexed in PubMed.

José Luis García Garmendia, data generation and compilation, data analysis, drafting of first manuscript. Federico Gordo Vidal, data generation and compilation, review of the text. Santiago Ramón Leal-Noval, data generation and compilation, review of the text. Rosario Amaya Villar, data generation and compilation, review of the text. Néstor Raimondi, data generation and compilation, review of the text. Ana Ochagavía Calvo, review of the text. José Garnacho Montero, original idea, data generation and compilation, and review of the text.

This study has received no financial support.

The authors declare that they have no conflicts of interest.