What is already known?
There was an urgent need for rapid dissemination of evidence during the early phases of the COVID-19 pandemic. However, the quality and timing of availability of evidence has not previously been described.
The COVID-19 pandemic is the biggest worldwide emerging health challenge in this century. This has created a huge need for research, from basic science to all phases of pharmacological studies and qualitative health science evaluation. While COVID-19 disease has been reported in children and young people (CYP) of all ages, including at birth,1–3 most confirmed cases have been in adults.
Infection with SARS-CoV-2 has taken a milder course in children than in adults: most infected children have presented with mild symptoms or have been asymptomatic.4–7 However, numerous publications have warned of unintended consequences to children in the form of adverse life events, poor access to education and widening inequality.8 The balance between the direct and indirect effects of the disease on children highlights the challenge of interpreting scientific certainty in complex systems and its impact on national and local public health decision making.
As the COVID-19 pandemic unfolded, the urgency of the need for evidence to inform policy making and practice came to the fore. As a result, an expert COVID-19 literature in children group was brought together, with membership from clinicians, the Royal College of Paediatrics and Child Health (RCPCH) and Don’t Forget the Bubbles (DFTB) (a not-for-profit educational website).9 The aim of this was to be able conduct a daily rapid and dynamic review process, balancing methodological rigour with the need to quickly synthesise and produce clinically useful output for clinicians on the front line.
This paper describes the publication trends of all the pertinent scientific literature relating to COVID-19 in children in the first 6 months of the global pandemic.
Data sources and search strategy
A comprehensive search of preprint and published literature was conducted daily across four databases between 1 January 2020 and 30 June 2020. After this point, the decision was made to move to weekly searches and a more selective review process due to the volume of publications and the less acute need for rapid dissemination.
The search strategy, developed in PubMed, Scopus, Ovid-Embase and, manually, on MedRxiv, consisted of keywords and Medical Subject Headings (MeSH) terms related to COVID-19.10 Synonyms, alternate spellings, abbreviations and historical terms were incorporated.
Observational studies including case reports, case series, cross-sectional studies, intervention studies (including randomised controlled trials) and cohort studies.
Inclusion and exclusion criteria
Studies that described either epidemiological data, clinical features, vertical transmission, neonatal outcomes, predictors of severity, or prognosis and complications of SARS-CoV-2 in CYP aged between 0 and 18 years old were deemed eligible. Only publications in English were included. Papers where adult and paediatric data were combined, and paediatric data could not be extracted, were excluded. Systematic reviews, other review articles, letters/communications that did not provide clinical data and studies with irredeemable methodological flaws as determined by consensus of at least two clinical academics were excluded.
Study screening and quality assessment: all references identified by searches were exported to endnote, and duplicates were removed. One reviewer screened titles and abstracts against the inclusion/exclusion criteria (NTDC). All potentially relevant articles were sent for full text review by one of a panel of 45 independent reviewers, who confirmed that inclusion criteria were met. We were unable to formally assess methodological quality and risk of bias due to feasibility constraints on the large volume of articles and the rapid nature of the review process. Quality and bias were informally, qualitatively assessed by the study reviewers and a clinical academic for written reviews.
Data extraction: 45 independent reviewers extracted key data from studies that met the inclusion criteria into a data extraction tool. This captured information about each paper’s date of publication (representing either the date of first online publication in a peer-reviewed journal or in some cases the date of publication in a preprint repository), number of children described and country of origin. In addition, each paper was categorised into one of the three main themes: epidemiology (Epi), clinical studies (Clin) and neonatal (Neo). Subsequently, further subcategories were added and applied both prospectively and retrospectively as the evidence base broadened (Epi: disease burden, Epi: transmission, Clin: clinical features, Clin: comorbidities, Clin: paediatric multisystem inflammatory syndrome – temporally associated with SARS-CoV-2 (PIMS-TS) and Clin: therapeutics).
All papers that met the inclusion criteria and deemed potentially relevant were summarised and published on the DFTB website.9 A narrative synthesis of all the summaries was published on the RCPCH website,11 updated on a weekly. To quantify the utility of the evidence and review output, data on the respective websites was collected over time, including Altmetric scores (an automatically calculated weighted count of all the attention a research output has received).
Following early concerns of the same patients appearing in multiple reports,12 all papers originating from China were further assessed for potential duplicate reporting of participants. Data were extracted regarding the institution(s) from which patients were recruited, the start and end dates for recruitment, the age range of included participants and any other limitations to the inclusion criteria (eg, studies limited to children admitted to intensive care). For each publication from China, the number of other papers with overlapping inclusion criteria and recruitment site(s) were determined. Papers for which data were missing regarding recruitment site or recruitment period were excluded from this analysis, as were papers reporting data at a regional or national level.
Between 1 January 2020 and 30 June 2020, a total of 45 453 papers were retrieved by the search, of which 476 satisfied our inclusion criteria (figure 1 and online supplemental file 1). The median number of search hits per week was 906 (IQR: 211–3345) and peaked at 5178 during the week commencing 18 May. The median number of papers meeting criteria for inclusion per week was 18 (IQR: 3–29). The number of search hits remained stable at around 4000 per week until the end of the study period, whereas the number of included papers declined as time went on, falling to 15 during the last week of June.
Table 1 summarises the distribution of papers included by study size (number of children included), continent and category. The cumulative number of children described in all publications totalled 41 396; however, there is likely to be substantial overlap of patients across publications.
The median number of children per paper was 6 (IQR 1–33). Nearly one-third of papers (30.2%) reported on a single child only, and a further 28.3% reported on between one and nine children (ie, 58% of studies included reported on nine or fewer children). Figure 2 shows the number of children reported on, in included studies, over time. The largest category of publications was classified under the theme ‘clinical features’, with a total of 190 (39.9%) publications, followed by 131 (27.5%) for ‘neonates’.
The number of publications by continent varied widely (table 1), with half of the publications from Asia (50.0%) and just under a third from Europe (29.6%). One hundred and eighty-one publications (38.0%) originated from China. The distribution of publications from different regions broadly followed the path of the pandemic, peaking in Asia first, followed by Europe and subsequently North America (figure 3 and figure 4). PIMS-TS papers peaked in May and June and were published almost exclusively from Europe and North America. Despite publications peaking in Asia first, only two publications regarding PIMS-TS came from Asia.13 14
Of the 181 publications from China, data were available for 142 (78.5%) regarding potential duplicate reporting of participants. Among the 142 studies with data available, 103 (72.5%) were found to have overlapping inclusion criteria and recruitment site(s) with at least one other paper (table 2). The median number of overlapping publications per paper was 2 (IQR 0–5, range 0–26). Among papers reporting data from Hubei Province (n=75), where SARS-CoV-2 was first reported, the median number of overlapping publications per paper was 5 (IQR 2–16). There were 23 studies reporting patients recruited from Wuhan Children’s Hospital.
Our prospective bibliographic analysis of COVID-19 publications relevant to children demonstrated a steady increase in the number of papers meeting the inclusion criteria over time, before peaking in May 2020 and tailing off in June. The rate of publication closely mirrored the trends in numbers of cases during the first phase of the pandemic in the northern hemisphere, following 1 month behind. While the number of included papers fell, the number of search hits continued to increase, reflecting fewer original research articles in proportion to review articles/guidelines/opinion articles.
Critically, the numbers of children included in studies was very small, with one-third of studies reporting on a single child. This is notable given seroprevalence studies have shown up to 4% of children within some of the worst affected European countries may have been infected.15 This almost certainly reflects under testing of cases of COVID-19 in children due to their milder disease phenotype.16 While large observational studies require resources and time to deliver, the huge numbers of case studies make a significant proportion of COVID-19 research of limited applicability to clinical practice, due to the inherent bias and unrepresentative nature of n=1 studies.
No interventional studies were reported. There was an extreme paucity of articles regarding therapeutics in children, which were all descriptive in nature and included no clinical trials. Children are commonly excluded from clinical trials of novel therapeutics until they have been conducted in adults, however given a number of trials such as RECOVERY,17 were predominantly tested repurposed pharmaceuticals that have a long safety record in children, delays in recruitment of children into high-quality trials only results in random assortments of therapies being given outside of a research context, without the safety provision or oversight of a formal clinical trial. It would be recommended to consider enrolling children in clinical trials earlier and to encourage flexible, rapidly deployable descriptive and interventional studies that can be rolled out using existing research networks and, where possible, generate large and generalisable clinical cohort information.
The importance of research networks and collectives becomes apparent; organisations such as the International Severe Acute Respiratory and Infection Consortium (ISARIC) demonstrated how collaboration can generate generalisable information relatively quickly. ISARIC-4C18 provided invaluable clinical data at the point of publication, but in the event of future pandemics, consideration should be given to publishing the basic data ‘real-time’.
A significant proportion of large studies were prepublished on medRxiv, a preprint server that allows papers to be made available prior to peer-review and publication in journals. This enabled information to be made available rapidly and often preceded journal publication which, during a period of relative data paucity made an impact on information availability.
Geographical spread of publications was uneven but appeared to follow the path of the pandemic. The absence of some clinical phenotypes, such as PIMS-TS, being reported in Asia, even subsequent to cases being described in Europe and North America, remains unexplained and warrants further investigation.
Analysis of papers from China suggested that there was likely to be substantial overlap of patients between some publications. Duplicate reporting falsely inflates the apparent size of the evidence base and has the potential to introduce bias when data across publications is synthesised in systematic reviews and meta-analyses. However, while overlap in inclusion criteria and recruitment site(s) can suggest the potential for duplicate reporting, the true extent of multiple counting cannot be ascertained without individual patient data. This highlights the need for formal, centralised data collection systems.
The prospective nature of this work was only possible due to the generosity of volunteer reviewers. Being able to access continuous, up-to-date information was important for frontline clinicians, who could access summaries directly online but also for policy makers, addressing questions such as when to open schools, which clinical groups to shield and how to manage COVID-19 and PIMS-TS most effectively.
The acute need for rapid, real-time evidence synthesis is reflected in the utilisation statistics for the DFTB and RCPCH web pages, including their use in international policy documents. The DFTB COVID-19 In Children evidence review page9 went live in March 2020 and was accessed 128 492 times between then and 30 June 2020. As of 30 June, it had an Altmetric score of 3139, has been cited numerous times in academic publications and referenced in 10 policy documents including from the WHO.19 The RCPCH research evidence summaries page11 was accessed 38 094 times between 9 April and 30 June, advised numerous policy outputs and was cited by, for example, UK Research and Innovation.20
While the large volumes of research produced provided benefits for sharing information quickly with a wider audience, judicious caution was used when considering the conclusions made by individual papers. We found cross-over publications where data from the same children had been reused.
Strengths and limitations
To our knowledge, this is the only comprehensive bibliographic overview of all the literature published on COVID-19 in children during the first 6 months of the pandemic. In addition, through the establishment of an expert review network, we were able to identify and formally review a select number of papers that would be most relevant to healthcare professionals working directly with children, organisations supporting clinicians, CYP and their families and policy makers. We have displayed how the research output, globally, evolved over the pandemic.
A decision was made, early in the programme, to exclude data that had not been published in traditional academic literature, due to limitations in formal search strategies and language translation. As a result, when certain organisations or governments published pertinent data that had not been peer reviewed, the data were not included in this report. This is worthy of consideration in the event of future pandemics, as some data were made publicly available by state publications in native languages significantly earlier than when it became available in the formal academic literature,15 and many countries published relevant documents in native language only. In addition, many national reports were published in PDF format only, impeding web searches. HTML publications would improve accessibility and dissemination of these important data. Systematic reviews and meta-analyses were not included given that the process in place was a rolling systematic review of the literature.
Considering the volume of search hits and limited resource, each paper was only reviewed by one independent reviewer. However, as all written reviews were subsequently screened and edited by clinical academics, we do not believe this to have significantly impacted on the papers sent for review and ultimately included in the evidence summaries that were displayed.
While children have been spared the worst of the clinical impact of COVID-19, evidence regarding features of the disease and its transmission within this population group is essential for healthcare workers, children, families, teachers, as well as local, regional and national policy makers. An expert group was formed between clinicians, RCPCH and DFTB to conduct daily rapid searches of key databases, identifying all published literature relating to CYP and selecting the most pertinent papers to be reviewed, summarised and made available to all, with a significant impact on international policy. Over the first 6 months of the pandemic, 476 papers were identified, describing (at most) 41 396 CYP affected by COVID-19. The number of relevant papers declined as the pandemic progressed, and it is noteworthy that just under one-third of the papers reported on a single child.
Data availability statement
Data are available on reasonable request. Data available on request.
All the reviewers who contributed to the review process: Alison Boast, Grace Leo, Dani Hall, Dan Yeoh, Melody Redman, Sarah Sloan, Tricia Barlow, Anne Bean, Maeve Kelleher, Victoria Dachtler, Irnthu Premadeva, Daniel Hawcutt, Lilian Nyirongo, Esther Alderson, Tessa Davis, Sunil Bhopal, Aimee Donald, Sarah Blackstock, Alice Armitage, Anne-Lise Goddings, Lyda Jadresic, Maham Zaman, Celia Avigdor, David Beverley, Hilary Smith, Vivienne van Someren, Brendan Harrington, Louise Tina Day, Sarah Hall, Alastair Falconer, Sarah Lee, Noel Murphy, Chris Lamming, Carlos de Sousa, Mike Hall, Nandhini Prakash, Maggie Fitzpatrick. Robert Scott Jupp, Bhupinder Sandhu, Rob Primhak and Richard Morton. The Children and Young People’s Transformation Team at National Health Service (NHS) England and NHS Improvement. The Research and Evidence Team at the Royal College of Paediatrics and Child Health for managing the literature searches and the Senior Officers for their advice and feedback. Tessa Davis from Don’t Forget the Bubbles for support with data analysis.