Next Article in Journal
Effects of Aquatic Therapy for Children with Autism Spectrum Disorder on Social Competence and Quality of Life: A Mixed Methods Study
Previous Article in Journal
Assessing Approaches of Human Inhalation Exposure to Polycyclic Aromatic Hydrocarbons: A Review
Reply published on 23 March 2021, see Int. J. Environ. Res. Public Health 2021, 18(6), 3326.

Comment on Choi, Y.-J., et al. Cellular Phone Use and Risk of Tumors: Systematic Review and Meta-Analysis. Int. J. Environ. Res. Public Health 2020, 17, 8079

Population Health Sciences, Bristol Medical School, University of Bristol, Bristol BS8 2PS, UK
Department Epidemiology and Public Health, Swiss Tropical and Public Health Institute, 4002 Basel, Switzerland
University of Basel, 4003 Basel, Switzerland
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2021, 18(6), 3125;
Received: 14 November 2020 / Accepted: 15 March 2021 / Published: 18 March 2021
We welcome the updated systematic review and meta-analysis of case-control studies of mobile phone use and cancer by Choi et al., which was recently published in this journal [1]. Given the uncertainties that remain to surround the issue of radiofrequency radiation exposure and cancer risk, regular synthesis of available epidemiological evidence continues to be important, and the synthesis published by Choi et al. provides a timely update. However, Choi et al. have made several peculiar decisions in their synthesis which result in difficulties in the inferences that can be made, and which deserve further discussion.
Firstly, the main meta-analysis shown in Figure 2 in [1] combined case-control studies for different benign and malign tumors, including those of the head, but also non-Hodgkin’s lymphoma and leukaemia, and provides one meta-analytic summary of these. It is not common practice to combine different outcomes with different aetiologies in one meta-analytic summary [2,3] and, given the substantial heterogeneity observed, it is highly questionable if the common risk estimate for diseases with different aetiologies that Choi et al. try to combine in their meta-analysis does exist (note that the arbitrary set of outcomes used by Choi et al. is not the same as ‘all cancers’, which is an aggregate outcome used in meta-analyses). It would be more appropriate to conduct separate meta-analyses by type of tumor, and Choi et al. have indeed done these as well. These results are provided in the Online Supplement (Table S3) and do not provide summary evidence of excess tumor risk for any particular individual tumour types.
Choi et al. further presented subgroup analyses of studies conducted by Hardell et al., studies by the INTERPHONE consortium, and a group of miscellaneous case-control studies. They identify interesting differences between those three subgroups, and conduct further analyses to explore possible reasons for the observed differences. Interestingly, Choi et al. fail to notice the most obvious conclusion from these subgroup analyses, in that both the INTERPHONE-related studies and miscellaneous studies are largely in agreement and do not point to an excess cancer risk from mobile phone use. Evidence of large excess cancer risks are almost exclusively based on the studies by the Hardell group; as already described in earlier meta-analyses [4,5]. In fact, relative excess risks of 90% (30–170%) and 70% (4–180%) reported by the Hardell group (Table 1 and Figure 2) associated with any mobile phone use are implausible high, and do not triangulate [6] with evidence from other epidemiological sources, such as prospective cohort studies [7,8] and incidence trends [9]. Incidence trend analyses are generally considered a weak study design but in this specific case of a clear change of exposure of virtually the whole population, limited confounding factors that may change over time and reliable cancer registries, incidence trends are important for evidence evaluation and plausibility considerations.
Even when exposure-response associations are observed (Table 3), and the INTERPHONE studies and miscellaneous studies provide relative consistent estimates (Odds Ratios of 1.25 (0.96–1.62) and 1.73 (0.66–4.48), respectively) of some excess risk associated with a, arbitrary, cumulative call time of at least 1000 h, the evidence from the Hardell studies similarly provides an implausibly high Odds Ratio of 3.65 (1.69–7.85); out of line with all evidence from other sources. The INTERPHONE team have spent considerable efforts trying to evaluate whether observed increased and decreased risks could be the result of recall and selection bias [10,11,12,13,14] and a recent study found some indication for reverse causality as an explanation for seemingly protective effects from mobile phone use [15]. It is therefore surprising that Choi et al. have not similarly discussed the likelihood of bias away from the Null in the Hardell studies. Disregarding the implausible risk reported by the Hardell group, a summary risk point estimate based on all other case-control studies for 1000+ cumulative hours of use would be in the order of 1.30–1.50, which triangulates much better with other lines of research.
Choi et al. argue that a plausible explanation for the observed differences could be that the Hardell studies are of better quality than those in the other two groups, based on individual appraisal of each study using the Newcastle-Ottawa Scale and National Heart, Lung, and Blood Institute quality assessment tool of case-control studies (Tables S1 and S2). The differences in rating within and between the three groups of case-control studies are minimal, but Choi et al. rated the methodological quality Hardell studies a little higher quality mainly because they had very high response rates and because they mostly classified as having excellent, blinded, assessment of exposure compared to the INTERPHONE and miscellaneous studies. This seems to be an error or misunderstanding in the use of these criteria. First, achievement of a high participation rate is an asset in an epidemiological study. However, to achieve a participation rate of over 80% in a population based case-control study in Western Countries, as reported in the Hardell papers, is highly unusual nowadays. Regardless, one would expect that in a study with such high participation rates, the proportion of mobile phones users in controls should closely match the official subscriber statistics, which was not the case for the Hardell studies [5]. Thus, serious concerns remain about how these high participation rates have been achieved or calculated.
Secondly, the blinding concept as rated by Choi et al. is inappropriate. Exposure assessment in the INTERPHONE studies was conducted by trained interviewers, which have been susceptible to interviewer bias because they could indeed probably not be blinded to case-control status [16]. However, it is highly unlikely this would have resulted in higher bias compared to the Hardell studies, in which exposure assessment was based on questionnaire-based self-reporting, by cases and controls, of mobile phone use who, by definition, are not blinded to their disease status. Methodological work suggests that both face-to-face interviews and self-administered questionnaires are susceptible to various ‘mode of administration’ biases, but that exposure assessment based on self-administered questionnaires are generally more susceptible to recall bias [16]. As such, the methodology of the Hardell studies should have been classified as being of comparable quality to the other case-control studies in this review, at most.
Choi et al. further looked at source of funding as a possible explanation for observed differences, but provided erroneous funding information. Only the Hardell studies received direct funding from interest groups such as the telecom industry [17,18] and pressure groups [19], while Hardell has also acted as an expert witness on behalf of the plaintiff in hearings involving mobile phone use and brain tumour risk in the past, presumably reimbursed [20], but this was not reported by Choi et al. In contrast, INTERPHONE studies’ industry funding was through a well-established firewall model to avoid influence of the funders on the researchers. There is empirical evidence from human experimental studies that such a funding structure has not resulted in biased study results but in higher study quality, whereas direct funding by interest groups may produce biased results [21,22]. Further, the three study groups only contribute to either ‘funded by industry’ or not (according to Choi et al.), which makes this analysis non-informative.
Given that observational epidemiological studies are susceptible to various biases, which can result in under as well as over-reporting of true effects, rigorous evaluation is needed to understand why the studies by the Hardell group provide different results than the majority of other case-control studies and with other groups of epidemiological literature. In the absence of direct evidence for any causes of these differences, triangulation of epidemiological studies susceptible to different types of biases [16], such as case-control studies, cohort studies, and ecological studies of cancer incidence, as well as with evidence from animal and laboratory studies is warranted. Although some uncertainties remain, most notably for highest exposed users and for new GHz frequencies used in 5G, we can be reasonably sure that the evidence has converged to somewhere in the range of an absence of excess risk to a moderate excess risk for a subgroup of people with highest exposure. Important, over time, the evidence had reduced the uncertainty regarding the cancer risk of mobile phone use.

Author Contributions

F.d.V. drafted the first outline. M.R. and F.d.V. collaborated on subsequent. All authors have read and agreed to the published version of the manuscript.


This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of Interest.


  1. Choi, Y.; Moskowitz, J.; Myung, S.; Lee, Y.; Hong, Y. Cellular Phone Use and Risk of Tumors: Systematic Review and Meta-Analysis. Int. J. Environ. Res. Public Health 2020, 17, 8079. [Google Scholar] [CrossRef] [PubMed]
  2. Borenstein, M.; Hedges, L.; Higgins, J.; Rothstein, H. When Does it Make Sense to Perform a Meta-Analysis? In Introduction to Meta-Analysis, 1st ed.; John Wiley & Sons, Ltd.: Chchester, UK, 2009; pp. 357–364. [Google Scholar]
  3. Higgins, J.; Thomas, J. Analysing data and undertaking meta-analyses. In Cochrane Handbook for Systematic Reviews of Interventions Version 6.2 (Updated February 2021); Higgins, J.P.T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M.J., Welch, V.A., Eds.; Cochrane: London, UK, 2021; Available online: (accessed on 17 March 2021).
  4. Lagorio, S.; Röösli, M. Mobile phone use and risk of intracranial tumors: A consistency analysis. Bioelectromagnetics 2014, 35, 79–90. [Google Scholar] [CrossRef] [PubMed]
  5. Röösli, M.; Lagorio, S.; Schoemaker, M.J.; Schüz, J.; Feychting, M. Brain and Salivary Gland Tumors and Mobile Phone Use: Evaluating the Evidence from Various Epidemiological Study Designs. Annu. Rev. Public Health 2019, 40, 221–238. [Google Scholar] [CrossRef] [PubMed][Green Version]
  6. Lawlor, D.A.; Tilling, K.; Smith, G.D. Triangulation in aetiological epidemiology. Int. J. Epidemiol. 2016, 45, 1866–1886. [Google Scholar] [CrossRef] [PubMed][Green Version]
  7. Benson, V.S.; Pirie, K.; Reeves Gillian, K.G.K.; Beral, V.; Green, J.; Schuz, J. Mobile phone use and risk of brain neoplasms and other cancers: Prospective study. Int. J. Epidemiol. 2013, 42, 792–802. [Google Scholar] [CrossRef] [PubMed][Green Version]
  8. Frei, P.; Poulsen, A.H.; Johansen, C.; Olsen, J.H.; Steding-Jessen, M.; Schüz, J. Use of mobile phones and risk of brain tumours: Update of Danish cohort study. BMJ 2011, 343, d6387. [Google Scholar] [CrossRef] [PubMed][Green Version]
  9. Karipidis, K.; Elwood, M.; Benke, G.; Sanagou, M.; Tjong, L.; Croft, R.J. Mobile phone use and incidence of brain tumour histological types, grading or anatomical location: A populationbased ecological study. BMJ Open 2018, 8, e024489. [Google Scholar] [CrossRef] [PubMed]
  10. Lahkola, A.; Salminen, T.; Auvinen, A. Selection bias due to differential participation in a case-control study of mobile phone use and brain tumors. Ann. Epidemiol. 2005, 15, 321–325. [Google Scholar] [CrossRef] [PubMed]
  11. Vrijheid, M.; Armstrong, B.K.; Bedard, D.; Brown, J.; Deltour, I.; Iavarone, I.; Krewski, D.; Lagorio, S.; Moore, S.; Richardson, L.; et al. Recall bias in the assessment of exposure to mobile phones. J. Expo. Sci. Environ. Epidemiol. 2009, 19, 369–381. [Google Scholar] [CrossRef] [PubMed][Green Version]
  12. Vrijheid, M.; Richardson, L.; Armstrong, B.K.; Auvinen, A.; Berg, G.; Carroll, M.; Chetrit, A.; Deltour, I.; Feychting, M.; Hours, M.; et al. Quantifying the Impact of Selection Bias Caused by Nonparticipation in a Case-Control Study of Mobile Phone Use. Ann. Epidemiol. 2009, 19, 33–41. [Google Scholar] [CrossRef] [PubMed]
  13. Vrijheid, M.; Cardis, E.; Armstrong, B.K.; Auvinen, A.; Berg, G.; Blaasaas, K.G.; Brown, J.; Carroll, M.; Chetrit, A.; Christensen, H.C.; et al. Validation of short term recall of mobile phone use for the Interphone study. Occup. Environ. Med. 2006, 63, 237–243. [Google Scholar] [CrossRef] [PubMed][Green Version]
  14. Vrijheid, M.; Deltour, I.; Krewski, D.; Sanchez, M.; Cardis, E. The effects of recall errors and of selection bias in epidemiologic studies of mobile phone use and cancer risk. J. Expo. Sci. Environ. Epidemiol. 2006, 16, 371–384. [Google Scholar] [CrossRef]
  15. Olsson, A.; Bouaoun, L.; Auvinen, A.; Feychting, M.; Johansen, C.; Mathiesen, T.; Melin, B.; Lahkola, A.; Larjavaara, S.; Villegier, A.S.; et al. Survival of glioma patients in relation to mobile phone use in Denmark, Finland and Sweden. J. Neurooncol. 2019, 141, 139–149. [Google Scholar] [CrossRef] [PubMed]
  16. Bowling, A. Mode of questionnaire administration can have serious effects on data quality. J. Public Health 2005, 27, 281–291. [Google Scholar] [CrossRef][Green Version]
  17. Hardell, L.; Mild, K.H.; Carlberg, M. Case-control study on the use of cellular and cordless phones and the risk for malignant brain tumours. Int. J. Radiat. Biol. 2002, 78, 931–936. [Google Scholar] [CrossRef]
  18. Hardell, L.; Hallquist, A.; Mild, K.H.; Carlberg, M.; Påhlson, A.; Lilja, A. Cellular and cordless telephones and the risk for brain tumours. Eur. J. Cancer. Prev. 2002, 11, 377–386. [Google Scholar] [CrossRef]
  19. Hardell, L.; Carlberg, M.; Söderqvist, F.; Mild, K.H. Case-control study of the association between malignant brain tumours diagnosed between 2007 and 2009 and mobile and cordless phone use. Int. J. Oncol. 2013, 43, 1833–1845. [Google Scholar] [CrossRef] [PubMed][Green Version]
  20. Wakeford, R. RE: Secret Ties to Industry and ConflictingInterests in Cancer Research. Am. J. Ind. Med. 2007, 50, 239–240. [Google Scholar] [CrossRef]
  21. Huss, A.; Egger, M.; Hug, K.; Huwiler-Müntener, K.; Röösli, M. Source of funding and results of studies of health effects of mobile phone use: Systematic review of experimental studies. Environ. Health Perspect. 2007, 115, 1–4. [Google Scholar] [CrossRef] [PubMed][Green Version]
  22. van Nierop, L.E.; Röösli, M.; Egger, M.; Huss, A. Source of funding in experimental studies of mobile phone use on health: Update of systematic review. Comptes Rendus Phys. 2010, 11, 622–627. [Google Scholar] [CrossRef]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Back to TopTop