1. Introduction
Cisplatin is widely used in everyday clinics as well as in clinical trials [
1,
2], and as a reference chemotherapeutic for the validation of new antineoplastic drugs and/or methods of treatment [
3]. The main limitation of cisplatin is its high toxicity [
4] and the risk of the development of intrinsic or acquired cancer cell resistance [
5]. Some reports have linked cisplatin therapy with an increased risk of second cancers [
6]. To overcome these problems, cisplatin is typically combined with other drugs or therapeutic methods [
7,
8,
9,
10]. Due to its simple structure and well-known pharmacological and toxicological profiles, cisplatin is also useful as a model drug in the search for new anticancer cures, especially for metal-based complexes, e.g., platinum-based complexes [
11]. Specifically, cisplatin is applied as a reference drug for preliminary in vitro tests of potential new antitumor drugs.
Cytotoxicity studies in vitro are the first biological tests performed for potential new therapeutic substances. Cytotoxicity is a general term for how toxic a substance is to cells, and IC
50 is a quantitative measure that specifies how much of a particular substance (e.g., a drug) is required to inhibit in vitro biological processes by 50%. In the context of cancer research, IC
50 determines the concentration of a chemical compound that can inhibit cancer cell growth by half, relative to cells grown without the compound. IC
50 is a very important measure that is also related to EC
50, the plasma concentration required to obtain 50% of the maximum effect in vivo. Hence, the relationship between in vitro and in vivo cytotoxicity can help to reject chemical compounds during the initial stage of clinical study [
12]. However, during data searching for our previous publication [
11], we observed a diversity in the literature on the cytotoxic effects of cisplatin in cancer cell lines, i.e., in published IC
50 values.
Primary cells and/or continuously growing cancer cells (cell lines) are primary in vitro models used for cytotoxicity tests. The cytotoxicity endpoint parameters include cell viability (e.g., trypan blue staining), cell membrane damage (e.g., LDH—lactate dehydrogenase assay), cell proliferation (e.g., Alamar Blue test), DNA damage (e.g., PCR (polymerase chain reaction), comet, halo, TUNEL (terminal deoxyribonucleotidyltransferase-mediated deoxyuridine triphosphate nick end labeling) assay, HPLC-electrospray tandem mass spectrometry, FISH (fluorescence in situ hybridization), FCM (flow cytometry), total protein content (sulforhodamine—SRB assay), mitochondrial function (including measurements of mitochondrial calcium, superoxide, mitochondrial permeability transition and membrane potential) or metabolic effects as indicators of the potential to cause toxicity to a cell culture e.g., MTT [3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyl tetrazolium bromide] assays) [
13,
14]. We used some of the methods described below for the meta-analyses in this review.
The MTT test is a colorimetric assay that is most frequently used to determine cytotoxicity. It is based on the activity of succinate dehydrogenase, a mitochondrial enzyme of living cells that converts the soluble tetrazolium salt, 3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyltetrazolium bromide, into its reduced form, insoluble formazan. The formazan crystal precipitates occur in small amounts or not in all damaged cells. To obtain reliable and reproducible results by using the MTT test, laboratory precision work is required at all stages of the test, particularly in the last stage of the assay, i.e., the dissolution of formazan crystals. Therefore, there many modifications to the MTT test have been introduced. One of these is CCK-8 assays, which use a tetrazolium salt as a substrate and which, under the influence of dehydrogenase, becomes converted into a colored, soluble compound instead of formazan crystals. In CCK-8 assays, a highly water-soluble tetrazolium salt is used; therefore, this test exhibits better detection sensitivity [
15].
SRB assays allow for the determination of the total amount of protein in the examined sample, which is directly proportional to the number of cells. The basis of this method is the electrostatic binding of sulforhodamine to proteins at an appropriate pH, depending on the qualitative composition of amino acids, after cell fixation with trichloroacetic acid [
16].
The assay of intracellular ATPs allows for the determination of the efficiency of mitochondrial energy processes, which reflect cell viability. The change in the ATP is proportional to the increase or decrease in the number of cells, as well as to the decrease in the efficiency of energy processes in cells. The determination of the number of ATPs can be based on bioluminescence occurring in the reaction with the luciferase catalyzing the oxidation of luciferin to oxyluciferin, with the participation of one ATP molecule [
17].
In this review, we have focused on the publications describing new anticancer compounds (mainly metal complexes), in which cisplatin was used as a reference drug. Cell lines like HeLa, HepG2, and MCF-7, selected as the subject of this study, are the most frequently used in cytotoxicity studies. Some published results have shown differences in the IC
50 values of carboplatin, etoposide, paraquat in an in vitro model based on human glioblastoma cells [
18]. Our aim was to investigate the reliability and reproducibility of cisplatin cytotoxicity in selected cancer cell lines (HeLa, HepG2 and MCF-7) based on the analysis of published data for 2018–2022 years in available databases such as Science Direct, Scopus, and PubMed. The possible reasons of the heterogeneity in results was also of interest.
3. Discussion
The phenomenon of the same drug differences in IC
50 cytotoxicity in the same cell line in vitro has already been described in previous studies [
18,
68]. Among the reasons for this diversity are cell density, cell culture time, and the method of cytotoxicity detection [
68]. To investigate further the variability of published results, we analyzed IC
50 values of cisplatin in three human cancer cell lines (HeLa, HepG2 and MCF-7). The obtained results were divided according to the duration of the cell culture time. Among the analytical methods determining the IC
50 value, the MTT test was a dominant, while the SRB, ATP, and CCK-8 methods were also used (
Tables SA1, SB1 and SC1 in Files S1–S3); the methods were briefly described in the Introduction.
The forest plots illustrate a large discrepancy between cisplatin IC
50 values. From many factors putatively impacting results, only cell culture time and analytical methods determining the IC
50 values are available in publications. Therefore, to perform a more homogenous evaluation of the collected publications, we sub-grouped them according to cell culture time. The 95% confidence intervals for specific cell lines used in different publications are disjoint, which results in significant differences between published results. This also revealed that, for various tumor lines, the range of IC
50 joined via meta-analysis is slightly different, suggesting that some cell lines are more resistant to cisplatin than others. For example, in the HepG2 line, cisplatin cytotoxicity after 48 h is 18.07 (95% CI, 11.10–25.03) and after 72 h, it is 10.93 (95% CI, 4.45–17.41), which shows that the confidence intervals in subgroups are overlapping. Similarly, we can show the overlapping of 95% confidence intervals for MCF-7 cells after 48 h [15.27 (95% CI 11.01–19.53), and after 72 h, it is 10.70 (95% CI, 6.00–15.44)], whereas the overall cytotoxicity is 13.38 (95% CI, 10.28–16.42) (
Figure 3). Finally, the overall cytotoxicity of the cisplatin in HeLa cell line after 48 h is 13.11 (95% CI, 8.46–17.76) (
Figure 1). Thus, the differences described above are not substantial between cell lines as well as between cell culture times, because the confidence intervals are not disjoint. The reason is the substantial diversity within three cell lines and within culture time subgroups. The I
2 indices which describe the proportion of total variation across studies are over 99.8% for HeLa and HepG2 and over 99.7% for MCF-7. Independently on the cell culture time, the values of I
2 are extremely high. Thus, for all examined cell lines, the heterogeneity cannot be attributed to cisplatin exposure time only. Furthermore, for all cell lines, overall and within time-subgroups, the results of the Q Cochran test for homogeneity were significant (
p < 0.0005). Notwithstanding, the significance of the Cochran Q test comparing variances after 48 h and 72 h was not obtained, confirming that this heterogeneity is not related to the cell culture time. Similarly to I
2 indices, other statistics measuring the heterogeneity such as τ
2 and H
2 confirm substantial heterogeneity within time-subgroups and within the overall effect. The prediction intervals are much wider than confidence intervals due to the substnatial heterogeneity τ
2, both for all publications and for the overall effects in the time-subgroups.
As confirmed by the asymmetry forest plots, most of the results of the Egger’s regression tests for funnel plot asymmetry are significant (e.g., HeLa, MCF-7 overall, and within 48 and 72 h subgroups and for HepG2 during the 48 h culture). Moreover, 0.95 confidence intervals for differences between individual IC50 values and the averaged values within time-subgroup IC50 illustrate significant deviations from these averaged values in most cases.
To the best of our knowledge, there are few publications that attempt to describe the quality of the IC
50 results. In the work of Damian et al. [
18], the IC
50 values of three different anticancer drugs: carboplatin, etoposide, and paraquat were tested against two glioblastoma lines—U87MG and U373MG—via the four analytical methods simultaneously (acid phosphatase, MTT, Almar Blue, and trypan blue). Different IC
50 values were obtained for each reference drug depending on the method used. For the cytotoxicity values determined using trypan blue, significant differences were observed compared to other methods. The obtained results allowed to the researchers to distinguish the advantages and limitations of each of the tested methods [
18].
In a more recent work by Arokia Femina et al. [
68], the IC
50 values of 5-fluorouracil tests available in the literature on 10 types of human cancer cell lines (AGS (gastric adenocarcinoma), DLD1 and SNU-C4 (colorectal adenocarcinoma), HCT116 (colorectal carcinoma), HT-29 and MKN28 (gastric adenocarcinoma), MKN45 and SGC7901 (gastric carcinoma), SK-MES-1 (lung carcinoma), SW620 (colon adenocarcinoma)) were compared by using MTT, CCK-8, SRB, and a clonogenic assay in 12–72 h cultures with different cell densities. A wide scatter of the 5-fluorouracil IC
50 results (values 1.46–289.7 µM) was observed. According to the authors of the study, these differences may be related to the type and proliferative potential of the cell line used, the method used, the seeding density, the drug exposure time, and its concentration.
Our research has focused on one of the most used anticancer drugs—cisplatin—which is the reference drug for many newly developed complexes with antiproliferative activities. Contrary to earlier authors, we conducted a broad review of the literature and ranked the results according to the three cell lines and the cultivation time. For the two tested HeLa and HepG2 lines, almost all results were obtained via the MTT method; only in the case of MCF-7 was the variety of methods used greater. The wide spread of IC50 values obtained for the HeLa (48 h) and HepG2 (48 h and 72 h) lines indicates a significant influence of factors other than the type of cell line, the duration of culture, and the method used. In our research, we did not consider the effect of cell culture density, although the condition of the tested cell line and the preparation of the drug may have an impact. When new complex compounds are tested, there is often a problem with their solubility, so a certain amount of solvent, e.g., dimethyl sulfoxide (DMSO) is added to the culture. The methodology of the work rarely mentions whether the same amount of the same solvent was added to the control or the reference drug culture. According to our results, the large overall heterogeneity is not due to different cell culture times, but is due to other factors that are difficult to determine from published studies.
Authors typically do not provide enough information about experimental conditions that might help to explain inconsistency within the findings. Furthermore, the quality of experimental data description differs among publications. The observed diversity of results might also be related to the quality of tested cancer cell lines. In different research centers, cancer cell lines, despite having the same origin, may have their own “story”; in particular, such as the cell culture’s passage, the pre-assay preparation of the cell line and reagents, the concentration of cells, and the cell culture medium, among others. Interestingly, in 2003, the report from Deutsche Sammlung von Mikroorganismen und Zellkulturen (DSMZ) reported significant contamination of cell lines. About 18% of 252 “new” hematopoietic cell lines were cross-contaminated (by other cell lines). The scale of this problem emphasized the value of good laboratory practice [
69]. Additionally, it would be helpful to provide some functional tests for used cell lines like free radicals and cytokine/chemokine production after exposure to cytotoxic drugs. Moreover, the methods and protocols need to be considered, e.g., colored compounds could interfere in a test with absorbance measurements.
On the other hand, determining the IC50 values of cisplatin might be not the aim of a study; this value is often presented only for comparison with a chemical compound of interest in tumor treatment. Nevertheless, unexpected inconsistency in results makes it questionable that we have reliable models to test new anti-cancer drugs’ cytotoxicity in vitro.
Further publications providing a better description of experimental conditions may help to determine the main factors affecting results and help to clarify the reasons for the inconsistency of published results.
4. Materials and Methods
4.1. Data Selection
Meta-analysis was performed according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement guidelines [
70].
4.1.1. Databases SEARCH Criteria
A combination of the key words, IC50, cytotoxicity, anticancer, antitumor, cisplatin, HeLa, HepG2, or MCF-7 cells was used. Databases searched: Science Direct, Scopus, and PubMed, for full-text research articles published between 2018 and 2022, and only journals in the field of chemistry, pharmacology, toxicology, or pharmaceutical science.
4.1.2. Records Identified
When searching Science Direct, n = 2817; Scopus, n = 2431; and PubMed, n = 691.4.1.3. Eligibility:
Records excluded as duplicates (n = 2925) and for other reasons (n = 2974). The reasons for excluding full text articles were as follows:
Cisplatin IC50 results in cell lines other than HeLa, HepG2, or MCF-7;
Cisplatin IC50 data for cell culture times other than 48 and 72 h;
Results obtained under unusual or special conditions (e.g., light irradiation in photodynamic therapy) or under culture conditions described as ambiguous;
Cisplatin IC50 results obtained using 3D spheroid cultures.
4.2. Studies Included in the Meta-Analysis
Forty studies were included in meta-analyses, based on the eligibility criteria. Of these studies, four reported results were based on MCF-7 and HepG2 cells, one was based on MCF-7 and HeLa cells, and three were based on HeLa and HepG2 cells. Only one publication reported results based on all three cell lines. To enhance results, a study published in 2016 describing cisplatin IC50 values for all three cancer cell lines was included.
4.3. Statistical Methods
Of interest was the reliability of the IC50 results for a specific cell line and for a specific culture period. In the same cell culture, cisplatin has a different IC50 value at 48 and 72 h; therefore, the effect of this drug may differ in studies with different experimental times. Cell culture time is a potential factor for cytotoxicity and was taken into account when performing the meta-analysis. For either HepG2 or MCF-7 cancer lines, each article published the results for one culture time point of interest i.e., 48 h or 72 h. In these cases, a meta-analysis was performed for two independent subgroups within the studies (48 h and 72 h). However, time subsets were not available for HeLa cell lines, as the vast majority of results were found for a culture time of 48 h. In order to examine whether the different IC50 values were due to real differences (heterogeneity), or whether the diversity of the IC50 results occurred by chance (homogeneity), the heterogeneity statistics of the cisplatin effects measured via IC50 were examined via meta-analysis. Measures of effect heterogeneity indicate the extent to which the differences between results of individual studies influenced the overall effect. The heterogeneity analysis was carried out separately for each time group.
Cochran’s homogeneity Q statistics are the weighted sum of squared differences between individual study effects and the pooled effect across studies, with weights the same as for the pooling method. Homogeneity was analysed by testing whether the variability between studies τ
2 was equal to zero. This was based on Cochran’s homogeneity test Q statistic with
p-value based on a chi-square distribution with
k-1 degrees of freedom (
k is the number of studies). The Q test for homogeneity hypothesis was used to obtain information about the presence or absence of heterogeneity (e.g., absence of heterogeneity if the test is non-significant). However, to report on the extent of this heterogeneity, other statistics were used. For example, the heterogeneity was assessed using the inter-study variance τ
2. Additionally, I
2 and H
2 indices were calculated to assess heterogeneity [
71]. I
2 = (H
2 − 1)/H
2 expresses the amount of variability in a meta-analysis that is explained by the inter-trial heterogeneity rather than sampling error. Unlike
Q, it does not necessarily depend on the number of studies included in the meta-analysis. I
2 index can be directly compared between meta-analyses with different numbers of studies and different types of outcome data [
72].
Because the studies chosen in the meta-analysis were from different sites and likely included results of different specifications (e.g., different methods of obtaining IC50), the random effects model was chosen, which assumes that there are meaningful differences between studies.
The estimation of the effect is achieved by the iterative method of computing the restricted maximum likelihood estimate (REML).
The truncated Knapp–Hartung method [
73] (truncates the value if it is less than 1 when estimating the variance–covariance matrix) was used to adjust the standard error. According to the published recommendations, Hartung–Knapp method for random effect meta-analysis provides more accurate error rates than the DerSimonian and Laird method, especially for only a few studies [
74]. This method is also recommended when the accuracies of the studies vary [
75].
Lower and upper bounds of the confidence intervals for individual publications and overall effect including subgroup effects were evaluated. Forest charts were added to illustrate the summary of results of meta-analyses and to give a visual impression of the degree of the heterogeneity of the studies.
Random effect weights were estimated using the inverse variance, including within-study
SEi2 and inter-study variance. The weight
wi of the study depends on the observed variability according to the formula:
where
SEi is the standard error within each study and
τ2 is the variance between the studies.
The variability in the obtained effects for each study is due to the sampling error SEi and the differences between the study populations τ2. Weights wi or relative weights wi/∑wi define the size of the squares in the forest plot. The random effect estimates a weighted average of the impact of each publication. The confidence interval for the effect depends on τ2.
In meta-analysis, it is important to assess the bias. Publication bias appears because studies with desirable results are more likely to be published. Consequently, published results may be biased in a certain direction. Analysis of the publication bias was performed using the Eggers’ regression-based test for meta-analysis with continuous outcomes [
76,
77]. The Egger’s test for asymmetry was performed by examining the linear regression of the standardized effect (
e/
SE) on the precision (1/
SE):
where
e is the estimated true effect,
SE is the standard error of effect, and
ε is a random noise. The size of
α (intercept) indicates the extent of the asymmetry. Eggers’ test estimates the statistics based on the t-distribution. Test of intercept
α = 0 is based on
t-distribution with
k-2 degrees of freedom. Additionally, to test for publication bias in the meta-analysis, a trim-and-fill analysis was applied (results not presented).
In addition, to assess the significance of the deviation from the mean value for a specific publication (also the 48 h and 72 h time subgroups), for each tumor cell line, and culture time considered, the difference between individual published IC50 values and averaged IC50 values for the respective time were calculated. It was assumed that for a specific cell line (HeLa, HepG2, MCF-7) and for a specific culture time (48 or 72 h), the correct IC50 value is approximated by the average of the relevant values.
Each individual difference can be viewed as an individual effect. Thus, the effect,
, is defined as the difference between the
i-th individual published IC
50 value and averaged IC
50 value
where
k is the number of publications, as the particular time of cell culture (48 or 72 h);
mi is the individual published IC
50, and
m is the assumed theoretical IC
50 assessed as the average of
mi. In addition, corresponding standard errors
were calculated from source data, i.e.,
SDi and
ni (given in
Tables SA1, SB1 and SC1 in Files S1–S3). The meta-analysis for deviances is given in numerical tables and corresponding forest diagrams. The prediction intervals are also presented in order to reflect the expected uncertainty in the summary effect when a new study was added to the meta-analysis. Prediction intervals for substantial heterogeneity τ
2 are much broader than confidence intervals.
The PS IMAGO PRO 9.0 package was used to create tables and figures in meta-analyses subsections. PS IMAGO PRO is an integrated tool for performing tasks in the field of statistical data analysis [
78]. Graphs were created using default options in the meta-analysis for numerical variables.
5. Conclusions
In 42 studies published between 2018 and 2022, we found an unexpected degree of diversity in cisplatin-related cytotoxicity values in MCF-7, HepG2, and HeLa tumour cell lines. After performing a meta-analysis using mixed-effect models, we observed a substantial degree of heterogeneity in the cisplatin cytotoxicity effects assessed via I2 indices at the 99.8% level in the HeLa cell line, at 48 h culturing. For a single cancer cell line such as HepG2 and MCF-7, splitting the data by cell culture times (48 and 72 h) resulted in the same degrees of diversity, as measured via I2. This indicates that experimental duration is not the main cause of this inconsistency. A substantial degree of heterogeneity was confirmed by other statistics such as τ2, H2, and the significant Q Cochran test for homogeneity. For all cancer cell lines considered, the differences between individual publications and the deviation of the IC50 values from the means of the time-subgroup values were often significant.
To determine the reasons for such diversity in the published results, the stratified analyses of a large series of reports with comprehensive descriptions of their experiment conditions would be helpful. Many factors can affect the quality of cytotoxicity test results, including cell line quality, study protocol validation, and the optimal selection of techniques.
The observed inconsistency in reported cytotoxicity results reduces confidence when comparing new compounds with published cisplatin IC50 values. The data available in the literature are too diverse and unreliable to serve as a reference. It is therefore advisable to carry out a separate reference control for each new experiment, and to not rely solely on the available literature data of IC50.