- freely available
Cancers 2013, 5(2), 491-510; doi:10.3390/cancers5020491
Published: 3 May 2013
Abstract: Hypoxia affects the tumor microenvironment and is considered important to metastasis progression and therapy resistance. Thus far, the majority of global analyses of tumor hypoxia responses have been limited to just a single omics level. Combining multiple omics data can broaden our understanding of tumor hypoxia. Here, we investigate the temporal change of the metabolite composition with gene expression data from literature to provide a more comprehensive insight into the system level in response to hypoxia. Nuclear magnetic resonance spectroscopy was used to perform metabolomic profiling on the MDA-MB-231 breast cancer cell line under hypoxic conditions. Multivariate statistical analysis revealed that the metabolic difference between hypoxia and normoxia was similar over 24 h, but became distinct over 48 h. Time dependent microarray data from the same cell line in the literature displayed different gene expressions under hypoxic and normoxic conditions mostly at 12 h or earlier. The direct metabolomic profiles show a large overlap with theoretical metabolic profiles deduced from previous transcriptomic studies. Consistent pathways are glycolysis/gluconeogenesis, pyruvate, purine and arginine and proline metabolism. Ten metabolic pathways revealed by metabolomics were not covered by the downstream of the known transcriptomic profiles, suggesting new metabolic phenotypes. These results confirm previous transcriptomics understanding and expand the knowledge from existing models on correlation and co-regulation between transcriptomic and metabolomics profiles, which demonstrates the power of integrated omics analysis.
Previous studies have revealed the impact of tumor hypoxia on the malignant properties and propagation features of cancer . Significant correlations between overexpression of hypoxia-inducible factor-1α (HIF-1α) and patient mortality have been found in brain, cervical, oropharyngeal, ovarian, lung, colorectal, and endometrial tumors [2,3]. Erb et al. studied the different grades of malignancy in oligodendrogliomas , the results showed that the most discriminate metabolic markers are associated with tumor hypoxia. Chan et al. used patient biopsies to investigate specific biomarkers for colorectal cancer. They observed that the most characteristic markers to discriminate colorectal tumor from normal tissue are hypoxia-related metabolites .
Breast cancer is one of the leading causes of cancer death in women, and worldwide approximately one in nine women suffer from this malignant disease . Many groups have used gene expression microarrays to study breast cancer tissue at the transcriptome level. It has been shown that the hypoxic microenvironment induces the expression of more than 100 genes which alter the tumor vitality, propagation, malignant progression, metastasis and resistance to chemotherapy or radiation treatment [7,8,9,10,11]. Similarly, protein arrays and proteomic profiling have been used to a more limited extent to study alterations at the proteome level. The most important molecules are hypoxia-inducible factors (HIFs), including oxygen labile α-subunit and stable β-subunit [6,12,13]. Under hypoxic condition, HIFα subunit (HIF1α, 2α, or 3α) becomes stabilized and dimerized with the HIF1β subunit to form the HIF heterodimeric complex, which binds to the so-called hypoxia responsive element (HRE) within target genes to activate gene expressions that change the nature of tumor cells .
Compared to genomics and proteomics, metabolomics is an emerging science. It is the omic science which quantifies the dynamic multiparametric metabolic response of living systems to patho-physiological stimuli or genetic modifications . Metabolomics studies use high field nuclear magnetic resonance spectroscopy (NMR) or mass-based spectrometry techniques in conjunction with chemometric methods to construct representative metabolic profiles for different physiological states. The metabolite profile comprises hundreds to thousands of endogenous organic metabolites. Comparing the disturbances in these profiles with the basal metabolic state may reveal potential biomarkers for physiological stimuli . There are few studies that have investigated the metabolite biotransformations in breast cancer or tumor hypoxia by NMR-based metabolomics [17,18,19]. Richardson et al. investigated the central carbon metabolism in breast cancer cell lines. The flux of carbon revealed the cellular transformation in different metabolic pathways . Morse et al. characterized choline-related metabolites in breast cancer cell lines and in vivo animal models . Weljie et al. studied the metabolic changes of breast cancer in vitro and in vivo . Amino acids such as leucine, threonine, lysine, phenylalanine, and pyruvate, lactate were found to be influenced by hypoxia. Troy et al. compared the metabolic profiles of HIF-1β-deficient and wild type Hepa-1 cells under hypoxia. From the changes in the quantities of amino acid- and choline-related compounds, they found that HIF-1 is not the only mechanism that regulates glycolysis .
Systems biology helps researchers reveal the complex mechanisms behind biological systems. Researches integrating multiple omics data offer insight into complex biological effects by systems biology approaches. Zhu et al. developed a network reconstruction approach to construct probabilistic networks with multiple omics data including metabolomics and transcriptomics data of yeast. They employed a causal regulator detection algorithm to identify the causal regulators in the networks . It also showed that systems biology is useful in cancer research. Jain et al. identified the key role of glycine in rapid cancer proliferation by integrating metabolomics data with preexisting gene expression data .
Breast cancer is currently the leading cause of womans’ death in many countries, and hypoxia is strongly associated with cancer progression and malignancy. Tumor hypoxia has been thoroughly investigated at the transcriptomic level for breast cancer. Currently, many studies focus on and discuss mechanisms of tumor hypoxia in breast cancer at the transcriptomic level. To better understand system responses to hypoxic perturbations in breast cancer, we designed a metabolic experiment to compare and integrate metabolomic and transcriptomic changes in a breast cancer cell under hypoxic conditions. We analyzed the metabolic changes that occurred as a function of time in MDA-MB-231 breast cancer cells undergoing hypoxia and normoxia by 1H-NMR. The metabolic differences were analyzed by principal component analysis (PCA) and were compared with the bioinformatics networks constructed in this study. To understand the predictive performance of identified metabolites, support vector machines (SVM)  are used to construct the prediction models. This study aims to integrate transcriptomic and metabolomic data to increase our understanding of tumor hypoxia in breast cancer.
2. Results and Discussion
2.1. NMR Metabolic Profiles
1H-NMR spectra of cell extracts obtained from the MDA-MB-231 breast cancer cell line under hypoxia and normoxia after 48 h are shown in Figure 1. The metabolites identified from the 1H-NMR spectrum included fatty acid (formate), carbohydrates (acetate, ethanol, lactate) and amino acids (alanine, phenylalanine, glutamate, glutamine, tyrosine), which are involved in the different metabolic pathways, such as energy metabolism (acetate, formate) and carbohydrate metabolism (ethanol, lactate).
Comparing the metabolite profiles of hypoxic to normoxic-treated cells, the metabolite levels of lactate, glutamine, and phenylalanine are increased, while those of myo-inositol, formate, tyrosine, creatine, glutamate, and alanine are decreased in hypoxic cells.
2.2. Influence of Hypoxia on Metabolic Profiles of MDA-MB-231 Cancer Cells
2.2.1. Temporal Changes in the Metabolic Pattern Induced by Hypoxic Treatment in MDA-MB-231 Breast Cancer Cells
The PCA results of the NMR spectra from MDA-MB-231 under hypoxic and normoxic treatment for 4 to 48 h are shown in Figure 2. The separation between hypoxia and normoxia was not obvious after 4 h treatment (Figure 2a). A clear separation with respect to hypoxia or normoxia treatment is clearly seen after 48 h treatment (Figure 2b). The trajectories of normoxic and hypoxic groups are in different directions. The trajectory of hypoxic group is shifted from the left to the lower right corner of the PCA plot (Figure 2c), whereas the trajectory of normoxic group is along the PC1 axis from 4 h to 48 h treatments (Figure 2d). Samples under 4 h normoxic condition separated from those under 48 h normoxic condition, and they were located on the opposite site of the PCA plot. Samples under 24 h normoxia located in the middle part of the plot and overlapped with both 4 h and 48 h normoxic samples. In contrast to the normoxic group, the samples exposed to 4 h, 24 h, and 48 h hypoxia showed distinct separation on the PCA plot. This observation indicates that there must be some specific metabolic differences between the two groups. Based on the result that 48 h treatment of hypoxia and normoxia showed distinct group separation (Figure 2b), we further compared the two 48 h-groups by PCA to investigate the metabolic differences.
From the loading plot of 48 h hypoxic and normoxic samples, we identified 17 metabolites that contribute to the separation of the two groups (Figure S1 and Table S1). The relative concentrations of glutamine, valine, and leucine are about 2 to 4 times higher in the hypoxic samples. Methionine, lactate and pyruvate are only 1.5 to 2 times higher in the hypoxic group. The relative concentrations of myo-inositol, creatine, creatine phosphate, proline, and alanine are 2 to 6 times lower in the hypoxic group. Glutamate, glycine, acetate, and ethanol are 1.3 to 2 times lower in the hypoxic group. In addition, we traced back the changes of these 17 metabolites in the 24 h and 4 h treatment groups, and found that the fold changes of glutamine and valine increased with longer hypoxia treatment. In contrast, the fold changes of acetate, taurine, alanine, creatine phosphate, and myo-inositol decreased when the hypoxia treatment became longer. The main energy source-glucose-showed slightly higher concentration in hypoxic groups after 24 h and 4 h hypoxic treatment. However, the concentration of glucose between the two groups after 48 h treatment is too low to be detected. The fold changes of the 17 metabolites at different time points were calculated and the results are presented in Figure 3 and Table S2. Median and standard derivation of 17 metabolites were provided in the Table S3 to show the scatter of the metabolite levels.
To understand the predictive performance of 17 identified metabolites, a support vector machine (SVM) is used to construct the prediction models (Table 1). The result showed that the identified metabolites can predict hypoxia and normoxia well at 48 h treatments. The prediction of hypoxia at 48 h showed 83.33% (training data) and 100% (testing data) accuracy, specificity and sensitivity while the prediction performance, especially specificity, for 24 h treatment was much lower. These results suggested that the metabolic changes of hypoxia occurred between 24 and 48 h. Detailed explanation of SVM testing was provided in the supplementary information.
|Table 1. The performance of predicting different time stages of hypoxia and normoxia in mixed samples.|
|Training data (LOOCV b)|
|SVM Model a||BAC c||Accuracy d||Specificity e||Sensitivity f|
a Models were trained by linear SVM classifier, and each model was trained using two-third of all samples (n = 12). b Leave-one-out cross validation (LOOCV) was performed on training sets. c BAC denotes the average of specificity and sensitivity. d Accuracy = [(TN + TP)/(TP + FP + FN + TN)]. e Specificity = [TN/(TN + FP)]. f Sensitivity = [TP/(TP + FN)].
2.2.2. Mechanism Discussion of Metabolic Changes
Tumor hypoxia affects tumor cell growth, apoptosis, differentiation, and metastasis by regulating several transcription factors such as hypoxia-inducible factor (HIF), nuclear factor (NF-κB), and p53 [27,28,29,30]. In addition, tumor hypoxia changes the cell metabolism by regulating the genes of metabolic enzymes such as lactate dehydrogenase (LDH), pyruvate kinase (PK), phosphoglycerate kinase (PGK), and glucose transporter (Glut). These changes increase the vitality and aggressiveness of tumor cell, and make tumor cell adapt to the severe hypoxic condition [31,32,33,34]. The exact values of the fold change were shown in Table S2. Both SVM testing results and the fold changes of metabolite concentration revealed the metabolic profiles between hypoxia and normoxia treatment were most distinct after 48 h treatment (Figure 3, Table S2). The differential metabolites at 48 h were further discussed to understand the effect of hypoxia on breast cancer cells, MDA-MB-231.
Lactate and pyruvate in the hypoxic group increased after 48 h treatment. This finding is coherent with the results of previous studies that hypoxia-inducible factor-1 (HIF-1) activates the genes encoded by glycolysis enzymes and glucose transporters under hypoxic condition [35,36,37,38,39]. The activation of glycolysis pathway increases the ATP production and finally increases the amount of pyruvate and lactate. The activation of LDH further converts pyruvate into lactate under hypoxic conditions . However, the changes of pyruvate and lactate in MDA-MB-231 under hypoxic treatment are only 1.80- and 1.99-fold, which is less significant in comparison with other metabolites. This may be due to the nature of MDA-MB-231 that the expression of HIF-1α in metastatic MDA-MB-231 was found to be higher than other non-metastatic breast cancer lines (MCF-7) . As HIF-1α was also induced in MDA-MB-231 after 48 h normoxic treatment, the concentrations of pyruvate and lactate were slightly increased. Therefore, the difference in fold changes of pyruvate and lactate between the two groups became small.
Glutamine and leucine are important amino acids which were found to be increased after hypoxia. Glutamine is the main source of nitrogen in tumor cells and is important for protein and DNA production. Breast cancer cells express the c-myc gene which regulates cell growth . Myc regulates glutaminolysis and produces NADPH that is required for de novo nucleotide biosynthesis . Besides, glutamine is also a major energy source other than glucose which is important in maintaining cell functions. Leucine is the precursor of adenosine triphosphate and is also essential to maintain cell proliferation. It was found that the uptake of radiolabeled glutamine and leucine in human neuroblastoma cells increased under hypoxic conditions . The increase of glutamine and leucine in MDA-MB-231 after hypoxia treatment might be due to the increase of uptake in order to provide sufficient energy for cell proliferation. Acetate, alanine, proline, creatine phosphate, taurine, and myo-inositol decreased after 48 h hypoxic treatment. The biological roles and the possible mechanisms of the various metabolic changes are discussed in the following paragraphs.
Acetate is the precursor of acetyl-CoA and fatty acids, and it plays an important role in cell metabolism. The incorporation of acetate into fatty acids has been investigated by radiolabeling methods. The results indicated that the incorporation of radiolabeled acetate into fatty acid increased under hypoxia in different cancer lines [45,46]. The increase of fatty acid synthesis is correlated with the activated expression of fatty acid synthetase (FAS) in tumor cells under hypoxic conditions [47,48]. In our study, the amount of acetate slightly decreased under hypoxia. This may be due to the increase of consumption of acetate to synthesize fatty acids.
The amount of alanine and proline decreased in MDA-MB-231 under hypoxia in our study. Alanine and proline are alternative sources of pyruvate . Pyruvate is a key metabolite in metabolic network that could be converted to carbohydrate, fatty acid, and amino acid. Under anaerobic condition, glucose converts to pyruvate and is further metabolized to lactate to produce ATP and NAD+. NAD+ is then reused in highly activated glycolysis. The decrease of alanine and proline might be due to the conversion to pyruvate to increase the ATP production and NAD+ generation for further conversion to lactate .
Creatine phosphate was found to be decreased under hypoxic treatment. The decrease is proposed to compensate for the lower efficiency of ATP production. Creatine phosphate kinase converts creatine phosphate and ADP to creatine and ATP. This reaction generates ATP as another energy source . The relative concentration of creatine phosphate was decreased significantly in MDA-MB-231 under hypoxic conditions to generate ATP for cancer proliferation.
The relative concentration of taurine in the hypoxic groups decreased as the duration of the hypoxic treatment became longer. Previous studies showed that taurine serves as an osmoregulator in brain, heart, and some tumor cells [52,53,54]. The hypo-osmotic insult induces the release of taurine from the cells. Schaffer et al. studied the relationship of chemical hypoxia and taurine release in neonatal cardiomyocytes. They concluded that the release of taurine in chemical hypoxic cells was caused by the increase of sodium in cells . Under hypoxia, the microenvironment causes acidosis in tumor cell, and it was found that Na+/H+ exchangers were upregulated for extrusion of protons to adapt to this condition . Based on these findings, we assume that the sodium content in MDA-MB-231 also increased under hypoxic treatment which resulted in the release of taurine from the tumor cells.
Myo-inositol is correlated with hormone signal transduction, detoxification, and osmoregulation under cell stress . In this study the content of myo-inositol was decreased after 48 h hypoxic treatment underwent a −4.29-fold change. This was proposed to be caused by the consumption of myo-inositol as osmoregulator and detoxification substance when the severe hypoxic condition occurred.
2.3. Inference of Metabolic Pathway Network from Transcriptome and Metabolome
In order to investigate the potential concordance from metabolomics study with pathways inferred from affected genes in other transcriptomics study, we proposed a metabolic pathway network method (Figure 4). The proposed method was as follows:
(1) Seventeen metabolites were quantified from our metabolomics study.
(2) We manually curated microarray data from the hypoxia study using the same cell line with this study . Bando et al. investigated the up-regulated genes of MDA-MB-231after 1, 3, 6, 12 and 24 h of hypoxia. Out of 12,625 genes, 66 genes were considered up-regulated (down-regulated genes were not provided). The corresponding proteins of these 66 genes were retrieved through the KEGG Markup Language (KGML) database . KGML files encompass the relationships between enzymes, compounds and the metabolic reactions in KEGG pathway maps. Enzymes and the corresponding metabolic reactions were extracted from KGML files by querying the Entrez gene ID of the 66 up-regulated genes, which were converted by the Database for Annotation, Visualization and Integrated Discovery (DAVID) program . Twelve pathways were included in the metabolic reactions (Table 2).
|Table 2. Pathways involved with metabolites from metabolome and transcriptome study. Metabolomics pathways inferred under hypoxia treatment were identified from our study. Transcriptomics pathways inferred under hypoxia treatment were obtained from the published study.|
|Arginine and proline metabolism||V||V|
|Pentose and glucuronate interconversions||V|
|Ascorbate and aldarate metabolism||V|
|Alanine, aspartate and glutamate metabolism||V|
|Glycine, serine and threonine metabolism||V|
|Cysteine and methionine metabolism||V|
|Valine, leucine and isoleucine biosynthesis||V|
|Taurine and hypotaurine metabolism||V|
|D-Glutamine and D-glutamate metabolism||V|
|Fructose and mannose metabolism||V|
|Synthesis and degradation of ketone bodies||V|
|Arachidonic acid metabolism||V|
|Linoleic acid metabolism||V|
|Metabolism of xenobiotics by cytochrome P450||V|
|Drug metabolism—cytochrome P450||V|
(3) The metabolic reactions and corresponding pathways of 17 quantified metabolites were also collected from KGML files, where there were 14 pathways corresponding to our metabolomics study (Table 2).
(4) To understand the differences between metabolomic and transcriptomic results, we compared the affected pathways of the metabolome and transcriptome. There were four common pathways: (glycolysis/gluconeogenesis, purine metabolism, arginine and proline metabolism, pyruvate metabolism). Also, to understand the relation between pathways, these pathways were laid out to specific position in the KEGG global metabolic map (Figure 5).
The proposed method could identify the consistent metabolic response from our direct identified metabolites and indirect metabolism pathways from expressed transcripts from literatures and past reports. Table 2 showed that the metabolism pathways from the quantified metabolites were highly related to the reported transcript literature results.
2.3.1. Pathways in Common between Metabolome and Transcriptome
We mapped our identified metabolites and up-regulated genes from previous studies on KEGG reference pathways. Comparison of the mapped pathways of the metabolome and transcriptome displays consistent and incongruous pathways (see Table 2). Consistent pathways are “glycolysis/gluconeogenesis” (KEGG map id: 00010), “pyruvate metabolism” (KEGG map id: 00620), “purine metabolism” (KEGG map id: 00230) and “prginine and proline metabolism” (KEGG map id: 00330). Up-regulation of glycolysis/gluconeogenesis and pyruvate metabolism is well known in tumor hypoxia. Cells generate ATPs by the glycolysis pathway instead of entering TCA cycle and electron transfer chain. The mapped purine metabolism metabolites are glutamine and glycine. These two metabolites are located on the periphery of the purine pathway map, connecting to “glycine, serine and threonine metabolism” (KEGG map id: 00260) or participating very upstream of purine synthesis. The enzyme mapped on is the pyruvate kinase, muscle (PKM2, NCBI GeneID: 5315, E.C.: 220.127.116.11), which catalyzes transferring phosphorus-containing groups reactions. Therefore, direct correlation from this mapped pathway to tumor hypoxia is not obvious. Expression of PKM2 has been demonstrated as necessary to promote aerobic glycolysis with reduced oxidative phosphorylation and provides growth advantages for tumor cells . It may be a pathway for expressing PKM2 to promote anaerobic glycolysis, which is also up-regulated under hypoxic conditions. In the arginine and proline metabolism, level of proline decreased at 48 h. As described in previous discussion, proline could be used as alternative energy source.
2.3.2. Pathways Difference between Metabolic Profiling and Transcriptome
Pathways on the right side of Figure 5 including glycolysis/gluconeogenesis and pyruvate metabolism are the major changes in the metabolomic responses. The difference includes amino acid metabolisms (glutamine, glutamate, alanine, glutathione and other amino acids, see Table 2) and nitrogen metabolism. In contrast, pathways on the left side of Figure 5 including glycolysis/gluconeogenesis and pyruvate metabolism reflect the transcriptomic responses. These pathways contain fructose and mannose metabolism, lipid metabolisms (arachidonic acid metabolism and linoleic acid metabolism) and retinol metabolism. The connections indicated the shared metabolites and enzymes in a pair of two metabolic pathways. Wider edge stands for more metabolites and enzymes shared in two connected pathways. The mapped metabolomic pathways are much higher interconnected than the mapped transcriptomic pathways (metabolomic pathway: 75 connections/91 pathway pairs, transcriptomic pathways: eight connections/eight pathway pairs). The connection between metabolomic pathways (32 connections) and the consistency pathways is also higher than that between transcriptomic pathways and the consistency pathways (three connections).
Like the dynamic Pachinko model , transcriptome and proteome can reveal how the metabolome transforms. The system is probabilistic determined by thermodynamics and kinetics. The outcome of the entire system is dependent on the collisions of molecules and previous metabolic transformation. Knowing the pins (key enzymes or transporter) and holes (the exit of the system) on the Pachinko model may not be enough for understanding the entire system. However, the metabolome points out the end phenotypes against stimuli and provides valuable information. The metabolome is needed to realize the entire system incorporating other omics data.
3. Materials and Methods
3.1. Cell Culture and Hypoxia Treatment
MDA-MB-231 Breast Cancer Cells were seeded onto a 10 cm dish at a density of 8 × 105 cells per dish. The cells were maintained in RPMI 1641 medium (GibcoTM-Invitrogen, Grand Island, NY, USA) with the addition of sodium bicarbonate, 10% fetal bovine serum, 1% penicillin and 1% streptomycin at 37 °C under humidified atmosphere containing 5% CO2 at 37 °C. After 2 days, the growth medium was replaced with fresh growth medium and the cells were placed in hypoxia chamber or normal incubator. For hypoxic treatment, cells were grown at 37 °C in a Hypoxia chamber (Anaerobic System PROOX model 110; BioSpherix, Redfield, NY, USA) at 0.5% O2 with a gas mixture consisting of 95% N2/5% CO2.
3.2. Extraction of Intra-Cellular Metabolite
Cells were harvested at 4 h, 24 h and 48 h after hypoxia or normoxia treatment. The culture medium was quickly removed and the cells were washed twice with 2 mL ice-cold PBS. After washing with PBS solution, the remaining cells were placed on ice and suspended in 600 mL ice-cold D2O for 30 min. D2O extract was transferred to Eppendorfs and centrifuged at 12,000 g at 4 °C for 5 min. The supernatant was collected and stored at −80 °C until use.
3.3. Sample Preparation for 1H-NMR Spectroscopy
Cell extract was thawed at room temperature. Two hundred and ninety-seven μL of cell extract was mixed with 3 μL of 5 mM sodium 3-trimethylsilyl-(2,2,3,3-2H4)-1-propionate (TMSP) in D2O (final concentration 0.05 mM). The D2O provided a NMR lock signal for the NMR spectrometer. One hundred micro-liter of the sample was then transferred into a 2 mm NMR tube.
3.4. Metabolome Analysis
3.4.1. NMR Analysis
Conventional 1H-NMR spectra of the cell extraction samples were obtained using a Bruker Avance 600 spectrometer (Bruker Biospin, Germany) operated at 600.04 MHZ at 25 °C. The one-dimensional 1H-NMR spectra were acquired using a standard NOESYPR1D pulse sequence (recycle delay-90°-t1-90°-tm-90°-acquisition; XWIN-NMR3.5) with a recycle delay time of 2 s, and a mixing time of 150 ms. The 90° pulse length was adjusted to ~4 μs at 0.17 dB and t1 was set to 4 μs, which provided an acquisition time of 2.72 s. For each sample, 128 free induction decays (FID) were collected using 32 k data points within 10 ppm, and the total data collecting time was 11 min. FIDs were then multiplied by an exponential weighting function corresponding to a line broadening of 0.3 Hz, and the data were zero-filled to 64 k data points.
3.4.2. Metabolite Identification
All acquired FIDs were Fourier transformed, phase corrected and aligned to the chemical shift of the alpha-glucose anomeric doublet at 5.223 ppm using ACD/Labs v.10.0 1D NMR Manager (Advanced Chemistry Development, Inc., Toronto, ON, Canada). The FIDs were further imported into R v. 2.8.1 for water deletion, scaling, baseline correction and normalization . The region of the peak containing H2O was removed within 4.5 to 5 ppm. Spectral intensities were scaled to the ratio of TMSP intensity at unit resolution in each NMR spectra. Then, an in-house baseline correction process and robust mean normalization were applied to each spectrum. The spectra region within 0.2 to 4.4 ppm was binned into 420 bins with a binning size of 0.01 ppm. Identification of metabolites was performed using the Profiler module of the Chenomx NMR Suite v.6.1 (Chenomx Inc., Edmonton, AB, Canada). The concentration of each metabolite was determined by Chenomx and was further normalized by the sum total concentration of each tested sample. Two-dimensional (2D) homonuclear j-resolved and total correlation spectroscopy (TOCSY) and heteronuclear multiple bond correlation (HMBC) NMR experiments were also used for peak assignment to specific metabolites.
3.5. Data Analysis
3.5.1. Statistical Analysis
Principal component analysis (PCA) was performed on these preprocessed data with mean centered using the stats package in R. PCA score plot visualizes the clustering of hypoxic group versus normoxic group with different treatments. The significant levels of peak regions in spectrum were assessed by each Principle Component (PC) loading, e.g., the Euclidean distance of PC1 and PC2 loadings. The potentially significant peak regions were marked out by setting a threshold of significant level.
To ensure the potentially significant peak regions were statistically significant, univariate analysis was performed on the integration of potential significant regions by paired-t test with α = 0.05 in R.
The fold changes of the identified compounds between tumor cells under hypoxic to normoxic condition were further computed from the concentration retrieved above. For each samples in either the hypoxia or normoxia group, the median of the concentrations was selected as a representative of the group for the fold change analysis. To calculate fold changes, we apply the following procedure: (a) Let the fold changes equals to the ratio of the concentration in hypoxia to that in normoxia; (b) Set the fold changes in log2 scale. The positive fold changes mean the higher concentration in hypoxia, and the negative fold changes mean the higher concentration in normoxia.
3.5.2. Classification Evaluation
To assess the predictive performance of identified metabolites, the concentration of identified metabolites were taken as features for each sample and predictive models were created using support vector machine (SVM). SVM maps the data from original feature spaces to high dimensions and searches for a hyper-plane to separate the hypoxia from the normoxia. We used the C interface to LIBSVM  to perform the training and test approach.
There were total 54 samples, nine samples each from hypoxia and normoxia treatment for 4 h, 24 h and 48 h (Table 3). For the classification of hypoxia at 4 h, 24 h and 48 h, samples with hypoxic treatment were labeled as positive items for this classification model (n = 9 for each time point) and samples with normoxic treatment were labeled negative (n = 9 for each time point). Six hypoxic samples and six normoxic samples were randomly chosen as the training set for each hypoxic classification model at 4 h, 24 h and 48 h.
|Table 3. Number of samples in each group.|
|4 h||24 h||48 h|
Leave-one-out cross validation (LOOCV) was performed on the training set to tune the kernel parameters. Cross validation was conducted with respect to the average of specificity and sensitivity (BAC), and the predictive performance was reported as specificity [TN/(TN + FP)], sensitivity [TP/(TP + FN)] and accuracy [TN + TP/(TP + FP + FN + TN)].
3.5.3. Construction of Pathway Network from Transcriptome and Metabolome
The transcriptomic data of MDA-MB-231 were extracted and curated from up-regulated genes under hypoxic condition . The affected metabolic pathways of the selected transcriptomic data and the identified metabolites in this study were extracted via the Kyoto Encyclopedia of Genes and Genomes (KEGG) Pathway  (Release 53.0). Metabolic pathways were compared and organized from the affected pathways of the transcriptome and metabolome as Table 2. All affected pathways of the transcriptome and metabolome were chosen to construct metabolic pathway network. Each pathway was one single node. The shared metabolites and enzymes in one pathway-pathway pair were used as the edge. Wider edges represented the larger number of shared metabolites and enzymes in two pathways. The pathway network was visualized by Cytoscape 2.6.3  and the relative position of each pathways was assigned according to KEGG global map (KEGG map id: 01100).
Our 1H-NMR spectroscopy experiments show the metabolic differences between hypoxia- and normoxia-treated MDA-MB-231. Through the PCA analysis, not only the discrimination of different condition groups is clear, but also the regions of NMR spectra which make the two groups different were discovered and verified by statistical analysis. Seventeen metabolites were quantified, which could be seen as potential hypoxic biomarkers to determine tumor hypoxic conditions.
Our time-dependent metabolic profile analysis showed that the discrimination between hypoxia and normoxia was ambiguous over 24 h but became clear over 48 h. This result suggests that the time point shown the change of metabolism might be between 24 and 48 h. In a previous transcriptomic study, the major changes were observed before 12 h . As metabolomic changes became prominent after transcriptomic changes, these changes can be aligned with physiological consequences.
Not only we can discover the metabolites associated with hypoxia and time-dependent biomarkers but also with the approach of a combined transcriptomics and metabolomics model, we constructed the metabolic pathway network with combining the metabolome and transcriptome for comparison of the mapped pathways of the two omics. The results indicated that there was some concordance in metabolism between the transcriptome and metabolome, but using a single platform would lead to only partial understanding of the hypoxia results. This integrated omics analysis could improve our understanding on the mechanisms of adaptation to hypoxia in breast cancer.
This work was supported by Colleges of Medicine and Electronic Engineering and Computer Science, National Taiwan University. This work was supported in part by the Excellent Research Projects of National Taiwan University under grant 98R0036-01. The authors thank the NTU Integrated Core Facility for Functional Genomics of National Research Program for Genomic Medicine of Taiwan for technical assistance. This study was supported by the National Science Council of Taiwan (NSC 101-2325-B-002-038). The authors thank the NTU Integrated Core Facility for Functional Genomics of National Research Program for Genomic Medicine of Taiwan for technical assistance.
- Hiraga, T.; Kizaka-Kondoh, S.; Hirota, K.; Yoneda, T. Hypoxia and hypoxia-inducible factor-1 expression enhance osteolytic bone metastases of breast cancer. Cancer Res. 2007, 67, 4157–4163, doi:10.1158/0008-5472.CAN-06-2355.
- Semenza, G.L. Defining the role of hypoxia-inducible factor 1 in cancer biology and therapeutics. Oncogene 2009, 29, 625–634, doi:10.1038/onc.2009.441.
- Zhong, H.; de Marzo, A.M.; Laughner, E.; Lim, M.; Hilton, D.A.; Zagzag, D.; Buechler, P.; Isaacs, W.B.; Semenza, G.L.; Simons, J.W. Overexpression of hypoxia-inducible factor 1 alpha in common human cancers and their metastases. Cancer Res. 1999, 59, 5830–5835.
- Erb, G.; Elbayed, K.; Piotto, M.; Raya, J.; Neuville, A.; Mohr, M.; Maitrot, D.; Kehrli, P.; Namer, I.J. Toward improved grading of malignancy in oligodendrogliomas using metabolomics. Magn. Reson. Med. 2008, 59, 959–965, doi:10.1002/mrm.21486.
- Chan, E.C.Y.; Koh, P.K.; Mal, M.; Cheah, P.Y.; Eu, K.W.; Backshall, A.; Cavill, R.; Nicholson, J.K.; Keun, H.C. Metabolic profiling of human colorectal cancer using high-resolution magic angle spinning nuclear magnetic resonance (HR-MAS NMR) spectroscopy and gas chromatography mass spectrometry (GC/MS). J. Proteome Res. 2009, 8, 352–361, doi:10.1021/pr8006232.
- Lundgren, K.; Holm, C.; Landberg, G. Hypoxia and breast cancer: Prognostic and therapeutic implications. Cell. Mol. Life Sci. 2007, 64, 3233–3247, doi:10.1007/s00018-007-7390-6.
- Tennant, D.A.; Duran, R.V.; Boulahbel, H.; Gottlieb, E. Metabolic transformation in cancer. Carcinogenesis 2009, 30, 1269–1280, doi:10.1093/carcin/bgp070.
- Brizel, D.M.; Scully, S.P.; Harrelson, J.M.; Layfield, L.J.; Bean, J.M.; Prosnitz, L.R.; Dewhirst, M.W. Tumor oxygenation predicts for the likelihood of distant metastases in human soft tissue sarcoma. Cancer Res. 1996, 56, 941–943.
- Raghunand, N.; Gatenby, R.A.; Gillies, R.J. Microenvironmental and cellular consequences of altered blood flow in tumours. Br. J. Radiol. 2003, 76, S11–S22, doi:10.1259/bjr/12913493.
- Fulda, S.; Debatin, K.M. HIF-1-regulated glucose metabolism a key to apoptosis resistance? Cell Cycle 2007, 6, 790–792, doi:10.4161/cc.6.7.4084.
- Weidner, N.; Semple, N.; Welch, W.R.; Folkman, J. Tumor angiogenesis and metastasis-correlation in invasive breast-carcinoma. N. Engl. J. Med. 1991, 324, 1–8, doi:10.1056/NEJM199101033240101.
- Galanis, A.; Pappa, A.; Giannakakis, A.; Lanitis, E.; Dangaj, D.; Sandaltzopoulos, R. Reactive oxygen species and HIF-1 signalling in cancer. Cancer Lett. 2008, 266, 12–20, doi:10.1016/j.canlet.2008.02.028.
- Brahimi-Horn, C.; Berra, E.; Pouyssegur, J. Hypoxia: The tumor’s gateway to progression along the angiogenic pathway. Trends Cell Biol. 2001, 11, S32–S36.
- Keith, B.; Johnson, R.S.; Simon, M.C. HIF1 alpha and HIF2 alpha: Sibling rivalry in hypoxic tumour growth and progression. Nat. Rev. Cancer 2012, 12, 9–22.
- Nicholson, J.K. Global systems biology, personalized medicine and molecular epidemiology. Mol. Syst. Biol. 2006, 2, doi:10.1038/msb4100095.
- Lenz, E.M.; Bright, J.; Knight, R.; Westwood, F.R.; Davies, D.; Major, H.; Wilson, I.D. Metabonomics with H-1-NMR spectroscopy and liquid chromatography-mass spectrometry applied to the investigation of metabolic changes caused by gentamicin-induced nephrotoxicity in the rat. Biomarkers 2005, 10, 173–187, doi:10.1080/13547500500094034.
- Hara, T.A.; Bansal, T.R.; de Grado, T.R. Effect of hypoxia on the uptake of [methyl-H-3]choline, [1-C-14] acetate and [F-18]FDG in cultured prostate cancer cells. Nucl. Med. Biol. 2006, 33, 977–984, doi:10.1016/j.nucmedbio.2006.08.002.
- Glunde, K.; Shah, T.; Winnard, P.T.; Raman, V.; Takagi, T.; Vesuna, F.; Artemov, D.; Bhujwalla, Z.M. Hypoxia regulates choline kinase expression through hypoxia-inducible factor-lot signaling in a human prostate cancer model. Cancer Res. 2008, 68, 172–180, doi:10.1158/0008-5472.CAN-07-2678.
- Yoshii, Y.; Furukawa, T.; Yoshii, H.; Mori, T.; Kiyono, Y.; Waki, A.; Kobayashi, M.; Tsujikawa, T.; Kudo, T.; Okazawa, H.; et al. Cytosolic acetyl-CoA synthetase affected tumor cell survival under hypoxia: The possible function in tumor acetyl-CoA/acetate metabolism. Cancer Sci. 2009, 100, 821–827, doi:10.1111/j.1349-7006.2009.01099.x.
- Richardson, A.D.; Yang, C.; Osterman, A.; Smith, J.W. Central carbon metabolism in the progression of mammary carcinoma. Breast Cancer Res. Treat. 2008, 110, 297–307, doi:10.1007/s10549-007-9732-3.
- Morse, D.L.; Carroll, D.; Day, S.; Gray, H.; Sadarangani, P.; Murthi, S.; Job, C.; Baggett, B.; Raghunand, N.; Gillies, R.J. Characterization of breast cancers and therapy response by MRS and quantitative gene expression profiling in the choline pathway. NMR Biomed. 2009, 22, 114–127, doi:10.1002/nbm.1318.
- Weljie, A.M.; Bondareva, A.; Zang, P.; Jirik, F.R. 1H-NMR metabolomics identification of markers of hypoxia-induced metabolic shifts in a breast cancer model system. J. Biomol. NMR 2011, 49, 185–193, doi:10.1007/s10858-011-9486-4.
- Troy, H.; Chung, Y.L.; Mayr, M.; Ly, L.; Williams, K.; Stratford, I.; Harris, A.; Griffiths, J.; Stubbs, M. Metabolic profiling of hypoxia-inducible factor-1 beta-deficient and wild type Hepa-1 cells: Effects of hypoxia measured by H-1 magnetic resonance spectroscopy. Metabolomics 2005, 1, 293–303.
- Zhu, J.; Sova, P.; Xu, Q.W.; Dombek, K.M.; Xu, E.Y.; Vu, H.; Tu, Z.D.; Brem, R.B.; Bumgarner, R.E.; Schadt, E.E. Stitching together multiple data dimensions reveals interacting metabolomic and transcriptomic networks that modulate cell regulation. PLoS Biol. 2012, 10, e1001301, doi:10.1371/journal.pbio.1001301.
- Jain, M.; Nilsson, R.; Sharma, S.; Madhusudhan, N.; Kitami, T.; Souza, A.L.; Kafri, R.; Kirschner, M.W.; Clish, C.B.; Mootha, V.K. Metabolite profiling identifies a key role for glycine in rapid cancer cell proliferation. Science 2012, 336, 1040–1044, doi:10.1126/science.1218595.
- Mahadevan, S.; Shah, S.L.; Marrie, T.J.; Slupsky, C.M. Analysis of metabolomic data using support vector machines. Anal. Chem. 2008, 80, 7562–7570, doi:10.1021/ac800954c.
- Greco, O.; Marples, B.; Joiner, M.C.; Scott, S.D. How to overcome (and exploit) tumor hypoxia for targeted gene therapy. J. Cell. Physiol. 2003, 197, 312–325, doi:10.1002/jcp.10374.
- Graeber, T.G.; Peterson, J.F.; Tsai, M.; Monica, K.; Fornace, A.J.; Giaccia, A.J. Hypoxia induces accumulation of P53 protein, but activation of a G(1)-phase checkpoint by low-oxygen conditions is independent of P53 status. Mol. Cell. Biol. 1994, 14, 6264–6277, doi:10.1128/MCB.14.9.6264.
- Koong, A.C.; Chen, E.Y.; Giaccia, A.J. Hypoxia causes the activation of nuclear factor kappa-B through the phosphorylation of I-kappa-B-alpha on tyrosine residues. Cancer Res. 1994, 54, 1425–1430.
- Semenza, G.L.; Wang, G.L. A nuclear factor induced by hypoxia via denovo protein-synthesis binds to the human erythropoietin gene enhancer at a site required for transcriptional activation. Mol. Cell. Biol. 1992, 12, 5447–5454.
- Freitas, I.; Bertone, V.; Griffini, P.; Accossato, P.; Baronzio, G.F.; Pontiggia, P.; Stoward, P.J. In situ lactate-dehydrogenase patterns as markers of tumor oxygenation. Anticancer Res. 1991, 11, 1293–1299.
- Semenza, G.L.; Jiang, B.H.; Leung, S.W.; Passantino, R.; Concordet, J.P.; Maire, P.; Giallongo, A. Hypoxia response elements in the aldolase A, enolase 1, and lactate dehydrogenase A gene promoters contain essential binding sites for hypoxia-inducible factor 1. J. Biol. Chem. 1996, 271, 32529–32537.
- Semenza, G.L.; Roth, P.H.; Fang, H.M.; Wang, G.L. Transcriptional regulation of genes encoding glycolytic-enzymes by hypoxia-inducible factor-1. J. Biol. Chem. 1994, 269, 23757–23763.
- Firth, J.D.; Ebert, B.L.; Pugh, C.W.; Ratcliffe, P.J. Oxygen-regulated control elements in the phosphoglycerate kinase-1 and lactate-dehydrogenase-A genes-similarities with the erythropoietin 3' enhancer. Proc. Natl. Acad. Sci. USA 1994, 91, 6496–6500, doi:10.1073/pnas.91.14.6496.
- Sivitz, W.I.; Lund, D.D.; Yorek, B.; Grovermckay, M.; Schmid, P.G. Pretranslational regulation of 2 cardiac glucose transporters in rats exposed to hypobaric hypoxia. Am. J. Physiol. 1992, 263, E562–E569.
- Mathupala, S.P.; Rempel, A.; Pedersen, P.L. Aberrant glycolytic metabolism of cancer cells: A remarkable coordination of genetic, transcriptional, post-translational, and mutational events that lead to a critical role for Type II hexokinase. J. Bioenerg. Biomembr. 1997, 29, 339–343, doi:10.1023/A:1022494613613.
- Minchenko, A.; Leshchinsky, I.; Opentanova, I.; Sang, N.L.; Srinivas, V.; Armstead, V.; Caro, J. Hypoxia-inducible factor-1-mediated expression of the 6-phosphofructo-2-kinase/fructose-2,6-bisphosphatase-3 (PFKFB3) gene—Its possible role in the Warburg effect. J. Biol. Chem. 2002, 277, 6183–6187.
- Airley, R.; Loncaster, J.; Davidson, S.; Bromley, M.; Roberts, S.; Patterson, A.; Hunter, R.; Stratford, I.; West, C. Glucose transporter Glut-1 expression correlates with tumor hypoxia and predicts metastasis-free survival in advanced carcinoma of the cervix. Clin. Cancer Res. 2001, 7, 928–934.
- Chen, C.H.; Pore, N.; Behrooz, A.; Ismail-Beigi, F.; Maity, A. Regulation of glut1 mRNA by hypoxia-inducible factor-1—Interaction between H-ras and hypoxia. J. Biol. Chem. 2001, 276, 9519–9525.
- Kolev, Y.; Uetake, H.; Takagi, Y.; Sugihara, K. Lactate dehydrogenase-5 (LDH-5) expression in human gastric cancer: Association with hypoxia-inducible factor (HIF-1 alpha) pathway, angiogenic factors production and poor prognosis. Ann. Surg. Oncol. 2008, 15, 2336–2344, doi:10.1245/s10434-008-9955-5.
- Robey, I.F.; Lien, A.D.; Welsh, S.J.; Baggett, B.K.; Gillies, R.J. Hypoxia-inducible factor-1 alpha and the glycolytic phenotype in tumors. Neoplasia 2005, 7, 324–330, doi:10.1593/neo.04430.
- Shiu, R.P.; Watson, P.H.; Dubik, D. C-myc oncognen expression in estrogen-dependent and estrogen-independent breast-cancer. Clin. Chem. 1993, 39, 353–355.
- Wise, D.R.; DeBerardinis, R.J.; Mancuso, A.; Sayed, N.; Zhang, X.Y.; Pfeiffer, H.K.; Nissim, I.; Daikhin, E.; Yudkoff, M.; McMahon, S.B.; et al. Myc regulates a transcriptional program that stimulates mitochondrial glutaminolysis and leads to glutamine addiction. Proc. Natl. Acad. Sci. USA 2008, 105, 18782–18787, doi:10.1073/pnas.0810199105.
- Soh, H.; Wasa, M.; Fukuzawa, M. Hypoxia upregulates amino acid transport in a human neuroblastoma cell line. J. Pediatr. Surg. 2007, 42, 608–612, doi:10.1016/j.jpedsurg.2006.12.010.
- Yoshii, Y.; Waki, A.; Furukawa, T.; Kiyono, Y.; Mori, T.; Yoshii, H.; Kudo, T.; Okazawa, H.; Welch, M.J.; Fujibayashi, Y. Tumor uptake of radiolabeled acetate reflects the expression of cytosolic acetyl-CoA synthetase: Implications for the mechanism of acetate PET. Nucl. Med. Biol. 2009, 36, 771–777, doi:10.1016/j.nucmedbio.2009.05.006.
- Loffler, M.; Schneider, F. Lipogenesis in ehrlich ascites tumor-cells under anaerobic culture conditions. J. Cancer Res. Clin. Oncol. 1979, 95, 115–122, doi:10.1007/BF00401005.
- Vavere, A.L.; Kridel, S.J.; Wheeler, F.B.; Lewis, J.S. 1-C-11-acetate as a PET radiopharmaceutical for imaging fatty acid synthase expression in prostate cancer. J. Nucl. Med. 2008, 49, 327–334, doi:10.2967/jnumed.107.046672.
- Furuta, E.; Pai, S.K.; Zhan, R.; Bandyopadhyay, S.; Watabe, M.; Mo, Y.Y.; Hirota, S.; Hosobe, S.; Tsukada, T.; Miura, K.; et al. Fatty acid synthase gene is up-regulated by hypoxia via activation of Akt and sterol regulatory element binding protein-1. Cancer Res. 2008, 68, 1003–1011, doi:10.1158/0008-5472.CAN-07-2489.
- Southam, A.D.; Easton, J.M.; Stentiford, G.D.; Ludwig, C.; Arvanitis, T.N.; Viant, M.R. Metabolic changes in flatfish hepatic tumours revealed by NMR-based metabolomics and metabolic correlation networks. J. Proteome Res. 2008, 7, 5277–5285.
- Israel, M.; Schwartz, L. The metabolic advantage of tumor cells. Mol. Cancer 2011, 10, 70:1–70:12.
- Rofstad, E.K.; Demuth, P.; Fenton, B.M.; Sutherland, R.M. P-31 nuclear magnetic-resonance spectroscopy studies of tumor energy-metabolism and its relationship to intracapillary oxyhemoglobin saturation status and tumor hypoxia. Cancer Res. 1988, 48, 5440–5446.
- Griffin, J.L.; Shockcor, J.P. Metabolic profiles of cancer cells. Nat. Rev. Cancer 2004, 4, 551–561, doi:10.1038/nrc1390.
- El-Sayed, S.; Bezabeh, T.; Odlum, O.; Patel, R.; Ahing, S.; MacDonald, K.; Somorjai, R.L.; Smith, I.C.P. An ex vivo study exploring the diagnostic potential of H-1 magnetic resonance spectroscopy in squamous cell carcinoma of the head and neck region. Head Neck 2002, 24, 766–772, doi:10.1002/hed.10125.
- Moreno, A.; Lopez, L.A.; Fabra, A.; Arus, C. H-1 MRS markers of tumour growth in intrasplenic tumour and liver metastasis induced by injection of HT-29 cells in nude mice spleen. NMR Biomed. 1998, 11, 93–106, doi:10.1002/(SICI)1099-1492(199805)11:3<93::AID-NBM520>3.0.CO;2-H.
- Schaffer, S.W.; Pastukh, V.; Solodushko, V.; Kramer, J.; Azuma, J. Effect of ischemia, calcium depletion and repletion, acidosis and hypoxia on cellular taurine content. Amino Acids 2002, 23, 395–400, doi:10.1007/s00726-002-0201-3.
- Gatenby, R.A.; Smallbone, K.; Maini, P.K.; Rose, F.; Averill, J.; Nagle, R.B.; Worrall, L.; Gillies, R.J. Cellular adaptations to hypoxia and acidosis during somatic evolution of breast cancer. Br. J. Cancer 2007, 97, 646–653, doi:10.1038/sj.bjc.6603922.
- Beckonert, O.; Monnerjahn, K.; Bonk, U.; Leibfritz, D. Visualizing metabolic changes in breast-cancer tissue using 1H-NMR spectroscopy and self-organizing maps. NMR Biomed. 2003, 16, 1–11, doi:10.1002/nbm.797.
- Bando, H.; Toi, M.; Kitada, K.; Koike, M. Genes commonly upregulated by hypoxia in human breast cancer cells MCF-7 and MDA-MB-231. Biomed. Pharmacother. 2003, 57, 333–340, doi:10.1016/S0753-3322(03)00098-2.
- Kanehisa, M.; Goto, S.; Kawashima, S.; Okuno, Y.; Hattori, M. The KEGG resource for deciphering the genome. Nucleic Acids Res. 2004, 32, D277–D280, doi:10.1093/nar/gkh063.
- Dennis, G., Jr.; Sherman, B.T.; Hosack, D.A.; Yang, J.; Gao, W.; Lane, H.C.; Lempicki, R.A. DAVID: Database for annotation, visualization, and integrated discovey. Genome Biol. 2003, 4, P3, doi:10.1186/gb-2003-4-5-p3.
- Christofk, H.R.; Vander Heiden, M.G.; Harris, M.H.; Ramanathan, A.; Gerszten, R.E.; Wei, R.; Fleming, M.D.; Schreiber, S.L.; Cantley, L.C. The M2 splice isoform of pyruvate kinase is important for cancer metabolism and tumour growth. Nature 2008, 452, 230–233, doi:10.1038/nature06734.
- Nicholson, J.K.; Wilson, I.D. Understanding “global” systems biology: Metabonomics and the continuum of metabolism. Nat. Rev. Drug Discov. 2003, 2, 668–676, doi:10.1038/nrd1157.
- R Development Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2008.
- Chang, C.; Lin, C. LIBSVM: A Library for Support Vector Machines. ACM Trans. Intell. Syst. Technol. 2001, 2, 27.
- Kanehisa, M.; Goto, S.; Furumichi, M.; Tanabe, M.; Hirakawa, M. KEGG for representation and analysis of molecular networks involving diseases and drugs. Nucleic Acids Res. 2010, 38, D355–D360, doi:10.1093/nar/gkp896.
- Shannon, P.; Markiel, A.; Ozier, O.; Baliga, N.S.; Wang, J.T.; Ramage, D.; Amin, N.; Schwikowski, B.; Ideker, T. Cytoscape: A software environment for integrated models of biomolecular interaction networks. Genome Res. 2003, 13, 2498–2504, doi:10.1101/gr.1239303.
© 2013 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).