Next Article in Journal
Dynamic Changes in Cell Wall Polysaccharides during Fruit Development and Ripening of Two Contrasting Loquat Cultivars and Associated Molecular Mechanisms
Next Article in Special Issue
Multi-Index Comprehensive Assessment Optimized Critical Flavonoids Extraction from Semen Hoveniae and Their In Vitro Digestive Behavior Evaluation
Previous Article in Journal
Preparation, Identification and Application of β-Lactoglobulin Hydrolysates with Oral Immune Tolerance
Previous Article in Special Issue
Deep Eutectic Solvent-Based Ultrasound-Assisted Strategy for Simultaneous Extraction of Five Macamides from Lepidium meyenii Walp and In Vitro Bioactivities
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Extraction of Ursolic Acid from Apple Peel with Hydrophobic Deep Eutectic Solvents: Comparison between Response Surface Methodology and Artificial Neural Networks

Department of Biology and Medicine, College of Food Science and Technology, Hunan Agricultural University, No. 1 Nongda Road, Furong District, Changsha 410128, China
*
Author to whom correspondence should be addressed.
Foods 2023, 12(2), 310; https://doi.org/10.3390/foods12020310
Submission received: 15 October 2022 / Revised: 29 December 2022 / Accepted: 30 December 2022 / Published: 9 January 2023
(This article belongs to the Special Issue Green Extraction, Separation, and Purification of Food Ingredients)

Abstract

:
Extracting ursolic acid (UA) from plant resources using organic solvents is incompatible with food applications. To address this, in this study, 15 edible hydrophobic deep eutectic solvents (HDESs) were prepared to extract UA from apple peel, the extraction conditions were optimized, and the optimization strategies were compared. It was found that the solubility of UA in the HDESs can be 9 times higher than the traditional solvent such as ethanol. The response surface optimization concluded that temperature had the greatest effect on the extraction and the optimized test conditions obtained as follows: temperature of 49 °C, time of 32 min, solid–liquid ratio of 1:16.5 g/mL, respectively. Comparing the response surface methodology (RSM) and artificial neural networks (ANN), it was concluded that ANN has more accurate prediction ability than RSM. Overall, the HDESs are more effective and environmentally friendly than conventional organic solvents to extract UA. The results of this study will facilitate the further exploration of HDES in various food and pharmaceutical applications.

1. Introduction

Pentacyclic triterpenes are based on 30 carbon skeletons, consisting of five six-membered rings or four five-membered rings and one five-membered ring, and are formed by the arrangement of epoxide squalene [1], which is one of the classes of triterpenes classified according to carbon skeletons [2]. It can be used as an active ingredient in the treatment of diabetes and its complications [3], as antibacterial agents, antibiofilm agents [4], is considered a potential drug for cancer, viruses, bacteria or protozoan infections [1]. Pentacyclic triterpenoids such as oleanolic acid, glycyrrhizic acid, glycyrrhetinic acid, ursolic acid (UA), betulin, and lupeol show a variety of biological activities [3].
As a kind of lipophilic natural pentacyclic triterpenoids, UA has anti-inflammatory [5], antibacterial, antioxidant [6], anti-fibrosis [7], and antiviral activities [8]. UA can induce autophagy to treat acute kidney injury [9], reduce cholesterol activity, and play a beneficial role in the regulation of intestinal microorganisms [10]. UA is present in many herbs, leaves, flowers, vegetables [11], and fruits such as lingonberries, cranberries, olives, loquat fruit, apples, etc. As far as we know, apple juice produces much unnecessary waste (apple pomace) during the production process [12]. Apple peels, as a waste by-product, contain phytochemicals such as UA [13]. For environmental considerations, it is, therefore, possible to utilize this unwanted waste by extracting UA from apple peels.
Traditional extraction methods of triterpenoids include impregnation, Soxhlet extraction, and hot reflux extraction. The traditional solvents for extracting pentacyclic triterpenes are mainly organic solvents, including methanol, ethanol, and hexane [11]. Kikowska et al. extracted pentacyclic triterpenoids from papaya callus with methanol [14]. Lourenço et al. extracted pentacyclic triterpenoids from birch bark with n-hexane and triterpenes in eucalyptus globulus with dichloromethane [15]. Xia et al. extracted pentacyclic triterpenoid UA and oleanolic acid from ligustrum lucidum fruit with 80% ethanol [16]. However, these traditional solvents have the disadvantages of being volatile, flammable, explosive, and non-environmental protection, so it is necessary to find a green, non-toxic, and low-volatile solvents to replace these traditional solvents.
Deep eutectic solvent (DES) is a new type of solvent which is obtained by mixing hydrogen donor (HBD) and hydrogen acceptor (HBA) in a certain molar ratio [17]. DES has the advantages of environmental friendliness, low volatility, low cost, easy preparation, sustainability, and biodegradability [18,19]. However, most early studies focused on hydrophilic DES, which limits the extraction of hydrophobic substances. In 2013, Abbott et al. first proposed a hydrophobic deep eutectic solvent (HDES) composed of capric acid and quaternary ammonium salts, which broadened the application of DES in extraction [20]. The extraction of bioactive substances with HDESs instead of traditional organic solvents has recently become a new trend. Ma et al. prepared five HDESs and then screened HDES for liquid-liquid microextraction of diphenylamine in fruits using n-Caprylic acid as HBD and menthol (Men) as HBA [21]. Khare et al. prepared 28 HDESs for achieving green extraction of mushroom ergosterol and obtained maximum extraction efficiency using Men: pyruvic acid [22]. Zhu et al. prepared a variety of HDESs using liquid-liquid microextraction combined with HPLC to determine eight synthetic pigments in beverage samples [23]. Silva et al. used HDES based on Men and thymol (Thy) instead of traditional organic solvents to extract pentacyclic triterpene from eucalyptus globulus bark. They found higher extraction rates than those extracted with methylene chloride [24]. However, the process optimization of this green extraction method is less systematically evaluated.
Response surface methodology (RSM) is an optimization method that uses mathematical and statistical methods to construct an optimal condition for assessing the independent variables’ significance and obtaining predicted response values [25]. Artificial neural networks (ANN) is a neural network inspired by the human brain capable of dealing with nonlinear problems without exact relationships, creating a nonlinear modeling approach between the independent and dependent variables, a highly interconnected structure [26,27]. There is accumulating evidence indicating that, compared with RSM, ANN can obtain more superior prediction results and optimization capabilities through its advantages, and the predicted results are closer to the actual values of the test [28,29]. Therefore, it is also necessary to systematically compare these two optimization methods in extracting bioactive substances with HDESs as a new generation of green solvents.
This experiment aims to extract UA from apple peel using natural Men-based HDESs and systematically compare the test effects of the two optimization methods. Firstly, the type of HDESs with the highest solubility was screened by testing the solubility of UA. Then the molar ratio of HBA and HBD of HDES was further screened to select the HDES with the highest solubility of UA. A single-factor test determined the optimum temperature, time, and solid–liquid ratio of HDES, and then the response surface optimization test was used to obtain the optimum test conditions. This study also used ANN to calculate several sets of data from response surface tests and compares the predictive power of these two methods by comparing their adjust coefficients of determination (Radj2) and root mean square errors (RMSE).

2. Materials and Methods

2.1. Materials

The apples (Red Fuji, Yantai, China) were purchased from a local supermarket. UA standard and decanoic acid (purity 99%) purchased from Shanghai Aladdin Biotechnology Company. Menthol (purity 95%) and thymol (purity 99%) were purchased from Wuhan Lanabai Pharmaceutical and Chemical Company. Methanol was purchased from China Ferton Reagent Company (HPLC grade). Pyruvic acid (purity 98%), levulinic acid (purity 99%) and propionic acid (purity 99.5%) were purchased from Shanghai Macklin Biotechnology Co., Ltd. (Shanghai, China). Formic acid (purity 98%), acetic acid (analytical purity), lactic acid (analytical purity), n-butanol (purity 99.5%), and anhydrous ethanol were purchased from China Pharmaceutical Group Chemical Reagent Co., Ltd. (Shanghai, China). Isopropanol (purity 99.7%) was purchased from Tianjin Institute of Chemical Reagents. Phosphoric acid (purity 90%) was purchased from Tianjin Comiou Chemical Reagent Co., Ltd. (Tianjin, China).

2.2. Preparation of HDESs

Based on the method of Peng et al., the HBD and HBA were placed in a reagent bottle and stirred at 80 °C for 30 min at 1 g with a collector thermostatic magnetic stirring bath, then a homogeneous liquid was formed and placed in at dry room temperature and set aside [30]. Silva et al. proposed the extraction of triterpenoid acids with Men and Thy [24]. Cai et al. proposed that DESs based on choline chloride and various hydrogen bond donors were chosen to extract UA, and they found that using organic acids as HBD could improve the extraction efficiency of UA [31]. The types of HBA and HBD in this study are shown in Table 1.

2.3. Viscosity of HDESs

Referring to Alhadid et al., the viscosity of all HDESs was measured using a Kinexus advanced rotational rheometer (Kinexus pro+, Malvern, UK) at room temperature 30 °C and atmospheric pressure with a gap of 0.1 mm and a shear stress of 1 Pa [36]. The test stopped after the shear viscosity was stabilized, and the data were averaged from the three data after stabilization. The viscosity of HDES with the best solubility at 30 °C, 40 °C, 50 °C, 60 °C, and 70 °C was then tested.

2.4. Solubility of UA

2.4.1. Screening Different Types of HDESs

The solubility of UA in 17 HDESs was determined according to the method proposed by Silva et al., with slight modifications [24]. The excess UA and HDES were swirled in a constant temperature magnetic stirrer at a speed of 1 g and a temperature of 60 °C for 30 min (at least three independent samples were prepared for the determination of solubility values and standard deviations). After the supersaturated undissolved UA and the supernatant were separated, 50 μL supernatant sample was taken. The sample was diluted with 495 μL methanol and permeated 0.22 μm through the membrane. The Agilent 1100 liquid chromatography system was used for HPLC determination. The system has a four-component solvent delivery system, an automatic sampler and a diode array detector (DAD). The detection wavelength is 210 nm. The separation is performed on a YMC-Pack ODS-A (250 × 4.60 mm) column with a column box and an automatic sampler maintained at 30 °C. The mobile phase was chromatographic grade methanol, and 0.1% phosphoric acid (92.5%:7.5%, v/v), and the flow rate was 1 mL/min. UA standard solutions of 20, 40, 80, 160, 320, and 640 μg/mL were prepared, and then, the standard curve was plotted with UA concentration as the horizontal coordinate and peak area (Y, mv/s) as the vertical coordinate (X, μg/mL). Then, the regression equation was established (Equation (1)), and the coefficient of determination R2 is 0.9997.
Y = 0.8248 X 33.913
The solubility was calculated by the following equation:
Solubility   ( mg mL ) = C sol × D 1000
where Csol represents the concentration of UA of the tested HDES solubility, D is the dilution multiple of the sample, and 1000 represents the unit conversion.
The HDES with the highest UA solubility was selected by solubility test, and then, the ratio of HBA and HBD of the best HDES were screened.

2.4.2. Screening the Ration of HBA and HBD of HDESs

The HDES with the highest UA solubility among the tested HDESs were further selected to test the UA solubility in HDES under different HBA and HBD molar ratios. The molar ratios of the selected HDESs are shown in Table 2. Except for the HDES of MT 4:1, which formed a clear colorless solid at room temperature, all other HDESs formed a uniform and stable clear colorless liquid, indicating that different ratios of HDESs were successfully prepared.

2.5. HDES Extracts UA from Apple Peels

The apple peels were placed in an oven and baked at 60 °C for 24 h to constant weight. The dried product was crushed and passed through a 60 mesh sieve. After passing through the sieve, the apple powder was baked at 60 °C for 1 h to prevent water reabsorption during the sieving. The 1 g sample was accurately weighed and added into 10 mL HDES solution and extracted with ultrasonic cleaner (KS3200DE, Kunshan Jielimei Ultrasonic Instrument Co., Ltd.) (Kunshan, China) at a power of 100 W and temperature of 50 °C for 30 min. The extract was placed in a high-speed centrifuge at 5550× g for 10 min, take 0.2 mL of supernatant with 1.8 mL of methanol for dilution, passed through a 0.22 μm membrane and then measured by HPLC to calculate the extraction yield (EY) of UA with the following formula.
EY   = M a s s   o f   u r s o l i c   a c i d M a s s   o f   a p p l e   p e e l × 100 %

2.6. Single-Factor Test for UA Extraction by HDES

According to the screening of Section 2.5, HDES with the highest solubility were selected to extract UA from apple peels. The optimal extraction time was obtained by changing the ultrasonic extraction time (15, 30, 45, 60, and 75 min) at the controlled temperature of 50 °C and the solid–liquid ratio of 1:14 g/mL. Then, the effect of extraction temperature (30, 40, 50, 60, and 70 °C) on extraction efficiency was explored when the solid–liquid ratio was 1:14 g/mL, and the extraction time was 30 min. Finally, the influence of different solid–liquid ratios (1:14, 1:16, 1:18, and 1:20 g/mL) on the extraction efficiency was explored at the time of 30 min and temperature of 50 °C. The most appropriate single factor conditions were determined, which provided conditions for response surface optimization experiment.

2.7. Response Surface Optimization of Extraction Process

Based on the results obtained from the single-factor test in Section 2.6., the optimal extraction conditions were selected, and a three-factor, three-level Box–Behnken design (BBD) response surface test was designed. The response surface optimization data were processed with design expert 13 to optimize the extraction conditions in Section 2.6. further, and the response surface optimization table (Table 3) was shown as follows:

2.8. ANN Modeling of Extraction Process

ANN comprises four parts: input, hidden layer, output layer, and output. The input layer of this experiment was composed of time, temperature, and solid–liquid ratio, and the output layer was the extraction yield of UA. The structure of ANN is shown in Figure 1.
In order to obtain better-predicted values and prevent model overfitting, the data is normalized, and the normalized formula is as follows.
  x normalized   value = y actual   value   y minimum   value y maximum   value y minimum   value
All the data involved in the training of ANN are derived from 17 sets of data from the RSM optimization experiment, in which the training set accounts for 70%, the validation set, and the test set account for 15%, respectively. The number of hidden layer neurons is calculated as follows:
h = n + l + a
where h is the number of neurons in the hidden layer, n is the number of network nodes in the input layer, l is the number of network nodes in the output layer, and a is a constant of 0~10.

2.9. Comparison of Predictive Capability of RSM and ANN

The accuracy of performance was evaluated by comparing Radj2 and RMSE between RSM and ANN. The calculation formula of Radj2 is as follows:
R adj 2 = 1 1 n ( Y cal Y Exp ) 2 1 n ( Y Avg Y Exp ) 2
The calculation formula of RMSE is as follows:
RMSE = 1 n ( Y cal     Y Exp ) 2 n
where Ycal is the calculated value of RSM or ANN, YExp is the true value of the experiment, YAvg is the average value of the experiment, and n is the number of experiments for 17.

2.10. Statistical Analysis

All experiments were conducted in triplicate. RSM design of experiments and model fit was performed in Design Expert 13 (Stat-ease, Inc., Minneapolis, MN, USA), ANN modeling was performed in MATLAB (R2018a, The MathWorks, Inc., Massachusetts, MA, USA), and the other statistical analysis was performed using GraphPad Prism 9 (GraphPad Software, San Diego, CA, USA). One-way analysis of variance (ANOVA) with Tukey’s test was used to determine the significant differences (p < 0.05) between the means was performed using IBM SPSS statistic 26.0 (International Business Machines Corporation, Armonk, NY, USA) software. All values are expressed as means ± standard deviations (S.D) unless stated otherwise.

3. Results and Discussion

3.1. Physical Properties of HDESs

This study determined the viscosity of 16 types of HDESs and the viscosity of the HDES with the most appropriate solubility at 30–70 °C to provide a reference for the solubility screening process. For most of the DES, their viscosity at room temperature is generally above 0.1 Pa [37]. High viscosity will affect the mass transfer during extraction and reduce the extraction efficiency, so the viscosity of HDES needs to be one of the reference factors when screening solvents. It can be seen from Figure 1a that increasing the chain length of fatty acids will increase the viscosity. From Men: formic acid, Men: acetic acid, Men: propionic acid, Men: capric acid, the viscosity of these fatty acids increases with the increase of chain length. This phenomenon also appeared in Men: isopropanol and Men: n-butanol. With the increase in carbon chain length, the viscosity also increased. The viscosity of binary eutectic is determined by hydrogen bond, van der Waals force, and electrostatic interaction [37]. Men: lactic acid has the highest viscosity, which may be enhanced by force formed between Men and lactic acid. Men: isopropanol has the lowest viscosity, probably isopropanol, as an organic alcohol, has a low viscosity of its own. Secondly, HDES composed of isopropanol and Men may have a low viscosity due to the low interaction force. The other viscosity of HDES are Men: levulinic acid > Men: pyruvic acid.
On the one hand, the physical properties of HDES can be changed by changing the size of the molar ratio between HBA and HBD, where the viscosity has different variations [38]. On the other hand, the viscosity of HDES is influenced by the size of cations, stacking and intermolecular interactions [39]. As can be seen from Figure 2a, HDES composed of Men and Thy, with this process of molar ratio from 2:1 to 1:6, the viscosity of HDES gradually increased with the increase of the molar proportion of HBD. However, at 1:8 molar ratio the viscosity of HDES shows an increasing trend.
The viscosity of DES decreases with increasing temperature [40]. As shown in Figure 2b, the viscosity became lower while increasing the temperature of HDES with MT 1:6 from 30 °C to 70 °C.
In summary, although the viscosity of HDES prepared in this experiment is higher than that of traditional organic solvents, the viscosity of hydrophobic DES is lower than DES. In general, the extraction process is carried out at temperatures above room temperature, so the viscosity of HDES decreases with temperature increase. Therefore, to selecting such HDES as an alternative to traditional organic reagents is feasible.

3.2. Solubility of UA

In this study, 16 kinds of Men-based HDESs were prepared to determine the solubility of UA, where the composition of HBA and HBD and the molar ratio of HBA and HBD are shown in Table 1 and Table 2. In order to select HDES with the highest UA solubility, solubility tests were performed on the 16 solvents, and all solubility tests were done at 60 °C. As can be seen from Figure 3a, MT 1:1 has the highest solubility among all HDESs, probably because Men and Thy have more similar polarity to UA, making UA better soluble in this HDES.
Among these solvents from HDES 1 to HDES 4, except Men: formic acid, the solubility of UA decreases as the alkyl chain increases. It is probably because as the alkyl chain increases, the steric effects of HDES composed of alkyl and Men increases. Moreover, it is known from Section 3.1. that the viscosity of HDES increases as the alkyl chain increases, and the high viscosity affects the mass transfer, thereby reducing the solubility of UA. The solubility of HDES 5 and HDES 6 also shows a tendency to decrease with the increase of carbon chains. The effect of branched chains on solubility is negligible when comparing the length of carbon chains. HDES 7, which is composed of Men: pyruvate, has the lowest solubility, probably because pyruvate contains a carbonyl group, which leads to its lower solubility. Men: lactate and Men: levulinic acid also showed higher solubility, but MT 1:1 had higher solubility than all other HDESs.
After preliminary solubility screening, it was determined that MT1:1 had the highest solubility, and then the molar ratio of HBA and HBD was further screened. As can be seen from Figure 3b, UA solubility increases with the increase of the molar ratio of HBD, reaching a peak at MT 1:6. Then, a decreasing trend started at MT 1:8. This is closely related to the viscosity of HDESs; as the proportion of HBD increases, the viscosity of HDES decreases. The low viscosity can accelerate the mass transfer between UA and HDES, thus increasing the solubility of UA.
In general, the solubility of the above HDESs prepared was more than nine times higher than that of traditional solvents such as ethanol. Compared with the traditional organic reagent, the solubility of HDESs is more considerable. Therefore, HDES with the highest solubility of MT 1:6 was selected for the following extraction experiments.

3.3. Single Factor Experiment

According to the results in Section 3.2, HDES of MT1:6 was used as the solvent for extracting UA from apple peels. Figure 4a shows the effect of time on the extraction yield. The results showed that with the increase of time, the extraction yield of UA increased gradually and reached the highest extraction yield in 30 min, then decreased with the further increase of extraction time. The change in UA extraction yield may be that increasing the time can increase the leaching of the extracted extract within a certain period. However, with the increase of ultrasonic time, the extraction temperature will rise, resulting in the destruction of UA after a long exposure to ultrasonic irradiation, causing the yields to decrease [41].
Figure 4b shows that the extraction yield reached the highest UA extraction yield at 50 °C, and then with the increase in temperature, the extraction yield showed a downward trend. On the one hand, viscosity decreases with increasing temperature. As mentioned in Section 3.2, increasing the temperature can reduce the viscosity of MT 1:6 HDES, and low viscosity can make UA more fully in contact with HDES, making the yield higher. On the other hand, at a temperature of 60 °C or 70 °C, as mentioned above, prolonged exposure to ultrasonic irradiation and high temperature will lead to the destruction of UA.
Figure 4c demonstrates the effect of the solid–liquid ratio on the extraction yield of UA. Within a specific range, the increase of extraction solvent can improve the extraction yield of UA. The UA extraction yield reached the highest value at 1:16 g/mL but decreased with the increase of the solid–liquid ratio to 1:20 g/mL. It may be because too much solvent will play a dilution role. In the process of ultrasonic-assisted extraction, the too high solid–liquid ratio will reduce the cavitation of ultrasonic [42]. Moreover, too much solvent will dissolve more impurities, compete with UA for dissolution, and reduce the extraction yield.
The most appropriate extraction conditions were initially screened for subsequent response surface optimization based on single-factor time, temperature, and solid–liquid ratio. The most appropriate conditions were time 30 min, temperature 50 °C, and the solid–liquid ratio of 1:16 g/mL.

3.4. Optimization Method

3.4.1. Response Surface Methodology

Response surface optimization is an excellent method to reduce the number of experiments to optimize the test. In this experiment, the most appropriate experimental conditions were determined by single factor test: time 30 min, temperature 50 °C, and solid–liquid ratio 1:16 g/mL. Then, according to the results of the single-factor experiment, the response surface optimization experiment of three factors and three levels of BBD was designed, and 17 test combinations were obtained, as shown in Table 4.
The correlation between the experimental data and the model was analyzed based on the ANOVA. The ANOVA table from Table 5 shows that the model p < 0.05, so it can indicate that the model is significant, while p > 0.05 for the lack of fit indicates that it is not significant. In general, C.V. is used to measure the reproducibility of model [28]. The C.V. of this model is 3.80% less than 10%, which indicates that the model is well reproduced. We can also see that the solid–liquid ratio has the most significant effect on the extraction yield of UA (p < 0.05). In contrast, temperature and time have no significant effect on the extraction yield of UA (p > 0.05). According to the size of the F-value, it can be seen that the solid–liquid ratio has the most significant effect on the extraction yield, then temperature, and finally, time. The R2 in this experiment was 0.9536, and the adjusted R2 was 0.8940 (>0.8), indicating that the experimental data fit well with the second-order polynomial equation. The multiple regression analysis of the data resulted in a second-order polynomial fit regression equation for UA extraction yield, which can be used to calculate UA extraction yield, and the equation is as follows:
Y = 11.44037 + 0.041617 A + 0.160512 B + 1.01406 C 0.000162 AB + 0.000492 AC + 0.00075 BC 0.000647 A 2 0.001703 B 2 0.032312 C 2
where Y is calculated value of response surface optimization. A, B, and C are time (min), temperature (°C), and the solid–liquid ratio (g/mL), respectively.
According to the results of ANOVA for the Quadratic model, the F and p-values of AB (time and temperature) were 0.9171 and 0.3701, respectively. The F and p-values of AC (time and solid–liquid ratio) were 0.3393 and 0.5785, respectively. BC’s F and p-values (temperature and solid–liquid ratio) were 0.3509 and 0.5722, respectively. AB has the most considerable F-value and the smallest p-value. In addition, it can be seen from Figure 5 that the surface of Figure 5a is steeper, indicating that the interaction of AB is greater. BC and AC have small p-values and large F-values. According to Figure 5b,c, it can be seen that their surfaces are not very steep compared to AB, indicating that the interaction is not apparent.
According to the response surface optimization, the predicted reaction conditions were obtained: time 32.334 min, temperature 49.238 °C, and solid–liquid ratio 1:16.514 g/mL. The calculated UA extraction yield was 1.544%. The optimal reaction conditions were calculated as follows: time 32 min, temperature 49 °C, and solid–liquid ratio 1:16.5 g/mL. The yield obtained by the final verification was 1.566%. Siani et al. extracted UA from dry apple peel with aqueous and hydroethanolic to a content of about 6.12 μg/g [43], which is lower than the content of UA in this study.

3.4.2. ANN Modeling

ANN is a multilayer feedback network model with a feed-forward back propagation algorithm. The ANN model in this study used the Levenberg–Marquardt algorithm in training the data. A two-layer feed-forward network with hidden sigmoid neurons and linear output neurons (fitnet), fits multidimensional mapping problems in extraction prediction arbitrarily well.
In this study, a trial-and-error approach was used to train the ANN model one after another, stopping the ANN training when the mean square error (MSE) value is the minimum and R is the maximum. The statement that it is not limited when choosing the number of neurons in the hidden layer is not true [44]. The number of neurons is not necessarily the more significant. The excessive number of neurons will lead to overfitting the model. According to this principle, an optimization of the ANN model was performed to determine the number of neurons of the ANN, and the final number of neurons obtained is the best when it is 6. Therefore, the structure of the ANN model for this experiment is 3–6–1. That is, the input layer is 3, the output layer is 2, and the number of neurons is 6.
When the neuron node was determined to be 6, RMSE and R were calculated according to Formulas (6) and (7). The MSE of the training set is 2.22 × 10−6, the MSE of the validation set is 0.000558, and the MSE of the test set is 0.00256. The accuracy of the experimental model can be demonstrated by the fact that the training and test set errors are close [26]. As seen in Figure 6, the MSE of the training set, the validation set, and the test set tends to be stable after four iterations, and the MSE difference between the training set and the validation set is slight. Therefore, it is considered that the model can be used to calculate the data of this study.
As shown in Figure 7a–d, after optimization according to ANN, the R of the training set is 0.99992, the R of the validation set is 0.99731, the R of the test set is 0.99899, and the R of all sets is 0.99229.

3.4.3. Comparison of Predictive Capability of RSM and ANN

The predicted and experimental values of RSM and ANN are shown in Table 4. It can be seen from these data intuitively that the predicted values and experimental values of ANN are closer than those of RSM. Then the prediction ability of the two methods was compared according to the size of the statistical parameters Radj2 and RMSE between RSM and ANN. Generally, the higher Radj2, the lower RMSE, and the more accurate the prediction model is [26]. The Radj2 and RMSE of RSM were 0.9535 and 0.03254, respectively. The Radj2 and RMSE of ANN were 0.9797 and 2.72249 × 10−5, respectively. It can be seen that the Radj2 of ANN is higher than RSM, and its RMSE is far less than RSM. Therefore, it can be proved that the ANN model can better calculate the experimental results than the RSM model, which is related to its general ability to approximate nonlinear systems [28]. In conclusion, it can be concluded that the ANN model is more reliable and accurate than RSM in its prediction capability and can be better used to calculate the test results of the extraction process of bioactive substances.

4. Conclusions

This study prepared 17 kinds of menthol-based HDESs composed of different HBA and HBD. This natural hydrophobic solvent was used for the first time to extract UA in apple peels to replace the traditional organic solvent. According to the solubility test, the solubility of UA in HDESs is significantly improved compared to that in the traditional organic solvent. The solubility of UA in ethanol is only 4.16 μg/mL, while the solubility of UA in HDES is up to 36.2 μg/mL. The response surface optimization concluded that temperature had the most significant effect on the extraction, and the optimized test conditions obtained were 49 °C, 32 min, and 1:16.5 g/mL, respectively. In this study, two different prediction methods, RSM and ANN, were used to calculate the values of 17 groups of experiments. According to the analysis of the data parameters in the two models, such as Radj2 and RMSE, ANN has a better prediction ability than RSM. Therefore, this work has the potential to become the foundation of an efficient technique for the extraction of UA for application in food industry. HDES is a promising extraction solvent, and its significance as a solvent is more than that. More studies need to pay attention to the coexistence of DES in the activity protection and efficacy promotion of plant-active ingredients.

Author Contributions

Conceptualization, C.Z.; investigation, H.L. and Y.L.; resources, C.Z., S.Q. and S.G.; writing—original draft preparation, H.L. and Y.L.; writing—review and editing, C.Z., M.S. and Y.L.; project administration, C.Z.; and funding acquisition, S.Q. and S.G. All authors have read and agreed to the published version of the manuscript.

Funding

This study was funded by the National Key Research and Development Program of China (2019YFC1604903) and the Natural Science Foundation of Hunan Province (grant number 2021JJ30321).

Data Availability Statement

Data is contained within the article.

Conflicts of Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

  1. Bachorik, J.; Urban, M. Biocatalysis in the Chemistry of Lupane Triterpenoids. Molecules 2021, 26, 2271. [Google Scholar] [CrossRef] [PubMed]
  2. Xu, C.; Wang, B.; Pu, Y.; Tao, J.; Zhang, T. Techniques for the analysis of pentacyclic triterpenoids in medicinal plants. J. Sep. Sci. 2018, 41, 6–19. [Google Scholar] [CrossRef] [PubMed]
  3. Alqahtani, A.; Hamid, K.; Kam, A.; Wong, K.H.; Abdelhak, Z.; Razmovski-Naumovski, V.; Chan, K.; Li, K.M.; Groundwater, P.W.; Li, G.Q. The pentacyclic triterpenoids in herbal medicines and their pharmacological activities in diabetes and diabetic complications. Curr. Med. Chem. 2013, 20, 908–931. [Google Scholar] [PubMed]
  4. Chung, P.Y. Novel targets of pentacyclic triterpenoids in Staphylococcus aureus: A systematic review. Phytomedicine 2020, 73, 152933. [Google Scholar] [CrossRef] [PubMed]
  5. Checker, R.; Sandur, S.K.; Sharma, D.; Patwardhan, R.S.; Jayakumar, S.; Kohli, V.; Sethi, G.; Aggarwal, B.B.; Sainis, K.B. Potent anti-inflammatory activity of ursolic acid, a triterpenoid antioxidant, is mediated through suppression of NF-kappaB, AP-1 and NF-AT. PLoS ONE 2012, 7, e31318. [Google Scholar] [CrossRef] [Green Version]
  6. Do Nascimento, P.G.; Lemos, T.L.; Bizerra, A.M.; Arriaga, Â.M.; Ferreira, D.A.; Santiago, G.M. Antibacterial and Antioxidant Activities of Ursolic Acid and Derivatives. Molecules 2014, 19, 1317–1327. [Google Scholar] [CrossRef]
  7. Huang, C.; Gan, D.; Luo, F.; Wan, S.; Chen, J.; Wang, A.; Li, B.; Zhu, X. Interaction Mechanisms Between the NOX4/ROS and RhoA/ROCK1 Signaling Pathways as New Anti- fibrosis Targets of Ursolic Acid in Hepatic Stellate Cells. Front. Pharm. 2019, 10, 431. [Google Scholar] [CrossRef] [Green Version]
  8. Tohme, M.J.; Gimenez, M.C.; Peralta, A.; Colombo, M.I.; Delgui, L.R. Ursolic acid: A novel antiviral compound inhibiting rotavirus infection in vitro. Int. J. Antimicrob. Agents 2019, 54, 601–609. [Google Scholar] [CrossRef]
  9. Zhao, J.; Zheng, H.; Sui, Z.; Jing, F.; Quan, X.; Zhao, W.; Liu, G. Ursolic acid exhibits anti-inflammatory effects through blocking TLR4-MyD88 pathway mediated by autophagy. Cytokine 2019, 123, 154726. [Google Scholar] [CrossRef]
  10. Hao, W.; Kwek, E.; He, Z.; Zhu, H.; Liu, J.; Zhao, Y.; Ma, K.Y.; He, W.S.; Chen, Z.Y. Ursolic acid alleviates hypercholesterolemia and modulates the gut microbiota in hamsters. Food Funct. 2020, 11, 6091–6103. [Google Scholar] [CrossRef]
  11. Lopez-Hortas, L.; Perez-Larran, P.; Gonzalez-Munoz, M.J.; Falque, E.; Dominguez, H. Recent developments on the extraction and application of ursolic acid. A review. Food Res. Int. 2018, 103, 130–149. [Google Scholar] [CrossRef] [PubMed]
  12. Vendruscolo, F.; Albuquerque, P.M.; Streit, F.; Esposito, E.; Ninow, J.L. Apple pomace: A versatile substrate for biotechnological applications. Crit. Rev. Biotechnol. 2008, 28, 1–12. [Google Scholar] [CrossRef] [PubMed]
  13. Cargnin, S.T.; Gnoatto, S.B. Ursolic acid from apple pomace and traditional plants: A valuable triterpenoid with functional properties. Food Chem. 2017, 220, 477–489. [Google Scholar] [CrossRef] [PubMed]
  14. Kikowska, M.A.; Chmielewska, M.; Włodarczyk, A.; Studzińska-Sroka, E.; Żuchowski, J.; Stochmal, A. Effect of Pentacyclic Triterpenoids-Rich Callus Extract of Chaenomeles japonica (Thunb.) Lindl. ex Spach on Viability, Morphology, and Proliferation of Normal Human Skin Fibroblasts. Molecules 2018, 23, 3009. [Google Scholar] [CrossRef] [Green Version]
  15. Lourenco, A.; Marques, A.V.; Gominho, J. The Identification of New Triterpenoids in Eucalyptus globulus Wood. Molecules 2021, 26, 3495. [Google Scholar] [CrossRef]
  16. Xia, E.Q.; Wang, B.W.; Xu, X.R.; Zhu, L.; Song, Y.; Li, H.B. Microwave-Assisted Extraction of Oleanolic Acid and Ursolic Acid from Ligustrum lucidum Ait. Int. J. Mol. Sci. 2011, 12, 5319–5329. [Google Scholar] [CrossRef]
  17. Smith, E.L.; Abbott, A.P.; Ryder, K.S. Deep eutectic solvents (DESs) and their applications. Chem. Rev. 2014, 114, 11060–11082. [Google Scholar] [CrossRef] [Green Version]
  18. Tang, W.; Liu, L.; Li, G.; Zhu, T.; Row, K.H. Optimal separation of phenol from model oils by forming deep eutectic solvents with quaternary ammonium salts. Korean J. Chem. Eng. 2016, 34, 814–821. [Google Scholar] [CrossRef]
  19. Abbott, A.P.; Capper, G.; Davies, D.L.; Rasheed, R.K.; Tambyrajah, V. Novel solvent properties of choline chloride/urea mixtures. Chem. Commun. 2003, 39, 70–71. [Google Scholar] [CrossRef] [Green Version]
  20. Van Osch, D.J.G.P.; Zubeir, L.F.; van den Bruinhorst, A.; Rocha, M.A.A.; Kroon, M.C. Hydrophobic deep eutectic solvents as water-immiscible extractants. Green Chem. 2015, 17, 4518–4521. [Google Scholar] [CrossRef]
  21. Ma, S.; Jin, X.; Wei, H.; Liu, Y.; Guo, M. Hydrophobic deep eutectic solvent-based ultrasonic-assisted liquid-liquid micro-extraction combined with HPLC-FLD for diphenylamine determination in fruit. Food Addit. Contam. Part A Chem. Anal. Control Expo. Risk Assess. 2021, 38, 339–349. [Google Scholar] [CrossRef] [PubMed]
  22. Khare, L.; Karve, T.; Jain, R.; Dandekar, P. Menthol based hydrophobic deep eutectic solvent for extraction and purification of ergosterol using response surface methodology. Food Chem. 2021, 340, 127979. [Google Scholar] [CrossRef] [PubMed]
  23. Zhu, S.; Zhou, J.; Jia, H.; Zhang, H. Liquid-liquid microextraction of synthetic pigments in beverages using a hydrophobic deep eutectic solvent. Food Chem. 2018, 243, 351–356. [Google Scholar] [CrossRef] [PubMed]
  24. Silva, N.; Morais, E.S.; Freire, C.S.R.; Freire, M.G.; Silvestre, A.J.D. Extraction of High Value Triterpenic Acids from Eucalyptus globulus Biomass Using Hydrophobic Deep Eutectic Solvents. Molecules 2020, 25, 210. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Huang, S.M.; Kuo, C.H.; Chen, C.A.; Liu, Y.C.; Shieh, C.J. RSM and ANN modeling-based optimization approach for the development of ultrasound-assisted liposome encapsulation of piceid. Ultrason. Sonochem. 2017, 36, 112–122. [Google Scholar] [CrossRef]
  26. Chouaibi, M.; Daoued, K.B.; Riguane, K.; Rouissi, T.; Ferrari, G. Production of bioethanol from pumpkin peel wastes: Comparison between response surface methodology (RSM) and artificial neural networks (ANN). Ind. Crops Prod. 2020, 155, 112822. [Google Scholar] [CrossRef]
  27. Musa, K.H.; Abdullah, A.; Al-Haiqi, A. Determination of DPPH free radical scavenging activity: Application of artificial neural networks. Food Chem. 2016, 194, 705–711. [Google Scholar] [CrossRef] [PubMed]
  28. Ciric, A.; Krajnc, B.; Heath, D.; Ogrinc, N. Response surface methodology and artificial neural network approach for the optimization of ultrasound-assisted extraction of polyphenols from garlic. Food Chem. Toxicol. 2020, 135, 110976. [Google Scholar] [CrossRef]
  29. Gammoudi, N.; Mabrouk, M.; Bouhemda, T.; Nagaz, K.; Ferchichi, A. Modeling and optimization of capsaicin extraction from Capsicum annuum L. using response surface methodology (RSM), artificial neural network (ANN), and Simulink simulation. Ind. Crops Prod. 2021, 171, 113869. [Google Scholar] [CrossRef]
  30. Peng, X.; Duan, M.-H.; Yao, X.-H.; Zhang, Y.-H.; Zhao, C.-J.; Zu, Y.-G.; Fu, Y.-J. Green extraction of five target phenolic acids from Lonicerae japonicae Flos with deep eutectic solvent. Sep. Purif. Technol. 2016, 157, 249–257. [Google Scholar] [CrossRef]
  31. Cai, C.; Wu, S.; Wang, C.; Yang, Y.; Sun, D.; Li, F.; Tan, Z. Deep eutectic solvents used as adjuvants for improving the salting-out extraction of ursolic acid from Cynomorium songaricum Rupr. in aqueous two-phase system. Sep. Purif. Technol. 2019, 209, 112–118. [Google Scholar] [CrossRef]
  32. Wang, L.-T.; Yang, Q.; Cui, Q.; Fan, X.-H.; Dong, M.-Z.; Gao, M.-Z.; Lv, M.-J.; An, J.-Y.; Meng, D.; Zhao, X.-H.; et al. Recyclable menthol-based deep eutectic solvent micellar system for extracting phytochemicals from Ginkgo biloba leaves. J. Clean. Prod. 2020, 244, 118648. [Google Scholar] [CrossRef]
  33. Zeng, C.; Liu, Y.; Ding, Z.; Xia, H.; Guo, S. Physicochemical properties and antibacterial activity of hydrophobic deep eutectic solvent-in-water nanoemulsion. J. Mol. Liq. 2021, 338, 116950. [Google Scholar] [CrossRef]
  34. Xu, Z.; Cai, Y.; Ma, Q.; Zhao, Z.; Yang, D.; Xu, X. Optimization of Extraction of Bioactive Compounds from Baphicacanthus cusia Leaves by Hydrophobic Deep Eutectic Solvents. Molecules 2021, 26, 1729. [Google Scholar] [CrossRef] [PubMed]
  35. Fan, X.-H.; Wang, L.-T.; Chang, Y.-H.; An, J.-Y.; Zhu, Y.-W.; Yang, Q.; Meng, D.; Fu, Y.-j. Application of green and recyclable menthol-based hydrophobic deep eutectic solvents aqueous for the extraction of main taxanes from Taxus chinensis needles. J. Mol. Liq. 2021, 326, 114970. [Google Scholar] [CrossRef]
  36. Alhadid, A.; Mokrushina, L.; Minceva, M. Influence of the Molecular Structure of Constituents and Liquid Phase Non-Ideality on the Viscosity of Deep Eutectic Solvents. Molecules 2021, 26, 4208. [Google Scholar] [CrossRef] [PubMed]
  37. Zhang, Q.; De Oliveira Vigier, K.; Royer, S.; Jerome, F. Deep eutectic solvents: Syntheses, properties and applications. Chem. Soc. Rev. 2012, 41, 7108–7146. [Google Scholar] [CrossRef] [PubMed]
  38. Naser, J.; Mjalli, F.; Jibril, B.; Al-Hatmi, S.; Gano, Z. Potassium Carbonate as a Salt for Deep Eutectic Solvents. Int. J. Chem. Eng. Appl. 2013, 4, 114–118. [Google Scholar] [CrossRef] [Green Version]
  39. Makoś, P.; Słupek, E.; Gębicki, J. Hydrophobic deep eutectic solvents in microextraction techniques—A review. Microchem. J. 2020, 152, 104384. [Google Scholar] [CrossRef]
  40. Sharma, G.; Sequeira, R.A.; Pereira, M.M.; Maity, T.K.; Chudasama, N.A.; Prasad, K. Are ionic liquids and deep eutectic solvents the same?: Fundamental investigation from DNA dissolution point of view. J. Mol. Liq. 2021, 328, 104384. [Google Scholar] [CrossRef]
  41. Xia, E.Q.; Yu, Y.Y.; Xu, X.R.; Deng, G.F.; Guo, Y.J.; Li, H.B. Ultrasound-assisted extraction of oleanolic acid and ursolic acid from Ligustrum lucidum Ait. Ultrason Sonochem. 2012, 19, 772–776. [Google Scholar] [CrossRef]
  42. Maran, J.P.; Priya, B.; Nivetha, C.V. Optimization of ultrasound-assisted extraction of natural pigments from Bougainvillea glabra flowers. Ind. Crops Prod. 2015, 63, 182–189. [Google Scholar] [CrossRef]
  43. Siani, A.C.; Nakamura, M.J.; Dos Santos, D.S.; Mazzei, J.L.; do Nascimento, A.C.; Valente, L.M. Efficiency and selectivity of triterpene acid extraction from decoctions and tinctures prepared from apple peels. Pharm. Mag. 2014, 10 (Suppl. S2), S225–S231. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Sha, W.; Edwards, K.L. The use of artificial neural networks in materials science based research. Mater. Des. 2007, 28, 1747–1752. [Google Scholar] [CrossRef]
Figure 1. Structure of ANN.
Figure 1. Structure of ANN.
Foods 12 00310 g001
Figure 2. The viscosity of different HDESs at 30 °C (a) and the viscosity of MT1:6 at 30~70 °C (b).
Figure 2. The viscosity of different HDESs at 30 °C (a) and the viscosity of MT1:6 at 30~70 °C (b).
Foods 12 00310 g002
Figure 3. The solubility of UA of HDESs composed of different types of HBA and HBD at 60 °C (a), and the solubility of MT1: 6HDES with different HBA and HBD molar ratios at 60 °C (b).
Figure 3. The solubility of UA of HDESs composed of different types of HBA and HBD at 60 °C (a), and the solubility of MT1: 6HDES with different HBA and HBD molar ratios at 60 °C (b).
Foods 12 00310 g003
Figure 4. The effects of different time (a), temperature (b), and solid–liquid ratio (c) on the extraction yield of UA.
Figure 4. The effects of different time (a), temperature (b), and solid–liquid ratio (c) on the extraction yield of UA.
Foods 12 00310 g004
Figure 5. The effect of time and temperature on the yield of UA extraction (a), the effect of solid–liquid ratio and temperature on the yield of UA extraction (b), and the effect of solid–liquid ratio and time on the yield of UA extraction (c).
Figure 5. The effect of time and temperature on the yield of UA extraction (a), the effect of solid–liquid ratio and temperature on the yield of UA extraction (b), and the effect of solid–liquid ratio and time on the yield of UA extraction (c).
Foods 12 00310 g005
Figure 6. Evolution of MSE during the ANN training phase.
Figure 6. Evolution of MSE during the ANN training phase.
Foods 12 00310 g006
Figure 7. Scatter plot of test data (target) compared with the computed neural network data in the training set (a), validation set (b), test set (c), and all sets (d).
Figure 7. Scatter plot of test data (target) compared with the computed neural network data in the training set (a), validation set (b), test set (c), and all sets (d).
Foods 12 00310 g007
Table 1. The molar ratio and aspect of HDES.
Table 1. The molar ratio and aspect of HDES.
AbbreviationHBAHBDMolar RatioAspectReference
MT 1:1MenThy1:1Transparent colorless liquid[24]
HDES 1Formic acid1:1White emulsion[32]
HDES 2Acetic acid1:1Transparent colorless liquid[22]
HDES 3Propionic acid1:1Transparent colorless liquid[32]
HDES 4Decanoic acid1:1Transparent colorless liquid[33]
HDES 5Isopropanol1:2Transparent colorless liquid[34]
HDES 6n-Butanol1:1Transparent colorless liquid[35]
HDES 7Pyruvic acid1:2Transparent yellow liquid[22]
HDES 8Lactic acid1:1Transparent colorless liquid[34]
HDES 9Levulinic acid1:1Transparent yellow liquid[34]
Table 2. Molar ratio of HBA and HBD.
Table 2. Molar ratio of HBA and HBD.
AbbreviationHBAHBDMolar RatioAspect
MT 4:1 4:1Transparent colorless solid
MT 3:1MenThy3:1Transparent colorless liquid
MT 2:12:1Transparent colorless liquid
MT 1:11:1Transparent colorless liquid
MT 1:21:2Transparent colorless liquid
MT 1:41:4Transparent colorless liquid
MT 1:61:6Transparent colorless liquid
MT 1:81:8Transparent colorless liquid
Table 3. Factors and levels of response surface analysis.
Table 3. Factors and levels of response surface analysis.
LevelsIndependent Variables
A (Time, min)B (Temperature, °C)C (Solvent to-Solid Ratio, g/mL)
−1154014
0305016
1456018
Table 4. Results of the Box–Behnken design (BBD) for the extraction yield of UA and ANN calculated.
Table 4. Results of the Box–Behnken design (BBD) for the extraction yield of UA and ANN calculated.
Run Factor EY/%RSM CalculatedANN Calculated
ABC
13040181.3611.31681.3277
23050161.5261.54021.5225
31560161.2091.18361.2090
44540161.2921.31361.2920
53050161.5671.54021.5225
64560161.2011.21211.2010
74550181.3661.38331.3660
83050161.521.54021.5225
91550181.2211.27671.2210
101540161.2031.18801.2030
113050161.5191.54021.5225
123060141.0941.13451.1115
133060181.331.29381.3300
144550141.2841.22451.3209
151550141.1981.17691.1980
163040141.1851.21731.1850
173050161.5781.54021.5225
Table 5. ANOVA statistics of the quadratic model for the extraction yields of UA.
Table 5. ANOVA statistics of the quadratic model for the extraction yields of UA.
SourceSum of SquaresdfMean SquareF-Valuep-Value
Model0.369190.041015.990.0007significant
A0.012210.01224.740.0658
B0.005410.00542.090.1917
C0.033410.033413.030.0086
AB0.002410..00240.91710.3701
BC0.000910.00090.33930.5785
AC0.000910.00090.35090.5722
A20.089110.089134.750.0006
B20.122010.122047.580.0002
C20.070310.070327.420.0012
Residual0.018070.0026
Lack of fit0.014830.00496.170.0556Not significant
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Li, H.; Liu, Y.; Guo, S.; Shi, M.; Qin, S.; Zeng, C. Extraction of Ursolic Acid from Apple Peel with Hydrophobic Deep Eutectic Solvents: Comparison between Response Surface Methodology and Artificial Neural Networks. Foods 2023, 12, 310. https://doi.org/10.3390/foods12020310

AMA Style

Li H, Liu Y, Guo S, Shi M, Qin S, Zeng C. Extraction of Ursolic Acid from Apple Peel with Hydrophobic Deep Eutectic Solvents: Comparison between Response Surface Methodology and Artificial Neural Networks. Foods. 2023; 12(2):310. https://doi.org/10.3390/foods12020310

Chicago/Turabian Style

Li, Haiyan, Yugang Liu, Shiyin Guo, Meng Shi, Si Qin, and Chaoxi Zeng. 2023. "Extraction of Ursolic Acid from Apple Peel with Hydrophobic Deep Eutectic Solvents: Comparison between Response Surface Methodology and Artificial Neural Networks" Foods 12, no. 2: 310. https://doi.org/10.3390/foods12020310

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop