Next Article in Journal
Application and Characterization of Gum from Bombax buonopozense Calyxes as an Excipient in Tablet Formulation
Previous Article in Journal
Development and Validation of a Successful Microbiological Agar Assay for Determination of Ceftriaxone Sodium in Powder for Injectable Solution

Pharmaceutics 2012, 4(3), 343-353; doi:10.3390/pharmaceutics4030343

Article
Design of Skin Penetration Enhancers Using Replacement Methods for the Selection of the Molecular Descriptors
Laurent Simon * and Beshoy Abdelmalek
Otto H. York Department of Chemical, Biological and Pharmaceutical Engineering, New Jersey Institute of Technology, Newark NJ 07102, USA; Email: bha5@njit.edu
*
Author to whom correspondence should be addressed; Email: laurent.simon@njit.edu; Tel.: +1-973-596-5263; Fax: +1-973-596-8436.
Received: 3 May 2012; in revised form: 25 June 2012 / Accepted: 28 June 2012 /
Published: 11 July 2012

Abstract

: Transdermal delivery of certain drugs is challenging because of skin barrier resistance. This study focuses on the implementation of feature-selection algorithms to design chemical penetration enhancers. A database, consisting of 145 polar and nonpolar chemicals, was chosen for the investigation. Replacement, enhanced replacement and stepwise algorithms were applied to identify relevant structural properties of these compounds. The descriptors were calculated using Molecular Modeling Pro™ Plus. Based on the coefficient of determination, the replacement methods outperformed the stepwise approach in selecting the features that best correlated with the flux enhancement ratio. An artificial neural network model was built to map a subset of descriptors from sixty-one nonpolar enhancers onto the output vector. The R2 value improved from 0.68, for a linear model, to 0.74, which shows that the improved framework might be effective in the design of compounds with user-defined properties.
Keywords:
skin penetration enhancer; replacement methods; neural networks

1. Introduction

Transdermal drug delivery is the transport of a therapeutic chemical substance into the body through the skin in the forms of ointment or a patch. The first transdermal drug-delivery system (TDDS), dating back to 1979, consisted of a scopolamine patch prescribed to treat motion sickness [1]. However, until the release of nicotine patches in 1991 [1], little attention was paid to this approach. Currently, there exists over 21 TDDSs. They are becoming more accepted, by the pharmaceutical industry, as viable drug-delivery methods.

The dermal route offers several advantages. Applying a transdermal patch is easy, painless and rapidly effective, as opposed to hypodermic injections or oraldrug-delivery systems, which usually require a long time for the medication to reach its target site. In spite of these benefits, only a limited number of drugs are administered using this method because of the skin barrier properties. The dermal membrane consists of three main layers of tissue: the epidermis (stratum corneum and viable epidermis), dermis and a subcutaneous region [2]. The skin is particularly resistant to the permeation of most external chemicals due to the presence of the stratum corneum, composed mainly of dead cells.

Chemical penetration enhancers (CPEs) are designed to stimulate transport across the skin. These sorption promoters (or accelerants) increase the drug flux and reduce the resistance of the skin barrier [3]. Penetration enhancers are added to the formulation of TDDSs to allow the permeation of molecules that would have crossed the dermal layer at a rate too slow for clinical applications or have been precluded from traversing the biological membrane. The effectiveness of an accelerant is measured by the Enhancement Ratio (ER), defined as the ratio of the flux in the presence of a fixed concentration of CPE to the delivery rate when the enhancer is not added to the formulation [4]. The application of CPEs has had positive impact on TDDSs. As a result, efforts have been devoted to the development of several numerical tools to help design CPEs that exhibit desired properties.

Previous studies suggest that the activity of a penetration enhancer is related to the structure of the drug as well as that of the enhancer [5]. A quantitative structure-activity relationship (QSAR) modeling framework can be adopted to derive mathematical relationships between the ER and the enhancers’ molecular descriptors (or features) [4]. However, the number and type of features to include in a model is generally not known a priori. In this study, three different methods for determining an optimal subset of molecular descriptors are investigated: the replacement method (RM), the enhanced replacement method (ERM) and the traditional forward stepwise regression. Artificial neural networks (ANNs) are then implemented to produce more accurate models. This contribution is the first to employ ERM and RM in selecting the best descriptors for transdermal applications.

The RM and ERM are provided to help identify the best combination of descriptors from a large pool of variables. Artificial neural networks are built to map a series of input patterns onto a set of output variables. When possible, these ANN models can be implemented to refine the initial QSAR models.

1.1. Replacement Method for the Selection of QSAR Descriptors

Given a multiple linear regression model:

Pharmaceutics 04 00343 i001

where xi is a feature (e.g., molecular weight) from a pool of size N and y is the output variable. The replacement method (RM) helps select a subset of M descriptors that result in the lowest standard deviation from Equation 1. In this procedure, an initial set of M features (V) is selected arbitrarily [6] from a population W. One of the descriptors in V is chosen (first path) and replaced one at a time by the remaining descriptors from W − V. The set that yields the smallest standard deviation (S) is saved [7]. The second step involves choosing the descriptor with the coefficient showing the greatest relative error and replacing it with features from the remaining set. Note that the replacement is made if it decreases S. This scheme is repeated until the set of descriptors remains unchanged. For each cycle, the descriptor optimized in the preceding cycle is not altered [6,8]. In the end, the best descriptor of the first path is obtained. The procedure continues for all of the M possible paths and the one with the smallest S is retained.

1.2. Enhanced Replacement Method for the Selection of QSAR Descriptors

The ERM can be applied to predict an optimal group of molecular descriptors from a large set. This technique consists of the implementation of two algorithms, the RM and a modified RM (or MRM), in the following sequence: RM, MRM and RM. The MRM follows the same strategy as the RM except that, in this case, the descriptor with the largest error is replaced in each step even when the substitution does not reduce the S value further [6].

1.3. Stepwise Methods for the Selection of QSAR Descriptors

The forward selection method assumes no variable in Equation (1) and adds one variable at a time [9]. A regressor is inserted if it improves the model as determined by the F-ratio statistic. The procedure continues until no selected predictor results in an F-ratio value larger than a user-defined threshold. In the backward elimination method, all the variables are included in Equation (1). The predictor with the smallest F-ratio is deleted. The process is repeated until the deletion of any feature would produce an F-ratio greater than a threshold. These techniques can also be implemented to find the best p-term equation using criteria that are based on the residual sum of squares [9].

1.4. Artificial Neural Networks

Artificial neural network (ANN) models were developed [10,11], as nonlinear predictors, to calculate an output vector from the QSAR descriptors. In this work, a feed-forward ANN model with back-propagation was built to estimate the enhancement ratio. Artificial neural networks include a set of weights, w (to be adjusted in a training phase), bias functions f (.) and parameter p:

Pharmaceutics 04 00343 i002

In a supervised ANN model, examples, representative of the process, are provided in the training phase. The best w and p, that minimize a performance criterion, are calculated. In the testing phase, new input patterns are supplied with the optimal w and p (Figure 1). Only one hidden-layer architecture is proposed for this application. There is also flexibility in the number of neurons associated with the hidden layer. Algorithms have been designed to prune large networks [12,13].

Pharmaceutics 04 00343 g001 1024
Figure 1. Artificial neural network model using one hidden layer and two nodes. The input vector is connected to the hidden neurons through weights w and p. The hidden layer and output variable are also connected. A nonlinear activation function σ is applied.

Click here to enlarge figure

Figure 1. Artificial neural network model using one hidden layer and two nodes. The input vector is connected to the hidden neurons through weights w and p. The hidden layer and output variable are also connected. A nonlinear activation function σ is applied.
Pharmaceutics 04 00343 g001 1024

1.5. Selection of Descriptors and Prediction of ER

Three skin penetration enhancer sets were analyzed in this study. These data consist of:

  • Sixty-one (61) nonpolar enhancers using hydrocortisone (HC) as a control in measuring ER (Table 1 of Iyer et al. [4]).

  • Forty-two (42) small and relative polar enhancers using hydrocortisone (HC) as control (Table 2 of Iyer et al. [4] except for AT-TERPENE/HC-7 and AT-TERPENE/HC-7).

  • Forty-two (42) nonpolar enhancers using Hydrocortisone Acetate (HCA) as a control (Table 3 of Iyer et al. [4]).

2. Experimental Section

The experimental method for measuring the ER was the same for all three datasets examined in this study [4]. The slope of the linear portion of the cumulative drug amount released versus the time corresponds to the steady-state flux (J, mg cm−2 h−1). The software MOLECULAR MODELING PRO™ Plus (MMP+) estimated the molecular descriptors of the 145 CPEs from their structures. MMP+ calculated approximately 114 descriptors. However, this number was reduced to 31 descriptors selected from the combined set used in studies of skin penetration enhancers provided in [4,5]. The ERM, RM and forward stepwise regression methods were introduced to discover significant relationships between a subset of structural parameters, taken from the 31 descriptors, and the ER. Statistical analyses were performed in MATLAB (The MathWorks Inc., Natick, MA, USA) using a QSAR/QSPR search algorithm Toolbox [6]. To minimize the possibility of correlation, the program was instructed to select eight descriptors, a number that was less than one-fifth the size of the supplied data [14]. A linear regression model was developed for each set and the following values were reported: N, the number of data; R2, the coefficient of determination; S, the standard deviation.

3. Results and Discussion

The relevant descriptors obtained by the enhanced replacement method were: hydrogen bond acceptor, polar surface area, moment of inertia, glass temperature (Tg), molar volume, radius of gyration (RG), dipole moment and polarity (Table 1). The applicability domain (AD) was not explicitly determined. However, the three data sets (i.e., Database or DB #1, DB #2, DB #3) obtained from Iyer et al. [4] were separated on the basis of polarity and the control drug (i.e., hydrocortisone and hydrocortisone acetate). In this sense, the data within a group, share a “common mechanism of action”.

The y-intercept and regression coefficients are also shown in the table. For example, the following equation was built for the 61 nonpolar enhancers (DB #1) using the ERM:

Pharmaceutics 04 00343 i003

The performance of the ERM was similar to that of the RM. The stepwise method produces the lowest R2 value (i.e., 0.671). Figure 2 illustrates both the experimental and predicted values.

Pharmaceutics 04 00343 g002 1024
Figure 2. Correlation between the linear predicted and experimental ER for the set of 61 nonpolar CPEs that enhanced the permeability of hydrocortisone. The enhanced replacement method (ERM) was applied for the selection of the descriptors.

Click here to enlarge figure

Figure 2. Correlation between the linear predicted and experimental ER for the set of 61 nonpolar CPEs that enhanced the permeability of hydrocortisone. The enhanced replacement method (ERM) was applied for the selection of the descriptors.
Pharmaceutics 04 00343 g002 1024

In the work of Iyer et al. [4], six classic QSAR descriptors yield an R2 of 0.732. Seventy-nine classic QSAR descriptors were drawn for the analyses. A quadratic model was fitted to the data. When using the forty-two small and relative polar enhancers (DB #2), R2 values for the different methods were less than 0.4. These findings agree with the results in Iyer et al. [4] because no meaningful model could be developed from classic QSAR descriptors. Figure 3 shows the calculated and experimentally determined ER values for the ERM.

Table Table 1. Results of the ERM, RM and the forward stepwise selection procedure applied to the three datasets (i.e., DB #1, 2 and 3). Databases 1 and 2 were composed of 61 nonpolar and 42 polar enhancers, respectively, with hydrocortisone serving as a control molecule. Forty-two compounds were in the third database. The R2, y-intercepts and coefficients of the multiple linear models are listed.

Click here to display table

Table 1. Results of the ERM, RM and the forward stepwise selection procedure applied to the three datasets (i.e., DB #1, 2 and 3). Databases 1 and 2 were composed of 61 nonpolar and 42 polar enhancers, respectively, with hydrocortisone serving as a control molecule. Forty-two compounds were in the third database. The R2, y-intercepts and coefficients of the multiple linear models are listed.
DB # 1 NonpolarDB # 2 PolarDB # 3 Nonpolar
N = 61 HydrocortisoneN = 42 HydrocortisoneN = 42 Hydrocortisone Acetate
ERMRMStepwiseERMRMStepwiseERMRMStepwise
R20.6830.6830.6710.3850.3610.3230.5220.4880.390
Intercept−120.81−120.81−132.43−14.98−14.506.39233.64181.96203.21
Molecular Descriptor
Density (g/cm3) −237.02
Dipole Moment (debye)−5.07−5.07−5.24
Dispersion (J/cm3)1/2 1.571.022.76 × 10−1
Number of Torsional Bonds −12.67−9.22
Energy of Cohesion (J/mol) 1.00 × 10−41.00 × 10−41.00 × 10−47.00× 10−4
Hydrophilic Lipophilic Balance (HLB)
Highest Occupied Molecular Orbital (HOMO) 1.02532.18449.36
Hydrogen Bond Acceptor80.4380.4383.28−3.74 127.2892.98
Hydrogen Bond Donor −9.81−1.63 −121.54
Hydrogen Bonding (J/cm3)1/2 4.36 × 10−1 −7.32 −5.83
Hydrophilic Surface Area (cm2/mol)
Kappa 2
Log P −1.15
Lowest Unoccupied Molecular Orbital (LUMO) −7.33 × 10−1−1.32
Mean Water of Hydration
Molar Volume (cm3/mol)−6.54 × 10−2−6.54 × 10−2
Molecular Length (Å) 1.65 7.276.72−2.56
Molecular Volumes(Å3) −4.27 × 10−1 2.64
Molecular Weight (Da) −5.53× 10−2 −1.85
Molecular Width (Å) −1.51 −1.71
Moment of Inertia (g cm2/mol)−1.70 × 10−3−1.70 × 10−3−2.00 × 10−3
Molar Refractivity (MR) −12.10−4.01
Percent Hydrophilic Surface Area
Polar Surface Area (Å2)−1.76−1.76−1.82 −6.53× 10−1
Polarity (J/cm3)1/220.6920.6921.01 3.69 × 10−14.05 6.44
Radius of Gyration (Å)16.9616.9612.04
Surface Area (Å2) 3.39× 10−1 2.44
Surface Tension (dynes/cm) −8.79× 10−2−6.90 × 10−2−8.83 × 10−2 2.22
Tg (C) −1.10× 10−1−1.10 × 10−1−1.01 × 10−1 3.83× 10−2
Tm (C)
Water Solubility (mol/L) 2.85× 10−1
Pharmaceutics 04 00343 g003 1024
Figure 3. Correlation between the linear predicted and experimental ER for the set of 42 polar CPEs from DB #2. The enhanced replacement method (ERM) was applied for the selection of the descriptors.

Click here to enlarge figure

Figure 3. Correlation between the linear predicted and experimental ER for the set of 42 polar CPEs from DB #2. The enhanced replacement method (ERM) was applied for the selection of the descriptors.
Pharmaceutics 04 00343 g003 1024

The third dataset (DB #3) yields R2 values of 0.522, 0.488 and 0.390 for the ERM, RM and stepwise method, respectively. The data are plotted in Figure 4. For the ERM, Based on 77 classic descriptors and applying a quadratic model, an R2 value of 0.738 was achieved in Iyer et al. [4]. It is customary to consider log(ER) instead of ER when modeling skin penetration enhancers [3]. However, this measure did not necessarily lead to improved outcomes. For example, based on the ERM algorithm, the molecular weight and Log(P) are determining factors in the formulation of enhancers from DB #1. Yet, a much lower R2 of 0.47 was observed.

Pharmaceutics 04 00343 g004 1024
Figure 4. Correlation between the linear predicted and experimental ER for the set of 42 nonpolar CPEs from DB #3. The enhanced replacement method (ERM) was applied for the selection of the descriptors.

Click here to enlarge figure

Figure 4. Correlation between the linear predicted and experimental ER for the set of 42 nonpolar CPEs from DB #3. The enhanced replacement method (ERM) was applied for the selection of the descriptors.
Pharmaceutics 04 00343 g004 1024

Replacement techniques (i.e., ERM and RM) are more effective than the stepwise method in selecting the best descriptors for predicting skin penetration enhancement. Given that a multiple linear model was adopted and a relatively small pool (31 versus 77 and 79 features) was chosen, this approach is comparable to other techniques published in the literature. The inclusion of features, such as 4D-fingerprint descriptors [4], may help improve the performance of replacement procedures.

The significance of the parameters to enhance drug permeation has been reported by other researchers. For example, Potts and Guy employed hydrogen bond acceptors and the molecular volume to predict skin permeability [15]. A smaller average dipole moment of the penetration enhancer causes the enhancement ratio to increase [16]. The radius of gyration, a topological index which describes the compactness of a molecule, affects the ability of the enhancer to penetrate the skin layers. This parameter was used by Zheng et al. [16]. In the design for skin penetration enhancers, the ERM may serve as a tool for selecting molecular structures that play a key role in promoting transdermal drug delivery. Artificial neural networks are then built to generate a more accurate model. Database #1 was selected for this application because it produced the best results. The Neural Networks package in Mathematica (Wolfram Reasearch, Inc., Champaign, IL, USA) was chosen for this study. The descriptors obtained by the ERM method were used as inputs to the neural networks. One hidden layer and two nodes were selected to capture the input-output relationship. A simple topology was adopted to prevent over-parameterization and to reduce the complexity of the final NN equation. The database was first randomized and divided into a training set, consisting of 75% of the compounds, and a testing set. Input and output patterns were scaled between 0.1 and 0.9. A feed-forward ANN model, composed of a sigmoid-type activation function, one hidden layer and two neurons, was constructed using eight inputs and one output (i.e., an 8:2:1 network). The total number of iterations was set at 150. To avoid over-training, the training phase was instructed to stop if the MSE did not decrease steadily for a number of consecutive iterations. It took 35 iterations to identify an input-output relationship (Figure 5). The optimized neural network function, based on the scaled vectors, is:

Pharmaceutics 04 00343 i004

where:

Pharmaceutics 04 00343 i005

and:

Pharmaceutics 04 00343 i006

The results of the training are shown in Figure 5 (R2 = 0.74). The ANN model outperformed Equation 1 when it is applied to the training set and was able to predict the testing set (Figure 6, R2 = 0.62).

Pharmaceutics 04 00343 g005 1024
Figure 5. Training results for DB #1 (N = 47 patterns). The ERM was used for the selection of the descriptors.

Click here to enlarge figure

Figure 5. Training results for DB #1 (N = 47 patterns). The ERM was used for the selection of the descriptors.
Pharmaceutics 04 00343 g005 1024
Pharmaceutics 04 00343 g006 1024
Figure 6. Testing results for DB #1 (N = 14 patterns). The enhanced replacement method was used for the selection of the descriptors.

Click here to enlarge figure

Figure 6. Testing results for DB #1 (N = 14 patterns). The enhanced replacement method was used for the selection of the descriptors.
Pharmaceutics 04 00343 g006 1024

An analytical ANN model makes it possible to design penetration enhancers based on user-defined performance criteria. The ERM-ANN approach can be extended to include the irritation potential (IP) of the CPE [5]. This strategy is suitable for estimating the flux enhancement ratio given a set of molecular descriptor values (x). The inverse problem [17], which consists of computing x necessary to meet a target ER, is more difficult. The reverse engineering of chemical structures from such descriptors is also a challenge [18]. However, robust techniques, such as the ERM, that identify the relevant features and powerful data-based modeling tools could play important roles in the process.

4. Conclusions

A database of skin penetration enhancers was assembled to compare the performance of replacement techniques to the more traditional stepwise method, when selecting the best molecular descriptors. The collection was divided into three sets that differ by the control compound and the polarity of the accelerants. R2 values of 0.683, 0.683 and 0.671, which corresponded to the enhanced replacement (ERM), replacement (RM) and forward stepwise methods, respectively, were computed for 61 nonpolar enhancers with hydrocortisone serving as the reference (DB #1). The other two case studies, composed of forty-two compounds each, yielded lower R2 values. Better predictions were obtained when the ERM-resulted selections for DB #1 were fed into a feed-forward back-propagation network model trained with only two hidden neurons (R2 = 0.74). The R2 value for the testing phase was found to be 0.62. Based on these results, the combined ERM and artificial neural network approach can be implemented in the design of chemical enhancers.

Acknowledgments

This work was supported in part by the Ronald E. McNair Post-Baccalaureate Achievement Program (U.S. Department of Education).

Conflict of Interest

The authors declare no conflict of interest.

References and Notes

  1. Prausnitz, M.R.; Langer, R. Transdermal drug delivery. Nat. Biotechnol. 2008, 26, 1261–1268. [Google Scholar] [CrossRef]
  2. Jiang, S.C.; Ma, N.; Li, H.J.; Zhang, X.X. Effects of thermal properties and geometrical dimensions. Burns 2002, 28, 713–717. [Google Scholar] [CrossRef]
  3. Ghafourian, T.; Zandasrar, P.; Hamishekar, H.; Nokhodchi, A. The effect of penetration enhancers on drug delivery through skin: A QSAR study. J. Contr. Release. 2004, 99, 113–125. [Google Scholar] [CrossRef]
  4. Iyer, M.; Zheng, T.; Hopfinger, A.J.; Tseng, Y.J. QSAR analyses of skin penetration enhancers. J. Chem. Inf. Model. 2007, 47, 1130–1149. [Google Scholar] [CrossRef]
  5. Karande, P.; Jain, A.; Ergun, K.; Kispersky, V.; Mitragotri, S. Design principles of chemical penetration enhancers for transdermal drug delivery. Proc. Natl. Acad. Sci. USA 2005, 102, 4688–4693. [Google Scholar]
  6. Mercader, A.G; Duchowicz, P.R.; Fernandez, F.M; Castro, E.A. Replacement method and enhanced replacement method versus the genetic algorithm approach for the selection of molecular descriptors in QSPR/QSAR theories. J. Chem. Inf. Model. 2010, 50, 1542–1548. [Google Scholar]
  7. Duchowicz, P.R.; Castro, E.A.; Fernández, F.M.; González, M.P. A new search algorithm for QSPR/QSAR theories. Normal boiling points of some organic molecules. Chem. Phys. Lett. 2005, 412, 376–380. [Google Scholar] [CrossRef]
  8. Mercader, A.G; Duchowicz, P.R.; Fernández, F.M.; Castro, E.A. Modified and enhanced replacement method for the selection of molecular descriptors in QSAR and QSPR theories. Chemometr. Intell. Lab. Syst. 2008, 92, 138–144. [Google Scholar] [CrossRef]
  9. Hocking, R.R. The analysis and selection of variables in linear regression. Biometrics 1976, 32, 1–49. [Google Scholar] [CrossRef]
  10. Neely, B.J.; Madihally, S.V.; Robinson, R.L.J.; Gasem, K.A.M. Nonlinear quantitative structure-property relationship modeling of skin permeation coefficient. J. Pharm. Sci. 2009, 98, 4069–4084. [Google Scholar] [CrossRef]
  11. Yerramsetty, K.M.; Neely, B.J.; Madihally, S.V.; Gasem, K.A.M. A skin permeability model of insulin in the presence of chemical penetration enhancer. Int. J. Pharm. 2010, 388, 13–23. [Google Scholar] [CrossRef]
  12. Tolstrup, N. Pruning of a large network by optimal brain damage and surgeon: an example from biological sequence analysis. Int. J. Neural. Syst. 1995, 6, 31–42. [Google Scholar] [CrossRef]
  13. Gorodkin, J.; Hansen, L.K; Krogh, A.; Svarer, C.; Winther, O. A quantitative study of pruning by optimal brain damage. Int. J. Neural. Syst. 1993, 4, 159–169. [Google Scholar] [CrossRef]
  14. Perkins, R.; Fang, H.; Tong, W.; Welsh, W.J. Quantitative structure-activity relationship methods: perspectives on drug discovery and toxicology. Environ. Toxicol. Chem. 2003, 22, 1666–1679. [Google Scholar] [CrossRef]
  15. Potts, R.O.; Guy, R.H. A predictive algorithm for skin permeability: The effects of molecular size and hydrogen bond activity. Pharmceut. Res. 1995, 12, 1628–1633. [Google Scholar] [CrossRef]
  16. Zheng, T.; Hopfinger, A.J.; Esposito, E.X.; Liu, J.; Tseng, Y.J. Membrane-interaction quantitative structure-activity relationship (MI-QSAR) analyses of skin penetration enhancers. J. Chem. Inf. Model. 2008, 48, 1238–1256. [Google Scholar] [CrossRef]
  17. Simon, L.; Fernandes, M. Neural network-based prediction and optimization of estradiol release from ethylene–vinyl acetate membranes. Comput. Chem. Eng. 2004, 28, 2407–2419. [Google Scholar] [CrossRef]
  18. Faulon, J.; Brown, W.M.; Martin, S. Reverse engineering chemical structures from molecular descriptors: how many solutions? J. Comput. Aided. Mol. Des. 2005, 19, 637–650. [Google Scholar] [CrossRef]
Pharmaceutics EISSN 1999-4923 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert