Next Article in Journal
Diffusion-Denoising Process with Gated U-Net for High-Quality Document Binarization
Next Article in Special Issue
The Effects of Angelica ternata Extract from Kyrgyzstan on the Formation of Candida albicans ATCC 10231 Biofilms
Previous Article in Journal
Seismic Behavior and Modeling of Ductile Composite Steel-Trussed Concrete Beam to Column Joints
Previous Article in Special Issue
Phytochemical Composition and Antioxidant Properties of Tambourissa ficus, a Mauritian Endemic Fruit
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Influence of Selected Factors on Antioxidant Properties of Chlorogenic Acid and Kaempferol as Well as Their Binary Mixtures

by
Małgorzata Olszowy-Tomczyk
* and
Dorota Wianowska
Department of Chromatography, Institute of Chemical Sciences, Faculty of Chemistry, Maria Curie-Skłodowska University in Lublin, Pl. Maria Curie-Skłodowska 3, 20-031 Lublin, Poland
*
Author to whom correspondence should be addressed.
Appl. Sci. 2023, 13(20), 11138; https://doi.org/10.3390/app132011138
Submission received: 24 August 2023 / Revised: 5 October 2023 / Accepted: 9 October 2023 / Published: 10 October 2023
(This article belongs to the Special Issue Bioactive Compounds from Natural Products - Volume II)

Abstract

:

Featured Application

The obtained results not only extend knowledge about the antioxidant properties of mixtures but can be helpful in the effective design of functional foods. This is a particularly important aspect from the point of view of both the manufacturer and the consumer, whose consciousness regarding the promotion and propagation of a healthy lifestyle has recently increased. The issue of the increase in the average age of many communities is also important. Taking into account the increasing tendency of the incidence of age-related diseases, more and more attention is paid to supplementing the body with natural products with high antioxidant activity.

Abstract

Plants are a rich source of substances with antioxidant properties, including high amounts of polyphenolic compounds, as well as other substances that do not necessarily have antioxidant properties. The presented paper concerns the evaluation of the antioxidant properties of polyphenolics and their binary mixtures in the presence of other components of the system that do not exhibit antioxidant properties. Model systems containing chlorogenic acid and kaempferol were used in the experiments, differing not only in the volume of the alcoholic antioxidant solution in the measurement system, but also in temperature, content of water, metal ions and hydrogen ions. The ability to neutralize free radicals in the systems was estimated using the ABTS method. In light of the presented data, the dominant resultant antioxidant effect in the mixture is antagonism. Its value depends not only on the mutual relations of individual compounds, but also on the presence of other components in the tested system, not only antioxidants. The greatest effect on the antioxidant properties was observed for systems containing iron ions, ethanol and in the presence of hydrogen ions. The obtained results indicate an extremely complex nature of the assessment of antioxidant properties, even in seemingly simple measurement systems.

1. Introduction

The formation of free radicals in living organisms is natural and most organisms are able to neutralize these reactive oxygen species themselves [1,2]. Nevertheless, in addition to their own neutralization system, they also use the antioxidant properties of certain nutrients for this purpose. Hence, the role of antioxidants in nutrition is an area of growing interest [3]. Supplementing the body with food containing substances with antioxidant properties is the basic way to protect it against the harmful effects of reactive oxygen species, including free radicals, which are the activators of many dangerous diseases [4,5]. Scientific research carried out around the world proves that reducing the number of free radicals with the help of a proper diet increases the body’s immunity, stops the development of some cancers and, most importantly, prolongs life [6].
Most of the food which is consumed by humans occurs in the form of complex mixtures containing at least several antioxidants, each of which may have a different antioxidant capacity. It is known that in the case of mixtures of antioxidants, their antioxidant potential is not always an additive value consisting of the antioxidant properties of individual components [7,8,9]. The results of performed experiments often indicate an antagonistic or synergistic interaction of antioxidant compounds [10,11,12]. Moreover, other components of mixtures which occur in nature, not only the antioxidants present in them, may affect the overall antioxidant properties of the entire mixture. The experiments carried out in simple measuring systems containing only single antioxidants have shown that factors such as metal ions, water or reaction solvent, as well as pH, influence the estimation of antioxidant activity of compounds in spectrophotometric methods (for example in DPPH, ABTS and β-carotene bleaching assays) [13,14,15]. The presence of the above-mentioned factors differentiating the measurement systems may also affect the resultant antioxidant effect of the mixture. There is no information in the literature about research performed in this regard. Therefore, the aim of this paper is to check how the selected factors affect the resultant antioxidant effect of a simple mixture containing only two antioxidants. Due to the fact that fruits and vegetables, which are components of the human diet, are a rich source of polyphenolic compounds (exogenous antioxidants), two popular compounds from this group were used in the research, i.e., kaempferol and chlorogenic acid. The choice of these antioxidants was dictated by the fact that they are present in commonly consumed plant products. For example, kaempferol is present in large amounts in onions, broccoli, chives, tea, grapes, tomatoes or strawberries. In turn, chlorogenic acid is a bioactive phenolic compound found in potatoes, eggplants, apples, plums or coffee beans [16,17]. Both of these antioxidants occur side by side in cherry and elderberry fruits [18,19,20]. The study of the influence of the presence of hydrogen ions, water, and metal ions on the properties of these compounds and their binary mixtures, as well as the temperature and type of solvent on these properties, was carried out using the ABTS method. The experiments used the concentrations of antioxidants in which these compounds typically occur in plants. In order to assess the impact of changing the concentration of each of them on the resultant antioxidant effect of the mixture, the tests were carried out at different volume ratios of the components of the measurement system.

2. Materials and Methods

2.1. Chemicals

Kaempferol, chlorogenic acid, 2,2′-azinobis(3-ethylbenzothiazoline-6-sulfonic acid) diammonium salt (ABTS) and potassium persulfate (di-potassium peroxdisulfate) were purchased from Sigma Aldrich (Poznań, Poland). Copper (II) chloride nine hydrate, disodium hydrogen phosphate dehydrate, iron (III) sulphate (VI) pentahydrate, phosphoric acid (V), sodium dihydrogen phosphate dihydrate, ethanol (EtOH) and methanol (MeOH) were purchased from the Polish Chemical Plant POCh (Gliwice, Poland). Water was purified on a Milli-Q system from Millipore (Millipore, Bedford, MA, USA).

2.2. Impact of Temperature

The effect of temperature on the antioxidant properties of kaempferol and chlorogenic acid and their binary mixtures was studied at different volume ratios of reactants in each mixture and at different temperatures (15 °C, 25 °C, 35 °C). For this purpose, methanolic solutions of chlorogenic acid and kaempferol with a concentration of 0.06 mg/mL were prepared, which were mixed in various volume ratios. The volumes of reagent solutions (antioxidants and radical) used in the one-component systems (containing a single antioxidant) and two-component systems (being a binary mixture) are presented in Table 1.

2.3. Impact of the Solvent

The influence of the solvent on the antioxidant properties of kaempferol and chlorogenic acid and their binary mixtures was examined using methanol and ethanol as the reaction medium. For this purpose, methanolic and ethanolic solutions of chlorogenic acid and kaempferol with a concentration of 0.06 mg/mL were prepared, which were mixed at different volume ratios. The effect of ethanol was determined using the same volumes of components of the tested systems as in methanol, with the difference that the solvent of all the solutions used to create the systems was ethanol. The volumes of reagent solutions (antioxidants and radicals) used in the one-component systems (containing a single antioxidant) and two-component systems (being a binary mixture) are presented in Table 1.

2.4. Influence of the Addition of Water, Metals and Hydrogen Ions on the Antioxidant Properties of the Examined Antioxidants and Their Binary Mixtures

The effect of the addition of water, metal ions and hydrogen ions on the antioxidant properties of kaempferol and chlorogenic acid and their binary mixtures was determined by adding 100 μL of the tested factor (water or an aqueous solution of copper (II) or iron (III) ions with an ion concentration of 0.001 mg/mL and 0.022 mg/mL, respectively, or buffer with a specific pH) to systems containing solutions of: antioxidant/antioxidants and radical cation. The components’ volume of the measuring systems containing both single antioxidants and their binary mixtures are presented in Table 1.

2.5. Preparation of the Radical Cation

The antioxidant properties of the examined antioxidants and their mixtures were determined using the ABTS method. It consists of monitoring changes in the color of the ABTS cation radical, which is formed as a result of the reaction of 2,2′-azinobis(3-ethylbenzenethiazoline-6-sulfonate) (ABTS) with potassium persulfate (K2S2O8). Generation of the radical cation lasts 12 h after mixing 5 mL of 7 mM ABTS solution with 88 µL of 140 mM K2S2O8 at room temperature, away from light [21]. Before each measurement, the radical cation formed is diluted with a solvent to an absorbance of 0.7 ± 0.02 at a wavelength of 744 nm. As a result of the reaction with individual antioxidants, the blue–green color of the solution disappears, which is related to the neutralization of the ABTS radical cation. Changes in the absorbance of the solution were monitored at 744 nm using a UV Probe-2550 spectrophotometer. Measurements were carried out in systems containing specific volumes of antioxidants or their two-component mixtures and solutions of factors differentiating the tested systems (the exact volumetric compositions of the measuring systems are presented in Table 1). Inhibition percentages (%I) defining the antioxidant properties of the tested substances and their mixtures were calculated from the equation
I ( % ) = ( 1 A 60 A 0 ) 100 %
where A0 and A60 are the values of ABTS●+ absorbance at 0 and 60 min of the radical neutralization reaction, respectively.

2.6. Statistical Analysis

All results are presented as mean values of five independent measurements ± standard deviation (SD). The one-way analysis of variance (ANOVA) and Fisher coefficient (F) value were used to assess the influence of experimental factors on the activity. The statistical analysis was performed using Excel (Microsoft Excel 2010).
If the calculated value of F (Fcal) exceeds the tabular value F (Fcrit), this indicates a statistically significant influence of the given parameter. To determine the significance of each Fisher coefficient, the p-values were used. The values were considered to be significantly different when the result of the compared parameters differed at the p = 0.05 significance level.

3. Results and Discussion

Figure 1, Figure 2, Figure 3 and Figure 4 show the antioxidant properties, expressed as percent inhibition, determined for the one-component systems containing either chlorogenic acid or kaempferol and for the systems that are binary mixtures thereof. The studies used methanol or ethanol solutions of the tested antioxidants at a concentration of 0.06 mg/mL, which were mixed with each other in different volume ratios to prepare their binary mixtures (20/80; 50/50; 80/20 v/v—see Table 1). Each of the presented graphs was created on the basis of data obtained in various measurement systems, which were differentiated due to the temperature (Figure 1), solvent used (Figure 2) or the presence of water (Figure 3), or metal ions or hydrogen ions (Figure 4). In each of the figures presented, the red curve shows the changes in antioxidant activity in systems containing a variable amount of chlorogenic acid, while the blue curve reflects the antioxidant properties observed in the systems with different kaempferol content. Additionally, each figure contains two curves—the purple one, which is called “the experimental curve”, and the green one, which is the so-called “calculated curve”. The former shows the antioxidant activity changes for the experimental systems containing different volumes of the test compounds in the binary mixtures and the latter presents the calculated value of the antioxidant activity in the mixtures. The calculated curve was constructed based on the summation of antioxidant activity obtained for individual antioxidants in one-component systems containing the same amount of the ingredient as in the two-component mixture. For clarification, for a mixture containing 20 µL of chlorogenic acid and 80 µL of kaempferol, the calculated percentage of inhibition is 46.12 as shown in point “d” in Figure 1A. This value was obtained after adding up the % of inhibition determined in the systems containing: 20 μL of chlorogenic acid solution, i.e., 12.07% (value in point “a” on the graph, single-component system marked No. 1 in Table 1) and 80 μL of kaempferol solution, i.e., 34.05% (value in point “b” on the graph, single-component system marked No. 6 in Table 1). Point “c” is the experimental value (Ie), the actual value obtained for a mixture containing 20 µL of chlorogenic acid and 80 µL of kaempferol. The difference between the experimental value, the value in point “c” (Ie), and the theoretical value, the value in point “d” (Ic), indicates the observed, resultant effect of the antioxidant effect of the mixture—if it is negative, we are dealing with antagonism, and if it is a positive value, we are dealing with synergism. It is worth noting at this point that in the plots, the antioxidant amounts in the measuring system are expressed as the volume of its solution of a given concentration in a 100 μL sample introduced to the system. The solution volume of the chlorogenic acid in the pair, is shown on the lower X axis. The volume of the kaempferol is shown on the upper X axis. Values on the axis titled “volume for kaempferol” (top axis) should be read from right to left.
Table 2, Table 3, Table 4 and Table 5 present the results of statistical analysis obtained for the differences between the inhibition percent determined experimentally (Ie) and the percentages of theoretical inhibition (Ic) being the sum of the inhibition percent determined for single antioxidants (for a solution in which the amount of the component, i.e., chlorogenic acid or kaempferol, corresponds to the amount present in the mixture). The tables contain data obtained for % of inhibition determined at different volume ratios of antioxidants in their mixture. In the interpretation of the results aimed at determining the resultant antioxidant effect of the mixture, it was assumed that the lack of a significant difference between the Ie and Ic values indicates the additive antioxidant effect of the mixture containing chlorogenic acid and kaempferol. On the other hand, statistically significant differences (Fcal > Fcrit) indicate an antagonistic or synergistic antioxidant effect of the mixture. In the case of the first of the two mentioned effects, there is a negative value of the difference between Ie and Ic, while its positive value is responsible for the second effect.

3.1. Influence of Temperature on the Resultant Effect of the Mixture’s Antioxidant Activity

As noted above, Figure 1 shows changes in the percentage of inhibition as a function of concentration (volume) obtained for systems containing single antioxidants (chlorogenic acid or kaempferol) and their binary mixtures. The experiments were carried out at three different temperatures, i.e., 15 °C (see Figure 1A), 25 °C (see Figure 1B) and 35 °C (data in Figure 1C).
As can be seen from the presented relationships, regardless of the temperature used, an increase in antioxidant properties (increase in % of inhibition) is observed for individual antioxidants with increasing concentration (volume) of the antioxidant in the measuring system. This is not surprising because the more antioxidant, the better the antioxidant properties. As for temperature, according to the literature [22], temperature is one of the important factors affecting antioxidant activity. As it results from the presented data, the temperature affects the % inhibition values obtained for systems containing single antioxidants. At a higher temperature, higher % inhibition values are observed for the same measurement system than at a lower temperature.
In the literature, any changes in antioxidant properties resulting from an increase in temperature are most often explained by changes in the oxidation initiation process (acceleration with increasing temperature) and/or changes in the mechanism of antioxidant action [23]. Undoubtedly, the temperature also affects the kinetics of the radical neutralization reaction by the antioxidant. It should be noted here that the measurements of antioxidant properties were carried out in the 60th minute of the reaction, i.e., after a certain time, not necessarily sufficient for the reaction to reach a steady state. Additionally, the increase in temperature affects the dissociation of the antioxidant, which is necessary for the neutralization reaction of the ABTS radical cation [24]. Moreover, under the influence of temperature, the structure of the solvent may change (breakdown of clusters in the form of which methanol is present), which may facilitate the transfer of the electron and/or hydrogen from the antioxidant to the radical, thus resulting in a faster reaction [25,26].
In light of the presented data, it seems more important to compare the position and course of the curves obtained for binary mixtures—experimental and computational. These curves do not overlap; the theoretical curve is above the experimental curve, which indicates the antagonistic antioxidant effect of the mixture containing chlorogenic acid and kaempferol. There are not many reports in the literature about the antagonistic effects of compounds in the mixture. For example, in [27] it was shown that the co-occurrence of alkaloids and saponins significantly reduces the antioxidant activity of the quinine tree (Rauvolfiacaffra sond) extracts (activity of alkaloids alone  =  63%; activity of alkaloids with saponins  = 15%). A similar effect was described in [28] for mixtures of β-carotene with flavonoids (daidzein, baicalein) or with green tea polyphenols ((−)-epicatechin (EC), (−)-epigallocatechin (EGC), (−)-epicatechingallate (ECG) and (−)-epigallocatechingallate (EGCG)). The last of the cited works is significant in that it confirms the results presented in this paper. It was shown that although α-tocopherol with caffeic or ferulic acid has a synergistic effect, its combination with chlorogenic acid shows antagonism. According to the results presented in this study in Figure 1, this effect is observed regardless of the measurement temperature used. This conclusion is also confirmed by the data in Table 2F > Fcrit at p < 0.05 are obtained in all tested systems.
Assuming that the size of the F value is an exponent of the size of the antagonistic effect, it can be stated that the greatest antagonism is observed at 15 and 35 °C (F values for these temperatures are the highest). Moreover, by comparing the F value with the change in the concentration of chlorogenic acid, it is possible to observe a temperature-independent increase in the F value (increase in the antagonistic effect) with an increase in the concentration of chlorogenic acid. At room temperature, for the two volume ratios, i.e., 20/80 and 50/50, the changes are the smallest because the F values only slightly exceed the Fcrit value.
It is difficult to unambiguously explain the observed antagonistic effect, because it is not only the result of the temperature effect on the individual components of the measurement system, but it may also be the effect of the interaction of the components (antioxidants) with each other. According to the literature, the antagonistic antioxidant effect may be the result of a weaker antioxidant being rebuilt by a stronger one [7]. However, in light of the presented results, the validity of this hypothesis in the case of the kaempferol–chlorogenic acid mixture is questionable. According to the data in Figure 1B (see blue and red curves), kaempferol is a slightly stronger antioxidant than chlorogenic acid. Therefore, it would be expected that with an increase in the amount of kaempferol and not chlorogenic acid, an increase in the antagonistic effect of antioxidants in the mixture would be observed. The presented data therefore suggest that the observed antioxidant antagonism in the mixtures of the tested compounds does not result from the interactions between the individual components of the mixture, but rather from the difference in the kinetics of the reaction of a given antioxidant with the ABTS cationic radical. It seems that in the binary systems, the availability of the ABTS cation radicals for the weaker antioxidant molecules (chlorogenic acid) is reduced; hence, the observed antagonism.

3.2. Influence of Solvent on the Resultant Effect of the Mixture’s Antioxidant Activity

Figure 2 shows changes in % inhibition as a function of concentration (volume) in systems containing single antioxidants (chlorogenic acid or kaempferol) and their binary mixtures. These data were obtained using different reaction solvents—methanol (Figure 2A) and ethanol (Figure 2B). At this point, it should be noted that the data in Figure 2A are a repetition of the data in Figure 1B, and this repetition was made intentionally. The methanolic system at 25 °C was taken as a reference system to facilitate the interpretation of the results.
The analysis of the presented data allows us to conclude that while there are no clear differences in the antioxidant properties of kaempferol and chlorogenic acid determined in different solvents in one-component systems (compare the course of two red and two blue curves in both figures), the comparison of the location and course of the experimental and theoretical curves in Figure 2A,B for mixtures leads to the conclusion that in both cases these curves do not coincide, which indicates the lack of additive antioxidant activity. Moreover, this effect is more pronounced in the case of ethanol. The location of the theoretical curve over the experimental one indicates the antagonistic antioxidant behavior of the mixture. The antagonism is confirmed by the data in Table 3. In the case of analogous double systems (mixtures of antioxidants with the same composition), in which ethanol was used instead of methanol as the reaction solvent, the F values are higher, and the p values are lower (see Table 3).
When analyzing the effect of different solvents, it should be taken into account that the influence of the alcohol type on the antioxidant activity of the binary mixture may be related to the structural differences of the bulk alcohols, which consist of hydrogen-bonded clusters. These clusters exist in thermodynamic equilibrium. Borowski et al. [25,26] showed that hepta-, hexa-, penta-, tetra- and trimeric cluster structures predominate in methanol at room temperature, while monomeric structures are present in small amounts. On the other hand, in ethanol, pentamer structures are mainly present. It cannot be ruled out that different alcohol structures may affect the kinetics of the neutralization reaction of the ABTS radical cation. And although this effect is not so visible in one-component systems, it is more pronounced in two-component systems and different clustering is responsible for the kinetics of the reaction, and thus for the antioxidant antagonistic effect.

3.3. Influence of Water Addition on the Resultant Effect of the Mixture’s Antioxidant Activity

Figure 3A shows changes in % inhibition as a function of concentration (volume) in systems containing single antioxidants (chlorogenic acid or kaempferol) and their binary mixtures for the system with methanol, while Figure 3B presents data obtained for individual antioxidants and their binary mixtures in measurement systems to which water was added (detailed volumetric components for all systems with water are presented in Table 1). Studies on the effect of water addition on the antioxidant properties of antioxidants commonly found in plants and their binary mixtures seems to be important because water is a very significant component of all organisms and food products, including plant products where it occurs in various amounts. It should be emphasized here that, similarly to the above, in order to facilitate the interpretation of the results, the data for methanol have been repeated.
According to the data presented in the figures, in the systems containing single antioxidants, the addition of water results in less effective properties for systems with kaempferol (compare both blue curves in Figure 3A,B) and better in systems with chlorogenic acid (compare red curves in both figures). No wonder, because kaempferol is a strongly hydrophobic compound and in systems containing water, its solubility decreases, which results in its poorer antioxidant properties [29]. On the other hand, the solubility of chlorogenic acid in water is higher (40 mg/mL) than in organic solvents (25 mg/mL), and hence its better antioxidant properties [30,31].
It seems more important to compare the experimental and theoretical curves in Figure 3A,B. In both cases, the theoretical curve is above the experimental curve, which indicates the antagonistic antioxidant effect of the pair of tested antioxidants. However, it can be clearly seen that the difference is greater in the case of systems with water (Figure 3B).
This observation is confirmed by the data contained in Table 4—in the case of systems with water, higher F values and lower p values are observed than for analogous systems in which only methanol is present. Hence, it can be concluded that water contributes to the resultant antioxidant effect of the chlorogenic acid/kaempferol mixture. The fact that the antagonistic effect is greater in the presence of water may undoubtedly result from the poorer solubility of the stronger antioxidant (kaempferol) and its more difficult ionization, which is necessary to transfer the electron and/or hydrogen to the radical. Moreover, the better solubility and the related better ionization of the weaker antioxidant (chlorogenic acid) do not compensate for the poorer antioxidant properties of kaempferol. The validity of this theory is confirmed by the fact that increasingly higher F values and lower p values are observed with the increase in the amount of chlorogenic acid in the tested measurement systems (see data in Table 4).

3.4. Influence of the Addition of Metal Ions and Hydrogen Ions on the Resultant Effect of the Mixture’s Antioxidant Activity

Figure 4 shows changes in % inhibition as a function of concentration (volume) for the single antioxidants (chlorogenic acid or kaempferol) and their binary mixtures in the systems with addition of water (Figure 4A), ferric ions (Figure 4B), cupric ions (Figure 4C), buffer about pH = 1(Figure 4D) and buffer about pH = 3 (Figure 4E) (detailed volumetric components for all water systems are presented in Table 1). The use of metal ions and buffer solutions in research to check their impact on the antioxidant properties of the systems seems justified, because natural systems (plant extracts) may differ not only in the content of active substances (antioxidants), but also in the content of metal ions and the presence of natural acids (variable pH). It should be emphasized here that Figure 3A is a repetition of Figure 3B, necessary to ensure clarity of the discussion. In the studies aimed at examining the effect of metal ions and hydrogen ions on the antioxidant properties of the binary mixture, it was decided to assume that the system containing water was the reference system. Such a procedure seems reasonable because when examining the influence of a specific factor (metal ion or hydrogen ion concentration), its specific amount was introduced into the tested system/systems in the form of aqueous solutions. It should be emphasized here that the concentrations of metal ions that can be found in plants were used in the experiments (assuming extraction of the entire amount of metal from 1 g of the plant with 50 mL of the extractant) [14].
Comparing the position of the experimental and theoretical curves in each figure, it can be seen that, as in previous studies, these curves do not coincide and that the experimental curve is under the theoretical curve. As is known, this indicates antagonism, which is confirmed by the data in Table 5.
As for the effect of metal ions, a greater one is observed for iron ions. In the systems containing mentioned iron ions, higher F values and lower p values are observed, respectively, compared to the same values for the reference system (with water). This confirms the increase in the antagonistic antioxidant effect in the chlorogenic acid/kaempferol mixture, and moreover, this effect is most pronounced with a higher content of chlorogenic acid (for a mixture containing 80 µL acid and 20 µL kaempferol, the greatest difference between the experimental and theoretical value is observed, F = 60.97). In the case of systems with copper ions, the antagonism effect is also visible, but compared to the reference system, it seems to be slightly smaller (lower F values).
It is difficult to clearly explain the observed resultant antagonistic effect of the mixture in the presence of metal ions. It seems to be the result of many phenomena. On one hand, the observed poorer performance of the mixture may result from the ability of both metals to reduce as a result of attaching an electron from the antioxidant, which may inhibit the radical neutralization reaction. On the other hand, these metals can be complexed by the ABTS cation radical itself as well as by the antioxidants themselves [32]. As the literature data show [33], ABTS complexes have an intense color, which can cause falsification of absorbance measurements and cause apparently poorer antioxidant properties [33]. Transition metal complexes with polyphenolic compounds are also known from the literature. Moreover, in most cases, these complexes usually turn out to be more active than the parent compounds [34]. On the other hand, less antagonism in the presence of copper ions may result from a greater (than in systems with iron) predominance of the formation of complexes with better antioxidant properties [35].
In addition to metal ions, the presence and concentration of hydrogen ions also influence the observed antagonism in the mixture containing chlorogenic acid and kaempferol. Greater differences between the theoretical and experimental values are observed for systems with pH = 1 (greater F values than in the reference system). This is not surprising because, as already mentioned, ABTS is a method in which the neutralization of the cation radical occurs after the ionization and deprotonation of the active groups in the antioxidant. As such, it is a pH dependent reaction and the ionization potential of phenolic compounds decreases with an increasing pH value. Thus, in an acidic environment, the poorer antioxidant properties observed for both single antioxidants and their mixtures may be the result of hindered ionization of the antioxidant and poorer antioxidant properties.

4. Conclusions

The paper presents and discusses the influence of selected factors, such as temperature, reaction environment and presence of water, and metal ions or concentrations of hydrogen ions on the antioxidant properties of chlorogenic acid and kaempferol, as well as their binary mixtures. The obtained results indicate that:
-
A mixture of the above-mentioned polyphenolic compounds has an antagonistic effect.
-
The magnitude of this antagonistic effect depends not only on the mutual quantitative relationships of individual antioxidants in the mixture, but also on the presence of other ingredients in the tested systems, which, although they do not show antioxidant properties, affect the assessment (such as those listed above).
-
It is difficult to clearly explain the cause of the observed effect of the mixture because it is the effect of the action of a specific factor on a given antioxidant and of antioxidants between themselves in the presence of this factor.
The obtained results indicate an extremely complex nature of the assessment of antioxidant properties, even in seemingly simple measurement systems, and that the antioxidant properties of the mixture depend on many factors. In order to correctly predict the antioxidant behavior of a mixture of natural polyphenols, it is necessary to take into account the impact of their interactions with each other as well as the interactions they enter into with other components of the measurement system, not necessarily having antioxidant properties. This is quite an important issue because improper selection of antioxidants may have a negative impact on the body and instead of counteracting oxidative stress, it may strengthen it [36,37].

Author Contributions

Conceptualization and methodology, M.O.-T.; writing—original draft preparation, M.O.-T.; writing—review and editing, D.W.; supervision, D.W. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

The authors would like to thank the Institute of Chemical Sciences of the Maria Curie-Skłodowska University in Lublin for creating the research infrastructure, without which this research would not be possible.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Zhang, C.; Wang, X.; Du, J.; Gu, Z.; Zhang, Y. Reactive oxygen species-regulating strategies based on nanomaterials for disease treatment. Adv. Sci. 2021, 8, 2002797. [Google Scholar] [CrossRef]
  2. Mirończuk-Chodakowska, I.; Witkowska, A.M.; Zujko, M.E. Endogenous non-enzymatic antioxidants in the human body. Adv. Med. Sci. 2018, 63, 68–78. [Google Scholar] [CrossRef]
  3. Kotha, R.R.; Tareq, F.S.; Yildiz, E.; Luthria, D.L. Oxidative stress and antioxidants—Acriticalreview on in vitro antioxidant assays. Antioxidants 2022, 11, 2388. [Google Scholar] [CrossRef]
  4. Zujko, M.E.; Witkowska, A.M. Dietary antioxidants and chronic diseases. Antioxidants 2023, 12, 362. [Google Scholar] [CrossRef]
  5. Hegazy, A.M.; El-Sayed, E.M.; Ibrahim, K.S.; Abdel-Azeem, A.S. Dietary antioxidant for disease prevention corroborated by the Nrf2 pathway. J. Complement. Integr. Med. 2019, 16, 20180161. [Google Scholar] [CrossRef]
  6. Kamal, N.; Ilowefah, M.A.; Hilles, A.R.; Anua, N.A.; Awin, T.; Alshwyeh, H.A.; Aldosary, S.K.; Jambocus, N.G.S.; Alosaimi, A.A.; Rahman, A.; et al. Genesis and mechanism of some cancer types and an overview on the role of diet and nutrition in cancer prevention. Molecules 2022, 27, 1794. [Google Scholar] [CrossRef]
  7. Chen, X.; Li, H.; Zhang, B.; Deng, Z. The synergistic and antagonistic antioxidant interactions of dietary phytochemical combinations. Crit. Rev. Food Sci. Nutr. 2022, 62, 5658–5677. [Google Scholar] [CrossRef]
  8. Joshi, T.; Deepa, P.R.; Sharma, P.K. Effect of Different Proportions of Phenolics on Antioxidant Potential: Pointers for Bioactive Synergy/Antagonism in Foods and Nutraceuticals. Proc. Natl. Acad. Sci. India Sect. B Biol. Sci. 2022, 92, 939–946. [Google Scholar] [CrossRef]
  9. Uduwana, S.; Abeynayake, N.; Wickramasinghe, I. Synergistic, antagonistic, and additive effects on the resultant antioxidant activity in infusions of green tea with bee honey and Citrus limonum extract as additives. J. Agric. Food Res. 2023, 12, 100571. [Google Scholar] [CrossRef]
  10. Dawidowicz, A.L.; Olszowy-Tomczyk, M.; Typek, R. Synergistic and antagonistic antioxidant effects in the binary cannabinoids mixtures. Fitoterapia 2021, 153, 104992. [Google Scholar] [CrossRef]
  11. Olszowy-Tomczyk, M.; Wianowska, D. Antioxidant properties of selected flavonoids in binary mixtures-considerations on myricetin, kaempferol and quercetin. Int. J. Mol. Sci. 2023, 24, 10070. [Google Scholar] [CrossRef]
  12. Zhang, L.; Virgous, C.; Si, H. Synergistic anti-inflammatory effects and mechanisms of combined phytochemicals. J. Nutr. Biochem. 2019, 69, 9–30. [Google Scholar] [CrossRef]
  13. Dawidowicz, A.L.; Olszowy, M. Influence of some experimental variables and matrix components in the determination of antioxidant properties by β-carotene bleaching assay: Experiments with BHT used as standard antioxidant. Eur. Food Res. Technol. 2010, 231, 835–840. [Google Scholar] [CrossRef]
  14. Dawidowicz, A.L.; Olszowy, M. Antioxidant properties of BHT estimated by ABTS assay in systems differing in pH or metal ion or water concentration. Eur. Food Res. Technol. 2011, 232, 837–842. [Google Scholar] [CrossRef]
  15. Olszowy-Tomczyk, M.; Paprotny, Ł.; Celejewska, A.; Szewczak, D.; Wianowska, D. Comparison of the antioxidant properties of serum and plasma samples as well as glutathione under environmental and pharmacological stress factors involving different classes of drugs. Environ. Toxicol. Pharmacol. 2022, 94, 103936. [Google Scholar] [CrossRef]
  16. Somerset, S.M.; Johannot, L. Dietary flavonoid sources in Australian adults. Nutr. Cancer 2008, 60, 442–449. [Google Scholar] [CrossRef]
  17. Wianowska, D.; Gil, M. Recent advances in extraction and analysis procedures of natural chlorogenic acids. Phytochem. Rev. 2019, 18, 273–302. [Google Scholar] [CrossRef]
  18. Mudge, E.; Applequist, W.L.; Finley, J.; Lister, P.; Townesmith, A.K.; Walker, K.M.; Brown, P.N. Variation of select flavonolsand chlorogenic acid content of elderberry collected throughout the eastern united states. J. Food Compost. Anal. 2016, 47, 52–59. [Google Scholar] [CrossRef] [PubMed]
  19. Średnicka-Tober, D.; Ponder, A.; Hallmann, E.; Głowacka, A.; Rozpara, E. The profile and kontent of polyphenols and carotenoids in local and commercial sweet cherry fruits (Prunusavium L.) and their antioxidant activity in vitro. Antioxidants 2019, 8, 534. [Google Scholar] [CrossRef]
  20. Amjad, L.; Sokouti, B.; Asnaashari, S. A systematic review of anticancer roles and mechanisms of kaempferol as a natural compound. Cancer Cell Int. 2022, 22, 260. [Google Scholar] [CrossRef]
  21. Nenadis, N.; Wang, L.-F.; Tsimidou, M.; Zhang, H.-Y. Estimation of scavenging activity of phenolic compounds using the ABTS•+ assay. J. Agric. Food Chem. 2004, 52, 4669–4674. [Google Scholar] [CrossRef] [PubMed]
  22. Réblová, Z. Effect of temperature on the antioxidant activityof phenolic acids. Czech J. Food Sci. 2012, 30, 171–177. [Google Scholar] [CrossRef]
  23. Yanishlieva, N.V. Inhibiting oxidation. In Antioxidants in Food—Practical Applications; Pokorný, J., Yanishlieva, N.V., Gordon, H., Eds.; Woodhead Publishing: Cambridge, UK, 2001; pp. 22–70. [Google Scholar]
  24. Christodoulou, M.C.; Orellana Palacios, J.C.; Hesami, G.; Jafarzadeh, S.; Lorenzo, J.M.; Domínguez, R.; Moreno, A.; Hadidi, M. Spectrophotometric methods for measurement of antioxidant activity in food and pharmaceuticals. Antioxidants 2022, 11, 2213. [Google Scholar] [CrossRef] [PubMed]
  25. Borowski, P.; Jaroniec, J.; Janowski, T.; Woliński, K. Quantum cluster equilibrium theory of hydrogen-bonded liquids: Water, methanol and ethanol. Mol. Phys. 2003, 101, 1413–1421. [Google Scholar] [CrossRef]
  26. Ludwig, R. Quantum clusters equilibrium theory of liquids: Molecular clusters and thermodynamics of liquid ethanol. Mol. Phys. 1999, 97, 465–477. [Google Scholar] [CrossRef]
  27. Milugo, T.K.; Omosa, L.K.; Ochanda, J.O.; Owuor, B.O.; Wamunyokoli, F.A.; Oyugi, J.O.; Ochieng, J.W. Antagonistic effect of alkaloids and saponins on bioactivity in the quinine tree (Rauvolfiacaffra sond): Further evidence to support biotechnology in traditional medicinal plants. BMC Complement. Altern. Med. 2013, 13, 285. Available online: http://www.biomedcentral.com/1472-6882/13/285 (accessed on 4 October 2023). [CrossRef]
  28. Phan, M.A.T.; Peterson, J.; Bucknall, M.; Arcot, J. Interactions between phytochemicals from fruits and vegetables: Effects on bioactivities and bioavailability. Crit. Rev. Food Sci. Nutr. 2017, 58, 1310–1329. [Google Scholar] [CrossRef]
  29. Bangar, S.P.; Chaudhary, V.; Sharma, N.; Bansal, V.; Ozogul, F.; Lorenzo, J.M. Kaempferol: A flavonoid with wider biological activities and its applications. Crit. Rev. Food Sci. Nutr. 2022, 62, 1–25. [Google Scholar] [CrossRef]
  30. Available online: https://fnkprddata.blob.core.windows.net/domestic/data/datasheet/CAY/70930.pdf (accessed on 4 August 2023).
  31. Available online: https://www.chembk.com/en/chem/Chlorogenic%20acid (accessed on 4 August 2023).
  32. Kim, J.; Lee, K.; Nam, J.S. Metal-polyphenol complexes as versatile building blocks for functional biomaterials. Biotechnol. Bioprocess. Eng. 2021, 26, 689–707. [Google Scholar] [CrossRef]
  33. Deng, W.; Tan, Y.; Fang, Z.; Xie, Q.; Li, Y.; Liang, X.; Yao, S. ABTS-multiwalled carbon nanotubes nanocomposite/Bi film electrode for sensitive determination of Cd and Pb by differential pulse stripping voltammetry. Electroanalysis 2009, 21, 2477–2485. [Google Scholar] [CrossRef]
  34. Jabeen, E.; Janjua, N.K.; Ahmed, S.; Ali, T.; Murtaza, I.; Ashraf, Z.; Masood, N.; Kalsoom, S. Antioxidant activity and hepatotoxicity of flavonoids and their metal complexes through co-administration of β-cyclodextrin. Chem. Sel. 2019, 4, 9420–9432. [Google Scholar] [CrossRef]
  35. Simunkova, M.; Barbierikova, Z.; Jomova, K.; Hudecova, L.; Lauro, P.; Alwasel, S.H.; Alhazza, I.; Rhodes, R.J.; Valko, M. Antioxidant vs. prooxidantproperties of the flavonoid, kaempferol, in the presence of Cu(II) ions: A ROS-scavenging activity, Fenton reaction and DNA damage study. Int. J. Mol. Sci. 2021, 22, 1619. [Google Scholar] [CrossRef] [PubMed]
  36. Liu, X.; Xu, Y.; Liu, F.; Pan, X.; Miao, L.; Zhu, Q.; Tan, S. The feasibility of antioxidants avoiding oxidative damages from reactive oxygen species in cryopreservation. Front. Chem. 2021, 9, 648684. [Google Scholar] [CrossRef] [PubMed]
  37. Li, S.; Fasipe, B.; Laher, I. Potential harms of supplementation with high doses of antioxidants inathletes. J. Exerc. Sci. Fit. 2022, 20, 269–275. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publication. s are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Figure 1. The antioxidant activity changes assessed by the ABTS method for the systems containing different volumes of chlorogenic acid solution (red line with squares), kaempferol solution (blue line with diamonds), and their binary mixtures (violet line with “x”). The green line with triangles corresponds to the expected activity values for the tested pairs of compounds. The experimental values are the mean values for n = 5. The experiments were carried out at three different temperatures, i.e., 15 °C (A), 25 °C (B) and 35 °C (C).
Figure 1. The antioxidant activity changes assessed by the ABTS method for the systems containing different volumes of chlorogenic acid solution (red line with squares), kaempferol solution (blue line with diamonds), and their binary mixtures (violet line with “x”). The green line with triangles corresponds to the expected activity values for the tested pairs of compounds. The experimental values are the mean values for n = 5. The experiments were carried out at three different temperatures, i.e., 15 °C (A), 25 °C (B) and 35 °C (C).
Applsci 13 11138 g001
Figure 2. The antioxidant activity changes assessed by the ABTS method for the systems containing different volumes of methanolic (A) and ethanolic (B) solutions of: chlorogenic acid (red line with squares), kaempferol (blue line with diamonds), and their binary mixtures (violet line with “x”). The green line with triangles corresponds to the expected activity values for the tested pairs of compounds. The experimental values are the mean values for n = 5.
Figure 2. The antioxidant activity changes assessed by the ABTS method for the systems containing different volumes of methanolic (A) and ethanolic (B) solutions of: chlorogenic acid (red line with squares), kaempferol (blue line with diamonds), and their binary mixtures (violet line with “x”). The green line with triangles corresponds to the expected activity values for the tested pairs of compounds. The experimental values are the mean values for n = 5.
Applsci 13 11138 g002
Figure 3. The antioxidant activity changes assessed by the ABTS method for the systems containing different volumes of methanolic solutions (A) and methanolic with water addition (B) solutions of chlorogenic acid (red line with squares) and kaempferol (blue line with diamonds), and their binary mixtures (violet line with “x”). The green line with triangles corresponds to the expected activity values for the tested pairs of compounds. The experimental values are the mean values for n = 5.
Figure 3. The antioxidant activity changes assessed by the ABTS method for the systems containing different volumes of methanolic solutions (A) and methanolic with water addition (B) solutions of chlorogenic acid (red line with squares) and kaempferol (blue line with diamonds), and their binary mixtures (violet line with “x”). The green line with triangles corresponds to the expected activity values for the tested pairs of compounds. The experimental values are the mean values for n = 5.
Applsci 13 11138 g003
Figure 4. Changes in antioxidant activity assessed by the ABTS method for systems containing different volumes of methanol solutions of chlorogenic acid (red line with squares), kaempferol (blue line with diamonds) and their binary mixtures (violet line with “x”), with the addition of water (A), ions iron (B), copper ions (C), buffer with pH = 1 (D) and buffer with pH = 3 (E). The green line with triangles corresponds to the expected activity values for the tested pairs of compounds. Experimental values are mean values for n = 5.
Figure 4. Changes in antioxidant activity assessed by the ABTS method for systems containing different volumes of methanol solutions of chlorogenic acid (red line with squares), kaempferol (blue line with diamonds) and their binary mixtures (violet line with “x”), with the addition of water (A), ions iron (B), copper ions (C), buffer with pH = 1 (D) and buffer with pH = 3 (E). The green line with triangles corresponds to the expected activity values for the tested pairs of compounds. Experimental values are mean values for n = 5.
Applsci 13 11138 g004
Table 1. Volumes of solutions used for the determination of the antioxidant properties of chlorogenic acid and kaempferol, and their binary mixtures.
Table 1. Volumes of solutions used for the determination of the antioxidant properties of chlorogenic acid and kaempferol, and their binary mixtures.
Components’ Volumes
in μL
System Number
123456789
Chlorogenic acid 1205080---205080
Kaempferol 1---205080805020
MeOH or EtOH or
(MeOH + chosen factor 2)
180
(80 + 100)
150
(50 + 100)
120
(20 + 100)
180
(80 + 100)
150
(50 + 100)
120
(20 + 100)
100
(0 + 100)
100
(0 + 100)
100
(0 + 100)
ABTS+• 12800
Total volume3000
1 Chlorogenic acid, kaempferol, ABTS+• dissolved in MeOH or EtOH. 2 Water or an aqueous solution of metal ions or a buffer with a certain pH; ABTS+• and antioxidants dissolved in MeOH.
Table 2. Statistical significance (F and p values) of the difference between the experimental inhibition percent (Ie) and calculated inhibition percent (Ic) for the binary mixtures of the chlorogenic acid and kaempferol at three different volume ratios and at three different temperatures, together with the difference (Ie-Ic) and with the observed effect (the resultant) antioxidant effect of the antioxidants in the mixture (Fcrit = 7.71).
Table 2. Statistical significance (F and p values) of the difference between the experimental inhibition percent (Ie) and calculated inhibition percent (Ic) for the binary mixtures of the chlorogenic acid and kaempferol at three different volume ratios and at three different temperatures, together with the difference (Ie-Ic) and with the observed effect (the resultant) antioxidant effect of the antioxidants in the mixture (Fcrit = 7.71).
Temperature (°C)System NumberVolume Ratio (v/v)F–Valuesp-Values(Ie-Ic)-ValuesObserved Effect
15 720/8034.910.0041negativeantagonism
850/5040.190.0032negativeantagonism
980/2056.490.0002negativeantagonism
25720/8013.830.0205negativeantagonism/additivism
850/5011.030.0293negativeantagonism/additivism
980/2021.070.0101negativeantagonism
35720/8034.470.0042negativeantagonism
850/5047.330.0023negativeantagonism
980/2056.980.0016negativeantagonism
Table 3. Statistical significance (F and p values) of the difference between the experimental inhibition percent (Ie) and calculated inhibition percent (Ic) for the binary mixtures of chlorogenic acid and kaempferol at three different volume ratios and in different solvent, together with the difference (Ie-Ic) and with the observed effect (the resultant) antioxidant effect of the antioxidants in the mixture (Fcrit = 7.71).
Table 3. Statistical significance (F and p values) of the difference between the experimental inhibition percent (Ie) and calculated inhibition percent (Ic) for the binary mixtures of chlorogenic acid and kaempferol at three different volume ratios and in different solvent, together with the difference (Ie-Ic) and with the observed effect (the resultant) antioxidant effect of the antioxidants in the mixture (Fcrit = 7.71).
Solvent TypeSystem NumberVolume Ratio (v/v)F-Valuesp-Values (Ie-Ic)-ValuesObserved Effect
methanol720/8013.830.0205negativeantagonism/additivism
850/5011.030.0293negativeantagonism/additivism
980/2021.070.0101negativeantagonism
ethanol720/8039.890.0032negativeantagonism
850/5030.020.0054negativeantagonism
980/2055.990.0017negativeantagonism
Table 4. Statistical significance (F and p values) of the difference between the experimental inhibition percent (Ie) and calculated inhibition percent (Ic) for the binary mixtures of chlorogenic acid and kaempferol at three different volume ratios and in the system with methanol and with methanol in the presence of water addition, together with the difference (Ie-Ic) and with the observed effect (the resultant) antioxidant effect of the antioxidants in the mixture (Fcrit = 7.71).
Table 4. Statistical significance (F and p values) of the difference between the experimental inhibition percent (Ie) and calculated inhibition percent (Ic) for the binary mixtures of chlorogenic acid and kaempferol at three different volume ratios and in the system with methanol and with methanol in the presence of water addition, together with the difference (Ie-Ic) and with the observed effect (the resultant) antioxidant effect of the antioxidants in the mixture (Fcrit = 7.71).
SolventSystem NumberVolume Ratio (v/v)F-Valuesp-Values (Ie-Ic)-ValuesObserved Effect
methanol720/8013.830.0205negativeantagonism/additivism
850/5011.030.0293negativeantagonism/additivism
980/2021.070.0101negativeantagonism
methanol with water (100 μL)720/8029.210.0056negativeantagonism
850/5033.960.0043negativeantagonism
980/2035.190.0040negativeantagonism
Table 5. Statistical significance (F and p values) of the difference between the experimental inhibition percent (Ie) and calculated inhibition percent (Ic) for the binary mixtures of chlorogenic acid and kaempferol at three different volume ratios and in the systems with different additives, together with the difference (Ie-Ic) and with the observed effect (the resultant) antioxidant effect of the antioxidants in the mixture (Fcrit = 7.71).
Table 5. Statistical significance (F and p values) of the difference between the experimental inhibition percent (Ie) and calculated inhibition percent (Ic) for the binary mixtures of chlorogenic acid and kaempferol at three different volume ratios and in the systems with different additives, together with the difference (Ie-Ic) and with the observed effect (the resultant) antioxidant effect of the antioxidants in the mixture (Fcrit = 7.71).
Factors Volume Ratio (v/v)F-Valuesp-Values (Ie-Ic)-ValuesObserved Effect
water20/8029.210.0056negativeantagonism
50/5033.960.0043negativeantagonism
80/2035.190.0040negativeantagonism
metal ions
Fe3+20/8027.030.0065negativeantagonism
50/5058.640.0015negativeantagonism
80/2060.970.0014negativeantagonism
Cu2+20/8028.800.0058negativeantagonism
50/5018.050.0132negativeantagonism
80/2017.890.0134negativeantagonism
hydrogen ions
pH = 120/8027.070.0065negativeantagonism
50/5054.670.0018negativeantagonism
80/20142.530.0002negativeantagonism
pH = 320/8035.680.0039negativeantagonism
50/5037.630.0036negativeantagonism
80/2060.080.0015negativeantagonism
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Olszowy-Tomczyk, M.; Wianowska, D. Influence of Selected Factors on Antioxidant Properties of Chlorogenic Acid and Kaempferol as Well as Their Binary Mixtures. Appl. Sci. 2023, 13, 11138. https://doi.org/10.3390/app132011138

AMA Style

Olszowy-Tomczyk M, Wianowska D. Influence of Selected Factors on Antioxidant Properties of Chlorogenic Acid and Kaempferol as Well as Their Binary Mixtures. Applied Sciences. 2023; 13(20):11138. https://doi.org/10.3390/app132011138

Chicago/Turabian Style

Olszowy-Tomczyk, Małgorzata, and Dorota Wianowska. 2023. "Influence of Selected Factors on Antioxidant Properties of Chlorogenic Acid and Kaempferol as Well as Their Binary Mixtures" Applied Sciences 13, no. 20: 11138. https://doi.org/10.3390/app132011138

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop