- freely available
Toxins 2011, 3(7), 911-919; doi:10.3390/toxins3070911
Published: 20 July 2011
Abstract: The uremic syndrome is attributed to the progressive retention of a large number of compounds which, under normal conditions, are excreted by healthy kidneys. The compounds are called uremic toxins when they interact negatively with biological functions. The present review focuses on a specific class of molecules, namely the family of protein-bound uremic toxins. Recent experimental studies have shown that protein-bound toxins are involved not only in the progression of chronic kidney disease (CKD), but also in the generation and aggravation of cardiovascular disease. Two protein-bound uremic retention solutes, namely indoxyl sulfate and p-cresyl sulfate, have been shown to play a prominent role. However, although these two molecules belong to the same class of molecules, exert toxic effects on the cardiovascular system in experimental animals, and accumulate in the serum of patients with CKD they may have different clinical impacts in terms of cardiovascular disease and other complications. The principal aim of this review is to evaluate the effect of p-cresyl sulfate and indoxyl sulfate retention on CKD patient outcomes, based on recent clinical studies.
Kidney function impairment leads to the progressive retention of a large number of compounds which, under normal conditions, are excreted via the urinary tract [1,2]. Owing to this accumulation, the retained molecules are called uremic retention solutes or, as adopted by the EUTox group, uremic toxins. It is notable that all these compounds exhibit characteristic biological and/or biochemical activities.
The uremic toxins can be classified according to their molecular weight (MW) and their protein-binding ability , the most convenient classification being (1) small MW water-soluble compounds; (2) protein-bound compounds; and (3) larger MW compounds, the so-called “middle molecules” .
Toxic effects seem to be induced by compounds which are difficult to remove by dialysis. This is particularly true for the second group, namely the protein-bound uremic toxins which are poorly eliminated by the commonly used dialysis techniques. The present review will focus on this specific class of molecules.
Our group and others have recently studied two prototypes of protein-bound uremic retention solutes, p-cresyl sulfate and indoxyl sulfate. As shown by us and others, their serum levels are elevated in advanced stages of chronic kidney disease (CKD) and correlate with glomerular filtration rate [5,6,7].
In patients with CKD, cardiovascular disease (CVD) is highly prevalent . In addition to classical Framingham risk factors, uremic toxins can be considered as nontraditional risk factors in this population. Thus p-cresyl sulfate and indoxyl sulfate have been shown to exert toxic effects in vitro . Moreover, results obtained in different CKD patient cohorts (both in hemodialysis and in predialysis patients) have identified these two uremic toxins as emerging mortality risk factors [5,6,7,10,11,12].
The aim of the present review is to provide new insight into the respective impacts of p-cresyl sulfate and indoxyl sulfate retention for the outcomes of patients with CKD, based on recent clinical studies.
2. Indoxyl Sulfate and p-Cresyl Sulfate Formation
The action of intestinal bacteria on undigested proteins gives rise to a variety of indoles and phenols that are conjugated by the liver before being excreted by the kidney. Indoxyl sulfate and p-cresyl sulfate are examples of these compounds. Their accumulation in the uremic state has a negative impact on many body functions. They are prototype members of the large group of protein-bound uremic toxins .
Briefly, tryptophan is metabolized into indole by intestinal bacteria which, after intestinal absorption, is further converted to indoxyl sulfate in the liver. P-cresol emanates from the metabolism of the amino acids tyrosine and phenylalanine by the intestinal flora. These amino acids are generated from dietary proteins and metabolized to 4-hydroxyphenylacetic acid, which is then decarboxylated to p-cresol [13,14]. During its passage through the intestinal mucosa, a cytosolic sulfotransferase metabolizes p-cresol to p-cresyl sulfate . P-cresol is present in the circulation largely in the form of its sulfate conjugate, p-cresyl sulfate.
In many clinical studies, authors reported data on p-cresol; however, p-cresol was determined after acidification, so that the concentrations measured were in fact those of the main retention solute, p-cresyl sulfate as each molecule of sulfate which is broken down by hydrolysis will generate one molecule of p-cresol .
The serum levels of these two toxins can be evaluated by reverse phase high performance liquid chromatography. This method evaluates 2 forms, namely the free and the total molecule .
3. Indoxyl Sulfate, p-Cresyl Sulfate and Renal Toxicity
Both toxins are excreted by the kidneys via proximal tubular secretion. Consequently, they accumulate in the blood of patients with impaired renal function [18,19]. Moreover, they cannot be efficiently removed by conventional haemodialysis, due to their high binding affinity for albumin.
Our research group and others have demonstrated in a cohort of patients with different CKD stages that total and free indoxyl sulfate and total and free para-cresyl sulfate serum levels are elevated in the advanced stages of CKD and that they correlate with glomerular filtration rate in pre-dialysis patients [5,6,7].
In addition, some data suggested that they could have a negative impact on the kidney. Indeed, it has been demonstrated that indoxyl sulfate plays a role in the progression of CKD by inducing an inflammatory reaction, with enhanced expression of profibrotic cytokines such as transforming growth factor beta 1 (TGF 1) [20,21]. Moreover, administration of indoxyl sulfate to 5/6-nephrectomized rats accelerated the development of renal fibrosis , and its administration to hypertensive rats reduced the expression of klotho and promoted cell senescence accompanied by renal fibrosis .
4. Indoxyl Sulfate, p-Cresyl Sulfate and Vascular Toxicity
These two toxins may also have a role in the development of uremia-related cardiovascular disorders.
Although it has been shown that p-cresol can affect endothelial barrier function, endothelial cell proliferation and wound repair [9,24], it remains to be seen whether p-cresyl sulfate also exerts harmful effects on vascular cells in vitro. If so, not only p-cresol but also its sulfate conjugate would exert direct actions on the cardiovascular system. Indeed, Schepers et al. observed that p-cresyl sulfate (but not p-cresol) had pro-inflammatory effects on non-stimulated leukocytes in vitro—suggesting that p-cresyl sulfate may contribute to the propensity of renal patients towards vascular damage .
In addition to its profibrotic effects, indoxyl sulfate may also favor the development of CVD by inhibiting endothelial cell repair and promoting the proliferation of vascular smooth muscle cells . Indoxyl sulfate may play an important role in endothelial dysfunction via the generation of oxidative stress and induction of endothelial senescence . A recent paper suggested that contrasting with CKD conditions (in which indoxyl sulfate has pro oxidant properties), under normal physiological conditions, it could act as an antioxidant. . The latter experimental and clinical data suggest a role of these uremic toxins in vascular dysfunction in CKD patients. This hypothesis is corroborated by recent personal clinical studies. Thus, we demonstrated a weak relationship of serum-free and total p-cresyl sulfate with vascular calcification in 139 patients with different CKD stages, whereas no relationship existed with arterial stiffness, as evaluated by pulse wave velocity measurements. Similarly, serum-free and total indoxyl sulfate levels correlated with aortic calcification, and in addition also with vascular stiffness [5,6].
It is noteworthy that the concentration of such hydrophobic uremic toxins can be reduced by medical intervention. Thus, the oral charcoal adsorbent AST-120 (Kremezin, Kureha Chemical Industry, Tokyo, Japan) has been shown to be capable of adsorbing uremic toxins and thereby attenuating the oxidative stress generated by them . In a study in which AST-120 was administered to pre-dialysis CKD patients for two years, a significant reduction in carotid intima-media thickness and PWV was reported in the AST-120 group, when compared with the non-AST 120 group . Moreover, a recent study in 40 CKD patients showed that the administration of AST-120 led to an improvement in endothelial dysfunction in patients with CKD, in association with a decrease in serum indoxyl sulfate levels and a reduction in circulating markers of oxidative stress . As AST 120 is not a specific adsorbent of indoxyl sulfate, presumably the serum concentrations of other organic compounds including p-cresyl sulfate levels have been decreased simultaneously with the decrease of serum indoxyl sulfate .
5. Indoxyl Sulfate, p-Cresyl Sulfate and Clinical Outcomes
Because of the available experimental and clinical evidence in favor of the involvement of the two toxins in the vascular damage observed in CKD, possible associations with cardiovascular outcomes and mortality have been sought in recent years.
In a personal study, we have indeed been able to show that higher serum indoxyl sulfate (both free and total molecules) was associated with an increase overall and cardiovascular mortality in patients at different CKD stages . This effect was independent of traditional risk factors, vascular stiffness and aortic calcification. In contrast, two other studies, one in dialysis patients and the other in predialysis patients, did not find an association between indoxyl sulfate levels and overall mortality [12,30]. However, in these two studies, the mortality rate was low and the main objective was not to examine overall mortality.
The impact of serum p-cresol/p-cresyl sulfate concentrations on outcomes has been studied in two distinct patient cohorts, namely end stage renal disease patients and predialysis CKD patients.
Hemodialysis patients. In 175 prevalent chronic hemodialysis patients, free, but not total, p-cresol concentrations have been found to be related to mortality in unadjusted and adjusted models . Moreover, in another group of non-diabetic hemodialysis patients, free p-cresol was significantly associated with cardiovascular disease . Finally, Lin et al. recently showed that serum-free and total p-cresol levels were strongly related to cardiovascular disease in dialysis patients .
Predialysis patients. Meijers et al. have recently demonstrated an association of serum-free p-cresol with cardiovascular events in a cohort of 499 patients with mild-to-moderate CKD . An interesting finding was that the association was independent of glomerular filtration rate and Framingham risk factors. Other authors found that high serum p-cresyl sulfate levels were associated with all cause mortality risk in a group of 268 predialysis patients . In the same vein, we observed in a group of patients with different stages of CKD that serum-free, but not total, p-cresyl sulfate was associated with overall and cardiovascular mortality risk independently of other risk factors .
Taken together, results from CKD cohorts in different centers and countries suggest different impacts of indoxyl sulfate and p-cresol/p-cresyl sulfate and their total and free forms, respectively, on the relative risk of cardiovascular and all-cause mortality. Only one report suggested that both serum total and free indoxyl sulfate concentrations were associated with negative patient outcomes, whereas the majority of reports found a negative impact only from the free fraction of p-cresol/p-cresyl sulfate, but not the total fraction, with the exception of one study .
To better understand these discrepancies, we compared in our cohort of 139 patients with different CKD stages, the correlation between serum levels of total and free fractions for each of these two toxins, as well as the percentage of binding defined as: (total − free)/total × 100. We found that the total and free serum concentrations of indoxyl sulfate were closely associated (r2 = 0.77, p < 0.001). An association also existed between the free and total serum levels of p-cresyl sulfate (r2 = 0.60, p < 0.001). However, the two associations, although being linear, differed from each other according to Wald’s test, as shown in Figure 1, indicating a different type of correlation between the free and the bound form among these two toxins.
Since we found that a higher free level of each of these 2 toxins was associated with a higher relative risk of mortality, we compared their per cent protein binding according to the median of their free fraction levels, as described in Tables 1 and 2. The results of this analysis showed that, with increasing free toxin levels, the per cent binding of indoxyl sulfate increased as well (Table 1). In contrast, the per cent binding of p-cresyl sulfate decreased with increasing free moiety concentrations (Table 2). These data suggest that the percentage of free p-cresyl sulfate levels increased differently at high toxin concentrations than the percentage of free indoxyl sulfate. This difference could play a role in the observed differences of toxicity between the two molecules.
|Table 1. Per cent protein binding according to levels of free indoxyl sulfate divided by median level.|
|Free Indoxyl Sulfate||p|
|Total||≤0.038 mg/100 mL|
(n = 70)
|>0.038 mg/100 mL|
(n = 69)
|Binding (%)||89 ± 8|
|83 ± 9|
|91 ± 5|
Data are presented as mean ± standard deviation (median). Binding percentage is defined as (Total − Free)/Total × 100.
|Table 2. Per cent protein binding according to levels of free p-cresyl sulfate divided by the median level.|
|Free p-Cresyl Sulfate||p|
|Total||≤0.051 mg/100 mL|
( n = 70)
|>0.051 mg/100 mL|
( n = 69)
|Binding (%)||91 ± 11|
|97 ± 4|
|85 ± 13|
Data are presented as mean ± standard deviation (median). Binding percentage is defined as (Total − Free)/Total × 100.
It is well known that indoxyl sulfate and p-cresyl sulfate are competitive binding substrates for the same albumin binding sites, reaching 95% of binding for both molecules . However, these latter data have been observed with separation techniques . Recently, the binding of p-cresyl sulfate and p-cresol with human serum albumin was studied using microcalorimetry . The authors found a moderate affinity of both p-cresyl sulfate and p-cresol toward human serum albumin at 25 °C which became relatively weak at the physiological temperature of 37 °C. The binding appears to involve principally van der Waals type interactions, the binding sites of the two molecules being the same or very close. The low fraction of protein-bound toxin, 13−20%, is therefore probably insufficient to incriminate strong binding as the main cause of poor removal of these toxins by hemodialysis. Thus p-cresyl sulfate binding to albumin appears to be less strong than previously reported .
Since there is an association between the circulating levels of these uremic toxins and the relative risk of cardiovascular events and mortality in patients with CKD, it will be interesting to test the hypothesis that an active reduction of their levels should lead to a decrease in cardiovascular and global outcomes. Interestingly, it has been preliminarily reported that AST-120 given to CKD patients prior to dialysis initiation improved their overall survival rates, in comparison with CKD patients to whom AST-120 was not administered .
As pointed out above, it has long been known that AST 120 can absorb many organic compounds. According to a recent study, it is able to reduce the plasma levels of several solutes retained in the uremic state, including hippurate, phenylsulfate, indoxyl sulfate and p-cresyl sulfate . However, the authors did not report which fraction, namely free or total, was evaluated. This does not weaken the evidence for a beneficial effect of AST-120. It only limits our ability to separate the effect on the reduction of the 2 uremic toxins under evaluation (both the free forms and the total molecules) from that on the reduction of other circulating solutes.
There is increasing experimental and clinical evidence in favour of the hypothesis that uremic toxins, and in particular protein-bound toxins, are involved not only in the progression of CKD, but also in the promotion of cardiovascular disease. Two protein-bound uremic retention solutes, namely indoxyl sulfate and p-cresyl sulfate, could play a prominent role. However, although the two of them belong to the same class of molecules, exert toxic effects on the cardiovascular system and accumulate in the serum of patients with CKD, they may have different impacts in terms of cardiovascular disease. One of the reasons for differing toxicities could be differences in the proportion of free and total fractions among the two toxins, resulting from differences in protein binding affinity and/or potential competition in protein binding. In our opinion, these hypotheses deserve to be examined by further studies.
- Vanholder, R.; de Smet, R. Pathophysiologic effects of uremic retention solutes. J. Am. Soc. Nephrol. 1999, 10, 1815–1823.
- Vanholder, R.; de Smet, R.; Hsu, C.; Vogeleere, P.; Ringoir, S. Uremic toxicity: The middle molecule hypothesis revisited. Semin. Nephrol. 1994, 14, 205–218.
- Vanholder, R.; Baurmeister, U.; Brunet, P.; Cohen, G.; Glorieux, G.; Jankowski, J. A bench to bedside view of uremic toxins. J. Am. Soc. Nephrol. 2008, 19, 863–870.
- Vanholder, R.; van Laecke, S.; Glorieux, G. What is new in uremic toxicity? Pediatr. Nephrol. 2008, 23, 1211–1221, doi:10.1007/s00467-008-0762-9. 18324423
- Liabeuf, S.; Barreto, D.V.; Barreto, F.C.; Meert, N.; Glorieux, G.; Schepers, E.; Temmar, M.; Choukroun, G.; Vanholder, R.; Massy, Z.A. Free p-cresylsulphate is a predictor of mortality in patients at different stages of chronic kidney disease. Nephrol. Dial. Transplant. 2010, 25, 1183–1191, doi:10.1093/ndt/gfp592. 19914995
- Barreto, F.C.; Barreto, D.V.; Liabeuf, S.; Meert, N.; Glorieux, G.; Temmar, M.; Choukroun, G.; Vanholder, R.; Massy, Z.A. Serum indoxyl sulfate is associated with vascular disease and mortality in chronic kidney disease patients. Clin. J. Am. Soc. Nephrol. 2009, 4, 1551–1558.
- Meijers, B.K.; Claes, K.; Bammens, B.; de Loor, H.; Viaene, L.; Verbeke, K.; Kuypers, D.; Vanrenterghem, Y.; Evenepoel, P. p-Cresol and cardiovascular risk in mild-to-moderate kidney disease. Clin. J. Am. Soc. Nephrol. 2010, 5, 1182–1189, doi:10.2215/CJN.07971109. 20430946
- Vanholder, R.; Massy, Z.; Argiles, A.; Spasovski, G.; Verbeke, F.; Lameire, N. Chronic kidney disease as cause of cardiovascular morbidity and mortality. Nephrol., Dial., Transplant. 2005, 20, 1048–1056, doi:10.1093/ndt/gfh813.
- Dou, L.; Bertrand, E.; Cerini, C.; Faure, V.; Sampol, J.; Vanholder, R.; Berland, Y.; Brunet, P. The uremic solutes p-cresol and indoxyl sulfate inhibit endothelial proliferation and wound repair. Kidney Int. 2004, 65, 442–451, doi:10.1111/j.1523-1755.2004.00399.x. 14717914
- Meijers, B.K.; Bammens, B.; de Moor, B.; Verbeke, K.; Vanrenterghem, Y.; Evenepoel, P. Free p-cresol is associated with cardiovascular disease in hemodialysis patients. Kidney Int. 2008, 73, 1174–1180, doi:10.1038/ki.2008.31. 18305466
- Bammens, B.; Evenepoel, P.; Keuleers, H.; Verbeke, K.; Vanrenterghem, Y. Free serum concentrations of the protein-bound retention solute p-cresol predict mortality in hemodialysis patients. Kidney Int. 2006, 69, 1081–1087.
- Lin, C.J.; Wu, C.J.; Pan, C.F.; Chen, Y.C.; Sun, F.J.; Chen, H.H. Serum protein-bound uraemic toxins and clinical outcomes in haemodialysis patients. Nephrol. Dial. Transplant. 2010, 25, 3693–3700.
- De Smet, R.; van Kaer, J.; van Vlem, B.; de Cubber, A.; Brunet, P.; Lameire, N.; Vanholder, R. Toxicity of free p-cresol: A prospective and cross-sectional analysis. Clin. Chem. 2003, 49, 470–478, doi:10.1373/49.3.470. 12600960
- Curtius, H.C.; Mettler, M.; Ettlinger, L. Study of the intestinal tyrosine metabolism using stable isotopes and gas chromatography-mass spectrometry. J. Chromatogr. 1976, 126, 569–580.
- Burchell, B.; Coughtrie, M.W. Genetic and environmental factors associated with variation of human xenobiotic glucuronidation and sulfation. Environ. Health Perspect. 1997, 105, 739–747.
- Vanholder, R.; Bammens, B.; de Loor, H.; Glorieux, G.; Meijers, B.; Schepers, E.; Massy, Z.; Evenepoel, P. Warning: The unfortunate end of p-cresol as a uraemic toxin. Nephrol. Dial. Transplant. 2011, 26, 1464–1467, doi:10.1093/ndt/gfr056. 21406546
- Meert, N.; Eloot, S.; Waterloos, M.A.; van Landschoot, M.; Dhondt, A.; Glorieux, G.; Ledebo, I.; Vanholder, R. Effective removal of protein-bound uraemic solutes by different convective strategies: A prospective trial. Nephrol. Dial. Transplant. 2009, 24, 562–570.
- Niwa, T. Phenol and p-cresol accumulated in uremic serum measured by HPLC with fluorescence detection. Clin. Chem. 1993, 39, 108–111. 8419031
- De Smet, R.; David, F.; Sandra, P.; van Kaer, J.; Lesaffer, G.; Dhondt, A.; Lameire, N.; Vanholder, R. A sensitive HPLC method for the quantification of free and total p-cresol in patients with chronic renal failure. Clin. Chim. Acta 1998, 278, 1–21, doi:10.1016/S0009-8981(98)00124-7. 9877120
- Niwa, T.; Ise, M. Indoxyl sulfate, a circulating uremic toxin, stimulates the progression of glomerular sclerosis. J. Lab. Clin. Med. 1994, 124, 96–104. 8035108
- Miyazaki, T.; Ise, M.; Hirata, M.; Endo, K.; Ito, Y.; Seo, H.; Niwa, T. Indoxyl sulfate stimulates renal synthesis of transforming growth factor-beta 1 and progression of renal failure. Kidney Int., Suppl. 1997, 63, S211–S214.
- Enomoto, A.; Takeda, M.; Tojo, A.; Sekine, T.; Cha, S.H.; Khamdang, S.; Takayama, F.; Aoyama, I.; Nakamura, S.; Endou, H.; et al. Role of organic anion transporters in the tubular transport of indoxyl sulfate and the induction of its nephrotoxicity. J. Am. Soc. Nephrol. 2002, 13, 1711–1720. 12089366
- Adijiang, A.; Shimizu, H.; Higuchi, Y.; Nishijima, F.; Niwa, T. Indoxyl sulfate reduces klotho expression and promotes senescence in the kidneys of hypertensive rats. J. Ren. Nutr. 2011, 21, 105–109.
- Cerini, C.; Dou, L.; Anfosso, F.; Sabatier, F.; Moal, V.; Glorieux, G.; de Smet, R.; Vanholder, R.; Dignat-George, F.; Sampol, J.; et al. P-cresol, a uremic retention solute, alters the endothelial barrier function in vitro. Thromb. Haemost 2004, 92, 140–150. 15213855
- Schepers, E.; Meert, N.; Glorieux, G.; Goeman, J.; van der Eycken, J.; Vanholder, R. P-cresylsulphate, the main in vivo metabolite of p-cresol, activates leucocyte free radical production. Nephrol. Dial. Transplant. 2007, 22, 592–596. 17040995
- Yu, M.; Kim, Y.J.; Kang, D.H. Indoxyl sulfate-induced endothelial dysfunction in patients with chronic kidney disease via an induction of oxidative stress. Clin. J. Am. Soc. Nephrol. 2011, 6, 30–39.
- Miyamoto, Y.; Watanabe, H.; Otagiri, M.; Maruyama, T. New insight into the redox properties of uremic solute indoxyl sulfate as a pro- and anti-oxidant. Ther. Apher. Dial. 2011, 15, 129–131.
- Fujii, H.; Nishijima, F.; Goto, S.; Sugano, M.; Yamato, H.; Kitazawa, R.; Kitazawa, S.; Fukagawa, M. Oral charcoal adsorbent (AST-120) prevents progression of cardiac damage in chronic kidney disease through suppression of oxidative stress. Nephrol. Dial. Transplant. 2009, 24, 2089–2095.
- Nakamura, T.; Kawagoe, Y.; Matsuda, T.; Ueda, Y.; Shimada, N.; Ebihara, I.; Koide, H. Oral ADSORBENT AST-120 decreases carotid intima-media thickness and arterial stiffness in patients with chronic renal failure. Kidney Blood Pressure Res. 2004, 27, 121–126.
- Wu, I.W.; Hsu, K.H.; Lee, C.C.; Sun, C.Y.; Hsu, H.J.; Tsai, C.J.; Tzen, C.Y.; Wang, Y.C.; Lin, C.Y.; Wu, M.S. P-Cresyl sulphate and indoxyl sulphate predict progression of chronic kidney disease. Nephrol. Dial. Transplant. 2011, 26, 938–947, doi:10.1093/ndt/gfq580. 20884620
- Meijers, B.K.; de Loor, H.; Bammens, B.; Verbeke, K.; Vanrenterghem, Y.; Evenepoel, P. P-Cresyl sulfate and indoxyl sulfate in hemodialysis patients. Clin. J. Am. Soc. Nephrol. 2009, 4, 1932–1938, doi:10.2215/CJN.02940509. 19833905
- Berge-Lefranc, D.; Chaspoul, F.; Calaf, R.; Charpiot, P.; Brunet, P.; Gallice, P. Binding of p-cresylsulfate and p-cresol to human serum albumin studied by microcalorimetry. J. Phys. Chem., B 2010, 114, 1661–1665.
- Ueda, H.; Shibahara, N.; Takagi, S.; Inoue, T.; Katsuoka, Y. AST-120 treatment in pre-dialysis period affects the prognosis in patients on hemodialysis. Renal Fail. 2008, 30, 856–860.
- Kikuchi, K.; Itoh, Y.; Tateoka, R.; Ezawa, A.; Murakami, K.; Niwa, T. Metabolomic search for uremic toxins as indicators of the effect of an oral sorbent AST-120 by liquid chromatography/tandem mass spectrometry. J. Chromatogr., B: Anal. Technol. Biomed. Life Sci. 2010, 878, 2997–3002, doi:10.1016/j.jchromb.2010.09.006.
© 2011 by the authors; licensee MDPI, Basel, Switzerland. This article is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).