An Inconvenient Truth: Transdermal Buffering Lotions Appear to Offer No Significant Performance Improvement

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Overview

The authors conducted a randomized, placebo-control trial comparing the effect of topical lotions on cycling performance that simulated trail rides.  Three lotion conditions – placebo, carnitine (TC), and bicarbonate (TBC) – where randomly assigned after the familiarization trial.  Carnitine and bicarbonate have been hypothesized (marketed) to buffer pH changes during the sprint efforts during the cycle route.  Well-trained cyclists (n=15) participated.  The authors reported that a clear treatment effect of the lotions did not exist.  The three randomized trials (TBC, TC, Placebo) produced significantly improved performance for the entire lap average power in Lap 5, averages for laps 2 through 4 for hill climb average power, and for sprint segment (except for TC) compared to the performance in the familiarization trial.  The authors concluded that the transdermal lotions were ineffective to enhance cycling performance as determined by the Zwift technology.

 

Major Concerns

While I agree with the authors’ results and the disingenuous notion of skin lotions delivering ergogenic benefits, the title statement – “…Transdermal Buffering Lotions Fail to Improve High-Intensity Cycling… “ - is very definitive for remote data collection and the number of subjects (almost exclusively males) that were tested.  The statistical approach only allows the authors to state that no difference was observed (failed to reject the statistical null hypothesis). Stating something like “absence of an any effect” in the title seems more appropriate.

What would strengthen this paper would be to include info on the post-hoc chance of a Type 2 error.  The authors have described how the sample size of 14 was determined; however, they lost data on a few subjects and that reduces the detection ability.  Since the authors make strong claims about the lack of veracity of the few supportive studies and the manufacturers of the lotions (and I don’t disagree with the authors’ points), the authors will want to have a bullet-proof case for their claims.

Buried within the Limitation section (line 358) and implied in the Methods (lines 439-451) is the fact that data were collected remotely, outside of the lab without research supervision.  While the reader understands the reasoning, it is a limitation and should be mentioned in the Abstract (line 15 add “for remote data collection” after “Using the Zwift online platform…” and line 119.  For the later add “a remote data collection system” after “Zwift RM cycling platform.”

 

Specific Comments/Questions/Suggestions

Intro

Lines 65-66, “Evidence for the use of both BC and BA under specific conditions, generally very high-intensity continuous or intermittent exercise of durations from 30-sec to 10-min…:” Evidence suggests what?  This seems like an incomplete thought.

 

Line 95 “Research on TC is even more limited but no less equivocal.”  Minor thought, can research be equivocal or are the results of research equivocal?

 

Results

Lines 137-139:  The authors explain that data for two or three subjects were lost.  How did the authors handle this for statistical hypothesis testing?  Was the final sample 12 or 13?  Please explain.

 

Table 2 at line 126-148:

• The footnotes identify where differences exist. Does the bolded data mean the same?  Please add to the footnote if bold means something different.
• In the footnotes, do the authors mean significantly different within the cell (row by column) for the same performance outcome? Using “row” here is a bit confusing.

 

Discussion

Lines 358-362 in Section 3.6 and in the Discussion lines 259-265:  For justification of using Zwift in this study, support for its reliability should be mentioned.  For this, the authors might reference Matta G, et al. The reproducibility of 20-min time-trial performance on a virtual cycling platform. Int J Sports Med 2022 Dec;43(14):1190-1195.  doi: 10.1055/a-1848-8478. Epub 2022 May 10 or similar reliability studies on this technology.

Line 427 “…complete 4 trials included an…:” A minor suggestion to revise to read “4 trials including…” or “4 trials, which included…”

Lines 485-495: Could the authors state the actual mass of TBC and TC applied in this study, to help the reader understand the dosage?  Taking the lotion containing sodium bicarbonate for example, if one assumed that the density of the lotion as 1.00 and the mixture was 100% bicarbonate, the dose would be 20 g, which would not be unreasonable compared to oral-dosage studies that showed performance enhancement (Grgic J. et al International Society of Sports Nutrition position stand: sodium bicarbonate and exercise performance. J Int Soc Sports Nutr. 2021 Sep 9;18(1):61.   doi: 10.1186/s12970-021-00458-w.).  However, the lotion contains more than just sodium bicarbonate* and likely there would not be 100% transdermal absorption or transport (that is unlikely in the intestinal tract as well).  I’ll leave it to the authors’ call if they include this speculation, since I’m sure the actual dose is proprietary.

*Taken from the manufacturer’s website, the ingredient list for APR PR (most to least): Sodium Bicarbonate, Poloxamer 407, Aqua (Water), Isopropyl Palmitate, Lecithin, Cetyl Alcohol, Propylene Glycol, Alcohol Denat, Benzyl Alcohol, Sodium Hydroxide, Polyglyceryl -4 Laurate, Sodium Lauryl Sulfate, Menthol, Sorbic Acid, Parfum

Lines 542-543 However, the study does support the use of…Zwift…:” I ‘m not sure how the design of the present study supports this conclusion.  Yes, data were consistent for coefficient of variation within a cell for specific performance outcomes (Table 2) and the absence of effects was consistent.  But the study wasn’t designed to test reliability.  I would accept the authors stating that Zwift in this case was feasible for remote, unsupervised data collection but additional research, i.e., remote vs in-lab replication confirmation is needed.

Author Response

The authors thank the reviewer for their time reviewing the paper and appreciate the helpful recommendations. We have made substantial revisions and believe we have addressed all of the reviewers concerns. Specific responses can be found below.

Major Concerns

1. While I agree with the authors’ results and the disingenuous notion of skin lotions delivering ergogenic benefits, the title statement – “…Transdermal Buffering Lotions Fail to Improve High-Intensity Cycling… “ - is very definitive for remote data collection and the number of subjects (almost exclusively males) that were tested.  The statistical approach only allows the authors to state that no difference was observed (failed to reject the statistical null hypothesis). Stating something like “absence of an any effect” in the title seems more appropriate.

RESPONSE: The authors thank the reviewer for their comment, and we understand their point of view. The title was not solely based on results but also on the overall body of evidence presented in table 3. Therefore, we have revised the title to reflect the broader context of evidence. We have also more clearly articulated that in the abstract and our own conclusions (Lines 569-570). While we prefer the revised title, if the reviewer still believes the title should be revised, we propose the following:

An Inconvenient Truth: Transdermal Buffering Lotions Appear to be No Better Than a Placebo for High-Intensity Cycling

1. What would strengthen this paper would be to include info on the post-hocchance of a Type 2 error.  The authors have described how the sample size of 14 was determined; however, they lost data on a few subjects and that reduces the detection ability.  Since the authors make strong claims about the lack of veracity of the few supportive studies and the manufacturers of the lotions (and I don’t disagree with the authors’ points), the authors will want to have a bullet-proof case for their claims.

RESPONSE: We thank the reviewer for this valuable suggestion. In response, we conducted a post-hoc power analysis as recommended using Kendall’s W and partial eta squares as our effect size metrics for non-parametric and parametric procedures respectively. The effect size metrics accounted for actual sample sizes obtained (ranging from 11 to 13 due to missing data for some variables) and the total number of conditions per comparison (k=2 to k=13). We report the computed power for each test and provide a weighted average power across all comparisons. Our findings demonstrate that most of our analyses were well powered- with an overall weighted average power of 0.897 for non-parametric effects and 0.854 for parametric effects, indicating a low likelihood of Type II error. 

We have included a description of this post-hoc analysis in the manuscript summarizing these findings for transparency. Please see lines 282 – 288.

119. Buried within the Limitation section (line 358) and implied in the Methods (lines 439-451) is the fact that data were collected remotely, outside of the lab without research supervision.  While the reader understands the reasoning, it is a limitation and should be mentioned in the Abstract (line 15 add “for remote data collection” after “Using the Zwift online platform…” and line 119.  For the later add “a remote data collection system” after “Zwift RM cycling platform.”

 RESPONSE: The authors appreciate the feedback. We have revised this in Line 202 and in the limitations section of the document for lines 534-538 to read: “Such virtual environments have been shown to produce reliable cycling performances [48] while creating exciting new opportunities to conduct large scale cycling and possibly running trials in controlled environment on virtual riding routes. At the time of the study design, it was believed that such a platform would expand the potential subject pool. However, converting interested participants into enrolled and completed participants proved to be very challenging. While MR remote studies hold promise, a significant limi-tation of this and future remote studies is the lack of physiological data like VO2 and blood lactate, or markers of acid-base balance. Thus, research like this can lose insightful data that aids in the interpretive process.”

 

Specific Comments/Questions/Suggestions

1. Intro
2. Lines 65-66, “Evidence for the use of both BC and BA under specific conditions, generally very high-intensity continuous or intermittent exercise of durations from 30-sec to 10-min…:” Evidence suggests what?  This seems like an incomplete thought.

RESPONSE: Revised to state:  The preponderance of evidence indicates that both BC and BA can improve performance in very high-intensity continuous or intermittent exercise of durations from 30-sec to 10-min [21–24]. More recent evidence, however, indicates that certain events lasting longer than 10-min may also benefit [22,23]

2. Line 95 “Research on TC is even more limited but no less equivocal.”  Minor thought, can research be equivocal or are the results of research equivocal?

 RESPONSE: Revised to state, “Research results on TC are no less equivocal.”

1. Results
2. Lines 137-139:  The authors explain that data for two or three subjects were lost.  How did the authors handle this for statistical hypothesis testing?  Was the final sample 12 or 13?  Please explain.

RESPONSE:  Recruitment occurred in stages based on researcher availability. Recruitment challenges coupled with academic responsibilities resulted in an initial lag. Despite the data lost to technical issues and report failures resulting in an intention to treat approach was adopted. However, to assist the reader we have provided some additional language to clarify this point: Lines 314-316: “Therefore, the n associated with each outcome measure: power, HR, and RPE; varied slightly between trials and laps, ranging from n = 14 – 15 among power and RPE data and n = 13 – 15 among HR data.”  We have provided greater details on this in the results, as well as added a CONSORT diagram (Figure 2) which addresses this question as well.

2. Table 2 at line 126-148:

• The footnotes identify where differences exist. Does the bolded data mean the same?  Please add to the footnote if bold means something different.
• In the footnotes, do the authors mean significantly different within the cell (row by column) for the same performance outcome? Using “row” here is a bit confusing.

RESPONSE:  Thank you for your notes, table footnotes have been revised for clarity.

1. Discussion
   1. Lines 358-362 in Section 3.6 and in the Discussion lines 259-265:  For justification of using Zwift in this study, support for its reliability should be mentioned.  For this, the authors might reference Matta G, et al. The reproducibility of 20-min time-trial performance on a virtual cycling platform. Int J Sports Med2022 Dec;43(14):1190-1195.  doi: 10.1055/a-1848-8478. Epub 2022 May 10 or similar reliability studies on this technology.

RESPONSE: The authors appreciate the feedback. We have revised this in Line 202, Lines 440-442: “Nonetheless, our participants demonstrated an ability to produce consistent performances after just a single familiarization trial and support prior work using MR platforms like Zwift [48,52].”

And in the limitations section of the document LINES 538-547: Such virtual environments have been shown to produce reliable cycling performances [48] while creating exciting new opportunities to conduct large scale cycling and possibly running trials in controlled environment on virtual riding routes. At the time of the study design, it was believed that such a platform would expand the potential subject pool. However, converting interested participants into enrolled and completed participants proved to be very challenging. While MR remote studies hold promise, a significant limitation of this and future remote studies is the lack of physiological data like VO₂ and blood lactate, or markers of acid-base balance. Thus, research like this can lose insightful data that aids in the interpretive process.”

 

2. Line 427 “…complete 4 trials included an…:” A minor suggestion to revise to read “4 trials including…” or “4 trials, which included…”

 

RESPONSE:  Thank you for your suggestion. We agree and have revised the language used in that line: “ Each participant completed 4 trials including an …”

3. Lines 485-495: Could the authors state the actual mass of TBC and TC applied in this study, to help the reader understand the dosage?  Taking the lotion containing sodium bicarbonate for example, if one assumed that the density of the lotion as 1.00 and the mixture was 100% bicarbonate, the dose would be 20 g, which would not be unreasonable compared to oral-dosage studies that showed performance enhancement (Grgic J.et al International Society of Sports Nutrition position stand: sodium bicarbonate and exercise performance. J Int Soc Sports Nutr. 2021 Sep 9;18(1):61.   doi: 10.1186/s12970-021-00458-w.).  However, the lotion contains more than just sodium bicarbonate* and likely there would not be 100% transdermal absorption or transport (that is unlikely in the intestinal tract as well).  I’ll leave it to the authors’ call if they include this speculation, since I’m sure the actual dose is proprietary.

*Taken from the manufacturer’s website, the ingredient list for APR PR (most to least): Sodium Bicarbonate, Poloxamer 407, Aqua (Water), Isopropyl Palmitate, Lecithin, Cetyl Alcohol, Propylene Glycol, Alcohol Denat, Benzyl Alcohol, Sodium Hydroxide, Polyglyceryl -4 Laurate, Sodium Lauryl Sulfate, Menthol, Sorbic Acid, Parfum

RESPONSE: Thank you for your comment. The authors reviewed prior literature and identified that the TBC lotion contains 33.2% HCO₃^-, while the TC is reported to contain 1.5% carnosine-complex. We originally included dosage information in volumetric measures as this was consistent with the packaging of both supplement lotions at the time. However, we have updated the method text to clarify dosing based on prior research descriptions and packaging recommendations*. The text (lines 229-240) now reads:

“Unlike prior research, our purpose was to test market claims; thus, samples of each supplement were provided to participants in accordance with single-dose packaging available for purchase at the time of testing for both TBC and TC. Precisely measured using an Ozeri ZK14-T digital scale (Ozeri Corp, San Diego, CA, USA), participants received 20 g of TBC (containing ~6.64 g of HCO₃ ^–), 15 g of TC (containing ~0.15 g of carnosine), and 15 g of the PLAC lotion (PLAC dosage was selected to provide participants with similar total lotion volumes between all lotion conditions). Each lotion was shipped to participants in clear, BPA-free plastic jars marked only with supplement letter codes X (TBC), Y (TC), or Z (PLAC). Blinded to which lotion they were to apply each trial; participants were instructed to use the entire contents of each container (scraping the interior of the container) to their legs and gluteus maximus no less than 1-hr prior to their expected trial start time.”

*Note, this approach to dosing resulted in similar relative dosing paradigms employed by other authors. Relative dose of TBC (AMP PR) Lotion (0.9036 g/kg BM) was calculated by accounting for the proportion of SB (33.2%) per gram of AMP PR Lotion to match the 0.3 g/kg BM oral SB dose (Gurton, Greally et al., 2023; McKay et al., 2020). This meant the same amount of SB was given for oral and topical treatments, with absolute SB dosage ranging from 19.7 to 28.4 g. As such, there were no differences in the amount of HCO3− (range: 12.3–17.8 g; assuming the composition of SB is 62.6% HCO3− and 27.3% Na+) delivered between SB-ORAL and SB-LOTION. The placebo-matched lotion was supplied by the manufacturer with SB removed but still containing 0.5% menthol as per commercially available PR Lotion.

4. Lines 542-543 However, the study does support the use of…Zwift…:” I ‘m not sure how the design of the present study supports this conclusion.  Yes, data were consistent for coefficient of variation within a cell for specific performance outcomes (Table 2) and the absence of effects was consistent.  But the study wasn’t designed to test reliability.  I would accept the authors stating that Zwift in this case was feasible for remote, unsupervised data collection but additional research, i.e., remote vs in-lab replication confirmation is needed.

RESPONSE: Thank you for your comment. We have softened our language to more accurately reflect our original intent: “However, the study does support the feasibility of MR platforms like Zwift for the completion of unsupervised, remote cycling-performance experimentation over a variety of virtual terrain settings.”

 

Reviewer 2 Report

Comments and Suggestions for Authors

This study is meaningful in that it sought to scientifically verify the effects of transdermal lotions (TBC, TC) actually used in the field. It has the strength of being systematically designed, thereby increasing the reliability of the interpretation of the results. However, there are a few areas that should be addressed. My specific comments are as follows:

1. Methods: Although the study period spanned approximately two years, only 15 participants’ data were ultimately included in the analysis. This may raise questions for readers regarding the research process. Therefore, it is necessary to clearly present the number of individuals initially recruited, the number who met eligibility criteria, the number who withdrew, and their reasons for withdrawal. To aid readers’ understanding, it is recommended to add a flowchart (e.g., CONSORT diagram) illustrating participant flow.
2. Results: While the study was designed primarily around performance outcomes, it did not include physiological measures (e.g., VO₂, blood lactate, acid–base balance), nor did it directly assess the dermal absorption of the lotions. This represents an important limitation in interpreting the results. Accordingly, this point should be explicitly mentioned in the Limitations section or included as a recommendation for Future Research.
3. Structure: The current manuscript does not align with the structure generally recommended by MDPI journals (Introduction → Methods → Results → Discussion → Conclusions). The manuscript should be revised to follow this format.

Author Response

The authors thank the reviewer for their time reviewing the paper and the recommendations made. We believe we have addressed all of the reviewers concerns, making substantial revisions to the manuscript. Specific responses can be found below.

Comments and Suggestions for Authors

1. This study is meaningful in that it sought to scientifically verify the effects of transdermal lotions (TBC, TC) used in the field. It has the strength of being systematically designed, thereby increasing the reliability of the interpretation of the results. However, there are a few areas that should be addressed. My specific comments are as follows:

1. Methods: Although the study period spanned approximately two years, only 15 participants’ data were ultimately included in the analysis. This may raise questions for readers regarding the research process. Therefore, it is necessary to clearly present the number of individuals initially recruited the number who met eligibility criteria, the number who withdrew, and their reasons for withdrawal. To aid readers’ understanding, it is recommended to add a flowchart (e.g., CONSORT diagram) illustrating participant flow.

RESPONSE: Thank you for your input here. The authors have addressed this oversight in the results, as well as revising the manuscript to better here with the recent BMJ Consort update (ref added) and the revision of our figure to conform to CONSORT guidelines. Noted in lines 136-138 and 291 plus figure 2.

1. Results: While the study was designed primarily around performance outcomes, it did not include physiological measures (e.g., VO₂, blood lactate, acid–base balance), nor did it directly assess the dermal absorption of the lotions. This represents an important limitation in interpreting the results. Accordingly, this point should be explicitly mentioned in the Limitations section or included as a recommendation for Future Research.

RESPONSE: The authors take note of this comment and have revised the text in Lines 534-538 to read: “Such virtual environments have been shown to produce reliable cycling performances [48] while creating exciting new opportunities to conduct large scale cycling and possibly running trials in controlled environment on virtual riding routes. At the time of the study design, it was believed that such a platform would expand the potential subject pool. However, converting interested participants into enrolled and completed participants proved to be very challenging. While MR remote studies hold promise, a significant limitation of this and future remote studies is the lack of physiological data like VO2 and blood lactate, or markers of acid-base balance. Thus, research like this can lose insightful data that aids in the interpretive process.”

1. Structure: The current manuscript does not align with the structure generally recommended by MDPI journals (Introduction → Methods → Results → Discussion → Conclusions). The manuscript should be revised to follow this format.

RESPONSE: Thank you noting this. After review of the guidelines and template, we noted that the updated template used reflected the structure we presented, but the guidelines do not. We have since restructured as suggested.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The authors have done a thorough job addressing nearly of my concerns.

I continue to have one sticking point.  As previously noted by this reviewer, the Limitation section (line 536) is the first clear disclosure that data were collected remotely, outside of the lab without research supervision.  “online platform” can be used in the lab or remotely, so thatalone does not inform complete.  Also, Lines 185-186 in the Methods state “All trials were conducted indoors within temperature-controlled settings,” and implies a lab-controlled environment. Please make it clear to the reader that data were collected remotely.  This should be mentioned two places earlier in the paper: in the Abstract at line 15 add “for remote data collection” after “Using the Zwift online platform…”   It should also be mentioned at line 119 as the purpose in introduced.  For the later add “a remote data collection system” after “Zwift RM cycling platform.”

Author Response

As previously noted by this reviewer, the Limitation section (line 536) is the first clear disclosure that data were collected remotely, outside of the lab without research supervision.  “online platform” can be used in the lab or remotely, so thatalone does not inform complete.  Also, Lines 185-186 in the Methods state “All trials were conducted indoors within temperature-controlled settings,” and implies a lab-controlled environment. Please make it clear to the reader that data were collected remotely.  This should be mentioned two places earlier in the paper: in the Abstract at line 15 add “for remote data collection” after “Using the Zwift online platform…”   It should also be mentioned at line 119 as the purpose in introduced.  For the later add “a remote data collection system” after “Zwift RM cycling platform.”

The authors appreciate your review of the most recent draft and have no disagreement with your recommendations. We have re-read the paper and made the following revisions below to address your concerns.

Line 15: "Data were collected remotely using Zwift online platform. Fifteen cyclists completed four trials..." 

Lines 119-120: "...during a ~50-km cycling trials conducted remotely using the Zwift MR cycling platform."

Lines 171-173: "While a single researcher (CRH) was aware of which condition was provided to each subject, all trials were completed remotely from across the continental United States..."

Lines 186-187: "All trials were conducted remotely indoors within temperature-controlled settings."