The Limitations of Reward Effects on Saccade Latencies: An Exploration of Task-Specificity and Strength
Round 1
Reviewer 1 Report
In two experiments, Dunne and colleagues tested if the effects of spatially defined reward on saccade latencies – faster saccades to rewarded as compared to non-rewarded spatial locations – transfer to more complex oculomotor tasks, that is, a remote distractor task (Experiment 1) and an antisaccade task (Experiment 2). The reward/conditioning task replicated the effects of reward on saccade latencies, but no transfer to the other (blockwise interleaved) tasks was observed.
This is an interesting study addressing a timely question. The manuscript is reasonably well written and the results (as far as I can tell based on the present report, see below) are pretty straightforward, but I have several questions and suggestions regarding the methods and results.
Major points:
- I would like to see the results for the remote distractor task and the antisaccade task - preferably in dedicated figures, but at the very least, the means and standard errors and not only the results of statistical tests should be reported. After all, these are the critical results with respect to the question addressed here. Showing that saccades towards the rewarded hemifield were indeed faster than saccades towards the unrewarded hemifield during the conditioning phase (Figures 4 and 5) is certainly important, but more of a manipulation check in this context.
Minor points:
- Some more information about the timing should be added in the methods section – for all tasks, only information about the fixation time prior to task onset is provided. Was there a time limit for saccade initiation/completion (i.e., for how long was the target presented)? For how long was the target present after the saccade (i.e., in the post-saccade colour)? When there was reward feedback, for how long was it shown?
- The figures could be substantially improved. The resolution appears to be pretty poor and the font size is unnecessarily small and difficult to read. Information about the timing would be helpful in Figures 1, 2 and 3.
- “A button press was required to start the next trial.” (page 4, line 161) – Does this also apply to the reward paradigm and the antisaccade task?
- I found that the description of the procedure was pretty confusing. The overall design of the experiments - including both tasks and all phases – is only clarified on page 5 (lines 178-183). I think it would be helpful if this was mentioned earlier, before each task is described separately.
- Page 5, lines 184-185: The statement “The antisaccade task was run for 6 blocks and consisted of three experimental phases:” is only followed by two phases. The post-preconditioning phase is missing here.
- Why was there a post-preconditioning phase for the antisaccade task but not for the remote distractor task?
- Why were one-way ANOVAs used to examine the interaction of phase and hemifield in the reward task in Experiment 1, but t-tests in Experiment 2, even though task and design were exactly the same?
- There is a typo in the title (“task-specficity”).
Author Response
Response to Reviewer 1 Comments
In two experiments, Dunne and colleagues tested if the effects of spatially defined reward on saccade latencies – faster saccades to rewarded as compared to non-rewarded spatial locations – transfer to more complex oculomotor tasks, that is, a remote distractor task (Experiment 1) and an antisaccade task (Experiment 2). The reward/conditioning task replicated the effects of reward on saccade latencies, but no transfer to the other (blockwise interleaved) tasks was observed.
This is an interesting study addressing a timely question. The manuscript is reasonably well written and the results (as far as I can tell based on the present report, see below) are pretty straightforward, but I have several questions and suggestions regarding the methods and results.
Many thanks to the reviewer for the positive feedback regarding the study.
Point 1: I would like to see the results for the remote distractor task and the antisaccade task - preferably in dedicated figures, but at the very least, the means and standard errors and not only the results of statistical tests should be reported. After all, these are the critical results with respect to the question addressed here. Showing that saccades towards the rewarded hemifield were indeed faster than saccades towards the unrewarded hemifield during the conditioning phase (Figures 4 and 5) is certainly important, but more of a manipulation check in this context.
Response 1: We appreciate this suggestion very much. The remote distractor latency data can be found in Figure 7 (Page 10 / Line 379) and the antisaccade latency data can be found in Figure 9 (Page 12 / Line 437). Figures have not been included for the accuracy data as although the data replicated often yielded manipulation results (greater proportion of errors on distractor trials compared to control trials; greater proportion of errors on antisaccade trials) there were no clear reward effects found for these tasks.
Point 2: Some more information about the timing should be added in the methods section – for all tasks, only information about the fixation time prior to task onset is provided. Was there a time limit for saccade initiation/completion (i.e., for how long was the target presented)? For how long was the target present after the saccade (i.e., in the post-saccade colour)? When there was reward feedback, for how long was it shown?
Response 2: We thank the reviewer for this suggestion. The reviewer is right to point out that the specific timings of the experimental procedure could be made clearer. In accordance with the reviewers comment we have now added additional information regarding the specific timings of experimental trials (Experiment 1 Reward Paradigm Page 4 / Line 163-166; Remote Distractor task Page 5 / Line 207-210; Experiment 2 Antisaccade task Page 8 / Line 266-270).
Point 3: The figures could be substantially improved. The resolution appears to be pretty poor and the font size is unnecessarily small and difficult to read. Information about the timing would be helpful in Figures 1, 2 and 3.
Response 3: We appreciate the reviewer’s comments and agree that the figures required improvement. New figures with larger text size and better resolution have been created Further to this, experimental timings have been added to figures 2, 3 and 5.
Point 4: “A button press was required to start the next trial.” (page 4, line 161) – Does this also apply to the reward paradigm and the antisaccade task?
Response 4: A button press was required for all trials. This information has now been included. For the reward paradigm this information can be found on Page 4 / Line 167. For the remote distractor task this information can be found on Page 5 / Line 210. For the antisaccade task this information can found on Page 8 / Line 271.
Point 5: I found that the description of the procedure was pretty confusing. The overall design of the experiments - including both tasks and all phases – is only clarified on page 5 (lines 178-183). I think it would be helpful if this was mentioned earlier, before each task is described separately.
Response 5: The procedure sections have been revised to improve the clarity of description. Experiments 1 and 2 now have separate sections outlining their individual procedures on Page 3 / Line 133 and Page 6 / Line 230 respectively. Also, as per the reviewer’s request, the previous experimental descriptions clarifying the entirety of the procedure have been moved to the start of each experimental section (Experiment 1: Page 3 / Line 134-139; Experiment 2: Page 6 / Line 231-240). We have also included 2 new figures to display the experimental procedure for Experiments 1 and 2. (Figure 1 Page 4 / Line 142; Figure 4 Page 7 / Line 249)
Point 6: Page 5, lines 184-185: The statement “The antisaccade task was run for 6 blocks and consisted of three experimental phases:” is only followed by two phases. The post-preconditioning phase is missing here.
Response 6: We thank the reviewer for this comment. We have added the missing post-preconditioning phase on Page 7 / Line 260
Point 7: Why was there a post-preconditioning phase for the antisaccade task but not for the remote distractor task?
Response 7: The referee is right to point out that the remote distractor task in Experiment 1 contains only 2 experimental blocks whereas the antisaccade task in Experiment 2 contains 3 experimental blocks. In the revised manuscript the lack of a post-preconditioning phase in the remote distractor task has been specifically mentioned on Page 6 / Line 231-232. Whilst we agree that the experimental protocol for this study would be improved by the addition of a post-preconditioning phase this was rectified in Experiment 2.
Point 8: Why were one-way ANOVAs used to examine the interaction of phase and hemifield in the reward task in Experiment 1, but t-tests in Experiment 2, even though task and design were exactly the same?
Response 8: We thank the reviewer for this comment and agree that it is important to be consistent. As both experiments 1 and 2 comprised previous PhD study data for the first author, it was important to display a range of statistical analysis skills. In the interests of consistency, we have revised the manuscript so that both experiments 1 and 2 are analysed using t-tests. The amended analysis can be found on Page 9 / Line 326-335.
Point 9: There is a typo in the title (“task-specficity”).
Response 9: We have corrected the text as suggested Page 1 / Line 3.
Reviewer 2 Report
Dunne, Ellison & Smith investigated how reward effects acquired for regular pro-saccades transfer to well-known remote distractor (Experiment 1) or anti-saccade conditions (Experiment 2). Rewarding saccades to one hemifield led to a reduction of pro-saccade latencies in that direction in both experiments, but had no effect on distractor or anti-saccade conditions. The authors conclude that reward effects do not reflect long-term, systemic changes to the oculomotor system.
The study is very interesting and timely, but I see one major conceptual issue: Since the remote distractor and anti-saccade conditions were measured after the conditioning and before the extinction phase of the pro-saccade condition it might be that the reward effect from the pro-saccade condition simply washed-out at the time of the remote distractor and anti-saccade conditions. This interpretation would be entirely consistent with the data because the extinction phase also showed no residual effects of reward. I think the current experimental paradigm does not allow to distinguish properly between a lack of transfer or a quick washout and the authors should at least discuss both alternatives.
In addition, I have a few recommendations about the presentation of the statistical results and relevant literature:
The manuscript currently displays only saccade latencies in the pro-saccade task in figures 4 and 5. Since the latencies in the remote-distractor and anti-saccade trials are the primary measure of interest, I suggest to show these values as well in those figures.
Descriptive values in the results section are only provided for significant effects. I recommend to report descriptive values for all experimental conditions, such that the readers get a more complete picture of the data.
There was a significant main effect of phase for pro-saccade latency in Experiment 1, but not in Experiment 2. Since the pro-saccade protocol was identical in both Experiments, I was wondering how those differences can be explained.
Manohar et al. (2015) reported that reward can modulate performance above and beyond the speed-accuracy trade-off, while it seems that performance is limited to the speed-accuracy trade-off in the current study.
Line 106: How was eye dominance assessed?
Author Response
Response to Reviewer 2 Comments
Point 1: The study is very interesting and timely, but I see one major conceptual issue: Since the remote distractor and anti-saccade conditions were measured after the conditioning and before the extinction phase of the pro-saccade condition it might be that the reward effect from the pro-saccade condition simply washed-out at the time of the remote distractor and anti-saccade conditions. This interpretation would be entirely consistent with the data because the extinction phase also showed no residual effects of reward. I think the current experimental paradigm does not allow to distinguish properly between a lack of transfer or a quick washout and the authors should at least discuss both alternatives.
Response 1: Many thanks to the reviewer for this positive feedback regarding the study which we appreciate. We agree that readers will hopefully find this study interesting. The reviewer is right to point out that conceptually the lack of measurement in the remote distractor task after the preconditioning phase of the reward paradigm of Experiment 1 does not permit us to eliminate the idea that perhaps the results found are due to a wash-out of the reward effect at the time of the remote distractor task. In the antisaccade task in Experiment 2 this was more tightly controlled for. Due to this conceptual issue in the remote distractor task we altered the methodology slightly in the antisaccade task to include a post-preconditioning phase to eliminate the conclusion that this result may simply be due to wash-out. Our previous paper (Dunne et al., 2015) had identified a period of reward facilitation when no rewards were present for around 3 blocks (180 trials). As such we have conducted an additional analysis binning the first 3 and last 3 blocks of the secondary tasks in Experiments 1 and 2 in order to analyse whether there was any transfer of the reward facilitation seen in the reward paradigm. The additional analysis revealed no effects (Experiment 1: Page 10 / Line 382-383; Experiment 2: Page 12 / Line 441-452). We have included the alternative explanation regarding a potential wash-out effect into our discussion of the data along with the discussion of this additional analysis (Page 14 / Line 537-544)
Point 2: The manuscript currently displays only saccade latencies in the pro-saccade task in figures 4 and 5. Since the latencies in the remote-distractor and anti-saccade trials are the primary measure of interest, I suggest to show these values as well in those figures.
Response 2: We appreciate this suggestion very much. The lack of figures provided for the remote distractor and antisaccade task data was an oversight on the part of the authors. As such these have now been introduced in the revised manuscript. The remote distractor latency data can be found in Figure 7 (Page 10 / Line 379) and the antisaccade latency data can be found in Figure 9 (Page 12 / Line 437). Figures have not been included for the accuracy data as although the data replicated often yielded manipulation results (greater proportion of errors on distractor trials compared to control trials; greater proportion of errors on antisaccade trials) there were no clear reward effects found for these tasks.
Point 3: Descriptive values in the results section are only provided for significant effects. I recommend to report descriptive values for all experimental conditions, such that the readers get a more complete picture of the data.
Response 3: We thank the reviewer for this comment and agree that it is important to provide readers with better understanding of the data. As such, means and standard deviations have been added to the non-significant results also. These can be found throughout the results section (Page 9 / Line 314 – Page 13 / Line 497)
Point 4: There was a significant main effect of phase for pro-saccade latency in Experiment 1, but not in Experiment 2. Since the pro-saccade protocol was identical in both Experiments, I was wondering how those differences can be explained.
Response 4: We thank the reviewer for this comment. The referee is right to point out that the main effect of Phase is not replicated across Experiments 1 and 2. However we would argue that the critical replication is of the interaction between Phase and Hemifield, showing the reward facilitation to the rewarded hemifield during the conditioning phase when compared with the other phases without incentive. The lack of replication of the main effect of Phase suggests a more general facilitation of saccades in Experiment 1 when rewards were present regardless of the hemifield they were directed to. As the reward paradigm did not differ between Experiments 1 and 2 it is interesting that this main effect was not replicated however there was a 7ms facilitation for saccades in the conditioning phase compared to the other phases. This result may further add to the argument this paper is making regarding the transient and specific nature of the effects of rewards and their potentially limited use with clinical populations.
Point 5: Manohar et al. (2015) reported that reward can modulate performance above and beyond the speed-accuracy trade-off, while it seems that performance is limited to the speed-accuracy trade-off in the current study.
Response 5: We appreciate this comment from the reviewer regarding the Manohar et al., (2015) study which has now been incorporated into the discussion when talking about the accuracy data (Page 14 / Line 508-512)
Point 6: Line 106: How was eye dominance assessed?
Response 6: We thank the reviewer for this comment and apologise for the lack of information regarding the assessment of eye dominance. This information has now been included on Page 4 / Line 152-156 and was relevant for both Experiments 1 and 2.
Round 2
Reviewer 1 Report
The authors have addressed all of my concerns.
Author Response
We appreciate the reviewer for the constructive comments provided. We thank the reviewer for the effort and time put into the review of the manuscript.
Reviewer 2 Report
The authors improved the manuscript considerably, but some minor aspects could be improved even further.
Figures 1 and 3: The arrangement of panels (e.g. b & c in figure 1) can be easily misinterpreted as implying that different types of trials are following directly after each other, although each panel represents a complete block of trials. Either all panels are displayed in one row or panels need to be segmented more clearly.
Figure 7: The different line styles are hard to discriminate, I recommend to use at least two different colors for rewarded and unrewarded conditions.
I do appreciate that the authors analyzed blocks 1-3 and 4-6 separately. However, I do not agree that p-values of .09 or .06 in Experiment 1 provide substantial evidence for a null effect. The discussion of washout effects in lines 537ff could be a bit more nuanced and more faithful to the actual results.
Author Response
Response to Reviewer 2 Comments
Point 1: Figures 1 and 3: The arrangement of panels (e.g. b & c in figure 1) can be easily misinterpreted as implying that different types of trials are following directly after each other, although each panel represents a complete block of trials. Either all panels are displayed in one row or panels need to be segmented more clearly.
Response 1: We appreciate the feedback from the reviewer in regards to these additional figures. Figure 1 and Figure 4 have now been amended and the segmentation between the different panels is clearer. These figures can be found on Page 4 / Line 141 and Page 7 / Line 248 respectively.
Point 2: Figure 7: The different line styles are hard to discriminate, I recommend to use at least two different colors for rewarded and unrewarded conditions.
Response 2: We thank the reviewer for this comment. Changes have been made to Figure 7 in order to make the rewarded and unrewarded data clearer. Rewarded lines are now displayed in green. Unrewarded lines are now displayed in red. Figure 7 can be found on Page 10 / Line 379.
Point 3: I do appreciate that the authors analyzed blocks 1-3 and 4-6 separately. However, I do not agree that p-values of .09 or .06 in Experiment 1 provide substantial evidence for a null effect. The discussion of washout effects in lines 537ff could be a bit more nuanced and more faithful to the actual results.
Response 3: We thank the reviewer for this comment. Although we believe in being cautious in regards to the interpretation of non-significance we have included an explanation more in-line with the actual results. This additional explanation can be found on Page 14 / Line 542-555.
