Development of the Dutch Chaplaincy Outcome Measure

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The authors present the findings of a robust mixed-methods effort to design a patient-reported outcome measure specific to chaplaincy care in the Netherlands. I commend the team on their dedication, attention to detail, and thoroughness in this process. It is clear that their efforts intentionally integrated existing scholarly research and utilized methodology appropriate for the question. I will also highlight a few details worth reviewing, section by section, below.

Introduction

Line 29 - 32: I would consider re-writing this sentence (begins with "More recently,...") to deemphasize the interdisciplinary delivery of spiritual care and emphasize the importance of their findings relative to your work. In its current structure, it seems counter to the content right before and right after. 

Lines 46 - 62: Well written and excellent points. 

Line 84: I wonder if the word choice to describe chaplains' work as "doing good" is the most appropriate? May want to consider "effective" or "purposeful" care instead of "doing good"?

Methods

Lines 112-116: When you first mention the "Knowledge Workplace on Meaning and Spiritual Care" I would name it as a collaborative then. It took me a while to follow what type of group it was and its composition. So perhaps just some reorganization of this paragraph.

Figure 1: I'm not sure what the phrases "does not work" are for on the figure. I also wonder if you could recreate the figure to focus in on the portions of the model you discuss in this manuscript. The whole figure as is seems unnecessary.

Table 1: I had to get into the results to understand who the Client council and Participants in workshop were as identified in this table. Consider identifying those individuals in the methods (perhaps around lines 123 - 129).

Line 156: Your sentence states "The first focus group session of each learning network was focused on goals..." Consider changing the second use of the word "focus" to something different for the ease of reading.

Line 173: Sentence beginning with "To this end,..." I assume you are saying all participatnts were asked to (individually??) choose? In its current format, it is not clear that you mean to describe this as an individual task.

Line 175: I don't think you need the "s" after the 5. Currently "top 5s" could be "top 5"

Line 177: "This resulted in a total score for each goal" -- Do you mean you added up all individual scores to create the total?

Line 184 - 186: The way it is currently written (sentence starting with "The framework of goals ...") left me wanting to know more about how you used them as a conceptual guide. Can you be more specific? What do you mean you used them as a conceptual guide?

Line 246: Consider change the word "various" to "different". The word "various" makes it sound like there is more variation than really possible for only 4 people.

Results

Section 3.1: I wonder if you could consider moving the parts about eh format of the PROM after section 3.2. Without knowing what statements were chosen it is hard to follow the formatting content. It just may make for an easier read. (In other words, split up the content in the current 3.1. Leave goals there and move the content about the delivery format.)

Figure 2: Is it possible to have an english version of the sorting grid too?

Line 488-489: I didn't follow the description of people "moving profiles." If previously described in the manuscript, you may want to briefly restate here.

Discussion

Overall, I found the discussion easy to read and follow, however I wanted the authors to extrapolate more on their arguements. For example, in lines 569-578 the authors discuss the importance of future research identifying out how much the outcomes depend on a chaplain or these outcomes could be addressed by other professionals. Reading this in context of their introduction, where they discuss the misalignment of other PROM versions with spiritual wellbeing (see lines 33 - 49; one of my favorite paragraphs by the way), lead me to wonder if they too missed the mark on chaplaincy-specific outcomes? Or do they think they were on the mark? Perhaps starting with how their items align or misalign with previous PROMs could help. Then argue whether or not they addressed the inconsistencies seen elsewhere. If they did infact struggle with the same challenges as other PROMS, how could they then make sense of that? Could it be that participating clients' had limited conversations with chaplains? Or is it that chaplaincy as a profession needs to define its areas of expertise and challenge external professionals to more directly establish those parameters? Or is it because of a small sample? I would encourage the team to make a proposition regarding what they think and if nothing else, to directly discuss how their PROM may have the same challenges as previous iterations. 

I did not see a Limitations section and would encourage its inclusion.

Author Response

Thank you for the compliments on the thoroughness of our work and the suggestions for revision. We explain how we have taken up your suggestions here. We hope to have addressed them to your satisfaction.

Line 29 - 32: I would consider re-writing this sentence (begins with "More recently,...") to deemphasize the interdisciplinary delivery of spiritual care and emphasize the importance of their findings relative to your work. In its current structure, it seems counter to the content right before and right after. 

We have restructured the first half of the introduction at the recommendation of reviewer 2 and this sentence was removed.

 

Line 84: I wonder if the word choice to describe chaplains' work as "doing good" is the most appropriate? May want to consider "effective" or "purposeful" care instead of "doing good"?

We decided to keep the phrase. The goals of a care profession reflect the good they want to do for their patients. PROs represent what that ‘good’ is considered to look like.

 

Lines 112-116: When you first mention the "Knowledge Workplace on Meaning and Spiritual Care" I would name it as a collaborative then. Perhaps just some reorganization of this paragraph.

We have moved the sentence on the timeframe, so the explanation of the Knowledge Workplace immediately follows its first mention.

 

Figure 1: I'm not sure what the phrases "does not work" are for on the figure. I also wonder if you could recreate the figure to focus in on the portions of the model you discuss in this manuscript. The whole figure as is seems unnecessary.

We have replaced the figure with only the first three steps of the cycle that are described in this study.

 

Table 1: I had to get into the results to understand who the Client council and Participants in workshop were as identified in this table. Consider identifying those individuals in the methods (perhaps around lines 123 - 129).

We have now mentioned them in the methods section as well (l.158-164).

 

We have adopted the sentence-level suggestions for (originally) lines 156, 173, 175, 177, 184-186, 246; rephrasing some of the sentences to make them more clear.

 

Section 3.1: I wonder if you could consider moving the parts about eh format of the PROM after section 3.2. Without knowing what statements were chosen it is hard to follow the formatting content. It just may make for an easier read. (In other words, split up the content in the current 3.1. Leave goals there and move the content about the delivery format.)

Because the way the statements were selected and constructed was so dependend on the choices in step 1 of the PROM cycle, we have kept the order as it was. We have included an introduction to the results section to make it clear how and why the section is structured the way it is. We have added some examples of outcomes to the methods section and examples of statements to section 3.2 to give an indication of the content of the NUGV early on in the text.

 

Figure 2: Is it possible to have an english version of the sorting grid too?

We have now inserted an English version and removed the Dutch version.

 

Line 488-489: I didn't follow the description of people "moving profiles." If previously described in the manuscript, you may want to briefly restate here.

We have rephrased the description, hoping it will be clearer.

 

Reading this in context of their introduction, where they discuss the misalignment of other PROM versions with spiritual wellbeing (see lines 33 - 49; one of my favorite paragraphs by the way), lead me to wonder if they too missed the mark on chaplaincy-specific outcomes? Or do they think they were on the mark? Perhaps starting with how their items align or misalign with previous PROMs could help. Then argue whether or not they addressed the inconsistencies seen elsewhere. If they did infact struggle with the same challenges as other PROMS, how could they then make sense of that? Could it be that participating clients' had limited conversations with chaplains? Or is it that chaplaincy as a profession needs to define its areas of expertise and challenge external professionals to more directly establish those parameters? Or is it because of a small sample? I would encourage the team to make a proposition regarding what they think and if nothing else, to directly discuss how their PROM may have the same challenges as previous iterations. 

Thinking about revisions in relation to this comment we realized we had made an error in our thinking and writing about PROMs. The outcomes in a PROM are not necessarily specific to a discipline, but they should reflect the goals of treatment by the discipline (which they might share with others). This means that it is not necessarily a problem if the items of the Lothian PROM or of the NUGV reflect outcomes that are also affected by other care professionals. Nevertheless, the desire in the field of chaplaincy is to understand how it differs from others. This, however, cannot be shown with an instrument but should be examined in further research, with this as the research question. We have adjusted the presentation of the issue of chaplaincy specificity throughout the manuscript and have extended on this in the discussion section (lines 649-674).

 

I did not see a Limitations section and would encourage its inclusion.

We did discuss various limitations already but had not specified it as a limitations section. We have now added the header.

 

Reviewer 2 Report

Comments and Suggestions for Authors

I appreciated the article and the efforts to strengthen chaplaincy outcomes. Some things about the current structure of the article and its argument posed challenges for me. By section:

Introduction

The first and second paragraphs jump right into the nitty-gritty without quite enough context for the reader to feel comfortable. I actually think that the third paragraph would be a better opening paragraph.

Regarding the different assessments available, I personally think that the main hazard of trying to find outcomes that are chaplaincy-specific is that there are not outcomes that are truly exclusive to chaplaincy. I understand and value the parsing of spiritual-wellbeing and emotional-wellbeing, but I'm still left unclear as to what the real difference is -- or, more importantly, what the authors assume about what the real difference is. This becomes even murkier when we get to Table 2 in the results section; how are any of these statements exclusively about spiritual wellbeing?

The discussion of PROs in the introduction felt like an early preview of methods, and I wonder if it perhaps belongs in that section.

Methods

This is a very minor critique, but really important for international readers: At line 125 we learn that one of the contexts for chaplaincy is "care for victims of earthquakes attributed to gas extraction." I am baffled by this. Is this common in the Netherlands? Does the extraction of natural gas actually cause earthquakes in that part of the world? Is this a community group rather than a healthcare context? I need more information because I don't understand this as a context and would never assume that it even could be a context.

Nominal group technique is appropriate for this study.

I appreciate how thorough the authors are with their step-by-step description of their process, but I think it could be streamlined. Not every stakeholder's viewpoint, especially about things that seem more aesthetic than methodological, necessarily contributed to the outcome presented. 

Results

Can we get a translation of Figure 2, or a more detailed explanation of what we are seeing and what we should conclude from it?

In sum, I would try the instrument described for research. I don't necessarily believe that it solves the problem it purports to solve at the beginning of the article. I do wonder if it is ultimately very specific to the Dutch context.

Author Response

Thank you for your suggestions. We address them here, hoping we have revised the manuscript to your satisfaction.

 

The first and second paragraphs jump right into the nitty-gritty without quite enough context for the reader to feel comfortable. I actually think that the third paragraph would be a better opening paragraph.

We have restructured the first half of the introduction to start with a general introduction of outcome research and its purpose, followed by a more to-the-point discussion of the current shortcomings of this research.

 

Regarding the different assessments available, I personally think that the main hazard of trying to find outcomes that are chaplaincy-specific is that there are not outcomes that are truly exclusive to chaplaincy. I understand and value the parsing of spiritual-wellbeing and emotional-wellbeing, but I'm still left unclear as to what the real difference is -- or, more importantly, what the authors assume about what the real difference is. This becomes even murkier when we get to Table 2 in the results section; how are any of these statements exclusively about spiritual wellbeing?

< !--StartFragment -->This is an important question, that cannot be answered in this study. In revising the manuscript we also realized we had made a mistake in our thinking and writing about PROMs. The outcomes in a PROM are not necessarily specific to a discipline, but they should reflect the goals of treatment by the discipline (which they might share with others). This means that it is not necessarily a problem if the items of the Lothian PROM or of the NUGV reflect outcomes that are also affected by other care professionals. Nevertheless, the desire in the field of chaplaincy is to understand how it differs from others. This, however, cannot be shown with an instrument but should be examined in further research, with this as the research question.  We have adjusted the presentation of the issue of chaplaincy specificity throughout the manuscript and have extended on this in the discussion section (lines 649-674).

 

The discussion of PROs in the introduction felt like an early preview of methods, and I wonder if it perhaps belongs in that section.

There is indeed some overlap with the methods section. However, we felt it was important to introduce these characteristics of PROM development in the introduction section to help the reader understand the process early on. We have now added a subtitle, to make the purpose of the section more clear.

 

At line 125 we learn that one of the contexts for chaplaincy is "care for victims of earthquakes attributed to gas extraction." I am baffled by this. Is this common in the Netherlands? Does the extraction of natural gas actually cause earthquakes in that part of the world? Is this a community group rather than a healthcare context? I need more information because I don't understand this as a context and would never assume that it even could be a context.

We felt it would go too far to address this in the main text. But we have added a note to the text, explaining a little bit of the background.

 

I appreciate how thorough the authors are with their step-by-step description of their process, but I think it could be streamlined. Not every stakeholder's viewpoint, especially about things that seem more aesthetic than methodological, necessarily contributed to the outcome presented. 

We disagee that some remarks were mainly aesthetic, given the non-negligable impact an answer format for example can have on the how people understand and answer questions. And all remarks have been weighed to come the final product. In addition, we think the remarks provide relevant information about the the context in which the instrument was developed.

 

Can we get a translation of Figure 2, or a more detailed explanation of what we are seeing and what we should conclude from it?

We have now inserted an English version of the figure.

 

I do wonder if it is ultimately very specific to the Dutch context.

This is a fair remark. We have added a limitations section including a discussion of the limitation of the findings to the Dutch context (l.595-600)

Reviewer 3 Report

Comments and Suggestions for Authors

This paper fills a real gap in the literature and a need in the world by articulating chaplaincy-specific outcomes for measuring health and spiritual care. It’s very well researched and written. The use of the four goals of chaplaincy is quite effective, especially as they become related to outcome statements.

One idea for revision is to introduce a few of the chaplaincy-specific statements earlier in the paper to give readers a better picture of what will be assessed. As a reader, I kept wanting to flip ahead to see the statements, which are a real center of energy and insight in the paper.

You might also provide a bit more context and description of the populations surveyed (demographics, spiritual/religious affiliation).

Author Response

Thank you for your compliments about the value of the work and your suggestions for revision. We address those here, hoping we have handled them to your satisfaction.

 

One idea for revision is to introduce a few of the chaplaincy-specific statements earlier in the paper to give readers a better picture of what will be assessed. As a reader, I kept wanting to flip ahead to see the statements, which are a real center of energy and insight in the paper.

We have added some examples of outcomes to the methods section and examples of statements to section 3.2 to give an indication of the content of the NUGV early on in the text.

 

You might also provide a bit more context and description of the populations surveyed (demographics, spiritual/religious affiliation).

Unfortunately, we did not gather socio-demographic data from the participants. We have now stated this in the methods section (l.162-163)

 

 

Reviewer 4 Report

Comments and Suggestions for Authors

This is an excellent study that will significantly advance our understanding of how and why to assess the impact of spiritual care. It is especially useful for how it can help guide the conundrum mentioned on page 2 of the manuscript: "patients' goals for a specific type of care may differ from those of the care provider." 

The concerns about an outcome measure reported by participants and discussed on page 8 and following are to be expected and the authors' engagement with those concerns is commendable. 

The only addition I would prefer to see would be a very clear note that a study of spiritual care in a Dutch framework is, of course, conditioned deeply by Dutch circumstances. Similar studies could produce much different results in different places, as would the results of this PROM if deployed in multiple places. 

Author Response

Thank you for your compliments about the quality and contribution of the study, and your suggested revision. We adress it here, hopefully to your satisfaction.

The only addition I would prefer to see would be a very clear note that a study of spiritual care in a Dutch framework is, of course, conditioned deeply by Dutch circumstances. Similar studies could produce much different results in different places, as would the results of this PROM if deployed in multiple places. 

This is a fair remark. We have added a limitations section including a discussion of the limitation of the findings to the Dutch context (l.595-600)

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

Thank you for the revisions. I think the paper is excellent. Check punctuation typos in lines 31 and 3, capitalization in line 9, and alphabetical order of references in 743-747.