Next Article in Journal
Developing and Evaluating Newsletters for Parent Engagement in Sustainability via Active Garden Education (SAGE)
Next Article in Special Issue
Therapists’ Emotional Responses in Individual Therapy with Depressed Adolescents: An Evaluation of the Data Structure of the Feeling-Word Checklist—28
Previous Article in Journal
eHealth Engagement on Facebook during COVID-19: Simplistic Computational Data Analysis
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Monitoring and Measurement in Child and Adolescent Mental Health: It’s about More than Just Symptoms

Jenna Jacob
1,* and
Julian Edbrooke-Childs
CORC, Applied Research and Evaluation, Anna Freud Centre, 4–8 Rodney Street, London N1 9JH, UK
Clinical, Education and Health Psychology, UCL, Gower Street, London WC1E 6AE, UK
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2022, 19(8), 4616;
Submission received: 4 April 2022 / Accepted: 8 April 2022 / Published: 12 April 2022
(This article belongs to the Special Issue Monitoring and Measurement in Child and Adolescent Mental Health)
Routine outcome monitoring (ROM) provides information to practitioners and others providing healthcare support to demonstrate the impact of interventions and for service evaluation. ROM usually takes the form of questionnaires that are regularly administered to patients, from which healthcare professionals can gauge to what extent therapy is progressing [1]. Such questionnaires can be used as clinical tools that may help facilitate better collaborative practice; to extrapolate discussions about how a young person or family sees progress and their experiences of the work with the practitioner [2]. ROM is widely used in youth mental health services [2,3,4,5] although not without its challenges, including limited resources, and lack of clinical “buy-in” [3,6,7,8,9]. The majority of ROM questionnaires are standardized, comprising fixed items that span difficulties identified by clinicians most often focused on symptomology [10] and often specific to certain population groups, e.g., age range; learning disabilities; perinatal.
Evidence suggests that clinicians are less accurately able to predict patient outcomes using clinical judgement alone, particularly when a patient is veering off a projected track of progress [11,12]. It is also important to consider the differences between the use of measures and the use of measures with feedback: of incorporating discussion about the measurement questions, and areas of tracking focus, into the work with young people, and families/carers, as relevant. The importance of using outcome measures in a considered way is a matter of ethical concern and has been advised for a number of years [2]. This is where the use of measures and the collection of data are conducted in a way that is meaningful to the client and practitioner, and is not just a bureaucratic exercise, absent of any discussion with the client. Further, central to this considered use of ROM is discussion and focus on areas of outcome tracking that are important to the young person [10,13,14,15]. When used as part of collaborative practice, shared decision-making, and to empower clients, the use of ROM has been perceived by clients as helpful, as a means to enhance communication and to stimulate reflection [2,16]. Goal setting as ROM has been described as a conduit for a shared understanding of difficulties and ways forward [17,18], and key to young people feeling heard, valued and understood [2,18,19].
In a review of evidence regarding the impact of feedback from outcome measures on treatment effectiveness, incorporating feedback from standardized measures into ongoing therapy sessions with youth has been evidenced as useful in just over half of all cases [20]. Of particular importance was the use of feedback to ensure better outcomes for those who are veering off a projected trajectory, and better outcomes are reported when feedback is provided to both the practitioner and the patient [20]. In a meta-analysis from around the same time, Tam and Ronan [21] found a small but significant effect size of 0.20 (Hedges’s g) for feedback with youth. However, the authors of both studies advise caution in interpretation due the considerable heterogeneity across the reviewed studies and outcome measures. As far as we are aware, there have been no more recent reviews of the literature to further these findings, and recent Cochrane reviews highlight a paucity of evidence generally in both adult and youth settings [22,23]. Research to date has prioritized the experiences of youth in the use of routine outcome measures, which is of paramount importance.
There are several examples of measurement-focused care worldwide e.g., [2,24,25,26]. However, these initiatives tend to focus on symptomology tracking, as opposed to more complementary elements of change and improvement, such as functioning or empowerment. Whilst a focus on symptoms can be important, it will not be the principle area of change for all. The challenges of relying heavily on symptomology are that other important areas of outcome may be missed. For example, evidence demonstrates that widely available youth outcome measures do not capture the gamut of goals set by youth for focus in therapy, including existential areas of improvement such as understanding oneself, being independent or responsible, and increasing confidence [14]. A recent systematic review also reported that clients of all ages can experience ROM as suspicious, and limited in its ability to fully capture their needs [16]. Further, focusing solely on only one area of change, namely symptomology, runs the risk of over- or under- reporting levels of change; varied levels of improvement have been demonstrated when focused on different areas of measurement. Evidence suggests that only around 16% of youths report consistent cross-domain improvement, and one in four young people have reported significant improvements in symptoms but not in functioning. Further, one in three young people reported significant goal progress but not significant symptom improvement [27], and when goal data were added to combined analysis of symptom-based outcome analysis, an increase in the overall level of measurable improvement and a reduction in the no measurable change category was reported for youths [28].
There are also issues with the inclusiveness of self-report measures in particular. As text-based methods, these outcome measures could exclude some young people, either due to cultural or linguistic reasons, or because they have learning disabilities. Such examples include the propensity for standardized outcome measures to be developed based on experiences, behaviors, and attitudes of young White people from the Global North, which is underpinned by a general lack of research focused on young people in the Global South [29], despite 85% of the world’s youth being in Global South countries [30]. Further, mental health and wellbeing measure questions have been found to not be age-appropriate [31], and young people have raised challenges associated with the readability of them [32], also highlighting the frequent disparities between chronological age and reading age. These factors render a mismatch between standardized questions and many young people’s understanding and interpretation of their own experiences, as well as the understanding and interpretation of the data derived from the measures. The measurement of patients who have experienced trauma is also often excluded from exploration (e.g., [33]), or not adequately capturing relevant areas of outcome [34]. This further highlights the importance of considering complementary areas and methods of ROM.
This Special Issue is focused on a wide range of ROM. This includes measurement considerations for tracking quality of life (Krause et al.) and empowerment (Harju-Seppänen et al.), contextualizing, and complexity factors (da Costa et al.), the exploration of goal setting for use with young people who have experienced trauma (Law et al.), and who are receiving mental health interventions in school (Duncan et al.), the exploration of mental and physical health outcomes for children and young people with chronic physical health difficulties (Hiremath et al.), and for those who use the dark web (Idelji-Tehrani et al.). Tying all of these studies together, the clear need for additional monitoring outside of traditional symptom tracking is required for youth mental health and wellbeing support. It has been long recommended that individualized measures, or those that seek to explore a holistic view of outcomes, are used alongside standardized symptom focused outcome measures to ensure a breadth of information is explored [35,36,37,38]. By reflecting on the monitoring and measurement in child and adolescent mental health through this wide range of methods and approaches, we hope that this Special Issue furthers learning in the field. Through this multifaceted exploration of ROM, we hope it helps to promote reflection, which is useful at all levels of service provision, from the use of measures by practitioners, to wider policy and research considerations, retaining at the heart a considered focus on young people as whole beings and with areas of important outcome measurement which reach beyond a sole focus on symptoms.

Author Contributions

Conceptualization, writing—original and subsequent draft preparation was conducted by J.J. and J.E.-C. All authors have read and agreed to the published version of the manuscript.


The authors received no funding for this research. Edbrooke-Childs reports grants from NHS England and NHS Improvement, outside of the submitted work. Edbrooke-Childs and Jacob report grants from the Wellcome Trust outside of the submitted work.

Conflicts of Interest

Both authors work on the Child Outcomes Research Consortium (CORC) project, based at the Anna Freud National Centre for Children and Families, which encourages the use of outcome measures amongst its members in youth mental health settings.


  1. Carlier, I.V.; Meuldijk, D.; Van Vliet, I.M.; Van Fenema, E.; Van Der Wee, N.J.; Zitman, F.G. Routine outcome monitoring and feedback on physical or mental health status: Evidence and theory. J. Eval. Clin. Pract. 2010, 18, 104–110. [Google Scholar] [CrossRef] [PubMed]
  2. Law, D.; Wolpert, M. Guide to Using Outcomes and Feedback Tools with Children, Young People and Families. Formally Known as the COOP Document; CAMHS Press: London, UK, 2014. [Google Scholar]
  3. Fleming, I.; Jones, M.; Bradley, J.; Wolpert, M. Learning from a Learning Collaboration: The CORC Approach to Combining Research, Evaluation and Practice in Child Mental Health. Adm. Policy Ment. Health 2014, 43, 297–301. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Jensen-Doss, A.; Smith, A.M.; Becker-Haimes, E.M.; Ringle, V.M.; Walsh, L.M.; Nanda, M.; Walsh, S.L.; Maxwell, C.A.; Lyon, A.R. Individualized Progress Measures Are More Acceptable to Clinicians Than Standardized Measures: Results of a National Survey. Adm. Policy Ment. Health 2017, 45, 392–403. [Google Scholar] [CrossRef] [PubMed]
  5. Waldron, S.M.; Loades, M.E.; Rogers, L. Routine outcome monitoring in CAMHS: How can we enable implementation in practice? Child Adolesc. Ment. Health 2018, 23, 328–333. [Google Scholar] [CrossRef]
  6. Batty, M.J.; Moldavsky, M.; Foroushani, P.S.; Pass, S.; Marriott, M.; Sayal, K.; Hollis, C. Implementing routine outcome measures in child and adolescent mental health services: From present to future practice. Child Adolesc. Ment. Health 2013, 18, 82–87. [Google Scholar] [CrossRef]
  7. Boswell, J.F.; Kraus, D.R.; Miller, S.D.; Lambert, M.J. Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions. Psychother. Res. 2015, 25, 6–19. [Google Scholar] [CrossRef]
  8. Hall, C.L.; Moldavsky, M.; Taylor, J.; Sayal, K.; Marriott, M.; Batty, M.J.; Pass, S.; Hollis, C. Implementation of routine outcome measurement in child and adolescent mental health services in the United Kingdom: A critical perspective. Eur. Child Adolesc. Psychiatry 2013, 23, 239–242. [Google Scholar] [CrossRef] [Green Version]
  9. Jensen-Doss, A.; Haimes, E.M.B.; Smith, A.M.; Lyon, A.R.; Lewis, C.C.; Stanick, C.F.; Hawley, K. Monitoring Treatment Progress and Providing Feedback is Viewed Favorably but Rarely Used in Practice. Adm. Policy Ment. Health 2018, 45, 48–61. [Google Scholar] [CrossRef]
  10. Krause, K.R.; Bear, H.A.; Edbrooke-Childs, J.; Wolpert, M. Review: What Outcomes Count? Outcomes Measured for Adolescent Depression Between 2007 and 2017. J. Am. Acad. Child Adolesc. Psychiatry 2019, 58, 61–71. [Google Scholar] [CrossRef] [Green Version]
  11. de Jong, K.; Van Sluis, P.; Nugter, A.; Heiser, W.J.; Spinhoven, P. Understanding the differential impact of outcome monitoring: Therapist variables that moderate feedback effects in a randomized clinical trial. Psychother. Res. 2012, 22, 464–474. [Google Scholar] [CrossRef]
  12. Hannan, C.; Lambert, M.J.; Harmon, C.; Nielsen, S.L.; Smart, D.W.; Shimokawa, K.; Sutton, S.W. A lab test and algorithms for identifying clients at risk for treatment failure. J. Clin. Psychol. 2005, 61, 155–163. [Google Scholar] [CrossRef] [PubMed]
  13. Hickie, I.B.; Scott, E.M.; Cross, S.P.; Iorfino, F.; Davenport, T.A.; Guastella, A.J.; Naismith, S.L.; Carpenter, J.S.; Rohleder, C.; Crouse, J.J.; et al. Right care, first time: A highly personalised and measurement-based care model to manage youth mental health. Med. J. Aust. 2019, 211, S3–S46. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Jacob, J.; Edbrooke-Childs, J.; Law, D.; Wolpert, M. Measuring what matters to patients: Using goal content to inform measure choice and development. Clin. Child Psychol. Psychiatry 2015, 22, 170–186. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Lavik, K.O.; Veseth, M.; Frøysa, H.; Binder, P.-E.; Moltu, C. What are “good outcomes” for adolescents in public mental health settings? Int. J. Ment. Health Syst. 2018, 12, 3. [Google Scholar] [CrossRef] [Green Version]
  16. Solstad, S.M.; Castonguay, L.G.; Moltu, C. Patients’ experiences with routine outcome monitoring and clinical feedback systems: A systematic review and synthesis of qualitative empirical literature. Psychother. Res. 2017, 29, 157–170. [Google Scholar] [CrossRef]
  17. Bennett, L.R. Adolescent Depression: Meeting Therapeutic Challenges through an Integrated Narrative Approach. J. Child Adolesc. Psychiatr. Nurs. 2012, 25, 184–194. [Google Scholar] [CrossRef]
  18. Martin, K. A Critical Realist Study of Shared Decision-Making in Young People’s Mental Health Inpatient Units. Ph.D. Thesis, University College London, London, UK, 2019. [Google Scholar]
  19. Feltham, A.; Martin, K.; Walker, L.; Harris, L. Using goals in therapy: The perspective of people with lived experience. In Working with Goals in Psychotherapy and Counselling; Cooper, D.L.M., Ed.; Oxford University Press: Oxford, UK, 2018; pp. 73–85. [Google Scholar]
  20. Gondek, D.; Edbrooke-Childs, J.; Fink, E.; Deighton, J.; Wolpert, M. Feedback from Outcome Measures and Treatment Effectiveness, Treatment Efficiency, and Collaborative Practice: A Systematic Review. Adm. Policy Ment. Health 2016, 43, 325–343. [Google Scholar] [CrossRef] [Green Version]
  21. Tam, H.; Ronan, K. The application of a feedback-informed approach in psychological service with youth: Systematic review and meta-analysis. Clin. Psychol. Rev. 2017, 55, 41–55. [Google Scholar] [CrossRef]
  22. Bergman, H.; Kornør, H.; Nikolakopoulou, A.; Hanssen-Bauer, K.; Soares-Weiser, K.; Tollefsen, T.K.; Bjørndal, A. Client feedback in psychological therapy for children and adolescents with mental health problems. Cochrane Database Syst. Rev. 2018, 8, CD011729. [Google Scholar] [CrossRef]
  23. Kendrick, T.; Moore, M.; Gilbody, S.; Churchill, R.; Stuart, B.; El-Gohary, M. Routine use of patient reported outcome measures (PROMs) for improving treatment of common mental health disorders in adults. Cochrane Database Syst. Rev. 2014, 7, CD011119. [Google Scholar] [CrossRef] [Green Version]
  24. Clark, D.M. Implementing NICE guidelines for the psychological treatment of depression and anxiety disorders: The IAPT experience. Int. Rev. Psychiatry 2011, 23, 318–327. [Google Scholar] [CrossRef] [PubMed]
  25. Morgan, M.A.; Coates, M.J.; Dunbar, J.A.; Reddy, P.; Schlicht, K.; Fuller, J. The TrueBlue model of collaborative care using practice nurses as case managers for depression alongside diabetes or heart disease: A randomised trial. BMJ Open 2013, 3, e002171. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Van Orden, M.; Hoffman, T.; Haffmans, J.; Spinhoven, P.; Hoencamp, E. Collaborative mental health care versus care as usual in a primary care setting: A randomized controlled trial. Psychiatr. Serv. 2009, 60, 74–79. [Google Scholar] [CrossRef] [PubMed]
  27. Krause, K.R.; Edbrooke-Childs, J.; Singleton, R.; Wolpert, M. Are We Comparing Apples with Oranges? Assessing Improvement Across Symptoms, Functioning, and Goal Progress for Adolescent Anxiety and Depression. Child Psychiatry Hum. Dev. 2021, 1–17. [Google Scholar] [CrossRef]
  28. Jacob, J.; Edbrooke-Childs, J.; da Silva, L.C.; Law, D. Notes from the youth mental health field: Using movement towards goals as a potential indicator of service change and quality improvement. J. Clin. Psychol. 2021, 1–14. [Google Scholar] [CrossRef]
  29. Woelbert, E.; Lundell-Smith, K.; White, R.; Kemmer, D. Accounting for mental health research funding: Developing a quantitative baseline of global investments. Lancet Psychiatry 2021, 8, 250–258. [Google Scholar] [CrossRef]
  30. Working Group on Youth. Youth and the State of the World 2000–2002; The United Nations: New York, NY, USA, 2003. [Google Scholar]
  31. Jensen, S.A.; Fabiano, G.A.; Lopez-Williams, A.; Chacko, A. The reading grade level of common measures in child and adolescent clinical psychology. Psychol. Assess. 2006, 18, 346–352. [Google Scholar] [CrossRef]
  32. Demkowicz, O.; Ashworth, E.; Mansfield, R.; Stapley, E.; Miles, H.; Hayes, D.; Burrell, K.; Moore, A.; Deighton, J. Children and young people’s experiences of completing mental health and wellbeing measures for research: Learning from two school-based pilot projects. Child Adolesc. Psychiatry Ment. Health 2020, 14, 35. [Google Scholar] [CrossRef]
  33. Kwan, B.; Rickwood, D.J. A systematic review of mental health outcome measures for young people aged 12 to 25 years. BMC Psychiatry 2015, 15, 279. [Google Scholar] [CrossRef] [Green Version]
  34. Hoffman, K.; Cole, E.; Playford, E.D.; Grill, E.; Soberg, H.L.; Brohi, K. Health Outcome after Major Trauma: What Are We Measuring? PLoS ONE 2014, 9, e103082. [Google Scholar] [CrossRef]
  35. Alves, P.C.G.; Sales, C.; Ashworth, M.; Faísca, L. “There Are Things I Want to Say but You Do Not Ask”: A Comparison between Standardised and Individualised Evaluations in Substance Use Treatment. Int. J. Ment. Health Addict. 2018, 18, 861–874. [Google Scholar] [CrossRef] [Green Version]
  36. Edbrooke-Childs, J.; Jacob, J.; Law, D.; Deighton, J.; Wolpert, M. Interpreting standardized and idiographic outcome measures in CAMHS: What does change mean and how does it relate to functioning and experience? Child Adolesc. Ment. Health 2015, 20, 142–148. [Google Scholar] [CrossRef] [PubMed]
  37. Sales, C.M.D.; Ashworth, M.; Ayis, S.; Barkham, M.; Edbrooke-Childs, J.; Faísca, L.; Jacob, J.; Xu, D.; Cooper, M. Idiographic patient reported outcome measures (I-PROMs) for routine outcome monitoring in psychological therapies: Position paper. J. Clin. Psychol. 2022; online ahead of print. [Google Scholar] [CrossRef]
  38. Wolpert, M.; Deighton, J.; De Francesco, D.; Martin, P.; Fonagy, P.; Ford, T. From ‘reckless’ to ‘mindful’ in the use of outcome data to inform service-level performance management: Perspectives from child mental health. BMJ Qual. Saf. 2014, 23, 272–276. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Jacob, J.; Edbrooke-Childs, J. Monitoring and Measurement in Child and Adolescent Mental Health: It’s about More than Just Symptoms. Int. J. Environ. Res. Public Health 2022, 19, 4616.

AMA Style

Jacob J, Edbrooke-Childs J. Monitoring and Measurement in Child and Adolescent Mental Health: It’s about More than Just Symptoms. International Journal of Environmental Research and Public Health. 2022; 19(8):4616.

Chicago/Turabian Style

Jacob, Jenna, and Julian Edbrooke-Childs. 2022. "Monitoring and Measurement in Child and Adolescent Mental Health: It’s about More than Just Symptoms" International Journal of Environmental Research and Public Health 19, no. 8: 4616.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop