2.1. Service and Setting
The service is a Non-Government Organisation (NGO), established in 2006 in a rural community in the New England region of New South Wales (NSW), Australia. Its broad objective is to provide alternative and positive pathways into adulthood for high-risk young people by providing a multi-component service that can target co-occurring risk factors. This objective is operationalised through a number of flexible activities, organised into five standardised core program components: (i) effective engagement, to optimise participation in the program; (ii) case management, to address participants’ immediate and practical needs, such as attending court or homelessness; (iii) diversionary activities, to reduce participants’ exposure to high-risk situations, such as night-time encounters with police in public places or volatile situations at home; (iv) personal development, identity and team identity, to improve participants’ personal coping strategies when they are in high-risk situations and their sense of connection to their peers and community; and (v) learning and skills, to increase their opportunities for active participation in education or training likely to lead to employment.
The model of standardisation (the five core program components) with built-in flexibility (the specific activities that operationalise each component which are selected and designed by staff) provides a mechanism to both standardise the intervention across multiple communities and tailor it to the resources available in different communities. This service has been implemented in different formats for high-risk young people in five different communities. For the first community, the service was delivered in a previously disused shed donated by the local council. For the second and third communities, services were provided as an outreach model through a combination of young people attending the shed in the first community, and staff from the first community providing activities in the second and third communities. For the fourth and fifth communities, high-risk young people accessed the service on a working, but largely disused, farm.
For the purpose of this paper, the five communities were clustered into three groups based on the different service delivery models: on-site based in a shed (community one); outreach (communities two and three); and on-site based on a farm (communities four and five).
2.2. Participants of the Service
Young people are eligible to participate in the service if they: (i) reside in a community where the service is available; (ii) are aged 14–21 years; and (iii) are currently experiencing more than one of the following behavioural risk factors: involvement in criminal activity; substance use; violent behaviour; homelessness; poor mental health and wellbeing; poor engagement with school (including suspensions and unexplained absences); and un- or under-employment.
Potential participants are referred from: individuals (self-referral, family members/primary caregivers, or a community member); local schools (because they are at risk of becoming completely and permanently disconnected from mainstream education); or another government or non-government agency (e.g., police, magistrate, NSW Department of Family and Community Services). Each referral is made using a standard expression of interest (EOI) form, comprising questions about the young person’s status in relation to the eligibility criteria. The EOI is reviewed by the program manager and at least two senior staff, each of whom provide a recommendation. The manager makes the final decision on placement. Young people who meet the inclusion criteria and are recommended for placement are then interviewed by senior staff. Those who demonstrate a commitment to personal growth and appear genuinely self-motivated to participate are invited to attend the program for one week on a trial basis. If the number of suitable referrals is greater than the places currently available in the program, they are placed on a waiting list. Those who do not meet the inclusion criteria, or are not invited for a trial placement, have this decision explained to them and their referring agent by a senior staff member in a face-to-face meeting, and they are given the option of being referred to a more appropriate agency.
Trial participants become program participants if they successfully complete their trial week and attend at least four days in the first month. Participants who leave the program are welcome to recommence when it suits them, and the program manager ensures there are vacancies in the program for this eventuality. This flexibility is designed to foster ownership of decisions and personal responsibility.
Prior to establishing the NGO and researcher partnership, the service’s intake procedure comprised the completion of a basic administrative form (e.g., emergency contact details), and the setting of priorities and goals for participants. Collaboration between service staff and the researchers resulted in the development of a new, practically relevant, and scientifically rigorous routine assessment tool that was embedded into the existing intake procedure to measure participants’ risk factors. Acknowledging the dearth of appropriate measures of risk factors with published evidence for their reliability and validity amongst young people, let alone high-risk young people [21
], this collaboration achieved a compromise between pragmatism and scientific rigor by using “best-evidence” (BE) measures of risk factors. These are psychometrically tested measures with published evidence for their reliability or validity amongst a similar youth population, or where this is not possible, a normative adult population. Where these were unavailable, but staff required the information nonetheless for program design or monitoring of participant risk factors, non-psychometrically tested assessment items were sourced from surveys that targeted a similar population (SP) group (e.g., the NSW Schools Students Health Behaviours Survey [22
]). Where BE or SP assessment items could not be identified, new (N) items were developed by the researchers in partnership with staff. Assessment items were organised into demographic characteristics and four domains of risk sourced from a classification developed by the authors in a previous systematic review of services for high-risk young people [15
Demographic characteristics (SP): Gender, date of birth, Aboriginal or Torres Strait Islander status, and community of residence.
Education and employment (SP):
Number of school suspensions in the past six months, frequency of school attendance, employment status, and receipt of a government financial benefit [22
Mental health and wellbeing (BE, SP, and N):
In line with previous research [23
], a summary measure of recent suicide ideation was based on positive endorsement of one or more of five yes/no items from the Psychiatric Symptom Frequency Scale [24
]: “In the past four weeks have you ever felt that life is hardly worth living?”; “In the past four weeks have you ever thought that you would be better off dead?”; “In the past four weeks have you ever thought about taking your own life?”; “In the past four weeks have you ever made plans to take your own life?”; and “In the past four weeks have you ever attempted to take your own life?”. Given staff highlighted the importance of this measure as a screening tool to identify participants experiencing current suicide ideation, the time period from the original scale, which assesses suicide ideation “in the past year”, was modified to assess suicide ideation “in the past four weeks”. Resilience was measured using a brief 10-item version (CD-RISC-10) of the original 25-item Connor-Davidson resilience scale [25
]. This reliable and valid scale assesses respondents’ perceptions of their ability to adapt to change, to deal with unexpected events, to cope with illness, injury, unpleasant feelings or obstacles, and to remain positive in stressful situations. The CD-RISC-10 was favoured by staff over the original version because it has been administered to youth samples [27
] and its brevity reduces its response burden. Items are scored on a scale from 0 (not true at all) to 4 (true nearly all the time) and summed to a total score ranging from 0 to 40. Higher scores reflect greater resilience. Mental distress was measured using the six-item Kessler Psychological Distress Scale (K6), where each item was again rated on a five-point scale from 0 (none of the time) to 4 (all of the time) and summed to a total score from 0 to 24 [29
]. A score of ≥5 indicates moderate mental distress and a score of ≥19 indicates serious mental distress.
General health and wellbeing items relating to frequency of fast food consumption in the past week and frequency of physical activity in the past week were sourced from the NSW Schools Students Health Behaviours Survey [22
], and a new item was developed to measure frequency of health service utilisation.
Substance use (BE and SP):
Risky drinking was measured using the AUDIT-C (comprising the first three items of the Alcohol Use Disorders Identification Test (AUDIT) [31
]) which has demonstrable evidence for its reliability and validity, and performs well as an abbreviated alcohol screening measure in integrated health-risk surveys delivered in non-medical, community settings [32
]. Given staff indicated a high proportion of participants were Indigenous and that there was a general lack of understanding amongst participants about standard drink sizes, a modified version of the original AUDIT-C wording was used, which has proven to be acceptable to Indigenous Australians [33
]. The third item of the AUDIT-C, which refers to heavy drinking, was also modified to reflect the Australian Alcohol Guidelines in place at the time of this study [35
] (see Table 1
for relevant modifications). Responses were scored as 0–4 and summed to a total ranging from 0 to 12. Risky drinking was measured using the validated Indigenous-specific AUDIT-C cut-off scores, and defined as a score of ≥5 [33
]. For smoking, respondents were asked if they were current, occasional, ex-, or non-smokers (SP item). Current smokers were also asked the two item Heaviness of Smoking Index (HSI): how many cigarettes per day do you smoke; and how soon after waking do you smoke your first cigarette [36
]. Using the standard classifications, a HSI score of 5 or more indicated high dependence. To measure illicit drug use, an abbreviated version of the illicit drug use questions in the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) questionnaire [37
] was developed to achieve a balance between standardisation and response burden. The eight ASSIST illicit drug questions (which ask about specific drug use) were summarised into five: “Have you ever used cannabis”, and if yes, “How often did you use cannabis in the past three months?”, “Have you ever used an illicit substance (that was not cannabis)?”, and if yes, “How often did you use an illicit substance in the past three months that was not cannabis?”, and “which illicit substance did you use that was not cannabis?”
Crime (N): Respondents were asked if they had ever committed a crime, had ever been a victim of crime, had a high risk of exposure to crime in the home (defined as having lived with someone who had ever been to prison, or having lived with someone who had been released from prison in the past 6 months), and had they ever been involved with the juvenile justice system (defined as ever having to appear in court as the person of interest, or ever having been detained in a juvenile facility).
2.4. Procedure to Optimise the Feasibility of Applying Best-Evidence Measures
The new best-evidence assessment tool was designed with several practical features to facilitate flexible implementation in a dynamic program environment. First, it was developed to be delivered by staff in electronic format via tablet or laptop. These electronic devices pique participants’ interest in the assessment tool and allow staff to implement it across all three modes of service delivery. Second, it was designed so that it could be delivered in four discrete sections rather than requiring participants to complete the full assessment in one sitting. Third, a bespoke database was developed, into which assessment responses are automatically downloaded and from which pre-formatted reports can be generated. This allows staff to easily track participants’ progress over time and modify program activities to their changing needs. This process also allows researchers efficient access to de-identified data from consenting participants. Fourth, to ensure the ongoing utility of the assessment tool, electronic automatic reminders were built into the database to remind senior staff when follow-up assessments were due. This replicates computerised clinical decision support systems used in health care services that have improved the performance of health practitioners [38
]. The program manager attended a one-day training session, facilitated by the researchers, on the new data collection procedures and management of the database. The manager then communicated this information to staff through their usual organisational processes.
For each participant, the intake procedure is initiated at the discretion of service staff, but it must occur within one month of their trial week to ensure risk factors are defined before behaviour change commences. Prior to commencing the intake procedure, participants are assured by staff that their responses are confidential. For young people who leave before qualifying as a participant but return at a later date, the intake procedure is re-initiated.
2.6. Ethical Considerations, Ethics Approval and Consent to Participate
All staff members are trained to screen for suicidal behaviours amongst participants. If, during administration of the assessment tool these behaviours are evident and the staff member feels the issue is beyond their training, the procedure is for the staff member to immediately stop the assessment, monitor their behaviour, and refer them on to a suicide call back service. If the participant does not want to talk to a staff member about this issue, and indicates he or she will not use the Suicide Call Back Service, the staff member will ask the young person for their permission to refer them to the School Counsellor, their local Headspace office, the community health centre, or other health professional qualified to address suicide-related issues.
Ethics approval (HC13055) was granted for this study by the University of New South Wales, University of New England, James Cook University, the University of Queensland, and the NSW Aboriginal Health and Medical Research Council. Participants were informed about the research study at the beginning of the formal intake procedure. They were assured that the inclusion of their data in the research was optional, that only de-identified data would be provided to the researchers from consenting participants, and that choosing not to provide consent would not impact their relationship with program staff, or their ability to attend the program. Only participants who provided signed consent (for participants 18 years or older), or assent and parental consent (for participants 17 years or younger), were eligible to participate in the study.