The Canadian Incidence Study of Reported Child Abuse and Neglect (CIS) is the only national source of data on child welfare services in Canada [1
]. With the exception of the National Incidence Study (NIS) conducted in the United States, the CIS is the only other serial study in North America that collects data regarding both the extent and nature of child maltreatment through the utilization of surveys of child welfare workers [2
]. Three cycles of the CIS have been completed to date (CIS-1998, CIS-2003, CIS-2008) [3
]. As part of the CIS cycles, provincial/territorial incidence studies explore the incidence of reported child maltreatment and the characteristics of children and families investigated by provincial/territorial child welfare services. The incidence studies have provided valuable information to guide policy-makers and practitioners on child maltreatment and the child welfare system’s response. Rates and characteristics of investigated maltreatment vary over time and between provinces. In Canada, rates of investigated child maltreatment have nearly doubled from 135,261 maltreatment investigations in 1998 to 235,842 maltreatment-related investigations in 2008, a rate of 39.2 maltreatment-related investigations per 1000 children [4
Despite the important contributions of the incidence studies, the data collected is cross-sectional, designed to provide national and provincial estimates [1
]. Gaps remain in our basic understanding of children and their families who are involved in child welfare services over time, including the long-term trajectories of children and the impact of services on children’s development and well-being [6
]. These gaps are barriers to making informed practice and policy decisions [1
]. Decision-making that is responsive to the needs of child welfare-involved children and families is dependent upon quality data, research evidence, and an organizational culture and climate that supports evidence to inform decisions [6
]. Formal university–child welfare agency partnerships offer a unique opportunity to begin to fill the gaps in the child welfare knowledge base and to link child welfare research with the realities practice [1
With resources from the Social Sciences and Humanities Research Council of Canada’s (SSHRC) Connections Grant (2015–2016), a formal partnership was developed between the University of Toronto, clinicians, policy analysts, and researchers from child welfare agencies across Ontario. This partnership was an opportunity to advance the evidence base with respect to service provision in Ontario and to contribute to a culture of knowledge and evidence that will eventually support more complex research initiatives. This article describes the university–child welfare agency partnership within a complex Ontario child welfare landscape. It discusses how timely, field-directed analyses of pressing issues experienced by this sector is critical to informing child welfare practice and policy responses. One of the rich sources of administrative data analyzed as a foundation for this partnership was the Ontario Child Abuse and Neglect Data System (OCANDS)—the first provincial child welfare data system in Ontario to track child welfare-involved children and families longitudinally. As an example of the type of analyses conducted by this partnership, the results of agency-driven analyses of recurrence using a taxonomy developed by university researchers are reported and implications are discussed.
2. Ontario Child Welfare: The Data, Practice and Policy Landscape
Child welfare in Canada is a mandatory service [9
] that is governed by legislation specific to each of Canada’s 10 provinces and three territories [2
]. The collection of maltreatment statistics in Canada varies across provincial and territorial jurisdictions. In Ontario, estimates of the cost for funding the child welfare system are over 1.4 billion dollars a year [12
]. There are 47 child welfare organizations in Ontario and each organization is a not-for-profit, independent legal entity, governed by an independent and volunteer community board of directors [12
]. Child welfare agencies across Ontario vary in the delivery of their services [9
] and in their information technology systems. A common province-wide information system, the Child Protection Information Network (CPIN) has yet to be fully implemented. The full implementation of CPIN across the province has been delayed due to challenges associated with integrating the different and independent information systems used by organizations to document case and financial management [12
]. Moreover, provincial information systems have not typically been used to generate the kind of data required to understand service trajectories, outcomes, and the impact of community level risk factors.
Researchers at the Factor-Inwentash Faculty of Social Work at the University of Toronto currently collect, support, and maintain the data of the only two available comprehensive child welfare data sources in the province: the Ontario Incidence Study of Reported Child Abuse and Neglect (OIS 1993, 1998, 2003, 2008 and 2013) [13
] and the Ontario Child Abuse and Neglect Data System (OCANDS). The OIS and OCANDS are critical to advancing knowledge of child welfare services across the province. The OIS is the only source of provincially aggregated child welfare data that describe the characteristics of children and families identified to the Ontario child welfare system and estimates the rate of reported maltreatment in the province. The OIS-2013 contains clinical and service information for an estimated 125,000 children identified to the Ontario child welfare system. OCANDS is the only system in Ontario to longitudinally track children and their families.
These two data sources offer distinct, yet complementary contributions. The OIS studies have informed several key policy initiatives in the Ontario child welfare sector, including the implementation of differential response models and the creation of specialized intimate partner violence teams. These data have also contributed to the understanding of risk assessments in child protection investigations [18
] and the recognition of opportunities for prevention and early intervention with children at risk of maltreatment [19
]. In contrast to the cross-sectional nature of OIS-2013 data, the longitudinal design of OCANDS allows for the assessment of trajectories and outcomes of children and families involved with the child welfare system.
2.1. A Promising and Underutilized Source of Provincial Child Welfare Data: OCANDS
To date, OCANDS contains data for over one million families identified to the child protection system since 2007. OCANDS extracts and standardizes administrative data from disparate information systems from 40 of 47 participating Ontario Children’s Aid Societies. OCANDS was created to provide Children’s Aid Societies (CASs) with a mechanism to develop a better understanding of the service provision, to track service outcomes, and to ultimately, improve the quality of services [22
]. OCANDS has been identified as a valuable but underutilized resource for the sector. The capacity of OCANDS to describe the experiences of children and families throughout their involvement with partner agencies has been identified as an essential component for growth in the Ontario child welfare sector. The OCANDS corresponds to the child welfare service continuum -initial allegation, investigation, corresponding decision-making tools (e.g., safety assessment, risk assessment, family strengths and needs, risk re-assessment, etc.), as well as child placement information (e.g., type and duration of each placement, etc.). Research-ready datasets can be constructed from these data in response to specific administrative or practice questions. OCANDS is a child-specific, event-level, longitudinal database.
Ontario child welfare is in the early stages of providing publically reported data regarding service provision. The shift towards enhanced accountability, transparency and continued quality improvement were factors that lead to the introduction of publically reported performance indicators in Ontario [23
]. In 2011, 24 Service Performance Indicators (SPIs) were endorsed by the province as a collection of metrics that would represent the key dimensions of child welfare. These service performance indicators reflected the key measurement constructs articulated in the National Outcomes Matrix (NOM) [25
] which focused on safety, permanence and wellbeing domains, in addition to agency management [26
]. In 2015, the Ministry of Children and Youth Services began publically reporting five performance indicators aggregated from two-thirds of Ontario child welfare organizations. Three of five Ontario’s publically reported SPIs are generated from OCANDS. OCANDS also has a dynamic web-based interface for individual child welfare agencies which affords them the ability to run their own queries with their data. The production and calculation of SPIs is the key task performed by OCANDS to date.
Researchers in the US have long noted the promise and potential associated with the use of administrative data for the field of child welfare (e.g., [27
]). Drake and Jonson-Reid [27
] suggest that a key benefit derived from the use of administrative data is the ability of administrative databases to facilitate reciprocal relationships between researchers, practitioners and policy makers. The unique perspective of child welfare practitioners can lead to valuable and innovative research questions [27
]. Accountability and linkability to other sources of administrative child welfare data are potential considerations for use [27
]. In Ontario, there is currently no readily available infrastructure or framework that would allow for the linking of child welfare data to other data sources in other sectors, including children’s mental health and youth justice.
The use of multiple information systems in the Ontario child welfare system has created significant barriers to access to longitudinal administrative data. With adequate resources for infrastructure, OCANDS could be linked with other provincial (e.g., Census) and specialized databases (e.g., Ontario’s Looking After Children), creating one of the largest and most complete child welfare datasets in North America. The Canadian Foundation for Innovation (CFI) has funded the infrastructure for the OCANDS and the Ontario Association of Children’s Aid Societies (OACAS) and the Ministry of Children and Youth Service (MCYS) has provided contract funding for the calculation of service performance indicators. Adequate resources, infrastructure, and sustainability are among the issues that must be addressed in order to ensure that the potential of OCANDS is realized and longitudinal child welfare data generated is meaningfully analyzed and translated into evidence of characteristics, trends, outcomes and trajectories of child -welfare involved children and families. Without adequate resources and investment, the potential of using administrative data will be largely unrealized.
2.2. Recurrence in Context
The use of administrative data can help child welfare agencies develop clinical and community profiles of children who are at greatest risk of placement disruption and long-term foster care, as well as profiles of families at the greatest risk of recidivism. The construction of these datasets permit statistical modelling methods that have yielded promising results in foster care research conducted in the United States [28
] and, more recently, in Canada with the first province-wide longitudinal studies being conducted in Quebec [32
]. These landmark Canadian studies have identified age-specific factors associated with the timing of placement, for whom placement, placement changes, and family reunification are most likely to occur in the province [32
]. There are also numerous examples in the extant literature that highlight the promise of using administrative data to explore recurrence (e.g., [35
]). Studies conducted in Quebec, Canada by Hélie and colleagues [36
] are also notable examples.
The majority of research that has addressed maltreatment recurrence originates from the US [41
]. US studies show wide variations with respect to rates of recurrence and have ranged from approximately 13% to 69% [41
]. There are numerous considerations when comparing studies or integrating study findings on recurrence. Notably, the extant literature is different in study methodology, including the definition of recurrence and the follow-up period of study which make comparisons difficult. Risk factors associated with child maltreatment recurrence include family, child, environmental factors [35
]. Factors related to recurrence include the type of maltreatment (e.g., neglect), chronicity (e.g., number of prior reports), younger children, caregiver risk factors (e.g., substance and alcohol misuse) and family and environmental factors (e.g., lack of social supports, socio-economic or financial status of family) [42
]. Disparate policy, legislative and practice contexts between Ontario other jurisdictions makes comparability and generalizability of recurrence findings challenging.
Recurrence has been used as an indicator for child safety in various jurisdictions, including, the United States, Australia, UK, Sweden, and Canada [41
]. In the US, national child welfare performance measures are authorized through federal legislation [42
]. In contrast to the recent emergence of public reporting of recurrence as indicators in Ontario, recurrence is considered an established indicator for child welfare system performance and functioning in the US [42
]. It is part of the US Federal Child and Family Services (CFSR) review process and annual reports by the federal government have defined recurrence as a further incident of substantiated or indicated maltreatment within a six-month period [42
]. Using this six-month definition, US recurrence rates have been shown to vary between states from 1% to 12% [42
]. The importance of conducting research that is applicable and generalizable to the Ontario child welfare context to inform practice and policy is warranted.
3. Supporting A University–Child Welfare Agency Partnership for Translating Data into Evidence: The SSHRC Connections Grant
The child welfare sector in Ontario has struggled to make progress in working with and utilizing existing data for the day-to-day operations of the agency and for the evaluation of child welfare practice and policy. This is a significant barrier to enhancing accountability and transparency for services provided. Administrative and Census data have the potential to assist researchers and service providers to better understand child welfare services; however, significant resources, research, and analytic training and capacity are needed to utilize these data [1
]. Trocmé and colleagues have noted that the road to evidence-based practice in the field of child welfare is complex and high stakes [1
]. There are challenges in the creation, mobilization and integration of evidence to guide child welfare practice and policy [1
]. The paucity of child welfare research in Canada has been attributed to several interrelated factors, including: limited resources and support for research; poor social work research training; minimal research and evaluation capacity within organizations; and, challenges inherent in conducting research with high-risk populations [1
]. There is also limited capacity and infrastructure to utilize data to conduct longitudinal research that informs our understanding of children’s long-term service trajectories and outcomes [1
]. Although there are examples noted in the extant literature of formal university–child welfare partnerships in the United States to forward evidence driven decisions in child welfare (e.g., [8
]); there are few in a Canadian child welfare context (see [1
] that are notable exceptions).
Formal university–child welfare agency partnerships can provide an opportunity to place the research infrastructure of the university and the expertise of scholars at the service of child welfare agencies [1
]. These partnerships can facilitate the building of capacity to co-create and mobilize research in ways that are relevant and accessible [1
] and help bridge research to practice [45
]. By strengthening data utilization capacity and culture within organizations, university-based research projects are, in turn, more likely to succeed. The literature converges on critical elements that contribute to the development and sustainability of successful collaborative community-academic partnerships: respect and trust amongst partners; having a shared vision, goals and/or mission; a relationship where partners work well together; reciprocity between partners; and, mutual benefits for all partners [46
Using existing sources of data to answer clinically relevant research questions is critical to building child welfare research capacity within agencies and universities, and to advancing the evidence base [9
]. Data becomes evidence by aligning questions with relevant information through the right analytic method [7
]. Over the course of its one year duration, this initiative built upon a previous SSHRC-funded research partnership that established a collaborative process for knowledge transfer and mobilization between the University of Toronto and the child welfare sector. Child welfare agency partners approached the research team to augment this work and to produce timely, relevant, field-directed analyses of pressing issues in the sector and to inform ongoing reform efforts in Ontario.
The overarching purpose of the SSHRC funded partnership initiative was to continue to support a developing culture of knowledge exchange in the Ontario child welfare sector that will eventually lead to more complex research initiatives. The key objectives of the connections grant included: (1) enhancing the capacity of service providers to access and analyze child welfare data to inform service and policy decisions; (2) integrating clinical expertise in service and policy decisions; and (3) developing a joint research agenda addressing high-priority knowledge gaps. These outcomes were achieved by providing an opportunity for personnel from the child welfare sector to work with the research team to pose questions, engage in a process of refinement, and disseminate the results of their findings. Over time and multiple iterations, the research to practice feedback loop can expand to accommodate more complex and rigorous research.
Through the resources provided by the SSHRC Connections grant, this partnership allowed the field to access OCANDS in order to address relevant policy and practice research questions. The OCANDS longitudinal design allows organizations to access, examine, and compare child protection data over time. The valuable linkages made between researchers and child welfare practitioners through timely and relevant knowledge exchange is vital to inform and direct effective child welfare policy and practice. Building upon previously SSHRC funded research mobilization initiatives; this initiative established a strong collaborative process and foundation for knowledge transfer and mobilization between the researchers at the University of Toronto and the Ontario child welfare sector.
Conceptual and Methodological Approach of the Partnership Initiative: A Collaborative Data Analysis and Support Approach
This partnership articulated a shared vision and common goals whereby agencies develop their research priorities and academics facilitate their implementation at the individual agency level, through collaboration across agencies and with the participation of our membership organizations. This partnership was driven by the assumption that successful partnerships between sectors must be reciprocal in nature [1
]. The posing and refinement of research questions in response to available data and statistical methods, which is typical of investigator-driven research, is combined with a collaborative approach that assumes that the utilization of research knowledge must be rooted in questions that routinely challenge decision-makers in the field [1
]. Stakeholders can provide insight that can assist in identifying key issues that inform research [46
]. Moreover, once stakeholders develop the necessary analytical skills, they are better positioned to set sustainable research priorities [1
]. Responses to research questions and the link between research and practice must be timely and relevant in order to ensure uptake. Thus, addressing the priorities and questions of partner organizations is critical to successful and sustainable partnerships [1
A model conceptualizing the community impacts of collaborative research partnerships in social sciences and health, the Impact Model informed this initiative, as it highlighted the importance of considering the impacts and key functions of this partnership, including: knowledge generation (e.g., activities that lead to increased knowledge); knowledge sharing (e.g., dissemination activities to enhance the utilization of evidence), and research and training (e.g., opportunities for the development of research skills) [49
]. Knowledge generation can result from and produce a range of products or outputs through reciprocal influence [49
]. This may include the learning acquired by researchers with respect to the types of research questions that reflect the needs and issues relevant to child welfare practitioners and the children and families they serve. This model considers a diverse range of knowledge-sharing mediums, such as websites and peer-reviewed publications [49
]. With respect to research education and training, partnerships may often provide training and development opportunities for university-based researchers and community members. Through the collaborative nature of partnerships, this may include engaging community partners in research designs or other aspects of the research. It may also include university-based researchers developing and refining research agendas and designs to align with the needs and realities of community partners.
Although this SSHRC partnership was funded for a short duration of one-year, the goal of the partnership was to move towards success in each of these key areas to build a sound foundation for other university–child welfare agency partnership initiatives. The assumptions that research impacts are multi-directional, multi-dimensional, and evolve over time underpin this model. By strengthening data utilization capacity and culture within child welfare agencies, university-based research projects are, in turn, more likely to succeed. Capacity building and sustainability are intrinsically linked [50
Recent analyses of Canadian and Ontario incidence studies have provided cross-sectional reporting of the overrepresentation of child welfare service involvement in Ontario by ethno-racial category and have revealed that Aboriginal children or First Nations children are dramatically overrepresented in the Ontario child welfare system [51
]. The overrepresentation of Aboriginal children in Ontario is reflective of larger trend in the Canadian child welfare system (e.g., [53
]). The partnership’s collaborative approach is also aligned with the Ownership, Control, Access and Possession (OCAP) principles [58
] and the Tri-Council Policy Statement [59
]. Research using OCAP principles and TCPS2 must be guided by the questions generated by Aboriginal communities.
4. An Example of Child Welfare Agency-Driven OCANDS Analyses: Recurrence by Urgent Protection and Chronic Need
Within the context of this university–child welfare agency partnership, many research questions emerged. Child welfare agencies identified recurrence as an important priority and agency-driven analyses were subsequently conducted on OCANDS generated recurrence Service Performance Indicators (SPI’s). Recurrence was noted as a serious concern requiring analyses in order to promote the safety and well-being of the children it serves. The publically reported SPIs revealed considerable variation between agencies. Agency members of the partnership wanted to better understand the types of investigations and closures that recur. This partnership represented a unique opportunity to understand the variation between agencies through detailed analyses.
At the request of each participating agency, fact sheets were prepared as a tool to help communicate the findings of child welfare agency-driven analyses to various stakeholders in their respective agencies. The fact sheets were jointly authored by child welfare agency personnel and the research team. With the consent of the participating agencies, information sheets were also prepared with aggregated SPI data. The information sheets developed were made publically available through an open access website, the Child Welfare Research Portal (http://cwrp.ca/
). The dissemination of knowledge generated by the partnership at the agency-level and in the broader public domain are indicators of positive impacts of this partnership.
Investigative trends have been emerging in Canada that suggest a shift from urgent protection concerns to an increased focus on the consequences of family dysfunction on child development and well-being [60
]. The number of investigations in Ontario has dramatically increased from approximately 47,000 in 1993 to 125,000 in 2013 [16
]. It has been hypothesized that the increase in investigations may be partly due to a shift in investigative practices, a greater understanding of the adverse consequences of maltreatment, and the types of investigated maltreatment [16
]. In addition, as a proportion of all investigations, urgent protection cases have decreased across CIS cycles of the [60
Understanding the profile of children and families and distinct service needs through the urgent chronic taxonomy was proposed by Trocmé and colleagues [60
] and was a construct identified as one that the partnership wanted to explore, in particular whether it could be used to understand recurrence. This framework categorizes investigations as either urgent protection, where a child’s acute safety is deemed to be at risk or, chronic need, where the child’s development or well-being are at risk due to family dysfunction [60
]. For the purpose of this analyses, investigations were classified as urgent protection if: (a) there was a child in the family younger than four and the investigation was for neglect or physical abuse; (b) the primary concern was sexual abuse; or (c) a child had sustained physical harm [60
]. Investigations deemed as fitting the chronic need classification focused on investigations relating to risk of ongoing family challenges impacting child development and well-being (e.g., caregiver capacity) and there was no acute threat to safety or forensic considerations. The data elements needed to construct this taxonomy were available (e.g., child age, closing and recurrence dates, alleged concerns, and physical harm) in OCANDS to map and harmonize for the purpose of these analyses.
Investigating child welfare workers can close a case following the initial investigation or the case can remain open and be transferred for further or ongoing child welfare services. Service Performance Indicator (SPI) 4 measures recurrence of verified investigation in the 12-month period after a case is closed at the investigation stage. Service Performance Indicator (SPI) 5 measures recurrence of verified investigation in the 12-month period after a case is closed from the ongoing child welfare services stage. Recurrence refers to families coming back into contact with the child welfare system after file closure. Verification refers to whether it is more probable than not that the originally alleged or new child protection concerns, which includes risk of harm, have occurred or currently exist [61
]. For the partnership, SPI 4 and SPI 5 cases were further disaggregated by either urgent protection or chronic need cases. Notably, the rate of recurrence is not equivalent to the rate of re-victimization. A verification decision can mean that the family requires service, or the child was in fact victimized. The verification of risk factors does not necessarily mean that a maltreatment incident occurred.
and Table 2
show the recurrence rates for cases classified as either urgent or chronic at both the index and recurrence intervention stages. The 2013/2014 fiscal year is used as an example. The number of cases initially categorized as chronic following investigation was approximately three times higher at 11,936 (76% of all cases) than those that were classified as urgent. Table 1
shows that of the 3759 cases classified as urgent protection at the time of case closure at the investigation stage, 4% recurred as urgent protection cases; whereas, 7% came back to the attention of child welfare authorities as chronic need. Chronic protection cases recurred as urgent at a rate of 2%; whereas, 11% recurred as chronic need. Urgent cases had an overall recurrence of 11%; whereas, chronic cases had an overall recurrence of 13%. The majority of cases did not recur after 12-months (88%).
provides data related to cases that were closed following on-going service provision. For SPI 5, the number of cases classified at initially as chronic at closing was almost five times higher at 4522 when compared to cases initially classified as urgent at closing 917 at ongoing services. Of the 917 cases classified as urgent when they were closed, 7% recurred as urgent protection; whereas, 11% recurred as chronic need. Of the 4552 cases classified as chronic at closing, 3% recurred as urgent; whereas, a much larger proportion, 14% recurred as chronic need.
Overall, the majority of cases (83%) did not recur within 12 months. Cases were more likely to return as chronic need cases. Whether exploring recurrence after case closure at investigation or at ongoing child welfare services, the greatest number of cases that do recur are chronic need cases returning as chronic need.