Development of a Brief Screener for Crosscutting Patterns of Family Maltreatment and Psychological Health Problems

Abstract

Prior work established the presence of six crosscutting patterns of clinically significant family maltreatment (FM) and psychological health (PH) problems among active-duty service members. Here, we develop a brief screener for these patterns via Classification and Regression Trees (CART) analyses using a sample of active-duty members of the United States Air Force. CART is a predictive algorithm used in machine learning. It balances prediction accuracy and model parsimony to identify an optimal set of predictors and identifies the thresholds on those predictors in relation to a discrete condition of interest (e.g., diagnosis of pathology). A 22-item screener predicted membership in five of the six classes (sensitivities and specificities > 0.96; positive and negative predictive values > 0.90). However, for service members at extremely high risk of clinically significant externalizing behavior, sensitivity and positive predictive values were much lower. The resulting 22-item brief screener can facilitate feasible, cost-effective detection of five of the six identified FM and PH problem patterns with a small number of items. The sixth pattern can be predicted far better than chance. Researchers and policymakers can use this tool to guide prevention efforts for FM and PH problems in service members.

1. Introduction

1.1. Overview

Most work studying either family maltreatment (FM) or psychological health (PH) problems has taken a variable-centered approach, examining associations among risk/protective factors and individual outcomes. Although this approach has established an enormous corpus of risk literature, it potentially obscures meaningful subgroups of individuals who exhibit distinct patterns of co-occurring problems. In contrast, person-centered approaches (e.g., latent class analysis [LCA]; Eshima, 2022) can identify these distinct subgroups, or latent classes, characterized by different patterns across multiple variables (Muthén & Muthén, 2012).

In earlier work (Lorber et al., 2018) summarized below, we applied LCA to examine symptoms above clinical diagnostic thresholds (i.e., clinically significant (CS) problems) for multiple family maltreatment (FM) and psychological health (PH) problems. We refer throughout this paper to the meshing of these problem sets as “crosscutting problems,” each of which has its own literature, with demonstrated, considerable prevalences and public health impacts. FM problems comprised partner physical (Segura et al., 2025; White et al., 2024) and emotional abuse (Lawrence et al., 2012) and child physical (Mathews et al., 2020; Teicher et al., 2016) and emotional abuse (Korolevskaia & Yampolskaya, 2023; Taillieu et al., 2016). PH problems comprised hazardous drinking (Kuntsche et al., 2017; MacKillop et al., 2022), prescription drug misuse (Schepis et al., 2022), suicidal thoughts (Cai et al., 2021) and suicide attempts (Moradi et al., 2021), posttraumatic stress (Atwoli et al., 2015), and depressive symptoms (Inungu et al., 2024; Zhang et al., 2024).

In creating a screener, we posit that FM and PH frequent common pathways may be assessed in a more parsimonious manner than an agglomeration of problem-centered question sets. Such screening instruments are important to identify at-risk individuals for (a) more detailed follow-up assessment and (b) appropriate referrals to preventive or clinical healthcare. In healthcare settings or when conducting a broad survey, a lengthy battery of assessments may not be feasible. Thus, the development of brief screeners for behavioral and mental health problems has become commonplace (Klinkman & Okkes, 1998; Mitchell et al., 2016).

1.2. Identified Classes

The Lorber et al. (2018) LCA identified a replicable six-class solution that cut across these behaviors/symptoms using two large samples of United States Department of the Air Force (DAF) active-duty SMs. We labeled these classes according to their risk of clinically significant internalizing (e.g., depression) and externalizing (e.g., partner abuse) behaviors. Five groups were ordinally arrayed across all indicators, with increasing risk of clinically significant FM and PH problems in smaller and smaller groups of individuals: Very Low CS-Internalizing/Externalizing Risk (Class 1; 63.3%), Low CS-Internalizing/Externalizing Risk (Class 2; 19.6%), Moderate CS-Internalizing/Externalizing Risk (Class 3; 9.8%), High CS-Internalizing/Externalizing Risk (Class 4; 3.7%), and Very High CS-Internalizing/Externalizing Risk (Class 5; 1.1%). A distinct sixth class had a markedly elevated risk for FM and other externalizing behaviors without elevations in posttraumatic stress or depression (Extremely High CS-Externalizing Risk, Class 6; 2.6%).

1.3. Theoretical Framework

Four foundational factors likely underlie the overlap of PH and FM problems. First, advocates of dimensional models (Caspi & Moffitt, 2018; Kotov et al., 2017; Krueger, 1999) theorize that overarching behavioral dimensions—such as internalizing (e.g., depression, PTSD) and externalizing (e.g., aggression, substance abuse) problems—contribute to multiple forms of symptom expression and maladaptive behavior reflected in person-based symptom clusters. Second, PH and FM may be causally linked with each other, sometimes through maladaptive coping. For example, alcohol abuse may arise as a coping response for depression or posttraumatic stress symptoms while simultaneously reducing aggressive inhibition (e.g., Freisthler & Gruenewald, 2013). Third, shared environmental risk factors and genetic vulnerabilities may simultaneously increase susceptibility to multiple problem outcomes. In service members (SMs), both pre-military (Gottschall et al., 2022) and military trauma exposure (Sadeh et al., 2017) are shared risk factors for mental health symptoms, aggressive behavior, and compromised family relationships (e.g., Creech & Misca, 2017). Fourth, neurobiological vulnerabilities (e.g., hypothalamic-pituitary-adrenal (HPA) axis dysregulation, executive functioning deficits) may predispose individuals to multiple forms of maladaptive behavior (Goodkind et al., 2015).

1.4. Implications for Screener Development

This work on profiles of crosscutting symptoms is not actionable without an accompanying measure that could practically screen for group membership in surveys and intervention settings. Such settings highly value parsimonious yet valid instruments; a crosscutting measure can emphasize parsimony by isolating items that classify group membership rather than symptom levels on myriad problems. However, we are unaware of any screening measures available for patterns of FM and PH problems’ co-occurrence in any population. Such a screener is needed for prevention (to plan for population-level interventions) and clinical (to match intervention strategies to individuals’ needs) uses. To illustrate, screening positive for membership in the Very High CS-Internalizing/Externalizing Risk and Extremely High CS-Externalizing Risk classes could be the basis for referrals for more intensive assessment and, potentially, clinical intervention (National Research Council and Institute of Medicine, 2009). In contrast, those who screen positive for membership in Moderate CS-Internalizing/Externalizing Risk and High CS-Internalizing/Externalizing Risk classes could be steered toward secondary/selective preventive interventions. To enhance the real-world viability of such a screener, brevity is paramount. Screener items would also ideally be of low perceived intrusiveness to minimize response bias and missing responses (e.g., asking about less severe, more common behaviors, such as slapping, rather than more severe, less common behaviors, such as choking).

Therefore, we aimed to develop a brief, minimally intrusive screener for detecting these crosscutting patterns that could (a) be easily administered with minimal and automated scoring, (b) be completed in 5 min or less (e.g., with fewer than 30 questions), and (c) reliably identify class membership. We sought an analytic approach that could identify the most predictive items for class membership to distill the complex, multi-dimensional LCA profiles into a small set of questions that can be used in survey or clinical screening contexts. To accomplish this task, we employed binary Classification and Regression Tree (CART) analysis (Strobl et al., 2009). CART makes no assumptions about which predictors are associated with the outcome and how (e.g., linear vs. nonlinear associations) and does not impose distributional assumptions about the outcome (Breiman et al., 1984; Hastie et al., 2009; James et al., 2021). Thus, it avoids mismatches between data and model assumptions that may lead to a distorted understanding of the phenomenon under study, hence non-optimal screeners.

We should note from the outset that (a) because the sample was a large, representatively drawn sample from the U.S. military that included extensive questions about FM and PH and (b) FM and PH problems are universal, it is well suited for the intended purposes. The broad racial and ethnic demographic representativeness of the sample to overall population makes it somewhat appropriate to generalize to the broader population. On the other hand, a volunteer military does not draw randomly from the civilian population, and the sample is younger and more heavily male than the general population.

2. Materials and Methods

2.1. Participants

We conducted a secondary data analysis using data from the 2011 DAF Community Assessment (CA), an anonymous, Internet-based survey of a stratified random sample of DAF active-duty SMs conducted at 91 installations worldwide in 2011. Stratification was by installation, by gender, and by pay grade (i.e., rank), ensuring representative sampling within installations and across gender and pay grades. All selected individuals were invited to voluntarily and anonymously participate in the survey. SM participation rate was 38.41% of those invited, N = 63,227.

This investigation was limited to SMs who were in romantic relationships (married or cohabiting with a partner) and had one or more children living in the same household, given our interest in patterns involving FM and PH problems. Our sample included 30,100 SMs with children in relationships from 91 installations. Among these, 741 (2.4%) were excluded because they contributed no information on any FM or PH measure. Most were men (84.6%) between 21 and 45 years of age (94.6%), who were married (95.8%) and had between 1 and 4 (M = 1.91, SD = 0.88) children. Few (7.5%) were deployed at the time of the survey. Air Force survey designers deliberately did not collect racial/ethnic identification to maintain respondent anonymity. The combination of race/ethnicity with other collected demographic variables (e.g., rank range, gender, length of service at installation) could have created sufficiently unique profiles to identify individual respondents, particularly at smaller installations where certain demographic intersections might apply to only one person.

2.2. Procedure

The CA survey and protocol were reviewed and approved before dissemination by the United States Department of the Air Force Survey Office and the United States Department of the Air Force Compliance Office. Informed consent was not required due to military regulations involving force-wide surveys. For this study, the New York University institutional review board ruled that, as an anonymous archival study, it was exempt from human subjects review per the United States Department of Health and Human Services Common Rule (45 CFR § 46.104[d][4]) stipulating that research involving the use of existing data, documents, records, or specimens is exempt if the information is recorded in a manner that does not permit the identification of individuals, directly or through identifiers linked to the subjects.

Randomly selected DAF SMs were invited to complete the anonymous, web-based DAF CA survey. Weekly e-mails were sent reminding the selected SMs to participate. Each base conducted a community-wide campaign encouraging participation. The survey took approximately 20–60 min and could be completed across multiple sessions.

2.3. Measures

The CA assessed FM, PH problems, and risk and protective factors at multiple ecological levels. The psychometrics of the measures have been reported in detail elsewhere (e.g., Lorber et al., 2017; Lorber et al., 2018). These measures are summarized in

Table 1. Description of instruments.

Scale	Items	M	SD	Min	Max	α	Scale	Items	M%	SD	Min	Max	α
Partner emotional abuse a	13	6.60%	–	0	1	–	Support from significant other	3	5.09%	0.93	1	6	0.84
Partner physical abuse victimization a	24	1.40%	–	0	1	–	Workgroup cohesion	6	4.16%	1.19	1	6	0.87
Partner physical abuse perpetration a	23	0.60%	–	0	1	–	Work relationship satisfaction	5	4.27%	0.99	1	6	0.73
Child emotional abuse a	16	3.20%	–	0	1	–	Community satisfaction	15	4.12%	0.98	1	6	0.86
Child physical abuse a	29	12.60%	–	0	1	–	Community unity	14	4.12%	0.91	1	6	0.95
Hazardous drinking a	10	7.70%	–	0	1	–	Support from neighbors	5	4.03%	1.29	1	6	0.87
Prescription drug misuse a	20	4.30%	–	0	1	–	Support from DAF agencies	4	4.26%	1.12	1	6	0.96
Suicidal thoughts a	5	2.20%	–	0	1	–	Social support	4	4.33%	1.52	1	6	0.95
Suicidal behavior	5	0.10%	–	0	1	–	Support for youth	3	4.25%	1.04	1	6	0.76
Depression	7	1.37	0.48	0	4	0.86	Supportive leadership	9	4.05%	1.02	1	6	0.9
Posttraumatic stress	4	0.36	0.92	0	4	–	Economic well-being	2	1.59%	0.81	1	5	0.87
Personal coping	9	4.18	0.48	1	4	0.89	Physical well-being	6	4.19%	0.71	1	6.33	0.73
Parent–child relationship satisfaction	3	5.12	0.74	1	6	0.73	Religious involvement	5	3.36%	0.99	1	5.67	0.69
Family coping	3	5.02	0.88	1	6	0.89	Community safety	4	5.15%	0.79	1	6	0.74

Note. Min = minimum; Max = maximum; ^a α of family maltreatment, prescription drug misuse, PTS symptoms, and suicidal thoughts were not computed, as they are index scores that do not make assumptions about the relations among the items (Streiner, 2003).

.

2.3.1. FM and PH Problems

Family maltreatment: Clinically significant physical and emotional abuse of intimate partners and children were operationalized in the Family Maltreatment Measure (Heyman et al., 2020). To be classified as either partner or child maltreating, individuals need to report (a) one or more acts of physical or emotional aggression and (b) significant harm or a high potential for harm (e.g., choking, using a weapon).

Hazardous drinking. Individuals who score ≥ 8 on the Alcohol Use Disorders Identification Test were classified as hazardous drinkers (Rumpf et al., 2002).

Prescription drug misuse: Participants were provided with a list of a wide variety of controlled prescription drugs (e.g., opiates). For each drug checked, respondents indicated (a) the frequency of use when s/he did not have a prescription and (b) the frequency of use at a dosage greater than prescribed. Prescription drug misuse was scored as present versus absent.

Suicidal thoughts and behavior: Using items from the Center for Disease Control and Prevention Youth Risk Behavior Surveillance System (Brener et al., 2002), respondents were classified as positive for suicidal thoughts if they reported, in the last year, (a) thoughts of ending his/her life (sometimes, frequently), (b) seriously considering attempting suicide (rarely, sometimes, frequently), or (c) planning suicide. Respondents were classified as positive for suicide attempts if they reported ever attempting suicide in the past year.

Depression: A brief version of the Center for Epidemiological Studies Depression Scale (CES-D; Mirowsky & Ross, 1992) assessed depressive symptoms in the past week.

Posttraumatic stress: The Primary Care PTSD Screen (Prins et al., 2003) assessed posttraumatic stress disorder symptoms in the last month.

Crosscutting classes of FM and PH problems: As described in the Introduction, based on the above 11 indicators, six distinct crosscutting patterns of FM and PH problems were explored and replicated in the LCA of Lorber et al. (2018): Class 1 (Very Low CI-Internalizing/Externalizing Risk), Class 2 (Low CS-Internalizing/Externalizing Risk), Class 3 (Moderate CS-Internalizing/Externalizing Risk), Class 4 (High CS-Internalizing/Externalizing Risk), Class 5 (Very High CS-Internalizing/Externalizing Risk), and Class 6 (Extremely High CS- Externalizing Risk). For the present analyses, individuals were assigned to their most likely classes based on posterior probabilities from the LCA. These class assignments were used as outcomes in the CART analyses. Additionally, individual FM and PH items and/or scales were used as predictors in the CART analyses.

2.3.2. Risk and Protective Factors

We included 17 risk and protective factors assessing four groups of risk factors: personal, family, work, and community adjustment. Item averages were computed for each scale (Nichols et al., 2023).

Personal adjustment: Personal coping items, drawn from the 2001 DAF Community Assessment (Bowen et al., 2003) and the General Self-Efficacy Scale (Scholz et al., 2002), reflect participants’ coping with stress, work, and family demands. Economic well-being items, drawn from the Social Change in Candida Survey (Krause & Baker, 1992), a financial strain scale (Vinokur et al., 1996), and a measure of family economic pressure (Conger et al., 1993), assess participants’ subjective self-reported financial stress ratings in the last 12 months. Physical well-being items, drawn from the Short Form-8 Health Survey (Ware et al., 2001), reflect participants’ ratings on their overall physical health patterns (e.g., sleep and diet) over the past month. Religious involvement items assess (a) the importance of spirituality, and (b) involvement in, and satisfaction with, a religious faith.

Family adjustment: “Parent–child relationship satisfaction items, adapted from the Relationship Satisfaction Scale (Simons et al., 1993), reflect satisfaction with child-related relationships and experiences. Intimate relationship satisfaction items, drawn from the Couple Satisfaction Index (Funk & Rogge, 2007), measure satisfaction in respondents’ current romantic relationships. Family coping items, drawn from the 2001 Community Assessment (Bowen et al., 2003), measure the extent to which families work together toward accomplishments, face challenges, or navigate difficult periods. Career support from significant other, a study-specific measure, reflects partners’ understanding and support of respondents’ work and careers in the DAF.”.

Workplace adjustment: Workgroup cohesion items, adopted from an Army Family Research Program (adapted from Caliber Associates, 2003; U.S. Army Community and Support Center, 1989), assess participants’ satisfaction with their supervisees, colleagues, and supervisors. Work satisfaction, a study-specific measure, reflects participants’ and their spouses’ satisfaction with work and life in the DAF. Satisfaction with the Air Force, a study-specific measure, assesses the satisfaction with the DAF for participants and their families.

Community adjustment: Community safety items were from the Community Indicators Survey (Princeton Survey Research Associates, 1999), reflecting perceived child safety and perceived crime- and violence-related safety in respondents’ neighborhoods, the military installation, and the surrounding civilian area. Satisfaction with community resources, a study-specific measure, assesses the satisfaction with DAF community and civilian community resources. Community cohesion items, adapted from the Community Capacity measure (Bowen et al., 2003), reflect participants’ sense of connectedness and shared mission with other members of the community. Support from neighbors items, adapted from the Social Capital Community Benchmark Survey (2000), assesses the support from people in the neighborhood. Support from formal DAF agencies, a study-specific measure, assesses participants’ satisfaction with official installation programs and the perceived quality of services. Social support items, adapted from readiness measures in the Army Family Research Program (Research Triangle Institute, 1990), assess the tangible support available from relatives, friends, neighbors, and colleagues. Community support for youth, a study-specific measure, measures opportunities for children to use their time well, and the degree of support and value for youth by installation leadership. Support from DAF leadership, a study-specific measure, assesses the level of support received from service leaders.

2.4. Analytic Strategy

To develop a screener that predicts membership in each crosscutting pattern of FM and PH problems (i.e., classes identified in the prior LCA), we followed two steps. In Step 1, we began with demographic information (i.e., gender, pay grade, marital status, number of children in the household) and a list of scale-level variables in CART analyses in the prediction of the six crosscutting patterns. These scale-level predictors included the 11 FM and PH problem variables used in constructing the crosscutting patterns, as well as the 17 risk and protective factors. We considered these variables because they are predictive of the outcome based on theory, empirical studies (Nichols et al., 2023), and how the crosscutting patterns were defined (Lorber et al., 2018). This step generated a list of primary and surrogate predictors and their cutoffs. Because the predictors are mainly scale-level variables (e.g., item averages), the list of involved items was long.

In Step 2, we used the primary predictors generated in Step 1 and reduced the number of involved items. For each scale with a factorial structure, item response theory (IRT) analyses were conducted to identify a subset of items with high “item information” (i.e., reliability in discriminating among individuals with varying levels of the construct being measured) that were retained in subsequent analyses. For the FM measures, the IRT approach could not be used because of skip patterns and complex scoring employed in these interview-like measures. Instead, we used brief screener versions of these measures developed by Heyman et al. (2021). Results were evaluated in terms of the balance of prediction accuracy and model parsimony (e.g., having fewer than 30 item-level predictors in the resultant screener).

We conducted CART analyses using the R rpart package (version 4.1-23; T. Therneau et al., 2023), which implements recursive partitioning to split the data into non-overlapping groups that are increasingly homogeneous in the outcome. At each node, the subset of observations defined by prior splits, CART evaluates all candidate splits and selects the primary predictor and cutoff that yield the greatest reduction in heterogeneity. When a primary predictor contains missing values, surrogate predictors will be identified to approximate the primary splits for those cases. Because our outcome is categorical, the Gini index was used to quantify heterogeneity in tree partitions. To shrink bias caused by missing values, we used surrogate variables in place of primary splitting variables for cases with missing primary splitting variable values (T. Therneau et al., 2023). Participants with an observed outcome and at least one predictor entered the tree analyses. The sample size in Step 2 was slightly smaller because subsets of predictors were under study, and participants with no observed values for all these predictors were excluded from further consideration. In both steps, the Gini index was used to quantify heterogeneity in tree partitions. We conducted 10-fold cross-validation (Kohavi, 1995; Kuhn & Johnson, 2013; Varma & Simon, 2006) for inducing and verifying tree models. We applied complexity parameter (cp) values to prune the resultant trees (Breiman et al., 1984) so that they were robust to random error in the data to avoid overfitting. For a detailed explanation of these procedures, see T. M. Therneau and Atkinson (2023). The R code is provided in Appendix A of the online supplement.

The prediction accuracy of the resulting screener was quantified by its sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for each crosscutting pattern (Rainio & Kannasto, 2024). Sensitivity measures the proportion of screener-based “true positives” among all those individuals who belong to a given class of crosscutting FM and PH problems; specificity measures the proportion of screener-based “true negatives” among all individuals who do not belong to a given class. PPV is the proportion of true positives among people who test positive for membership in a given class; NPV is the proportion of true negatives among people who test negative for membership in a given class.

2.5. Missing Data

The rate of missing values ranges from 3.2% (suicidal behavior) to 63.2% (child physical abuse) in crosscutting pattern indicators (M = 22.6%) and from 5.4% (personal coping) to 25.0% (social support) in other scale-level variables (M = 13.2%). The crosscutting pattern indicator measures were later in the CA and thus tended to have higher missing data rates. In our CART analyses, the R rpart package identifies surrogate variables correlated with the predictors that have missing values and uses them when the primary split variable is unavailable. We assumed that these surrogates capture sufficient information to approximate the primary split for such cases.

3. Results

3.1. Step 1: Scale-Level Variable Selection

In Step 1 (n = 30,100), we used all the scale-level and demographic variables in CART analyses to predict the six crosscutting patterns, one pattern at a time. All participants were included in this step. Four scale-level primary predictors were retained: PTS symptoms, depression, partner physical abuse victimization, and partner emotional abuse victimization. The sensitivity, specificity, PPV, and NPV of predicting Classes 1 through 5 (Very Low through Very High CS-Internalizing/Externalizing Risk) were all close to 1 (

Table 2. Sensitivity and specificity for predicting patterns of crosscutting FM and PH problems using all scale-level predictors vs. the brief screener.

Class	Sensitivity	Specificity	PPV	NPV	Prevalence
All Scale-Level Predictors
1. Very Low Clinically Significant-Internalizing/Externalizing Risk	0.996	0.978	0.989	0.992	65.6%
2. Low Clinically Significant-Internalizing/Externalizing Risk	0.981	0.996	0.983	0.996	19.4%
3. Moderate Clinically Significant-Internalizing/Externalizing Risk	0.993	0.999	0.989	0.999	9.3%
4. High Clinically Significant-Internalizing/Externalizing Risk	0.963	1.000	0.998	0.999	3.5%
5. Very High Clinically Significant-Internalizing/Externalizing Risk	1.000	0.999	0.901	1.000	1.1%
6. Extremely High Clinically Significant- Externalizing Risk	0.595	0.997	0.690	0.995	1.2%
22-Item Brief Screener
1. Very Low Clinically Significant-Internalizing/Externalizing Risk	0.999	0.978	0.987	1.000
2. Low Clinically Significant-Internalizing/Externalizing Risk	0.991	0.991	0.966	0.998
3. Moderate Clinically Significant-Internalizing/Externalizing Risk	0.974	0.998	0.989	0.997
4. High Clinically Significant-Internalizing/Externalizing Risk	0.963	0.999	0.999	0.999
5. Very High Clinically Significant-Internalizing/Externalizing Risk	1.000	0.998	0.901	1.000
6. Extremely High Clinically Significant- Externalizing Risk	0.255	0.997	0.582	0.991

Note. PPV = Positive Predictive Value; NPV = Negative Predictive Value; CS = clinically significant.

). Despite high specificity and NPV, the sensitivity and PPV in predicting Class 6 (Extremely High CS-Externalizing Risk) were much lower.

3.2. Step 2: Reducing the Number of Items

In Step 2, we aimed to reduce the number of items. The majority of the participants were included in Step 2 (n = 28,254; 93.4% of the sample). Some participants were excluded because of the presence of missing values in all four resultant scale-level predictors. These four scale-level predictors (i.e., PTS symptoms, depression, partner physical abuse victimization, and partner emotional abuse victimization) were composites of 48 item-level variables. We kept all 4 PTS and 7 depression items, as these measures were brief. To assess partner physical abuse victimization, instead of using all 24 items, we used only the seven items identified in the partner abuse screener developed by Heyman et al. (2021); a binary variable was created that indicated whether a participant endorsed any of the seven behaviors, scored as 1/0. A 4-item composite reflecting distress due to emotional aggression was also retained. The statistical properties of the resulting 22-item screener (Appendix B of the online supplement) are presented in Table 2.

The predictors and cutoffs for predicting membership in five of the six groups (Classes 1–5) are straightforward. Specifically, depressive symptoms alone provided close to perfect prediction of membership in the five CS-Internalizing/Externalizing Risk groups: Very Low, Low, Moderate, High, and Very High CS-Internalizing/Externalizing Risk. Participants who had depression < 1.38 were classified into the Very Low CS-Internalizing/Externalizing Risk group (Class 1). Participants who had depression ≥ 1.38 and <1.85 were classified into the Low CS-Internalizing/Externalizing Risk group (Class 2). Those who had depression ≥ 1.85 and <2.38 were classified into the Moderate CS-Internalizing/Externalizing Risk group (Class 3). Those who had depression ≥ 2.38 and <3.07 were classified into the High CS-Internalizing/Externalizing Risk group (Class 4). Finally, those who had depression ≥3.07 were classified into the Very High CS-Internalizing/Externalizing Risk group (Class 5), as illustrated in Figure 1A (Node 3).

The prediction of the Extremely High CS-Externalizing Risk pattern (Class 6) was more complex (Figure 1B). The retained predictors were partner physical abuse victimization, depressive symptoms, partner emotional abuse-related distress (depression and/or stress), and PTS symptoms. As shown in Figure 1B, the tree began with partner physical abuse victimization. Among those who reported no partner physical abuse victimization, if they also reported partner emotional abuse-related distress scores ≥ 1.25, depression < 1.5, and ≥2 PTS symptoms, they were classified into the Extremely High CS-Externalizing Risk group (Node 8). Among those who reported partner physical abuse victimization, those with depression < 1.79 and partner emotional abuse-related distress scores ≥ 1.75 were also classified into the Extremely High CS-E Risk group (Node 13). A scoring algorithm is provided in Appendix C of the online supplement.

In Step 2, the sensitivity, specificity, PPV, and NPV of predicting Classes 1 through 5 (Very Low through Very High CS-Internalizing/Externalizing Risk) were very high (Table 2). Despite high specificity and NPV, sensitivity, and PPV in predicting Class 6 (Extremely High CS-Externalizing Risk) were even lower than they had been in Step 1, with sensitivity less than half its former value.

4. Discussion

Using CART analyses with a large representative sample of the DAF, a brief screener was developed to efficiently predict the risk of clinically significant crosscutting patterns of FM (e.g., child physical abuse) and PH problems (e.g., suicidality). As we discuss in more detail below, the screener we developed can be used to efficiently and soundly screen for these patterns of FM and PH problems, and steer people toward appropriately matched services.

Using only a 7-item version of the CES-D depression scale, these analyses were able to distinguish with excellent fidelity among individuals in five classes of increasingly high risk for clinically significant levels of all FM and PH variables, cutting across the internalizing and externalizing behavior spectra. Sensitivities and specificities exceeded 0.96; PPV and NPV exceeded 0.90. Our prior work demonstrated that these classes are distinct and replicable patterns of maladaptation (Lorber et al., 2018).

It was more challenging to classify individuals who belonged to the sixth group—individuals with extremely high risk for externalizing behaviors (e.g., partner physical abuse perpetration) yet relatively low levels of depressive and PTS symptoms. However, our screener was capable of identifying a practically significant proportion of people at risk for this low base rate, extreme pattern of problematic behavior. Additional items beyond the seven-item CES-D scale noted above were needed, including items tapping PTS symptoms, victimization by partner physical abuse, and emotional distress related to victimization by partner emotional abuse—22 items in total. Despite specificity and NPV above 0.99, the corresponding sensitivity (0.26) and PPV (0.58) were notably lower. On the one hand, the lower sensitivity and PPV imply that the resultant screener is not ideal for detecting membership in the Extremely High Clinically Significant Externalizing class. On the other hand, given the low (1.2%) prevalence of this class in the sample, the screener we developed is still valuable. The sensitivity estimate implies that the screener would be able to detect about one-fourth of these extremely high-risk individuals in a given sample. Furthermore, the PPV estimate implies that, for an individual who screens positive for membership in this class, there would be a higher than 50% chance that this person indeed belongs to a group of people with extremely high risk for a very destructive pattern of behavior. Thus, the screener is nearly 50 times more likely to correctly identify this extreme risk group as opposed to having no available screening tool and identifying them purely by chance. Considering the highly destructive nature of this group’s behavior—to self, others, and unit cohesion—an inexact screener that flags possible membership in this group would likely still prove useful to military prevention planners.

The brief screener would take only a few minutes to complete. This efficiency may result in a correspondingly higher reporting rate, hence higher data quality, as survey completion tends to drop with increasing survey length. Moreover, its scoring is straightforward. The screener would dramatically reduce the testing burden on both the test administrators and participants, in contrast to the ≥256-item battery used to assess crosscutting patterns of FM and PH problems in Lorber et al. (2018). That burden was likely reflected in the rate of missing data, which increased as participants in the present sample progressed through the survey.

The cost of the detection of false positives and false negatives was equally weighted in our analyses. This implied that we considered the financial, safety, or social consequences of these two types of false predictions to be equivalent. The optimal weighting of these two types of errors depends on the purposes for which a screener might be used. It should be considered in the context of the intended application. To illustrate, in a military clinical setting, false positives may be affordable. An SM may screen positive for membership in the Extremely High CS-Externalizing Risk class and be referred for further evaluation, upon which it is determined that s/he does not have problems that merit clinical attention. However, a false negative may lead to substantially more adverse consequences for the SM, their family, workplace, and community, comprising not only direct financial burdens on the healthcare and justice systems, but also lost productivity, career implications, lower return on investment for career development and training, and legal fees (e.g., Strenio, 2019). Depending on the intended application, future research may incorporate the unequal weighting of false positives and false negatives using results from cost-effectiveness analysis. The weights can be incorporated into CART analyses by providing a pre-defined loss matrix, which reflects the relative importance of these two types of false detection to a particular setting (Williams, 2011). This matrix guides a classification tree by weighting how much to penalize each incorrect classification in a given choice of a given split. Because FM and PH problems are often hidden from official systems (including intervention services) until a serious and costly incident occurs (e.g., suicide attempt, partner abuse arrest; Heyman et al., 2011), by identifying at-risk individuals early, investment in well-tuned screening (with appropriate intervention) could save systems money by offsetting the economic and social costs of PH and FM progression. A carefully defined loss matrix would require further investigation of the financial cost of further testing, administrative expense, and other practical issues. This effort may potentially increase the sensitivity of resultant screeners.

4.1. Limitations

There are notable limitations of this study. First, it is important to understand the limitations of our CART analyses. We built a reasonable, rather than “best,” decision tree. Alternative approaches, such as random forest (Breiman, 2001), have been proposed and may be considered in future screener development and refinement efforts.

Second, more effort is needed to improve our screener’s accuracy in predicting membership in the Extremely High Clinically Significant Externalizing Risk group (Class 6). The severity of the consequences associated with this group’s very elevated risk of suicidal and violent behavior suggests the importance of further refining the screener to classify them more accurately. However, as we argue above, the screener identifies members of this group at a rate far greater than chance. Thus, administering the screener, its limitations notwithstanding, is much more informative than not administering it.

Third, the military study sample may constrain the generalization of our findings to other population groups. In this study, we used a DAF active-duty sample comprising young heterosexual couples with children. Their racial and ethnic distribution is more diverse than that of civilians. We caution against applying our findings to other populations without further evaluation.

Fourth, our data were cross-sectional. Future research on the predictive validity of the screener is needed.

Fifth, in the DAF community-wide survey, child abuse came toward the end of a very long survey that often took up to an hour, uncompensated, to complete if the respondent was in a relationship and had children, leading to survey termination before completion. In addition, even in an anonymous survey that contextualized acts of parent–child aggression by letting parents provide their motivations for the behavior (e.g., discipline), such questions are sensitive and may have contributed to the rate of missing data. In this study, missing information could be partially recovered using surrogate variables. Ideally, improvements in measurement and survey design would increase the response rates of the child physical abuse item, thereby enhancing screening performance.

Sixth, data were limited to self-report, which is subject to both the strengths and limitations of that method. Future work should bolster self-report with other measurement approaches where possible, further improving the precision of the screener.

Seventh, data were collected in 2011 (and were the same as that used in the Lorber et al., 2017, LCA). If the interrelationships among FM and PH problems, or their relation to screening items, has changed in the intervening period, then the screener’s concurrent and predictive validity would have degraded. Although we surmise this is unlikely, replication in both civilian and military samples, is important to assess for generalizability and reproducibility.

Finally, the DAF’s decision to highly curtail demographic assessment prevented any subgroup analyses based on race or ethnicity. The sample was representatively drawn from a highly diverse population, lending credence to its general application to American samples. However, future research should investigate measurement invariance (Putnick & Bornstein, 2016)—including sensitivity and specificity—across different racial, ethnic, geographical, sexual or gender minority, or other noteworthy subpopulations.

4.2. Implications for Use

Notwithstanding the above limitations, the 22-item screener presented here should facilitate sound and efficient detection of most of the crosscutting patterns of FM and PH problems identified in our earlier research. Stakeholders can use it as the first step in a multi-stage screening process (e.g., Winchester et al., 2013). Our screener likely would not replace assessment batteries or more in-depth measures of specific constructs of interest. Rather, its utility lies in its ability to efficiently identify individuals most likely to be members of the elevated-risk classes, thus facilitating indicated, in-depth second-stage assessment of FM and PH problems. Optimal screener usage, however, depends on the specific resources, goals, and ethical considerations of each setting. Although our findings establish the screener’s potential, interventionists must ultimately balance their own needs and specific system requirements to determine how to best integrate it into their efforts to identify and address FM and PH problems.

Individuals who screen positive for membership in groups with elevated risk (e.g., Classes 3–6) should likely be referred for more detailed assessment and appropriately matched preventive or clinical intervention. The ability to identify members of classes 5 (“Very High Clinically Significant Internalizing and Externalizing Risk”) and 6 (“Extremely High Clinically Significant Externalizing Risk”) is particularly significant. These small groups of people, together constituting 2.3% of the present sample, are at substantial risk of harming themselves (e.g., suicide) and others (e.g., violence). FM and PH problems tend to be concealed, thus going undetected and untreated (Heyman et al., 2011). The screener can be used to detect such high-need individuals who might otherwise go undetected and unserved.

In addition to the higher risk groups of people identified by the screener, the identification of membership in Class 2 is also of practical significance. Although labeled as “Low Clinically Significant Internalizing/Externalizing Risk,” they are a large group of people (nearly 20% of the present sample) who have elevations of FM and PH problem risk that distinguish them from the best-adjusted majority. Following the logic of selective prevention, they may benefit from a relatively low-intensity preventive intervention with no further assessment. This work should help researchers and stakeholders to make an informed choice about integrating our new tool into FM and PH problem intervention efforts.