Next Article in Journal
Organization of Community Mental Health Services for Persons with a Severe Mental Illness and Comorbid Somatic Conditions: A Systematic Review on Somatic Outcomes and Health Related Quality of Life
Previous Article in Journal
Toward a More Comprehensive Index of Youth Cigarette Smoking: Average Number of Cigarettes Smoked per Day among Students in the United States over Two Decades
Previous Article in Special Issue
The Influence of Parenting Style and Time Management Tendency on Internet Gaming Disorder among Adolescents
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Comparison of Cut Points for Measuring Risk Factors for Adolescent Substance Use and Antisocial Behaviors in the U.S. and Colombia

by
Eric C. Brown
1,*,
Pablo Montero-Zamora
1,
Francisco Cardozo-Macías
1,
María Fernanda Reyes-Rodríguez
2,
John S. Briney
3,
Juliana Mejía-Trujillo
4 and
Augusto Pérez-Gómez
4
1
Department of Public Health Sciences, Miller School of Medicine, University of Miami, Miami, FL 33136, USA
2
School of Psychology, Universidad El Bosque, Bogotá 110121, Colombia
3
Social Development Research Group, School of Social Work, University of Washington, Seattle, WA 98115, USA
4
Corporación Nuevos Rumbos, Bogotá 110111, Colombia
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2021, 18(2), 470; https://doi.org/10.3390/ijerph18020470
Submission received: 31 October 2020 / Revised: 30 December 2020 / Accepted: 5 January 2021 / Published: 8 January 2021
(This article belongs to the Special Issue Risk Factors for Adolescent Substance Use and Addictive Behaviors)

Abstract

:
As the identification and targeting of salient risk factors for adolescent substance use become more widely used globally, an essential question arises as to whether U.S.-based cut points in the distributions of these risk factors that identify “high” risk can be used validly in other countries as well. This study examined proportions of youth at “high” risk using different empirically derived cut points in the distributions of 18 measured risk factors. Data were obtained from large-scale samples of adolescents in Colombia and the United States. Results indicated that significant (p < 0.05) differences in the proportions of “high” risk youth were found in 38.9% of risk factors for 6th graders, 61.1% for 8th graders, and 66.6% for 10th graders. Colombian-based cut points for determining the proportion of Colombian youth at “high” risk were preferable to U.S.-based cut points in almost all comparisons that exhibited a significant difference. Our findings suggest that observed differences were related to the type of risk factor (e.g., drug specific vs. non-drug specific). Findings from this study demonstrate the need for collecting large-scale national data on risk factors for adolescent substance use and developing country-specific cut points based on the distributions of these measures to avoid misidentification of youth at “high” risk.

1. Introduction

Substance use and addiction is still one of the most important public health problems facing youth worldwide [1]. From a prevention science perspective [2], the identification and targeting of risk factors, as precursors and causal mechanisms for adolescent substance use, has been shown to be the predominant and most effective way to address this problem [3,4,5]. Despite a wealth of information [6] and multiple registries of efficacious and effective programs to prevent youth substance use [7], uniform procedures to identify and address salient risk factors for youth substance use in communities are sorely lacking. Prevention program implementation and technical assistance systems that guide community coalitions and local agencies toward effective action [8,9] often rely on the community diagnosis model [10] to connect prevention initiatives with appropriate evidence-based programs.
The community diagnosis model suggests that risk factor data, collected via youth surveys administered in schools or communities, can be used to create risk factor profiles that identify the proportions of youth in the school or community that are at “high” risk for substance use and other youth antisocial behaviors [11]. Such risk factor profiles have been shown to be valid and useful tools for prevention programming by community coalitions [12]. A hypothetical example of a risk factor profile for a specific school is shown in Figure 1 where each bar in the figure shows the percentage of youth identified as being at “high” risk for each measured risk factor, based on a particular decision rule for determining what “high” risk is. In this example, the most elevated risk factors are Poor Family Management (in the family domain) and Parental Attitudes Favorable for Drug Use (in the family and peer-individual domains). Ideally, elevated risk factors should be linked subsequently to appropriate evidence-based prevention programs following an action plan derived from a local community prevention coalition or local agency. Often, school district, county, and state averages are included in the profiles for comparative purposes. Recent extensions of the community diagnosis model are being applied in the international arena for national [13] and cross-national [14,15,16] comparisons.
Strengths of the community diagnosis model include having a source of data that is derived from the perceptions of youth in the community (which might be different than the perceptions of risk by adults in the community), and the ability for “high” risk to be easily calculated and interpreted from psychometrically validated risk factor measures. However, the community diagnosis model is not without its limitations. Feinberg et al. [10] note that time lags between data collection and action plan formation (which can take years), uniform weighting of risk factors in prevention planning (i.e., some risk factors correlate more strongly with outcomes than others), and comparisons of elevated risk factors are usually made vis-a-vis other risk factors in a profile and not against a “gold standard” level of risk.
An important caveat to this model is the need for valid estimation of the proportion of youth at “high” risk for each risk factor that makes up the risk profile. As the use of prevention systems that rely on the community diagnosis model are increasing worldwide, an essential question arises as to the international generalizability of cut points in the distributions of risk factor measures to identify “high” risk populations. To the extent that prevention programs and strategies are developed to confront specific risk factors for adolescent substance use, the efficacy of these interventions may be undermined by incorrect identification of relevant risk factors in a community, school, or other setting. This study examined the cross-national validity of U.S.-based cut points in measures of risk factors for adolescent substance use by comparing proportions of youth at “high” risk using different empirically derived cut points obtained from large-scale samples of youth in Colombia and the United States.

2. Methods

2.1. Study Design

This study is a secondary analysis of two large-scale epidemiologic data sets of youth risk factors for youth substance use, collected from youth in the United States and Colombia, respectively. U.S. data were provided by Bach Harrison L.L.C. (http://www.bach-harrison.com/), a survey research and evaluation firm that specializes in the analysis of data related to youth-based risk and protective factors. Colombian data were provided by the Corporación Nuevos Rumbos (https://www.nuevosrumbos.org/), a research and evaluation organization that specializes in the prevention of substance abuse in youth and adult populations, and the Universidad El Bosque, in Bogotá, Colombia (https://www.unbosque.edu.co/). All data sources relied on a common survey instrument, the Communities That Care Youth Survey (CTC-YS) [17], which was developed at the University of Washington and adapted in Colombia [18] to provide a validated community-based assessment tool for identifying salient risk and protective factors for youth health and behavior problems (e.g., substance use, violent and delinquent behaviors, risky sexual behaviors, and mental health problems) as part of the Communities That Care (CTC) prevention system.
CTC [19] is an evidence-based prevention “operating system”, designed to mobilize a community for effective prevention of youth antisocial behaviors. Experimental and quasi-experimental trials of CTC have demonstrated the efficacy of the system to impact targeted risk and protective factors [4,20] and related youth health and behavior problems (i.e., alcohol and drug use, violent and delinquent behaviors) [21,22]. In the U.S., hundreds of communities in dozens of states have implemented CTC, and many countries abroad (i.e., Australia, Brazil, Canada, Chile, Colombia, Croatia, Germany, India, Mexico, the Netherlands, and Sweden) have adapted the CTC system or the CTC-YS for use [20,23]. For example, in 2012, the Corporación Nuevos Rumbos began to implement an adapted version of CTC (called Comunidades Que se Cuidan) as a potential framework for a national system of youth substance use prevention in Colombia [24].
An important element of the CTC system is the targeting of salient risk and protective factors for intervention via high-fidelity implementation of a set of community-tailored prevention programs and strategies. The original empirically-identified formula for optimal bifurcation in the grade-specific distributions of risk and protective factor measures used in the CTC-YS was developed by Arthur et al. [11] with cut point calculated at Md + (0.15 × MAD), where MAD = mean absolute deviation about the median (Md), a measure of central tendency of a distribution around its median [25]. Under the assumption of a Gaussian distribution for a risk factor, this equation would yield an estimated 44% of youth above the “high” risk cut point. Minor adjustments to this formula have been made to account for the coarseness in the distributions of risk factor measures with a small number of items; however, the original Arthur et al. [11] equation stands as the basis for the calculation of proportions of youth at high risk using the CTC-YS and similar measures.

2.2. Measures

The CTC-YS [17] is a youth self-administered, paper-and-pencil questionnaire designed to be completed in a 50-min classroom period. The survey includes questions on student demographic characteristics (i.e., age, gender, race and ethnicity, family composition, and parental education); lifetime and 30-day measures of alcohol, marijuana, cigarette, and other drug use; heavy episodic drinking (i.e., five or more drinks in a row); past-year delinquency; and risk and protective factors in community. The U.S.-based CTC-YS has identified 28 potential risk factors for potential inclusion in its school-based surveys [26], however, different versions of the CTC-YS used in the U.S. and abroad contain different numbers and sets of risk and protective factors. From the data provided by Bach Harrison L.L.C. and the Corporación Nuevos Rumbos, we identified 18 risk factors across community, school, family, and peer-individual domains that had comparable items and response options. A list of risk factors, items, and country-specific cut points, are included in Annex I (Found in supplementary files, Table S1). A comparison of Spanish-language cultural equivalency between the English U.S.-based survey and the Spanish Colombian-based survey was conducted by the study investigative team who were fluent in both languages. Examinations of CTC-YS scale reliability for both the U.S.-based [17,26] and Colombian-based [18] risk and protective factor measures have demonstrated acceptable levels of internal consistency (i.e., Cronbach alphas > 0.80; [13]) for items in risk factor scales. The validity of U.S.-based CTC-YS measures and their cut points has been studied extensively and has shown good measurement properties across different age, sex, and racial and ethnic groups as well as good concurrent and predictive validity [27,28].

2.3. Participants

Participants in the U.S.-based sample were taken from a database of over 1,300,000 adolescents (Grades 6 to 12) from 11 different states in the U.S. The present study included only a sub-sample from students who were in grades 6, 8, and 10 as these three cohorts represented different developmental periods for prevention and validated cut points have not yet been developed for odd numbered grades (i.e., 7th, 9th, and 11th grades). Depending on the risk factor measure in question, sample sizes used in this study ranged from 138,035 to 373,985 6th-grade adolescents (Md = 319,477); 205,139 to 498,658 8th-grade adolescents (Md = 437,966); and 216,120 to 490,248 10th-grade adolescents (Md = 437,184). U.S.-based data were collected from 2010 to 2012 by state and local agencies. Because only aggregated data were available for this study, no information on students’ gender or age was available; however, no demographic restrictions were imposed on student participation in the surveys.
Participants in the Colombian sample were obtained from a database of over 80,000 adolescents (Grades 6 to 11) from 14 departments and 33 municipalities in Colombia. Following the U.S sub-samples, the present study included only Colombian students at grades 6th, 8th, and 10th to facilitate comparisons. Depending on the risk factor measure in question, sample sizes used in this study ranged from 3811 to 13,280 6th-grade adolescents (Md = 10,644); 1278 to 12,133 8th-grade adolescents (Md = 9569); and 2610 to 9941 10th-grade adolescents (Md = 7829). Colombian-based data were collected in 2012 to 2013 and 2016 to 2018 by the Corporación Nuevos Rumbos. Just over half (52.1%) of the sample was female and the average age of students was M = 14.0 years (SD = 1.76).

2.4. Statistical Analyses

Analyses consisted of calculating three sets of proportions of youth at “high” risk for each of the 18 risk factors: (1) the proportion of Colombian youth at “high” risk using cut points based on the distributions of risk factors observed in the Colombian data (COL1), (2) the proportion of Colombian youth at “high” risk using cut points observed in the U.S. data (COL2), and (3) the proportion of U.S. youth at “high” risk using the U.S.-based cut points (USA2). Each of these sets of proportions was compared with each other, by student grade (see Table 1, Table 2 and Table 3). For each comparison, odds ratios (ORs) and their 95% confidence intervals [29] were used to assess the magnitude of the difference in the pairs of proportions [30]. ORs whose 95% confidence interval did not include 1.00 were considered to be significantly (p < 0.05) different (i.e., an OR of 1.00 indicated that there was no difference in the proportions of youth at “high” risk for the pair in question). ORs were then transformed to Cox d effect sizes [31] to facilitate the interpretation of magnitudes in the differences between proportions.
The procedure for comparing the proportions of youth at “high” risk followed two steps (see Figure 2). First, if there was not a statistically significant difference in the proportions of youth between COL1 and COL2, the pair of cut points were considered to have an equivalent ability to discriminate between the proportions, with indeterminant preference for Colombian-based versus U.S.-based cut points for the Colombian sample of youth. Second, if there was a statistical difference in these proportions of youth between COL1 versus COL2, then ORs obtained in comparing (a) COL1 versus USA2 and (b) COL2 versus USA2 were inspected and the comparison that showed a nonsignificant difference in their respective proportions was preferred in favor of the alternate comparison that showed a significant difference. If (a) COL1 versus USA2 and (b) COL2 versus USA2 both exhibited nonsignificant ORs, then cut points were again determined to be equivalent with indeterminant preference between the Colombian-based versus U.S.-based cut points. If (a) COL1 versus USA2 and (b) COL2 versus USA2 were both significant, the smaller difference between the proportions (as indicated by the Cox d effect size for the difference in pairs) was considered preferential.
For example, as shown in Table 1, for the risk factor Laws and Norms Favorable to Drug Use, 40.0% of 6th-grade youth in the Colombian sample were identified as being at “high” risk using the Colombian-based cut points (COL1), 86.4% of 6th-grade of youth in the Colombian sample were identified as being at “high” risk using the U.S.-based cut points (COL2), and 48.0% of 6th-grade youth in the U.S. sample were identified as being at “high” risk using the U.S.-based cut points (USA2). Comparing COL1 versus COL2 (i.e., 0.400 and 0.864, respectively) indicated that the proportions were significantly different (i.e., 95% confidence interval = 0.103–0.107) with 6th-grade youth in the Colombian sample being almost 90% (OR = 0.105) less likely to be identified as “high” risk using the U.S.-based cut points compared to the Colombian-based cut points. In this case, both (a) COL1 versus USA2 and (b) COL2 versus USA2 were statistically significant (i.e., 95% confidence intervals = 0.718–0.726 and 6.83–6.93, respectively) with dCox = 0.197 and 1.17, respectively. Thus, given our decision rules, the smaller dCox effect size for COL1 versus USA2 suggested that the Colombian-based cut points showed preferential ability to identify “high” risk youth compared to the U.S.-based cut points for this grade and risk factor.

3. Results

The proportions of students at “high” risk for each of the 18 examined risk factors, their ORs, 95% confidence intervals, and dCox effect sizes are presented in Table 1, Table 2 and Table 3, for 6th-, 8th-, and 10th-grade students, respectively. In general, the various proportions of youth considered to be at “high” risk for each risk factor showed appreciable departure from the “optimal” proportion of 0.440 that would be expected under a normal distribution for each measure, owing to the coarseness of item response options used in these measures. Significant differences in the comparisons of proportions for COL1 versus COL2 were found for 7 out of the 18 (38.9%) risk factor comparisons for 6th graders, 11 out of 18 (61.1%) comparisons for 8th graders, and 12 out of 18 (66.6%) comparisons for 10th graders. Results indicated that across the 18 examined risk factors, the Colombian-based cut points were preferable to the U.S.-based cut points in all of the 7 significant 6th-grade comparisons, all but 1 (i.e., Parental Attitudes regarding Antisocial Behavior) of the 11 significant 8th-grade comparisons, and all of the 12 significant 10th-grade comparisons.
Among the 7 significant differences in the 6th-grade comparisons of proportions of youth at “high” risk, 6 indicated that using the U.S.-based cut points would overestimate the predicted percentages of youth at “high” risk by an average difference across all risk factors of 27.3% (corresponding to an average dCox = 0.746). One risk factor, Low Commitment to School, showed an underestimation of 17.0% (dCox =0.451) by U.S.-based cut points in the observed proportion of Colombian youth at “high” risk compared to Colombian-based cut points.
Among the 11 8th-grade comparisons that demonstrated a significant difference in proportions of youth at “high” risk, 10 risk factors indicated a preference for the Colombian-based cut points, with 8 of them indicating that the U.S.-based cut points would overestimate the predicted percentages of Colombian youth at “high” risk (average difference = 17.5%; average dCox = 0.446), and 2 of them (Low Commitment to School and Family History of Antisocial Behavior) indicating that the U.S.-based cut points would underestimate the predicted percentages of Colombian youth at “high” risk (average difference = 22.1%; average dCox = 0.610). One risk factor, Parental Attitudes Favorable Towards Antisocial Behavior, indicated a preference for the use of the U.S.-based cut points, with an underestimation in the percentage of Colombian youth at “high” risk of 17.8% (dCox = 0.438) by the Colombian-based cut points.
Among the 12 significant 10th-grade comparisons, all favored the Colombian-based cut points, with 6 risk factors demonstrating a overestimation in the percentages of Colombian youth at “high” risk (average difference = 18.3%; average dCox = 0.453) and 6 risk factors demonstrating an underestimation in the percentages of Colombian youth at “high” risk (average difference = 17.8%; average dCox = 0.525) when using the U.S.-based cut points.

4. Discussion

The widespread use of epidemiologic risk and protective factor data for adolescent substance use and the selection of evidence-based prevention programs. and strategies that are based on these data, represent a significant advance in the ability to effectively discriminate “high” risk subgroups and combat this health problem. However, as is unfortunately too common in public health, the valid measurement of these constructs lags behind their operationalization. Apparently, such is the case with the international use of cut points in the distributions of risk factors for adolescent substance use, at least for the use of cut points for Colombian youth calculated from distributions of risk factors from U.S. adolescents. Results of this study point to the need for collecting large-scale national data on youth risk factors for adolescent substance use and developing country-specific cut points in the distributions of these measures. While the use of cut points in the distributions of risk factors to identify “high” risk as targets of preventive intervention is a valuable tool in local prevention programming, the placement of these cut points has important implications for research.
Results indicated that over half (55.5%) of all comparisons between the proportions of Colombian youth at “high” risk using Colombian-based cut points versus the proportion of Colombian youth at “high” risk using U.S.-based cut points showed a significant difference, and that the numbers of differences in these proportions increased as grade level increased (38.9% for 6th graders, 61.1% for 8th graders, and 66.6% for 10th graders). The primary implication of the findings from this study suggests that international comparison of youth at “high” risk may be invalidated by the use of parameters (e.g., the median of the distribution and the mean absolute deviation from median) from distributions that are not based on the population in question. Findings from this study showed a general trend for U.S.-based cut points to overestimate the proportions of Colombian youth at “high” risk in 6th and 8th grades. This was due to right-ward shifts in the distributions of most of the risk factors for these Colombian adolescents compared to the 6th- and 8th-grade samples of U.S. adolescents used in this study, which represents higher average levels of risk for these youth in Colombia than in the U.S. Consequently, a 6th- or 8th-grade youth considered to be “high” risk by U.S. standards would probably be considered to be “normal” risk by Colombian standards. For the 10th-grade samples of adolescents in this study the picture was less clear, with half of the comparisons demonstrating an overestimation in the percentages of Colombian youth at “high” risk and half of the comparisons demonstrating an underestimation in the percentages of Colombian youth at “high” risk. Interestingly, risk factors that showed an overestimation in the proportions of Colombian youth at “high” risk using U.S.-based cut points tended to be related specifically to drug use (e.g., Laws and Norms Favorable to Drug Use, Parental Attitudes for Drug Use, and Friends Use of Drugs), and risk factors that showed an underestimation in the proportions of Colombian youth at “high” risk using U.S.-based cut points tended to be non-drug specific (e.g., Family Conflict, Poor Family Management, and Rewards for Antisocial Involvement).
It is important to note that decisions made by local community-specific health coalitions and agencies in Colombia using risk profiles that were U.S.-based cut points are not necessarily invalidated by the findings from this study. The relative comparison of levels of “high” risk across an array of risk factor measures within a locality can still be useful even when the cut points that define “high” risk are consistently overestimated or underestimated. In other words, the use of “non-native” cut points may not change the selection of salient risk factors in a community as long as all the risk factors in the local profile use the same formula for determining “high” risk. The analog is as if one uses the wrong time zone in setting up a series of meetings but uses a common clock to measure the durations of the various meetings. Thus, the internationalization of the Communities That Care prevention system remains a viable approach to the effective mobilization of communities for the implementation of evidence-based prevention; however, care should be exercised to ensure that measures have the requisite cross-national equivalency.
The present study has several limitations. As the focus of this study was to compare currently existing cut points used in the U.S. and Colombia, and not to derive new cut points for this comparison, which would require individual-level data. Moreover, our secondary data sources did not provide an objective measurement for testing if cut points validly identified youth at risk (e.g., assessing sensitivity and specificity of the cut points). Another limitation of this study was not having individual-level student data in the U.S. sample (e.g., year of data collection, gender, age, and race and ethnicity) that would have allowed for a deeper psychometric comparison of U.S. and Colombian data. The derivation of county-specific cut points should be based on large samples that are as representative to the national population as possible, which also are costly and exceedingly rare. Moreover, having longitudinal data at the individual level would allow for a rigorous test of predictive validity of risk factor cut points. The large-scale samples used in this study allowed for estimation of population distributions in the U.S. and Colombia that were as representative and generalizable as possible, and consequently served to strengthen the conclusions produced in this study. Another limitation exists in the focus of this study on the relative proportions of youth identified as being at high risk given application of the Arthur et al. [11] formula for determining optimal cut points in the distributions of the CTC-YS risk factors. Our study allowed the median (Md) and mean absolute deviation from the median (MAD) to vary as a function of the Colombian-based data used in this study; however, we did not examine whether another formula (i.e., values different from Md + 0.15 × MAD) would have resulted in better sensitivity and specificity vis-a-vis a “gold standard” criterion for identifying youth at “high” risk. Such an inquiry was beyond the scope of the present study. Additionally, the study was limited by the use of youth self-report data collected via school-based administration, which may be subject to reporting and selection biases, as students may be influenced by social desirability and school populations may be impacted by high rates of school dropout. These threats to the validity of our study are tempered by the rigid data collection protocols used in both U.S. and Colombian schools with monitoring of survey completion rates and standard human subject protections, and procedures that underscore the confidentiality and anonymity of collected data; however, as school dropout in upper grades is a widespread problem in Colombia, more research is needed to ensure that school-based data are representative of their respective communities. Finally, deeper psychometric analyses that elucidate the differences found in this study are needed to fully understand the reasons behind our observed differences, for instance the under- and over-estimation of proportions of youth at “high risk” by type of risk factor (e.g., drug specific vs. not-drug specific).

5. Conclusions

This study is the first to statistically compare empirically derived cut points in the distributions of risk factors between large-scale samples of adolescents from multiple grades using comparable measures. Findings from this study demonstrate the need for collecting risk factor data from large-scale samples of youth in countries that rely on community assessment of prevention needs and developing country-specific cut points in the distributions of these risk factors. Although U.S.-based cut points provide a good starting point for the selection of salient risk factors for prevention programming, differences in population distributions of risk factor measures between countries can obscure the estimation of proportions of youth at “high” risk in countries that use cut points derived from non-native sources of data.

Supplementary Materials

The following are available online at https://www.mdpi.com/1660-4601/18/2/470/s1, Table S1: Risk Factors Items and Cut Points Construct Dictionary.

Author Contributions

Principal Investigator of this study and primary contributor to the conceptualization and production of the manuscript, E.C.B. Co-Investigator of this study and second contributor to the conceptualization, data analysis, and production of the manuscript, P.M.-Z. Contributed to the collection, analysis, and management of the Colombian-based data, and production of this manuscript, F.C.-M. Contributed to the management and analysis of the Colombian-based data, M.F.R.-R. Contributed to the conceptualization of this study, and the analysis and management of the U.S.-based data, J.S.B. Director of Prevention for the Corporación Nuevos Rumbos, Co-Investigator of this study, supervised the collection of the Colombian-based data, and contributed to the production of this manuscript, J.M.-T. Principal Investigator of the Colombian site for the study, Director of the Corporación Nuevos Rumbos, and contributed to the production of this manuscript, A.P.-G. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by the National Institute on Drug Abuse (NIDA; #DA031175), the Colombian Ministry of Health and Social Protection and the Colombian Institute on Family Welfare, and the Universidad El Bosque (PCI 2015-8175). The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies.

Institutional Review Board Statement

The Corporación Nuevos Rumbos Ethics Committee approved the Colombian data used in this study. U.S. data were provided in aggregated form from Bach Harrison L.L.C. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed Consent Statement

Signed informed consent was obtained from all individual participants when required by state law.

Data Availability Statement

The aggregated data that support the findings of this study are available from the corresponding author (E.C.B), upon reasonable request to the firm of Bach Harrison L.L.C., Survey Research and Evaluation Services (http://www.bach-harrison.com/), the Corporación Nuevos Rumbos (https://www.nuevosrumbos.org/), and the Universidad El Bosque, in Bogotá, Colombia (https://www.unbosque.edu.co/).

Acknowledgments

We gratefully acknowledge the cooperation of the participating communities and schools in the United States and Colombia; and the firm of Bach Harrison L.L.C., Survey Research and Evaluation Services, for their valuable assistance to this study. This paper is dedicated to the memory of Michael W. Arthur.

Conflicts of Interest

Juliana Mejía-Trujillo and Augusto Pérez-Gómez are Directors of the Corporación Nuevos Rumbos, which holds the license for Comuniadades Que se Cuidan in Colombia. John S. Briney is a member of the University of Washington’s Center for Communities That Care, which holds the license for “Communities That Care Plus” in the United States.

References

  1. Institute for Health Metrics and Evaluation. GBD Compare Data Visualization. 2020. Available online: http://vizhub.healthdata.org/gbd-compare (accessed on 17 December 2020).
  2. Botvin, G.J. Advancing prevention science and practice: Challenges, critical issues, and future directions. Prev. Sci. 2004, 5, 69–72. [Google Scholar] [CrossRef] [PubMed]
  3. Catalano, R.F.; Haggerty, K.P.; Hawkins, J.D.; Elgin, J. Prevention of substance use and substance use disorders: The role of risk and protective factors. In Clinical Manual of Adolescent Substance Abuse Treatment; Kaminer, Y., Winters, K.C., Eds.; American Psychiatric Publishing: Washington, DC, USA, 2011; pp. 25–63. [Google Scholar]
  4. Hawkins, J.D.; Brown, E.C.; Oesterle, S.; Arthur, M.W.; Abbott, R.D.; Catalano, R.F. Early effects of Communities That Care on targeted risks and initiation of delinquent behavior and substance use. J. Adolesc. Health 2008, 43, 15–22. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Spoth, R.; Guyll, M.; Lillehoj, C.J.; Redmond, C.; Greenberg, M. Prosper Study of Evidence-Based Intervention Implementation Quality by Community-University Partnerships. J. Community Psychol. 2007, 35, 981–999. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. National Institute on Drug Abuse. Preventing Drug Use Among Children and Adolescents: A Research-Based Guide for Parents, Educators, and Community Leaders, 2nd ed.; Department of Health and Human Services: Bethesda, MD, USA, 2003.
  7. Mihalic, S.F.; Elliott, D.S. Evidence-based programs registry: Blueprints for Healthy Youth Development. Eval. Program Plan. 2015, 48, 124–131. [Google Scholar] [CrossRef] [Green Version]
  8. Mitchell, R.E.; Florin, P.; Stevenson, J.F. Supporting community-based prevention and health promotion initiatives: Developing effective technical assistance systems. Health Educ. Behav. 2002, 29, 620–639. [Google Scholar] [CrossRef]
  9. Stith, S.; Pruitt, I.; Dees, J.; Fronce, M.; Green, N.; Som, A.; Linkh, D. Implementing community-based prevention programming: A review of the literature. J. Prim. Prev. 2006, 27, 599–617. [Google Scholar] [CrossRef]
  10. Feinberg, M.E. Community epidemiology of risk and adolescent substance use: Practical questions for enhancing prevention. Am. J. Public Health 2012, 102, 457–468. [Google Scholar] [CrossRef]
  11. Arthur, M.W.; Briney, J.S.; Hawkins, J.D.; Abbott, R.D.; Brooke-Weiss, B.L.; Catalano, R.F. Measuring risk and protection in communities using the Communities That Care Youth Survey. Eval. Program Plan. 2007, 30, 197–211. [Google Scholar] [CrossRef]
  12. Eisenberg, N.; Brown, E.C.; Pérez-Gómez, A.; Mejía-Trujillo, J.; Paredes-Aguilar, M.; Cardozo-Macias, F.; de Maipo, F.S.C.; Guttmannova, K. Community utilization of risk and protective factor data for prevention planning in Chile and Colombia. Health Promot. Int. 2020, daaa075. [Google Scholar] [CrossRef] [PubMed]
  13. Montero, P.Z.; Reyes, M.R.; Cardozo, F.M.; Brown, E.C.; Pérez, A.G.; Mejía, J.T.; Toro, J.B.; Paredes, M.A. Adolescent substance use and its association with risk and protective factors. An exploratory analysis of the large-scale school survey of Comunidades Que se Cuidan, Colombia. Adicciones 2020, 32, 105–115. [Google Scholar]
  14. Beyers, J.M.; Toumbourou, J.W.; Catalano, R.F.; Arthur, M.W.; Hawkins, J.D. A cross-national comparison of risk and protective factors for adolescent substance use: The United States and Australia. J. Adolesc. Health 2004, 35, 3–16. [Google Scholar]
  15. Oesterle, S.; Hawkins, J.D.; Steketee, M.; Jonkman, H.; Brown, E.C.; Moll, M.; Haggerty, K.P. A cross-national comparison of risk and protective factors for adolescent drug use and delinquency in the United States and the Netherlands. J. Drug Issues 2012, 42, 337–357. [Google Scholar] [CrossRef] [Green Version]
  16. Brown, E.C.; Eisenberg, N.; Hawkins, J.D.; Oesterle, S.; Toumbourou, J.W.; Catalano, R.F.; Herrenkohl, T.I.; Pérez Gómez, A.; Trujillo, J.M.; Briney, J.S.; et al. Risk factors for youth alcohol and drug use: Cross-national measurement equivalency among Australia, Colombia, Germany, and the United States. Manuscript under review; available from the first author.
  17. Arthur, M.W.; Hawkins, J.D.; Pollard, J.A.; Catalano, R.F.; Baglioni, A.J., Jr. Measuring risk and protective factors for substance use, delinquency, and other adolescent problem behaviors: The Communities That Care Youth Survey. Eval. Rev. 2002, 26, 575–601. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Mejía-Trujillo, J.; Pérez-Gómez, A.; Reyes-Rodríguez, M.F. Implementación y adaptación en Colombia del sistema preventivo Communities that Care. Addicciones 2015, 27, 253–264. [Google Scholar] [CrossRef] [PubMed]
  19. Hawkins, J.D.; Catalano, R.F., Jr. Communities That Care: Action for Drug Abuse Prevention, 1st ed.; Jossey-Bass: San Francisco, CA, USA, 1992; Volume xxii, p. 247. [Google Scholar]
  20. Toumbourou, J.W. Implementing Communities That Care in Australia: A community mobilisation approach to crime prevention. Trends Issues Crime Crim. Justice 1999, 122, 1–6. [Google Scholar]
  21. Hawkins, J.D.; Oesterle, S.; Brown, E.C.; Abbott, R.D.; Catalano, R.F. Youth problem behaviors 8 years after implementing the Communities That Care prevention system. A community-randomized trial. JAMA Pediatr. 2014, 168, 122–129. [Google Scholar] [CrossRef]
  22. Feinberg, M.E.; Jones, D.; Greenberg, M.T.; Osgood, D.W.; Bontempo, D. Effects of the Communities That Care model in Pennsylvania on change in adolescent risk and problem behaviors. Prev. Sci. 2010, 11, 163–171. [Google Scholar] [CrossRef] [Green Version]
  23. Jonkman, H.B.; Haggerty, K.P.; Steketee, M.; Fagan, A.A.; Hanson, K.; Hawkins, J.D. Communities That Care, core elements and context: Research of implementation in two countries. Soc. Dev. Issues. 2008, 30, 42–57. [Google Scholar]
  24. Pérez-Gómez, A.; Mejía-Trujillo, J.; Brown, E.C.; Eisenberg, N. Adaptation and implementation of a science-based prevention system in Colombia: Challenges and achievements. J. Community Psychol. 2016, 44, 538–545. [Google Scholar] [CrossRef] [Green Version]
  25. Hays, W.L. Statistics, 4th ed.; Holt, Rinehart & Winston: Fort Worth, TX, USA, 1988. [Google Scholar]
  26. Brown, E.C.; Graham, J.W.; Hawkins, J.D.; Arthur, M.W.; Baldwin, M.M.; Oesterle, S.; Briney, J.S.; Catalano, R.F.; Abbott, R.D. Design and analysis of the Community Youth Development Study longitudinal cohort sample. Eval. Rev. 2009, 33, 311–334. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. Arthur, M.W.; Brown, E.C.; Briney, J.S.; Hawkins, J.D.; Abbott, R.D.; Catalano, R.F.; Becker, L.; Langer, M.; Mueller, M.T. Examination of substance use, risk factors, and protective factors on student academic test score performance. J. Sch. Health. 2015, 85, 497–507. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Briney, J.S.; Brown, E.C.; Hawkins, J.D.; Arthur, M.W. Predictive validity of established cut points for risk and protective factor scales from the Communities That Care Youth Survey. J. Prim. Prev. 2012, 33, 249–258. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  29. Fleiss, J.L.; Levin, B.; Paik, M.C. Statistical Methods for Rates and Proportions, 3rd ed.; John Wiley & Sons: New York, NY, USA, 2013. [Google Scholar]
  30. McHugh, M.L. The odds ratio: Calculation, usage, and interpretation. Biochem. Med. 2009, 19, 120–126. [Google Scholar] [CrossRef]
  31. Sánchez-Meca, J.; Marín-Martínez, F.; Chacón-Moscoso, S. Effect-size indices for dichotomized outcomes in meta-analysis. Psychol. Methods 2003, 8, 448–467. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Risk Factor Profile for a School.
Figure 1. Risk Factor Profile for a School.
Ijerph 18 00470 g001
Figure 2. Steps for comparing the proportions of youth at “high” risk.
Figure 2. Steps for comparing the proportions of youth at “high” risk.
Ijerph 18 00470 g002
Table 1. Comparison of proportions of sixth-grade students at “high” risk, odds ratios, 95% confidence intervals, and effect sizes using Colombian and United States cut-points.
Table 1. Comparison of proportions of sixth-grade students at “high” risk, odds ratios, 95% confidence intervals, and effect sizes using Colombian and United States cut-points.
Risk FactorProportion at “High” RiskCOL1 vs. COL2COL1 vs. USA2COL2 vs. USA2
COL1COL2USA2OR95% CIdCoxOR95% CIdCoxOR95% CIdCox
Laws and Norms Favorable to Drug Use0.4000.8640.4800.105[0.103, 0.107]1.370.722[0.718, 0.726]0.1976.88[6.83, 6.93]1.17
Perceived Availability of Drugs0.4160.6400.4530.401[0.391, 0.411]0.5540.860[0.855, 0.865]0.0912.15[2.13, 2.16]0.463
Perceived Availability of Handguns0.2140.2140.2631.00[0.933, 0.1.07]0.0000.763[0.755, 0.771]0.1640.763[0.755, 0.771]0.164
Community Disorganization0.4760.4760.4991.00[0.952, 1.05]0.0000.912[0.904, 0.920]0.0560.912[0.904, 0.920]0.056
Low Commitment to School0.4500.2800.2112.10[0.203, 2.17]0.4513.06[3.03, 3.08]0.6781.45[1.44, 1.47]0.227
Parental Attitudes for Drug Use0.3110.3110.1141.00[0.969, 1.03]0.0003.51[3.47, 3.55]0.7613.51[3.47, 3.55]0.761
Parental Attitudes for ASB0.3840.3840.3771.00[0.969, 1.03]0.0001.03[1.02, 1.04]0.0181.03[1.02, 1.04]0.018
Poor Family Management0.4980.6580.4830.516[0.503, 0.528]0.4011.06[1.06, 1.07]0.0362.06[2.05, 2.07]0.438
Family History of ASB0.3140.8540.4800.078[0.075, 0.081]1.540.496[.492, 0.500]0.4256.34[6.27, 6.40]1.12
Family Conflict0.3250.3250.3891.00[0.957, 1.05]0.0000.756[0.751, 0.726]0.1690.756[0.751, 0.762]0.169
Rewards for Antisocial Involvement0.2530.2530.2451.00[0.938, 0.107]0.0001.04[1.03, 1.05]0.0261.04[1.03, 1.05]0.026
Positive Attitudes for ASB0.3950.4000.4000.979[0.950, 1.01]0.0130.979[0.974, 0.985]0.0131.00[0.994, 1.01]0.000
Positive Attitudes for Drug Use0.3860.5160.1890.590[0.575, 0.604]0.3202.70[2.68, 2.72]0.6014.57[4.54, 4.61]0.922
Perceived Risk of Drug Use0.4300.5500.4450.617[0.601, 0.634]0.2920.941[.936, 0.946]0.0371.52[1.52, 1.53]0.255
Friends’ Use of Drugs0.4690.4690.1971.00[0.973, 1.03]0.0003.60[3.57, 3.63]0.7763.60[3.57, 3.63]0.776
Gang Involvement0.2150.2150.1281.00[0.935, 1.07]0.0001.87[1.83, 1.90]0.3781.87[1.83, 1.90]0.378
Early Initiation of Drug Use0.1940.1940.2341.00[0.930, 1.08]0.0000.788[0.780, 0.796]0.144.788[0.780, 0.796]0.144
Interaction with Antisocial Peers0.4240.4240.3361.00[0.952, 1.05]0.0001.45[1.44, 1.47]0.2271.45[1.44, 1.47]0.227
Note. COL1 = The proportion of Colombian youth at “high” risk using cut points based on the distributions of risk factors observed in the Colombian data. COL2 = The proportion of Colombian youth at “high” risk using cut points observed in the U.S. data. USA2 = The proportion of U.S. youth at “high” risk using the U.S.-based cut points. ASB = antisocial behavior. OR= odds ratio. CI = confidence interval. dCox = Cox effect size.
Table 2. Comparison of proportions of eight-grade students at “high” risk, odds ratios, 95% confidence intervals, and effect sizes using Colombian and United States cut points.
Table 2. Comparison of proportions of eight-grade students at “high” risk, odds ratios, 95% confidence intervals, and effect sizes using Colombian and United States cut points.
Risk Factor Proportion at “High” RiskCOL1 vs. COL2COL1 vs. USA2COL2 vs. USA2
COL1COL2USA2OR95% CIdCoxOR95% CIdCoxOR95% CIdCox
Laws and Norms Favorable to Drug Use0.4670.6980.4000.379[0.371, 0.388] 0.5881.31[1.31, 1.32] 0.0723.47[3.45, 3.49] 0.327
Perceived Availability of Drugs0.4640.564 0.4540.669[0.651, 0.688]0.2431.04[1.04, 105]0.0111.56[1.55, 1.56]0.116
Perceived Availability of Handguns0.3570.3570.3671.00[0.946, 0.1.06]0.0000.958[0.801, 1.14]0.0110.958[0.801, 1.14]0.011
Community Disorganization0.3940.5300.4360.577[0.550, 0.605]0.3340.841[0.834, 0.848]0.0461.46[1.45, 1.47]0.099
Low Commitment to School0.4510.2790.4622.12[0.205, 2.20]0.4560.957[0.952, 961]0.0120.451[0.448, 0.453]0.210
Parental Attitudes for Drug Use0.4920.4920.2371.00[0.974, 1.03]0.0003.12[3.10, 3.14]0.2993.12[3.10, 3.14]0.299
Parental Attitudes for ASB0.3780.5560.4910.485[0.472, 0.499]0.4380.630[6.27, 6.33]0.1221.30[1.29, 1.30]0.069
Poor Family Management0.4490.5220.4730.746[0.726, 0.767]0.1770.908[0.903, 0.913]0.0251.22[1.21, 1.22]0.052
Family History of ASB0.4730.2030.4563.52[3.26, 3.81]0.7631.07[1.06, 1.08]0.0180.304[0.301, 0.307]0.314
Family Conflict0.4090.4090.4931.00[0.960, 1.04]0.0000.712[0.707, 0.716]0.0900.712[0.707, 0.716]0.090
Rewards for Antisocial Involvement0.4610.4610.4561.00[0.952, 0.105]0.0001.02[1.01, 1.03]0.0051.02[1.01, 1.03]0.005
Positive Attitudes for ASB0.4410.4410.4681.00[0.970, 1.03]0.0000.897[0.892, 0.901]0.0290.897[0.994, 1.01]0.029
Positive Attitudes for Drug Use0.4840.7750.4370.272[0.267, 0.278]0.7881.21[1.20, 1.21]0.0504.44[4.41, 4.46]0.392
Perceived Risk of Drug Use0.3820.5110.3790.592[0.574, 0.609]0.3181.01[1.01, 1.02]0.0031.71[1.70, 1.72]0.142
Friends’ Use of Drugs0.4530.7280.4790.309[0.302, 0.317]0.7110.901[0.896, 0.905]0.0282.91[2.90, 2.93]0.281
Gang Involvement0.2260.2260.1811.00[0.934, 1.07]0.0001.32[1.30, 1.34]0.0731.32[1.30, 1.34]0.073
Early Initiation of Drug Use0.3200.3200.4511.00[0.939, 1.06]0.0000.573[0.566, 0.579]0.1470.573[0.566, 0.579]0.147
Interaction with Antisocial Peers0.3910.5590.4480.507[0.484, 0.530]0.4120.791[0.786, 0.796]0.0621.56[1.55, 1.57]0.117
Note. COL1 = The proportion of Colombian youth at “high” risk using cut points based on the distributions of risk factors observed in the Colombian data. COL2 = The proportion of Colombian youth at “high” risk using cut points observed in the U.S. data. USA2 = The proportion of U.S. youth at “high” risk using the U.S.-based cut points. ASB = antisocial behavior. OR= odds ratio. CI = confidence interval. dCox = Cox effect size.
Table 3. Comparison of proportions of tenth-grade students at “high” risk, odds ratios, 95% confidence intervals, and effect sizes using Colombian and United States cut points.
Table 3. Comparison of proportions of tenth-grade students at “high” risk, odds ratios, 95% confidence intervals, and effect sizes using Colombian and United States cut points.
Risk Factor Proportion at “High” RiskCOL1 vs. COL2COL1 vs. USA2COL2 vs. USA2
COL1COL2USA2OR95% CIdCoxOR95% CIdCoxOR95% CIdCox
Laws and Norms Favorable to Drug Use0.4540.6940.4840.367[0.357, 0.376]0.6080.886[0.882, 0.891]0.0732.42[2.40, 2.43] 0.535
Perceived Availability of Drugs0.4410.4410.4751.00[0.966, 1.04]0.0000.872[0.867, 0.877]0.083 0.872[0.867, 0.877] 0.083
Perceived Availability of Handguns0.4330.4330.4551.00[0.946, 1.06]0.0000.915[0.908, 0.921]0.054 0.915[0.908, 0.921] 0.054
Community Disorganization0.4670.6150.4680.548[0.522, 0.576]0.3640.996[0.987, 1.00]0.0021.82[1.80, 1.83] 0.362
Low Commitment to School0.4070.2270.4872.34[2.24, 2.44]0.5150.723[0.719, 0.727]0.1970.309[0.308, 0.311]0.711
Parental Attitudes for Drug Use0.4200.6340.3960.418[0.407, 0.429]0.5291.10[1.10, 1.11]0.0602.64[2.63, 2.66]0.589
Parental Attitudes for ASB0.3770.3770.3491.00[0.964, 1.04]0.0001.13[1.12, 1.14]0.0731.13[1.12, 1.14]0.073
Poor Family Management0.4700.3890.4931.39[1.34, 1.44]0.2010.912[0.907, 0.917]0.0560.655[0.651, 0.658]0.257
Family History of ASB0.4800.0920.4789.11[8.00, 10.4]1.3391.01[1.00, 1.02]0.0050.111[0.109, 0.112]1.33
Family Conflict0.4460.2740.3992.13[2.02, 2.25]0.4591.21[1.20, 1.22]0.1170.568[0.565, 0.572]0.342
Rewards for Antisocial Involvement0.4480.3330.4211.63[1.53, 1.73]0.2941.12[1.11, 1.12]0.0670.687[0.682, 0.692]0.228
Positive Attitudes for ASB0.4460.3140.4101.76[1.70, 1.83]0.3421.16[1.15, 1.16]0.0890.659[0.655, 0.662]0.253
Positive Attitudes for Drug Use0.4480.6280.4530.481[4.68, 4.93]0.4440.980[0.975, 0.985]0.0122.04[4.54, 4.61]0.432
Perceived Risk of Drug Use0.4410.4410.4751.00[0.966, 1.04]0.0000.872[0.867, 0.877]0.0830.872[0.867, 0.877]0.083
Friends’ Use of Drugs0.4650.6020.4810.575[0.559, 0.559]0.3360.938[0.933, 0.943]0.0391.63[1.62, 1.64]0.297
Gang Involvement0.1950.1950.1831.00[0.922, 1.08]0.0001.08[1.07, 1.10]0.0471.08[1.07, 1.10]0.047
Early Initiation of Drug Use0.3620.3620.6321.00[0.941, 1.06]0.0000.330[0.332, 0.328]0.6710.330[0.332, 0.328]0.671
Interaction with Antisocial Peers0.3700.5460.4550.488[0.464, 0.513]0.4340.703[0.699, 0.708]0.2131.44[1.43, 1.45]0.221
Note. COL1 = The proportion of Colombian youth at “high” risk using cut points based on the distributions of risk factors observed in the Colombian data. COL2 = The proportion of Colombian youth at “high” risk using cut points observed in the U.S. data. USA2 = The proportion of U.S. youth at “high” risk using the U.S.-based cut points. ASB = antisocial behavior. OR= odds ratio. CI = confidence interval. dCox = Cox effect size.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Brown, E.C.; Montero-Zamora, P.; Cardozo-Macías, F.; Reyes-Rodríguez, M.F.; Briney, J.S.; Mejía-Trujillo, J.; Pérez-Gómez, A. A Comparison of Cut Points for Measuring Risk Factors for Adolescent Substance Use and Antisocial Behaviors in the U.S. and Colombia. Int. J. Environ. Res. Public Health 2021, 18, 470. https://doi.org/10.3390/ijerph18020470

AMA Style

Brown EC, Montero-Zamora P, Cardozo-Macías F, Reyes-Rodríguez MF, Briney JS, Mejía-Trujillo J, Pérez-Gómez A. A Comparison of Cut Points for Measuring Risk Factors for Adolescent Substance Use and Antisocial Behaviors in the U.S. and Colombia. International Journal of Environmental Research and Public Health. 2021; 18(2):470. https://doi.org/10.3390/ijerph18020470

Chicago/Turabian Style

Brown, Eric C., Pablo Montero-Zamora, Francisco Cardozo-Macías, María Fernanda Reyes-Rodríguez, John S. Briney, Juliana Mejía-Trujillo, and Augusto Pérez-Gómez. 2021. "A Comparison of Cut Points for Measuring Risk Factors for Adolescent Substance Use and Antisocial Behaviors in the U.S. and Colombia" International Journal of Environmental Research and Public Health 18, no. 2: 470. https://doi.org/10.3390/ijerph18020470

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop