Student Characteristics, Institutional Factors, and Outcomes in Higher Education and Beyond: An Analysis of Standardized Test Scores and Other Factors at the Institutional Level with School Rankings and Salary
Abstract
:1. Introduction
2. Brief Historical Review Focused on the Role of Cognitive Aptitudes
2.1. Using Other Methods: Twin Studies and a Natural Experiment
2.2. Estimates of the Teacher’s Contribution to Student Achievement
2.3. Estimates of Teacher and School Effects Using Methods Focused on Forward Causal Inference
2.4. Estimates of the Students’ Contribution to Student Achievement
2.5. Higher Education
3. This Study
3.1. Measurement of Student Cognitive Characteristics
3.2. Analytic Plan
4. Method
4.1. Data and Analytic Sample
4.2. Variables
4.3. Statistical Methods
5. Results
6. Discussion
6.1. Limitations
6.2. Findings Replicate and Extend Those in K-12 to Higher Education and Also Historically
6.3. Part of Student Outcomes May Be Due to Selection, but Teachers and Institutions Still Matter
6.4. Conclusions and Future Directions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Appendix A
Variable | N | Mean | SD | Min | Max |
---|---|---|---|---|---|
SAT average | 1300 | 1060.170 | 136.849 | 712 | 1555 |
Admission rate | 1996 | 0.672 | 0.205 | 0.042 | 1 |
Total undergraduate enrollment | 6340 | 2450.591 | 5509.977 | 1 | 100,011 |
Faculty salary/month | 4190 | 6495.023 | 2426.603 | 216 | 27,570 |
Retention at 4-year institution | 2109 | 0.729 | 0.158 | 0.062 | 1 |
% Students ever borrowed loan | 5382 | 0.775 | 0.222 | 0.010 | 0.986 |
% Students received Pell grant | 324 | 0496 | 0.222 | 0 | 1 |
Cost per academic year | 3590 | 25,851.390 | 14,439.030 | 4259 | 85,308 |
Diversity index | 707 | 49.335 | 17.033 | 0 | 89.2 |
Location | 6627 | 1.814 | 0.947 | 1 | 4 |
1 = city | 3180 | ||||
2 = Suburb | 2025 | ||||
3 = Town | 897 | ||||
4 = Rural | 525 | ||||
Control | 7068 | 2.139 | 0.838 | 1 | 3 |
1 = public | 2056 | ||||
2 = private non-profit | 1970 | ||||
3 = private for-profit | 3042 | ||||
Salary 6 years after graduation | 5450 | 31,383.650 | 11,626.790 | 11,800 | 151,500 |
Salary10 years after graduation | 5259 | 39,399.140 | 17,901.610 | 14,600 | 250,000 |
THE U.S. ranking | 983 | 470.236 | 250.563 | 1 | 800 |
Lumosity ranking | 443 | 226.072 | 131.155 | 1 | 456 |
Variable | N | Mean | SD | Min | Max |
---|---|---|---|---|---|
SAT average | 1301 | 1056.328 | 121.778 | 660 | 1395 |
Fall 2013 acceptance rate | 1293 | 0.655 | 0.171 | 0.077 | 1 |
Total enrollment | 1200 | 7856.723 | 9908.471 | 161 | 77,329 |
Endowment | 1073 | 2.01 × 108 | 5.64 × 108 | 65,712 | 8.27 × 109 |
Average freshman retention rate | 1290 | 0.751 | 0.108 | 0.420 | 0.970 |
Room and board | 1210 | 9822.236 | 2383.622 | 1032 | 23,386 |
% Graduating students ever borrowed loan | 962 | 0.691 | 0.149 | 0.083 | 0.990 |
Average student debt | 947 | 28,119.670 | 6373.642 | 5500 | 50,275 |
Diversity index | 462 | 49.138 | 16.494 | 9.3 | 89.2 |
Location | 1106 | 2.522 | 1.000 | 1 | 4 |
1 = city | 219 | ||||
2 = Suburb | 287 | ||||
3 = Town | 404 | ||||
4 = Rural | 196 | ||||
Control | 1226 | 1.782 | 0.975 | 1 | 3 |
1 = public | 478 | ||||
2 = private non-profit | 745 | ||||
3 = private for-profit | 3 | ||||
Early-career salary | 851 | 44,622.210 | 5735.902 | 30,800 | 68,600 |
Mid-career salary | 851 | 76,564.390 | 13,439.65 | 41,000 | 131,800 |
THE U.S. ranking | 975 | 484.673 | 239.351 | 15 | 800 |
Lumosity ranking | 422 | 238.768 | 125.014 | 15 | 456 |
Panel A: College Scorecard Data 2016–2017 | ||||
---|---|---|---|---|
Coefficient for SAT Score | Robust Standard Error | N | R2 | |
U.S. News national ranking | −0.384 *** | 0.011 | 187 | 0.798 |
U.S. News Liberal Arts ranking | −0.367 *** | 0.014 | 122 | 0.825 |
Revealed preferences ranking | −0.484 * | 0.203 | 92 | 0.156 |
THE World ranking | −0.981 *** | 0.065 | 149 | 0.450 |
Critical thinking ranking | 0.592 *** | 0.043 | 42 | 0.684 |
Panel B: U.S. News Data 2014 | ||||
Coefficient for SAT Score | Robust Standard Error | N | R2 | |
U.S. News national ranking | −0.453 *** | 0.016 | 180 | 0.739 |
U.S. News Liberal Arts ranking | −0.398 *** | 0.020 | 166 | 0.741 |
Revealed preferences ranking | −0.187 *** | 0.025 | 76 | 0.268 |
THE World ranking | −0.996 *** | 0.126 | 136 | 0.284 |
Critical thinking ranking | 0.610 *** | 0.034 | 68 | 0.716 |
References
- Abdulkadiroglu, Atila, Joshua Angrist, and Parag Pathak. 2014. The elite illusion: Achievement effects at Boston and New York exam schools. Econometrica 82: 137–96. [Google Scholar] [CrossRef] [Green Version]
- Angoff, William H., and Eugene G. Johnson. 1990. The differential impact of curriculum on aptitude test scores. Journal of Educational Measurement 27: 291–305. [Google Scholar] [CrossRef]
- Asbury, Kathryn, and Jonathan Wai. 2020. Viewing education policy through a genetic lens. Journal of School Choice 14: 301–15. [Google Scholar] [CrossRef]
- Atteberry, Allison C., and Andrew J. McEachin. 2020. Not where you start, but how much you grow: An addendum to the Coleman Report. Educational Researcher 49: 678–85. [Google Scholar] [CrossRef]
- Avery, Christopher N., Mark E. Glickman, Caroline M. Hoxby, and Andrew Metrick. 2013. A revealed preference ranking of U.S. colleges and universities. The Quarterly Journal of Economics 128: 425–67. [Google Scholar] [CrossRef]
- Bailey, Drew H., Greg J. Duncan, Flavio Cunha, Barbara R. Foorman, and David S. Yeager. 2020. Persistence and fadeout of educational intervention effects: Mechanisms and potential solutions. Psychological Science in the Public Interest 21: 55–97. [Google Scholar] [CrossRef] [PubMed]
- Baumert, Jurgen, Oliver Ludtke, Ulrich Trautein, and Martin Brunner. 2009. Large-scale student assessment studies measure the results of process knowledge acquisition: Evidence in support of a distinction between intelligence and student achievement. Educational Research Review 4: 165–76. [Google Scholar] [CrossRef]
- Binet, Alfred, and Theodore Simon. 1905. Méthode nouvelle pour le diagnostic du niveau intellectuel des anormaux. L’Année Psychologique 11: 191–244. [Google Scholar] [CrossRef]
- Bowman, Nicholas A. 2013. How much diversity is enough? the curvilinear relationship between college diversity interactions and first-year student outcomes. Research in Higher Education 54: 874–94. [Google Scholar] [CrossRef]
- Brown, Matt I., Jonathan Wai, and Christopher F. Chabris. 2021. Can you ever be too smart for your own good? Comparing linear and nonlinear effects of cognitive ability on life outcomes. Perspectives on Psychological Science 16: 1337–59. [Google Scholar] [CrossRef]
- Butler, Heather A., Christopher Pentoney, and Mabelle P. Bong. 2017. Predicting real-world outcomes: Critical thinking ability is a better predictor of life decisions than intelligence. Thinking Skills & Creativity 25: 38–46. [Google Scholar] [CrossRef]
- Byrne, Brian, William L. Coventry, Richard K. Olson, Sally J. Wadsworth, Stefan Samuelsson, Stephen A. Petrill, Erik G. Willcutt, and Robin Corley. 2010. “Teacher effects” in early literacy development: Evidence from a study of twins. Journal of Educational Psychology 102: 32–42. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Caplan, Bryan D. 2018. The Case against Education: Why the Education System Is a Waste of Time and Money. Princeton: Princeton University Press. [Google Scholar]
- Carroll, John B. 1993. Human Cognitive Abilities: A Survey of Factor Analytic Studies. Cambridge: Cambridge University Press. [Google Scholar]
- Cash, Ceilidh B., Jessa Letargo, Steffen P. Graether, and Shoshanah R. Jacobs. 2017. An analysis of the perceptions and resources of large university classes. CBE Life Sciences Education 16: ar33. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Ceci, Stephen J. 1991. How much does schooling influence general intelligence and its cognitive components? A reassessment of the evidence. Developmental Psychology 27: 703–22. [Google Scholar] [CrossRef]
- Chabris, Christopher F. 2007. Cognitive and neurobiological mechanisms of the law of general intelligence. In Integrating the Mind: Domain General versus Domain Specific Processes in Higher Cognition. Edited by Maxwell J. Roberts. New York: Psychology Press, pp. 449–91. [Google Scholar]
- Chetty, Raj, John N. Friedman, and Jonah E. Rockoff. 2014. Measuring the impacts of teachers II: Teacher value-added and student outcomes in adulthood. American Economic Review 104: 2633–79. [Google Scholar] [CrossRef] [Green Version]
- Chetty, Raj, John N. Friedman, Nathaniel Hilger, Emmanuel Saez, Diane Whitmore Schanzenbach, and Danny Yagan. 2011. How does your kindergarten classroom affect your earnings? Evidence from Project STAR. The Quarterly Journal of Economics 126: 1593–660. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Chingos, Matthew M., Katharine M. Lindquist, and Grover J. "Russ" Whitehurst. 2014. School Superintendents: Vital or Irrelevant? Washington, DC: Brookings Institution. Available online: https://www.brookings.edu/research/school-superintendents-vital-or-irrelevant/ (accessed on 23 March 2022).
- Coleman, James S., Ernest Q. Campbell, Carol J. Hobson, James McPartland, Alexander M. Mood, Frederic D. Weinfeld, and Robert L. York. 1966. Equality of Educational Opportunity. Washington, DC: Government Printing Office. [Google Scholar]
- Coleman, William, and Edward E. Cureton. 1954. Intelligence and achievement: The “Jangle Fallacy” again. Educational and Psychological Measurement 14: 347–51. [Google Scholar] [CrossRef]
- Cunha, Jesse M., and Trey Miller. 2014. Measuring value-added in higher education: Possibilities and limitations in the use of administrative data. Economics of Education Review 42: 64–77. [Google Scholar] [CrossRef] [Green Version]
- Dale, Stacy Berg, and Alan B. Krueger. 2002. Estimating the payoff to attending a more selective college: An application of selection on observables and unobservables. The Quarterly Journal of Economics 117: 1491–527. [Google Scholar] [CrossRef] [Green Version]
- Deary, Ian J., Steve Strand, Pauline Smith, and Cres Fernandes. 2007. Intelligence and educational achievement. Intelligence 35: 13–21. [Google Scholar] [CrossRef]
- Detterman, Douglas K. 2016. Education an intelligence: Pity the poor teacher because student characteristics are more significant than teachers or schools. The Spanish Journal of Psychology 19: e93. [Google Scholar] [CrossRef] [Green Version]
- Dobbie, Will, and Rolad G. Fryer Jr. 2014. The impact of attending a school with high-achieving peers: Evidence from the New York City exam schools. American Economic Journal: Applied Economics 6: 58–75. [Google Scholar] [CrossRef] [Green Version]
- Dynarski, Susan M. 2018. Evidence on New York City and Boston Exam Schools. Brookings Institution. Available online: https://www.brookings.edu/research/evidence-on-new-york-city-and-boston-exam-schools/ (accessed on 23 March 2022).
- Dynarski, Susan, Joshua Hyman, and Diane Whitmore Schanzenbach. 2013. Experimental evidence on the effect of childhood investments on postsecondary attainment and degree completion. Journal of Policy Analysis and Management 32: 692–717. [Google Scholar] [CrossRef] [Green Version]
- Egalite, Anna J. 2016. How family background influences student achievement. EducationNext 16. Available online: https://www.educationnext.org/how-family-background-influences-student-achievement/ (accessed on 23 March 2022).
- Engelhardt, Lena, Frank Goldhammer, Oliver Ludtke, Olaf Koller, Jurgen Baumert, and Claus H. Carstensen. 2021. Separating PIAAC competencies from general cognitive skills: A dimensionality and explanatory analysis. Studies in Educational Evaluation 71: 101069. [Google Scholar] [CrossRef]
- Firkowska, Anna, Antonina Ostrowska, Magdalena Sokolowska, Zena Stein, Mervyn Susser, and Ignacy Wald. 1978. Cognitive development and social policy. Science 200: 1357–62. [Google Scholar] [CrossRef]
- Frey, Meredith C., and Douglas K. Detterman. 2004. Scholastic assessment or g? The relationship between the Scholastic Assessment Test and general cognitive ability. Psychological Science 15: 373–78. [Google Scholar] [CrossRef]
- Gamoran, Adam, and Daniel A. Long. 2007. Equality of educational opportunity: A 40-year retrospective. In International Studies in Educational Inequality, Theory and Policy. Edited by Richard Teese, Stephen Lamb, Marie Duru-Bellat and Sue Helme. Dordrecht: Springer. [Google Scholar] [CrossRef]
- Gelman, Andrew, and Guido Imbens. 2013. Why Ask Why? Forward Causal Inference and Reverse Causal Questions. NBER Working Paper 19614. Available online: https://www.nber.org/papers/w19614 (accessed on 23 March 2022).
- Goldhaber, Dan. 2015. Exploring the potential of value-added performance measures to affect the quality of the teacher workforce. Educational Researcher 44: 87–95. [Google Scholar] [CrossRef]
- Gottfredson, Nisha C., A. T. Panter, Charles E. Daye, Walter A. Allen, Linda F. Wightman, and Meera E. Deo. 2008. Does diversity at undergraduate institutions influence student outcomes? Journal of Diversity in Higher Education 1: 80–94. [Google Scholar] [CrossRef] [Green Version]
- Grasby, Katrina L., Callie W. Little, Brian Byrne, William L. Coventry, Richard K. Olson, Sally Larsen, and Stefan Samuelsson. 2019. Estimating classroom-level influences on literacy and numeracy: A twin study. Journal of Educational Psychology 112: 1154–66. [Google Scholar] [CrossRef]
- Grömping, Ulrike. 2007. Estimators of Relative Importance in Linear Regression Based on Variance Decomposition. The American Statistician 61: 139–47. [Google Scholar] [CrossRef]
- Hair, Nicole L., Jamie L. Hanson, Barbara L. Wolfe, and Seth D. Pollack. 2015. Association of child poverty, brain development, and academic achievement. JAMA Pediatrics 169: 822–29. [Google Scholar] [CrossRef]
- Hart, Sara A., Callie Little, and Elsje van Bergen. 2021. Nurture might nature: Cautionary tales and proposed solutions. Npj Science of Learning 6: 2. [Google Scholar] [CrossRef]
- Heckman, James J. 2000. Policies to foster human capital. Research in Economics 54: 3–56. [Google Scholar] [CrossRef] [Green Version]
- Hunt, Earl. 2009. Good news, bad news, and a fallacy: A review of outliers: The story of success. Intelligence 37: 323–24. [Google Scholar] [CrossRef]
- Huntington-Klein, Nick. 2020. Human capital versus signaling is empirically unresolvable. Empirical Economics 60: 2499–531. [Google Scholar] [CrossRef]
- Jencks, Christopher, Marshall Smith, Henry Acland, Mary Jo Bane, David Cohen, Herbert Gintis, Barbara Heyns, and Stephan Michelson. 1972. Inequality: A Reassessment of the Effect of Family and Schooling in America. New York: Basic Books. [Google Scholar]
- Johnson, Wendy, Thomas J. Bouchard Jr., Robert F. Krueger, Matt McGue, and Irving I. Gottesman. 2004. Just one g: Consistent results from three test batteries. Intelligence 32: 95–107. [Google Scholar] [CrossRef]
- Judge, Timothy A., Ryan L. Klinger, and Lauen S. Simon. 2010. Time is on my side: Time, general mental ability, human capital, and extrinsic career success. Journal of Applied Psychology 95: 92–107. [Google Scholar] [CrossRef] [Green Version]
- Kaufman, Scott Barry, Matthew R. Reynolds, Xin Liu, Alan S. Kaufman, and Kevin S. McGrew. 2012. Are cognitive g and academic achievement g one and the same g? An exploration on the Woodcock-Johnson and Kaufman tests. Intelligence 40: 123–38. [Google Scholar] [CrossRef]
- Kelley, Truman L. 1927. Interpretation of Educational Measurements. Yonkers: World Book Company. [Google Scholar]
- Koenig, Katherine A., Meredith C. Frey, and Douglas K. Detterman. 2008. ACT and general cognitive ability. Intelligence 36: 153–60. [Google Scholar] [CrossRef]
- Kulkarni, Siddharth, and Jonathan Rothwell. 2015. Beyond College Rankings: A Value-Added Approach to Assessing Two- and Four-Year Schools. Washington, DC: Brookings Institution, Available online: https://www.brookings.edu/research/beyond-college-rankings-a-value-added-approach-to-assessing-two-and-four-year-schools/ (accessed on 23 March 2022).
- Kuncel, Nathan R., Sarah A. Hezlett, and Deniz S. Ones. 2004. Academic performance, career potential, creativity, and job performance. Can one construct predict them all? Journal of Personality and Social Psychology 86: 148–61. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Lakin, Joni M., and Jonathan Wai. 2020. Spatially gifted, academically inconvenienced: Spatially talented students experience less academic engagement and more behavioural issues than other talented students. British Journal of Educational Psychology 90: 1015–38. [Google Scholar] [CrossRef] [PubMed]
- Lakin, Joni M., and Jonathan Wai. 2022. Developing student aptitudes as an important goal of learning. Gifted Child Quarterly 66: 95–97. [Google Scholar] [CrossRef]
- Li, Hongbin, Lingsheng Meng, Xinzheng Shi, and Binzhen Wu. 2012. Does attending elite colleges pay in China? Journal of Comparative Economics 40: 78–88. [Google Scholar] [CrossRef]
- Light, Audrey. 2001. In-school work experience and the returns to schooling. Journal of Labor Economics 19: 65–93. [Google Scholar] [CrossRef]
- Lohman, David F. 1993. Teaching and testing to develop fluid abilities. Educational Researcher 22: 12–23. [Google Scholar] [CrossRef]
- Lortie-Forgues, Hugues, and Matthew Inglis. 2019. Rigorous large-scale RCTs are often uninformative: Should we be concerned? Educational Researcher 48: 158–66. [Google Scholar] [CrossRef] [Green Version]
- Lubinski, David, and Camilla P. Benbow. 2020. Intellectual precocity: What have we learned since Terman? Gifted Child Quarterly 65: 3–28. [Google Scholar] [CrossRef]
- Lubinski, David. 2020. Understanding educational, occupational, and creative outcomes requires assessing intraindividual differences in abilities and interests. Proceedings of the National Academy of Sciences 117: 16720–22. [Google Scholar] [CrossRef]
- Luchman, Joseph N. 2015. Determining subgroup difference importance with complex survey designs: An application of weighted dominance analysis. Survey Practice 8: 1–10. [Google Scholar] [CrossRef] [Green Version]
- Luchman, Joseph N. 2021. Determining relative importance in Stata using dominance analysis: Domin and domme. The Stata Journal 21: 510–38. [Google Scholar] [CrossRef]
- Lykken, David T. 1968. Statistical significance in psychological research. Psychological Bulletin 70: 151–59. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Maranto, Robert, and Jonathan Wai. 2020. Why intelligence is missing from American education policy and practice, and what can be done about it. Journal of Intelligence 8: 2. [Google Scholar] [CrossRef] [Green Version]
- Martins, Pedro S., and Ian Walker. 2006. Student Achievement and University Classes: Effects of Attendance, Size, Peers, and Teachers. IZA Discussion Paper No. 2490. Available online: https://ssrn.com/abstract=955298 (accessed on 23 March 2022).
- Montenegro, Maximiliano, Paula Clasing, Nick Kelly, Carlos Gonzalez, Magdalena Jara, Rosa Alarcón, Augusto Sandoval, and Elvira Saurina. 2016. Library resources and students’ learning outcomes: Do all the resources have the same impact on learning? The Journal of Academic Librarianship 42: 551–56. [Google Scholar] [CrossRef] [Green Version]
- Neill, Christine. 2015. Rising student employment: The role of tuition fees. Education Economics 23: 101–21. [Google Scholar] [CrossRef]
- Neyt, Brecht, Eddy Omey, Dieter Verhaest, and Stijn Baert. 2018. Does student work really affect educational outcomes? A review of the literature. Journal of Economic Surveys 33: 896–921. [Google Scholar] [CrossRef] [Green Version]
- Piantadosi, Steven, David P. Byar, and Sylvan B. Green. 1988. The ecological fallacy. American Journal of Epidemiology 127: 893–904. [Google Scholar] [CrossRef]
- Pike, Gary R., George D. Kuh, Alexander C. McCormick, Corinna A. Ethington, and John C. Smart. 2011. If and when money matters: The relationships among educational expenditures, student engagement and students’ learning outcomes. Research in Higher Education 52: 81–106. [Google Scholar] [CrossRef]
- Quiroga, Maria Angeles, Alice Diaz, Francisco J. Roman, Jesus Privado, and Roberto Colom. 2019. Intelligence and video games: Beyond “brain games”. Intelligence 75: 85–94. [Google Scholar] [CrossRef]
- Quiroga, Maria Angeles, Sergio Escorial, Francisco J. Roman, Daniel Morillo, Andrea Jarabo, Jesus Privado, Miguel Hernandez, Borja Gallego, and Roberto Colom. 2015. Can we reliably measure the general factor of intelligence (g) through commercial video games? Yes, we can! Intelligence 53: 1–7. [Google Scholar] [CrossRef]
- Ree, Malcolm James, and James A. Earles. 1991. The stability of g across different methods of estimation. Intelligence 15: 271–78. [Google Scholar] [CrossRef]
- Revelle, William, Joshua Wilt, and David M. Condon. 2011. Individual differences and differential psychology: A brief history and prospect. In The Wiley-Blackwell Handbook of Individual Differences. Edited by Tomas Chamorro-Premuzic, Sophie von Stumm and Adrian Furnham. Hoboken: Wiley-Blackwell, pp. 3–38. [Google Scholar]
- Rindermann, Heiner. 2007. The g-factor of international cognitive ability comparisons: The homogeneity of results in PISA, TIMSS, PIRLS, and IQ-tests across nations. European Journal of Personality 21: 667–706. [Google Scholar] [CrossRef]
- Ritchie, Stuart J., and Elliot M. Tucker-Drob. 2018. How much does education improve intelligence? A meta-analysis. Psychological Science 29: 1358–69. [Google Scholar] [CrossRef]
- Roksa, Josipa, Cindy Ann Kilgo, Teniell L. Trolian, Ernest T. Pascarella, Charles Blaich, and Kathleen S. Wise. 2017. Engaging with diversity: How positive and negative diversity interactions influence students’ cognitive outcomes. The Journal of Higher Education (Columbus) 88: 297–322. [Google Scholar] [CrossRef]
- Roohr, Katrina Crotts, Margarita Olivera-Aguilar, and Ou Lydia Liu. 2021. Value Added in Higher Education: Brief History, Measurement, Challenges, and Future Directions. In Learning Gain in Higher Education (International Perspectives on Higher Education Research, Vol. 14). Edited by Christina Hughes and Malcolm Tight. Bingley: Emerald Publishing Limited, pp. 59–76. [Google Scholar] [CrossRef]
- Schlotter, Martin, Guido Schwerdt, and Ludger Woessman. 2011. Econometric methods for causal evaluation of education policies and practices: A non-technical guide. Education Economics 19: 109–37. [Google Scholar] [CrossRef]
- Schmidt, Frank L. 2017. Beyond questionable research methods: The role of omitted relevant research in the credibility of research. Archives of Scientific Psychology 5: 32–41. [Google Scholar] [CrossRef]
- Schmidt, Frank L., and John Hunter. 2004. General mental ability in the world of work: Occupational attainment and job performance. Journal of Personality and Social Psychology 86: 162–73. [Google Scholar] [CrossRef] [Green Version]
- Schult, Johannes, and Jorn R. Sparfeldt. 2016. Do non-g factors of cognitive ability tests align with specific academic achievements? A combined bifactor modeling approach. Intelligence 59: 96–102. [Google Scholar] [CrossRef]
- Sims, Sam, Jake Anders, Matthew Inglis, and Hugues Lortie-Forgues. 2020. Quantifying ‘Promising Trials’ Bias in Randomized Controlled Trials in Education. Working Paper No. 20-16. London: University College London, Centre for Education Policy and Equalising Opportunities (CEPEO). [Google Scholar]
- Singer, Judith D. 2019. Reshaping the arc of quantitative educational research: It’s time to broaden our paradigm. Journal of Research on Educational Effectiveness 12: 570–93. [Google Scholar] [CrossRef]
- Snow, Richard E. 1996. Aptitude development and education. Psychology, Public Policy, and Law 2: 536–60. [Google Scholar] [CrossRef]
- Spearman, Charles. 1904. “General intelligence,” objectively determined and measured. The American Journal of Psychology 15: 201–92. [Google Scholar] [CrossRef]
- Steyvers, Mark, and Robert J. Schafer. 2020. Inferring latent learning factors in large-scale cognitive training data. Nature Human Behavior 4: 1145–55. [Google Scholar] [CrossRef] [PubMed]
- Stinebrickner, Todd R., and Ralph Stinebrickner. 2007. The causal effect of studying on academic performance. In National Bureau of Economic Research Working Paper 13341; Cambridge, MA: National Bureau of Economic Research. [Google Scholar] [CrossRef]
- The Chronicle of Higher Education. forthcoming. Diversity Indexes. The Chronicle of Higher Education. Available online: www.chronicle.com/package/diversity-indexes/ (accessed on 23 March 2022).
- Thorndike, Robert M., and David F. Lohman. 1990. A Century of Ability Testing. Chicago: The Riverside Publishing Company. [Google Scholar]
- U.S. Department of Education College Scorecard. 2017–2018. Available online: https://collegescorecard.ed.gov/ (accessed on 23 March 2022).
- Wai, Jonathan, and Drew H. Bailey. 2021. How intelligence research can inform education and public policy. In The Cambridge Handbook of Intelligence and Cognitive Neuroscience. Edited by Aron K. Barbey, Sherif Karama and Richard J. Haier. Cambridge: Cambridge University Press, pp. 434–47. [Google Scholar] [CrossRef]
- Wai, Jonathan, and Joni M. Lakin. 2020. Finding the missing Einsteins: Expanding the breadth of cognitive and noncognitive measures used in academic services. Contemporary Educational Psychology 63: 101920. [Google Scholar] [CrossRef]
- Wai, Jonathan, and Kaja Perina. 2018. Expertise in journalism: Factors shaping a cognitive and culturally elite profession. Journal of Expertise 1: 57–78. Available online: https://www.journalofexpertise.org/articles/volume1_issue1/JoE_2018_1_1_Wai_Perina.html (accessed on 23 March 2022).
- Wai, Jonathan, Matt I. Brown, and Christopher F. Chabris. 2018. Using standardized test scores to include general cognitive ability in education research and policy. Journal of Intelligence 6: 37. [Google Scholar] [CrossRef] [Green Version]
- Wai, Jonathan. 2013. Investigating America’s elite: Cognitive ability, education, and sex differences. Intelligence 41: 203–11. [Google Scholar] [CrossRef]
- Whitehurst, Grover J. “Russ”, Matthew M. Chingos, and Michael R. Gallaher. 2013. Do School Districts Matter? Washington, DC: Brookings Institution. Available online: https://www.brookings.edu/research/do-school-districts-matter/ (accessed on 23 March 2022).
- Winitzky-Stephens, Jessie R., and Jason Pickavance. 2017. Open educational resources and student course outcomes: A multilevel analysis. International Review of Research in Open and Distributed Learning 18: 35–49. [Google Scholar] [CrossRef] [Green Version]
- Wolf, Alison. 2003. Does Education Matter? Myths about Education and Economic Growth. London: Penguin Global. [Google Scholar]
- Wolf, Patrick, ed. 2019. School Choice: Separating Fact from Fiction. New York: Routledge. [Google Scholar]
Dataset | U.S. News 2014 | College Scorecard 2017–2018 |
---|---|---|
Variables | ||
Student characteristics | ||
SAT score | Average SAT score | Average SAT score |
Outcomes | ||
Earning | PayScale early-career | 6 years after graduation |
PayScale mid-career | 10 years after graduation | |
Ranking | ||
THE U.S. Ranking | THE U.S. Ranking | |
Lumosity Ranking | Lumosity Ranking | |
Institutional factors | ||
Cost of attending | Room and board | Cost per academic year |
Admission, enrollment, and completion | Total enrollment | Total enrollment |
Acceptance rate | Admission rate | |
Six-year completion rate | ||
Average freshmen retention rate | Retention at 4-year-institution | |
University resources | Endowment | Faculty average salary/month |
% Graduating students ever borrowed loan | % Students ever borrowed | |
Average student debt | % Students received Pell grants | |
Diversity | Diversity index | Diversity index |
Panel A: College Scorecard Data 2016–2017 | ||||||
6-year Salary | 10-year Salary | |||||
Model 1 (SAT) | Model 2 (Institutional factors) | Model 3 (SAT + institutional factors) | Model 1 (SAT) | Model 2 (Institutional factors) | Model 3 (SAT + institutional factors) | |
Coefficient for SAT score | 44.228 *** | n/a | 16.387 * | 65.634 *** | n/a | 22.039 * |
(3.767) | (7.955) | (5.299) | (9.651) | |||
N | 341 | 341 | 341 | 341 | 341 | 341 |
R2 | 0.419 | 0.569 | 0.587 | 0.470 | 0.641 | 0.653 |
Panel B: U.S. News Ranking Data 2014 | ||||||
Early-career Salary | Mid-career Salary | |||||
Model 1 (SAT) | Model 2 (Institutional factors) | Model 3 (SAT + institutional factors) | Model 1 (SAT) | Model 2 (Institutional factors) | Model 3 (SAT + institutional factors) | |
Coefficient for SAT score | 29.207 *** | n/a | 19.093 *** | 75.686 *** | n/a | 40.299 *** |
(3.853) | (6.442) | (5.702) | (10.983) | |||
N | 264 | 264 | 264 | 264 | 264 | 264 |
R2 | 0.296 | 0.378 | 0.416 | 00.412 | 0.525 | 0.560 |
Panel A: College Scorecard Data 2016–2017 | Coefficient for SAT Score | Standard Error | N | R2 | |
THE U.S. ranking | Model 1 (SAT) | −1.527 *** | 0.086 | 280 | 0.526 |
Model 2 (institutional factors) | n/a | 280 | 0.754 | ||
Model 3 (SAT + institutional factors) | −0.585 ** | 0.147 | 280 | 0.770 | |
Lumosity ranking | Model 1 (only SAT) | −0.847 *** | 0.041 | 204 | 0.514 |
Model 2 (institutional factors control) | n/a | 204 | 0.585 | ||
Model 3 (SAT + institutional factors) | −0.460 *** | 0.129 | 204 | 0.615 | |
Panel B: U.S. News Data 2014 | Coefficient for SAT Score | Standard Error | N | R2 | |
THE U.S. ranking | Model 1 (SAT) | −1.717 *** | 0.075 | 269 | 0.562 |
Model 2 (institutional factors) | n/a | 269 | 0.746 | ||
Model 3 (SAT + institutional factors) | −0.745 *** | 0.126 | 269 | 0.778 | |
Lumosity ranking | Model 1 (SAT) | −0.929 *** | 0.050 | 199 | 0.564 |
Model 2 (institutional factors) | n/a | 199 | 0.518 | ||
Model 3 (SAT + institutional factors) | −0.742 *** | 0.088 | 199 | 0.626 |
Variable | Model 1 R2 | Model 2 R2 | Model 3 R2 | ||
---|---|---|---|---|---|
Panel A: College Scorecard dataset 2017–2018 | |||||
6-year salary | 0.419 | 0.569 | 0.581 | 0.721 | 0.979 |
10-year salary | 0.470 | 0.641 | 0.653 | 0.720 | 0.982 |
THE U.S. ranking | 0.526 | 0.754 | 0.770 | 0.683 | 0.979 |
Lumosity ranking | 0.514 | 0.585 | 0.615 | 0.836 | 0.951 |
Panel B: U.S. News Data 2014 | |||||
Early-career salary | 0.296 | 0.378 | 0.416 | 0.712 | 0.909 |
Mid-career salary | 0.412 | 0.525 | 0.560 | 0.736 | 0.938 |
THE U.S. ranking | 0.562 | 0.746 | 0.778 | 0.722 | 0.959 |
Lumosity ranking | 0.564 | 0.518 | 0.626 | 0.901 | 0.827 |
Coefficients | Standard Error | Standardized Dominance Statistic | Ranking | |
---|---|---|---|---|
Panel A: mean salary six years after graduation | ||||
Average SAT score | 16.387 ** | 5.249 | 0.188 | 2 |
Cost per academic year | −0.135 | 0.096 | 0.028 | 8 |
Admission rate | −3136.935 | 1883.976 | 0.016 | 9 |
Retention at 4-year-institution | 18,836.437 ** | 6150.669 | 0.157 | 3 |
Completion rate | −6873.158 | 4395.932 | 0.127 | 5 |
Total enrollment | −0.1372 ** | 0.043 | 0.040 | 7 |
Faculty average salary/month | 1.113 *** | 0.228 | 0.152 | 4 |
% Students ever borrowed | 11325.510 | 8586.535 | 0.005 | 11 |
% Students received Pell grants | −20,357.645 *** | 3811.716 | 0.221 | 1 |
Diversity index | 82.190 *** | 23.575 | 0.055 | 6 |
Location | n/a | n/a | 0.010 | 10 |
School control | n/a | n/a | 0.000 | 12 |
Panel B: Mean salary ten years after graduation | ||||
Average SAT score | 22.039 ** | 6.747 | 0.177 | 2 |
Cost per academic year | 0.005 | 0.124 | 0.043 | 8 |
Admission rate | −5679.633 * | 2421.490 | 0.026 | 9 |
Retention at 4-year-institution | 28,917.060 *** | 7905.503 | 0.178 | 1 |
Completion rate | −2061.146 | 5650.127 | 0.149 | 5 |
Total enrollment | −0.228 *** | 0.055 | 0.044 | 7 |
Faculty average salary/month | 1.514 *** | 0.293 | 0.155 | 4 |
% Students ever borrowed | 10,353.670 | 11,036.340 | 0.005 | 11 |
% Students received Pell grants | −20,776.170 *** | 4899.228 | 0.163 | 3 |
Diversity index | 103.223 *** | 30.302 | 0.049 | 6 |
Location | n/a | n/a | 0.011 | 10 |
School control | n/a | n/a | 0.000 | 12 |
Panel C: THE U.S. ranking | ||||
Average SAT score | −0.585 *** | 0.130 | 0.152 | 4 |
Cost per academic year | −0.009 *** | 0.003 | 0.119 | 5 |
Admission rate | 59.200 | 60.210 | 0.031 | 9 |
Retention at 4-year-institution | −108.600 | 208.100 | 0.153 | 2 |
Completion rate | −543.9 *** | 116.100 | 0.177 | 1 |
Total enrollment | −0.002 | 0.001 | 0.091 | 6 |
Faculty average salary/month | −0.033 *** | 0.007 | 0.153 | 3 |
% Students ever borrowed | 203.800 | 230.400 | 0.007 | 11 |
% Students received Pell grants | −331.400 *** | 101.200 | 0.062 | 7 |
Diversity index | −1.249 ** | 0.569 | 0.038 | 8 |
Location | n/a | n/a | 0.019 | 10 |
School control | n/a | n/a | 0.000 | 12 |
Panel D: Lumosity ranking | ||||
Average SAT score | −0.460 *** | 0.120 | 0.258 | 1 |
Cost per academic year | 0.004 | 0.002 | 0.046 | 6 |
Admission rate | −83.449 | 51.189 | 0.023 | 8 |
Retention at 4-year-institution | −165.522 | 202.047 | 0.171 | 3 |
Completion rate | −162.704 | 100.635 | 0.157 | 4 |
Total enrollment | 0.001 | 0.001 | 0.032 | 7 |
Faculty average salary/month | −0.006 | 0.005 | 0.068 | 5 |
% Students ever borrowed | 14.267 | 183.907 | 0.006 | 11 |
% Students received Pell grants | 346.395 ** | 116.398 | 0.221 | 2 |
Diversity index | −0.327 | 0.646 | 0.008 | 10 |
Location | n/a | n/a | 0.009 | 9 |
School control | n/a | n/a | 0.000 | 12 |
Coefficients | Standard Error | Standardized Dominance Statistic | Ranking | |
---|---|---|---|---|
Panel A: Pay-Scale early-career | ||||
Average SAT score | 19.092 ** | 7.011 | 0.283 | 1 |
Total enrollment | −0.021 | 0.038 | 0.095 | 5 |
Acceptance rate | 4311.067 | 2195.387 | 0.028 | 8 |
Average freshmen retention | 10,510.41 * | 5154.387 | 0.200 | 2 |
Endowment | 8.82 × 10−7 *** | 2.25 × 10−7 | 0.119 | 3 |
Room and board | 0.153 | 0.141 | 0.044 | 7 |
% graduating students ever borrowed loan | −4187.357 | 3344.803 | 0.094 | 6 |
Average student debt | 0.177 * | 0.075 | 0.028 | 9 |
Diversity index | 89.793 *** | 25.093 | 0.098 | 4 |
Location | n/a | n/a | 0.008 | 11 |
School control | n/a | n/a | 0.004 | 10 |
Panel B: Pay-scale mid-career | ||||
Average SAT score | 40.299 *** | 10.757 | 0.273 | 1 |
Total enrollment | −0.037 | 0.074 | 0.084 | 4 |
Acceptance rate | 7188.705 | 4375.667 | 0.036 | 8 |
Average freshmen retention | 38,653.78 *** | 11,396.04 | 0.268 | 2 |
Endowment | 9.93 × 10−7 | 6.12 × 10−7 | 0.069 | 7 |
Room and board | 0.841 ** | 0.293 | 0.099 | 3 |
% graduating students ever borrowed loan | −6762.375 | 5265.526 | 0.076 | 5 |
Average student debt | 0.279 * | 0.133 | 0.015 | 9 |
Diversity index | 175.039 *** | 48.531 | 0.074 | 6 |
Location | n/a | n/a | 0.003 | 11 |
School control | n/a | n/a | 0.005 | 10 |
Panel C: THE U.S. Ranking | ||||
Average SAT score | −0.745 *** | 0.124 | 0.253 | 2 |
Total enrollment | −0.002 | 0.001 | 0.120 | 3 |
Acceptance rate | −72.737 | 61.716 | 0.041 | 8 |
Average freshmen retention | −1054.54 *** | 137.275 | 0.299 | 1 |
Endowment | −2.58 × 10−8 * | 1.09 × 10−8 | 0.084 | 4 |
Room and board | −0.014 *** | 0.004 | 0.070 | 5 |
% graduating students ever borrowed loan | 2.295 | 71.240 | 0.043 | 7 |
Average student debt | −0.008 *** | 0.002 | 0.020 | 9 |
Diversity index | −3.025 *** | 0.628 | 0.053 | 6 |
Location | n/a | n/a | 0.006 | 11 |
School control | n/a | n/a | 0.010 | 10 |
Panel D: Lumosity ranking | ||||
Average SAT score | −0.742 *** | 0.104 | 0.432 | 1 |
Total enrollment | −0.001 | 0.001 | 0.073 | 4 |
Acceptance rate | −142.921 ** | 53.496 | 0.042 | 6 |
Average freshmen retention | −273.882 * | 131.068 | 0.255 | 2 |
Endowment | −8.90 × 10−9 | 4.55 × 10−9 | 0.078 | 3 |
Room and board | −0.004 | 0.004 | 0.028 | 7 |
% graduating students ever borrowed loan | −27.626 | 59.679 | 0.060 | 5 |
Average student debt | 0.003 | 0.002 | 0.018 | 8 |
Diversity index | 0.616 | 0.594 | 0.013 | 9 |
Location | n/a | n/a | 0.001 | 10 |
School control | n/a | n/a | 0.000 | 11 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wai, J.; Tran, B. Student Characteristics, Institutional Factors, and Outcomes in Higher Education and Beyond: An Analysis of Standardized Test Scores and Other Factors at the Institutional Level with School Rankings and Salary. J. Intell. 2022, 10, 22. https://doi.org/10.3390/jintelligence10020022
Wai J, Tran B. Student Characteristics, Institutional Factors, and Outcomes in Higher Education and Beyond: An Analysis of Standardized Test Scores and Other Factors at the Institutional Level with School Rankings and Salary. Journal of Intelligence. 2022; 10(2):22. https://doi.org/10.3390/jintelligence10020022
Chicago/Turabian StyleWai, Jonathan, and Bich Tran. 2022. "Student Characteristics, Institutional Factors, and Outcomes in Higher Education and Beyond: An Analysis of Standardized Test Scores and Other Factors at the Institutional Level with School Rankings and Salary" Journal of Intelligence 10, no. 2: 22. https://doi.org/10.3390/jintelligence10020022