# Panel Data Models for School Evaluation: The Case of High Schools’ Results in University Entrance Examinations

## Abstract

**:**

## 1. Introduction

^{®}17 [1]. By using exemplars, we provide a guide for social scientists new to the area of panel data analysis. Due to data availability, the analysis of student performance in university admission tests was restricted to Andalusian high schools. The nine public universities in Andalusia (southern Spain) are part of the Distrito Único Andaluz, a regional admissions system. To apply for bachelor’s degrees at these universities, students submit an ordered list of their preferred undergraduate programs to the centralized system, which fills the available places using the highest university access marks.

## 2. Background

## 3. Methodology

**β**is the K × 1 vector of coefficients on the set of explanatory variables. However, due to the limitations of the information contained in the administrative database used, our analysis only considers a single explanatory variable (average course grades of the last two years, 11th and 12th grades).

_{it}, and they are often called “idiosyncratic errors”, and (ii) ${\mu}_{i}$, school-specific time-invariant characteristics that are not directly observable to the econometrician but influence student learning outcomes (termed “unobserved heterogeneity” in panel data econometrics). Such factors can be regarded as time-invariant, and at the same time, it is extremely hard to measure them. The fact that we have repeated measurements of the same units allows us to control for their unknown characteristics that are constant over time. The failure to account for these unobserved individual differences leads to bias in the resulting estimates [11]. Depending on our assumptions about this latter term, different estimation procedures are available [12]. If the assumption is that the unobserved heterogeneity is uncorrelated with the explanatory variables in the model, random effects (REs) estimation is used to assess the relationship between the explanatory variables and educational outcomes (mu can be taken as random). Additionally, the standard assumption is that e behaves like a random variable and is uncorrelated with

**x**. Otherwise, with correlated heterogeneity, we have to use other techniques that have become known as fixed effects (FEs) estimation. This variant is called the fixed-effects model, as early treatments modeled these effects as parameters ${\mu}_{1},\dots ,{\mu}_{N}$ to be estimated [13]. FEs remove the effect of those time-invariant characteristics from the predictor variables, so we can assess the predictors’ net effect. The key insight is that if the unobserved variable does not change over time, then any changes in the dependent variable must be due to influences other than these fixed characteristics [14]. So, the estimated coefficients of the fixed-effects models cannot be biased because of omitted time-invariant characteristics. Unlike the fixed-effects model, the rationale behind the random-effects model is that the variation across units is assumed to be random and uncorrelated with the predictors or independent variables included in the model. If we believe that differences across entities have some influence on the dependent variable, then we should use random effects. Because the estimate of the slope parameters (

**β**) differs across the different estimation methods, a frequently asked question in empirical research is which model to use: the fixed-effects model or the random-effects model. Although sometimes researchers prefer random-effects models merely because they simply want to obtain the effects of time-invariant variables, this is not a sufficient justification. A formal Hausman test can be used to test whether or not the school-specific heterogeneity is fixed. Whether ${\mu}_{i}$ is assumed to be fixed or random is crucial to obtaining unbiased parameter estimates. The null hypothesis is that H

_{0}: random effects (REs) are consistent and efficient. If the test fails to reject the null hypothesis, as the p-value is greater than 0.05, we would select the RE model: the school-specific heterogeneity, though present in the data, is not correlated with the explanatory variables and can very well be taken as random; the RE estimators will be consistent and efficient. Otherwise, if we reject H

_{0}, then Cov$\left({\mathit{x}}_{it},{\mu}_{i}\right)\ne 0$, and it would be wiser to use the fixed-effects (FEs) estimator to obtain unbiased estimates [12].

## 4. Results

#### 4.1. Data and Results

_{1}is the coefficient for this independent variable; and ${e}_{it}$ is the error term.

**β**

_{1}that indicates how much Y changes over time, on average per school, when X increases by one unit in Equation (2) or the average effect of X over Y when X changes across time and between schools by one unit in Equation (3). However, not controlling for unobserved school-specific effects leads to bias in the resulting estimates. The variable μ

_{i}captures all unobserved factors affecting Y that do not change over time. In Equation (2), μ

_{i}(i = 1…N) is an unknown intercept for each high school (N school-specific intercepts). This implies that all behavioral differences between high schools are fixed over time and are represented as parametric shifts of the regression function. The intercept is allowed to vary from high school to high school, while the slope parameter is assumed to be constant in both the school and time dimensions. In Equation (3), intercepts vary randomly among schools, and μ

_{i}is a random disturbance term that is assumed to be constant over time. The term μ

_{i}is a stochastic variable that embodies the unobservable or non-measurable school differences. Essentially, the effect is thought to be a random school effect rather than a fixed parameter. In short, we might try to discern whether there is a difference in achievement between high schools in the comunidad autónoma of Andalucía. Instead of including every high school in the equation (as we would have in the fixed-effects model using dummy variables) one can randomly sample high schools and assume that the effect is randomly distributed across high schools but constant through time. The first step in our analysis is, therefore, deciding whether to estimate a fixed- or random-effects panel model. The results of the Hausman specification test supported the fixed-effects (FEs) model for public high schools and the random-effects (REs) model for private high schools (Table 2). Serial correlation is not a problem in micro panels (with very few years), and taking group means can remove heteroskedasticity. In all cases, for both public and private high schools, the results indicate that students’ scores on the Selectividad exams are explained by their grades in the Bachillerato. These latter grades are a summary indicator of the academic performance during two school years, which reflects, among other factors, student effort and socioeconomic characteristics. The grade point average of private high schools has a larger effect on the Selectividad examination results in the Technological B. and Social Sciences B. compared to public secondary schools. For example, a one-unit increase in high school GPAs increases the Selectividad exam scores by almost 0.70 points in the Social Sciences B. in private schools versus 0.62 in public schools. However, in the Bachillerato of Health Sciences, the value of the estimated β

_{1}is greater for public high schools compared to private ones: as GPAs increase by 1 unit, standardized test scores increase by 0.70 and 0.68 points, respectively. Nevertheless, the results in the Selectividad exams vary between high schools not only by the observed student characteristics (GPAs) but also by unobserved characteristics (time-invariant unobserved heterogeneity). Analyzing the “rho” value in Table 2, an important portion of the variance of the outcomes on the Selectividad exams is due to unobserved characteristics that differ between high schools. “rho” is known as the intraclass correlation (% of the variance due to differences across panels). We should highlight the results obtained for the Bachillerato of Health Sciences and the Bachillerato of Social Sciences in private high schools (RE estimation). If ${\mu}_{i}$ is conceptualized as a random variable in RE estimation, it can be interpreted as a randomly varying intercept in Equation (3) that captures unmodeled unit-specific heterogeneity of Y’s level (e.g., the heterogeneity of Selectividad test scores). The above 50 percent variation in high school test scores is explained by the constant over time school effect. That is, educational outcomes in the Selectividad vary more among high schools than within a high school on different occasions. This greater dispersion (variability) in the test scores across private high schools would indicate that some exceptional schools set a very high benchmark for the group of private schools. However, how does that help to infer grade inflation? From a short panel, we cannot infer that Bachillerato course grades are inflated in those private high schools with poor performance on the Selectividad exams. They would be inflated if this pattern holds up over several years.

#### 4.2. Discussion

## 5. Conclusions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- StataCorp. Stata Statistical Software: Release 17; StataCorp LP: College Station, TX, USA, 2021. [Google Scholar]
- Sanders, J.R.; Davidson, E.J. A model for school evaluation. In International Handbook of Educational Evaluation; Kellaghan, T., Stufflebeam, D.L., Eds.; Springer: Berlin/Heidelberg, Germany, 2003; pp. 807–826. [Google Scholar]
- OECD. OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes; OECD Publishing: Washington, DC, USA, 2013. [Google Scholar]
- Coutinho-Pereira, M.; Moreira, S. Efficiency of secondary schools in Portugal: A stochastic frontier analysis. Banco Port. Econ. Bull.
**2007**, 13, 101–117. [Google Scholar] - Zwick, R. Higher education admission testing. In Educational Measurement, 4th ed.; Brennan, R.L., Ed.; American Council on Education/Praeger Series on Higher Education: Washington, DC, USA, 2006; pp. 647–679. [Google Scholar]
- Bejar, I.I.; Blew, E.O. Grade inflation and the validity of the Scholastic Aptitude Test. Am. Educ. Res. J.
**1981**, 18, 143–156. [Google Scholar] [CrossRef] - Gershenson, S. Grade Inflation in High Schools (2005–2016); Thomas, B., Ed.; Fordham Institute: Washington, DC, USA, 2018. [Google Scholar]
- Dowling, A. Output Measurement in Education (Australia Council for Educational Research, December 2008). Available online: https://research.acer.edu.au/cgi/viewcontent.cgi?article=1001&context=policy_analysis_misc (accessed on 19 November 2022).
- Walpole, M.; McDonough, P.M.; Bauer, C.J.; Gibson, C.; Kanyi, K.; Toliver, R. This test is unfair: Urban African American and Latino high school students’ perceptions of standardized college admission tests. Urban Educ.
**2005**, 40, 321–349. [Google Scholar] [CrossRef] - Wooldridge, J.M. Introductory Econometrics: A Modern Approach; Cengage Learning: Mason, OH, USA, 2015. [Google Scholar]
- Baltagi, B.H. Panel data methods. In Handbook of Applied Economic Statistics; Ullah, A., Giles, D.E.A., Eds.; CRC Press: Boca Raton, FL, USA, 1998; pp. 311–323. [Google Scholar]
- Andreß, H.J.; Golsch, K.; Schmidt, A.W. Applied Panel Data Analysis for Economic and Social Surveys; Springer: Berlin/Heidelberg, Germany, 2013. [Google Scholar]
- Cameron, A.C.; Trivedi, P.K. Microeconometrics: Methods and Applications; Cambridge University Press: Cambridge, UK, 2005. [Google Scholar]
- Stock, J.H.; Watson, M.W. Introduction to Econometrics; Addison Wesley: Boston, MA, USA, 2003. [Google Scholar]
- Lazear, E.P. Educational production. Q. J. Econ.
**2001**, 116, 777–803. [Google Scholar] [CrossRef] - Johnson, R.L. The development and use of school profiles. In International Handbook of Educational Evaluation; Kellaghan, T., Stufflebeam, D.L., Eds.; Springer: Berlin/Heidelberg, Germany, 2003; pp. 827–842. [Google Scholar]

**Table 1.**High school grades and university entrance examination scores: descriptive statistics. Andalusian high schools, 2005–2008.

Total Observations ^{(a)} | Percentage | High Schools’ Average GPA (Bachillerato) | High Schools’ Average Test Scores (Selectividad) | High Schools’ Average GPA (Bachillerato) | High Schools’ Average Test Scores (Selectividad) | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|

2005–2008 | 2005–2008 | 2005 | 2006 | 2007 | 2008 | 2005 | 2006 | 2007 | 2008 | |||

Technological B. | ||||||||||||

Private high school | 627 | 23.93 | 7.58 | 6.15 | 7.57 | 7.51 | 7.57 | 7.68 | 6.36 | 5.90 | 6.20 | 6.12 |

Public high school | 1993 | 76.07 | 7.47 | 5.90 | 7.40 | 7.45 | 7.46 | 7.54 | 6.09 | 5.73 | 5.94 | 5.84 |

Difference of means (private–public) | 0.12 | 0.25 | 0.17 | 0.06 | 0.11 | 0.14 | 0.27 | 0.17 | 0.26 | 0.28 | ||

Is the diff. statistically significant at 5%? ^{§} | Yes | Yes | Yes | No | Yes | Yes | Yes | Yes | Yes | Yes | ||

Health Sciences B. | ||||||||||||

Private high school | 638 | 23.85 | 7.54 | 6.19 | 7.39 | 7.48 | 7.54 | 7.73 | 6.15 | 6.25 | 6.13 | 6.24 |

Public high school | 2037 | 76.15 | 7.41 | 5.96 | 7.32 | 7.35 | 7.42 | 7.55 | 5.93 | 6.00 | 5.95 | 5.97 |

Difference of means (private–public) | 0.13 | 0.23 | 0.07 | 0.12 | 0.12 | 0.18 | 0.22 | 0.25 | 0.19 | 0.27 | ||

Is the diff. statistically significant at 5%? ^{§} | Yes | Yes | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | ||

Social Sciences B. | ||||||||||||

Private high school | 643 | 23.86 | 7.11 | 6.01 | 7.08 | 7.11 | 7.10 | 7.15 | 5.82 | 6.02 | 6.01 | 6.17 |

Public high school | 2052 | 76.14 | 6.99 | 5.69 | 7.03 | 6.98 | 6.97 | 7.00 | 5.61 | 5.66 | 5.68 | 5.80 |

Difference of means (private–public) | 0.12 | 0.32 | 0.05 | 0.13 | 0.13 | 0.15 | 0.21 | 0.36 | 0.33 | 0.37 | ||

Is the diff. statistically significant at 5%? ^{§} | Yes | Yes | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes |

^{§}Mean comparison test. diff.—mean(private)–mean(public); H

_{0}—diff. = 0.

^{(a)}There are around 160 private high schools and 520 public high schools. For each high school (group), we have about four observations (an average score per year). Source: author’s calculations.

**Table 2.**Andalusian high schools’ results in university entrance examinations: panel data estimates.

Technological B. | Health Sciences B. | Social Sciences B. | |||||||
---|---|---|---|---|---|---|---|---|---|

Private High Schools | Private High Schools | Private High Schools | |||||||

Coef. | Std. Err. | Coef. | Std. Err. | Coef. | Std. Err. | ||||

High schools’ average GPA in the Bachillerato | 0.6864 | ** | 0.0385 | 0.6792 | ** | 0.0350 | 0.6964 | ** | 0.0400 |

Constant | 0.9364 | ** | 0.2942 | 1.0670 | ** | 0.2661 | 1.0506 | ** | 0.2870 |

sigma_μ | 0.4332 | 0.4002 | 0.4154 | ||||||

sigma_e | 0.4706 | 0.3838 | 0.3774 | ||||||

rho | 0.4588 | 0.5209 | 0.5478 | ||||||

Wald chi2(1) | 318.05 | 376.76 | 302.45 | ||||||

Prob > chi2 | p < 0.001 | p < 0.001 | p < 0.001 | ||||||

Number of obs. | 625 | 638 | 642 | ||||||

Number of groups | 160 | 164 | 166 | ||||||

Dep. var. = high schools’ average test scores (Selectividad) | |||||||||

Random-effects GLS regression | Random-effects GLS regression | Random-effects GLS regression | |||||||

Hausman test | chi2(1) = 0.50; Prob > chi2 = 0.4795 | chi2(1) = 2.59; Prob > chi2 = 0.1078 | chi2(1) = 0.15; Prob > chi2 = 0.6941 | ||||||

Public high schools | Public high schools | Public high schools | |||||||

Coef. | Std. Err. | Coef. | Std. Err. | Coef. | Std. Err. | ||||

High schools’ average GPA in the Bachillerato | 0.6763 | ** | 0.0224 | 0.6967 | ** | 0.0215 | 0.6197 | ** | 0.0228 |

Constant | 0.8503 | ** | 0.1679 | 0.7972 | ** | 0.1596 | 1.3550 | ** | 0.1596 |

sigma_μ | 0.4839 | 0.4176 | 0.3894 | ||||||

sigma_e | 0.5087 | 0.4311 | 0.3939 | ||||||

rho | 0.4750 | 0.4840 | 0.4943 | ||||||

F(1;1464) | 908.98 | ** | |||||||

F(1;1510) | 1050.95 | ** | |||||||

F(1;1528) | 740.10 | ** | |||||||

Number of obs. | 1988 | 2033 | 2051 | ||||||

Number of groups | 523 | 522 | 522 | ||||||

Dep. var. = high schools’ average test scores (Selectividad) | |||||||||

Fixed-effects (within) regression | Fixed-effects (within) regression | Fixed-effects (within) regression | |||||||

Hausman test | chi2(1) = 6.90; Prob > chi2 = 0.0086 | chi2(1) = 17.51; Prob > chi2 = 0.0000 | chi2(1) = 32.62; Prob > chi2 = 0.0000 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Salas-Velasco, M.
Panel Data Models for School Evaluation: The Case of High Schools’ Results in University Entrance Examinations. *Stats* **2023**, *6*, 312-321.
https://doi.org/10.3390/stats6010019

**AMA Style**

Salas-Velasco M.
Panel Data Models for School Evaluation: The Case of High Schools’ Results in University Entrance Examinations. *Stats*. 2023; 6(1):312-321.
https://doi.org/10.3390/stats6010019

**Chicago/Turabian Style**

Salas-Velasco, Manuel.
2023. "Panel Data Models for School Evaluation: The Case of High Schools’ Results in University Entrance Examinations" *Stats* 6, no. 1: 312-321.
https://doi.org/10.3390/stats6010019