Distinguishing Contemporaneous Rule Recovery from Financial Risk Forecasting in Higher Education Institutions: A Controlled Synthetic-Panel Experiment

Highlights

What are the main findings?

• The proposed panel-based framework evaluates HEI financial risk as a multi-class temporally validated early-warning task, integrating revenue-structure, expenditure-pressure, financial-health, and institutional/contextual indicators.
• Financial-health indicators provide the strongest direct classification signal, whereas broader multidimensional indicators mainly improve contextual interpretation and governance relevance rather than substantially increasing point-classification performance.

What are the implication of the main finding?

• HEI financial risk early warning should move beyond static ratio-based assessment by combining panel-data modeling, temporal and institutional validation, leakage-aware evaluation, class-balanced performance metrics, and interpretable risk evidence.
• RPA can serve as a traceable execution layer rather than a forecasting model, supporting the conversion of risk classifications into standardized alerts, review procedures, responsibility assignment, audit trails, and feedback-based governance updates.

Abstract

High classification performance in financial-risk early-warning research may reflect recovery of a constructed contemporaneous rating rule rather than prediction of an independent future outcome. This study distinguishes these two forms of evidence through a controlled synthetic-panel experiment situated in the context of higher education institutions (HEIs). Task 1 is a contemporaneous positive-control audit in which the four components defining a deterministic three-class rating are supplied to the classifiers. Task 2 uses predictors measured at year $t-1$ to classify the rating at year t through strict rolling-origin evaluation under prespecified Weak, Moderate, and Strong temporal-persistence conditions. In Task 1, the four fitted classifiers achieved held-out Macro-F1 values of 0.9968–1.0000, demonstrating near-complete recovery of the disclosed rating rule but providing no prospective forecasting evidence. In Task 2, the best mean annual Macro-F1/Macro-AUC increased from 0.480/0.696 under Weak persistence to 0.549/0.747 under Moderate persistence and 0.633/0.824 under Strong persistence. The Weak–Moderate–Strong ordering was observed for both primary metrics across all four fitted classifiers, while higher model complexity provided no consistent advantage within the disclosed synthetic mechanism. These findings confirm contemporaneous rule recoverability and sensitivity to deliberately embedded temporal persistence only within the controlled experiment. They do not establish predictive validity, transportability, or decision benefit in real HEIs. The study contributes a reproducible task-to-claim approach that aligns target construction, predictor–target overlap, information timing, rolling-origin evaluation, probability quality, and model performance with the inferences that the resulting evidence can legitimately support. Real-world validation would require source-indexed longitudinal HEI data, independently adjudicated post-origin outcomes, verified information-availability dates, external testing, and prospective decision evaluation.

1. Introduction

Higher education institutions (HEIs) operate under changing conditions of enrollment, funding, expenditure, and accountability. Their financial sustainability is therefore shaped by both internal resource structures and external institutional pressures [1,2,3]. Financial monitoring can support the timely identification of vulnerability and the prioritization of professional review [4,5]. However, the evidential meaning of such monitoring depends on the relationship among outcome definition, predictor timing, validation design, and intended use.

The distinction between current-state classification and early warning is especially important. Early warning requires predictors available before a defined forecast origin and an outcome observed after that origin  [6,7,8]. A contemporaneous rating may summarize current financial conditions [9], but it does not by itself demonstrate advance prediction. If variables used to construct a rating are also supplied to a classifier, a high performance may reflect the recovery of the rating rule rather than prediction of an independent later outcome  [10,11,12]. A random held-out split cannot resolve this information-timing problem when target construction overlaps with the predictors. Prospective claims therefore require explicit temporal alignment and evaluation on later untouched periods [13,14].

This distinction is also consequential for machine-learning comparisons. Flexible models can approximate nonlinear thresholds and interactions more closely than restrictive parametric specifications [15], but algorithmic complexity cannot create temporal validity or outcome independence. The apparent performance may also depend on the persistence of the underlying process: a lagged predictor should be more informative when the preceding state is more strongly carried into the next period. Without a controlled mechanism, however, the effects of target overlap, temporal persistence, data quality, and model flexibility are difficult to distinguish.

To isolate these validity problems, the present study uses a fully disclosed synthetic-panel experiment as a methodological test bed. Simulation studies are appropriate when a known data-generating mechanism is required to isolate method behavior under controlled conditions [16,17]. The purpose of the experiment is not to estimate the prevalence of financial risk among HEIs but to vary a known temporal mechanism while retaining complete control over target construction and information timing. Three balanced panels represent Weak, Moderate, and Strong temporal-persistence conditions, with the persistence coefficient fixed at $\rho =0.20$, $0.50$, and $0.80$, respectively. Each panel contains 3000 neutral synthetic units observed annually from 2015 to 2025. The three scenarios reuse the same units and random-number stream, with only $\rho$ varying across scenarios. This common-random-number strategy supports a more controlled comparison of the three prespecified conditions [18]. The experiment neither reproduces nor samples the financial conditions of identifiable HEIs. Public higher-education and financial materials inform only the broad interpretation of the synthetic variables.

The study objective is to determine how predictor–target overlap and prespecified temporal persistence affect the performance and interpretation of multiclass financial-risk classification within a reproducible controlled design. Two analytical tasks establish distinct evidence boundaries. Task 1 supplies the four same-year components used to construct a deterministic three-class rating and tests whether fitted classifiers recover that rule on held-out synthetic units. It is a positive-control audit rather than a forecasting test. Task 2 uses 12 predictors measured at year $t-1$ to classify the rating at year t through expanding-window rolling-origin evaluation across eight test years from 2018 to 2025. Task 2 therefore evaluates one-year-ahead predictability within the disclosed synthetic mechanism, but it does not establish real-world HEI early-warning performance.

Against this background, the study addresses two research questions:

RQ1.

To what extent can the deterministic three-class rating be recovered when its four target-defining components are supplied contemporaneously to the classifiers, and what does this reveal about the interpretation of target-inclusive performance?

RQ2.

How do one-year-ahead classification performance, discrimination, probability quality, and annual stability vary across prespecified Weak, Moderate, and Strong temporal-persistence conditions under rolling-origin evaluation, and do the four fitted classifiers exhibit the same scenario ordering on the two primary metrics?

Table 1 connects the research questions with their corresponding analytical tasks, evidence, and claim boundaries.

Table 1. Alignment of research questions, analytical tasks, and evidential boundaries.

RQ	Analytical Task	Evidence Obtained	Claim Boundary
RQ1	Task 1: contemporaneous rule-recovery audit	Unit-held-out Macro-F1 and complementary discrimination and probability metrics for the majority-class benchmark and four fitted classifiers.	Diagnoses predictor–target overlap and rule-recovery capacity; does not provide prospective early-warning evidence.
RQ2	Task 2: ${\mathbf{X}}_{i,t-1}\to Y_{i,t}$	Eight annual rolling-origin tests per scenario; Macro-F1 and Macro-AUC as primary outcomes, with balanced accuracy, Brier score, ECE, and annual variation as complementary evidence.	Measures temporal predictability within the prespecified synthetic mechanism; does not establish real-institution risk prevalence, transportability, or deployment utility.

The study makes three bounded contributions. First, it provides a controlled demonstration of why target-inclusive same-year classification must be distinguished from fixed-horizon prediction. Second, it shows how a common-random-number design can isolate the effect of prespecified temporal persistence while holding synthetic units and innovation streams constant. Third, it links task definition to prespecified primary and complementary metrics, annual evaluation, archived probability outputs, and explicit claim boundaries within a reproducible reporting design. These contributions are methodological: the study provides neither a sectoral estimate of HEI financial risk nor validation of an operational early-warning system.

Any future use of automated workflows for data checks, version control, case routing, or audit recording would require separate prospective operational and decision-focused validation under accountable human oversight  [19,20,21,22,23]. Automation cannot validate a constructed outcome, convert synthetic evidence into empirical validity, or replace professional judgement.

The remainder of the paper is organized as follows. Section 2 reviews early-warning validity, predictor–target overlap, temporal evaluation, and comparative model assessment and then develops the conceptual framework and prespecified checks. Section 3 specifies the synthetic data-generating process, the deterministic rating, analytical variables, integrity audits, Task 1 and Task 2 designs, model configurations, evaluation measures, and reproducibility controls. Section 4 reports the integrity checks, scenario diagnostics, task-specific performance, annual Task 2 results, and the contrast between Tasks 1 and 2. Section 5 interprets the findings and defines a staged path from synthetic experimentation to empirical and operational validation. Section 6 summarizes the principal findings, methodological contributions, and evidential boundaries.

2. Literature Review and Conceptual Framework

This section develops the conceptual basis for the controlled experiment. It first distinguishes financial monitoring from prospective early warning. It then examines constructed outcomes and predictor–target overlap, temporal persistence and rolling-origin validation, and the relationship among model flexibility, discrimination, and probability quality. The section concludes by synthesizing these issues into a conceptual framework and two prespecified checks that guide the subsequent methods and interpretation.

2.1. HEI Financial Monitoring and Prospective Early Warning

HEI financial monitoring commonly considers liquidity, solvency, budgetary control, revenue structure, and broader financial sustainability [24,25]. These dimensions are institutionally consequential, because resource and revenue structures are associated with credit assessment and academic output [26], as well as research capacity and institutional competitiveness [27,28]. Institutional and contextual information may also assist in interpretation, because HEIs differ in scale, mission, funding exposure, and operating environments [29].

HEI studies commonly analyze annual financial and institutional records [25,30]. In such applications, however, the year attached to a record is not necessarily the date on which that information became available for prediction. Failure to distinguish the observation period, release date, forecast origin, and outcome period can introduce look-ahead bias or target leakage. Researchers may also use expert- or threshold-based ratings as tractable outcomes. Such ratings can support current-state monitoring, but their construction must be disclosed, and their defining inputs must not be mistaken for independent predictors [31].

Substantive relevance does not determine predictive status. A measure observed in the same period as a rating can describe the current conditions, support rule-consistency monitoring, or inform professional review. Early-warning evidence is narrower: predictors must be available before a defined forecast origin, and the outcome must be observed after that origin. A multi-year panel can support temporal analysis, but longitudinal structure alone does not guarantee prospective validity [32]. The observation time, reporting time, and model-availability time of each field must therefore be distinguished.

This study consequently treats “early warning” as an evidential claim rather than a label that can be attached to any financial classifier. Task 1 is a contemporaneous rule-recovery audit. Task 2 imposes a one-year information boundary, but its outcome remains synthetic and rule-derived. Task 2 is therefore methodologically closer to forecasting, while remaining insufficient to support claims about actual institutional distress or real-world HEI early-warning performance.

2.2. Constructed Outcomes, Target Overlap, and Positive-Control Audits

Outcome construction is a measurement decision. A researcher-defined rating is reproducible only when its components, thresholds, class order, overrides, and boundary conditions are disclosed. If those components are subsequently reused as model inputs, predictor–target overlap is created. Flexible classifiers may then achieve very high held-out performance, because the construction rule remains stable across the split and not because the inputs independently anticipate a later event. This form of label leakage can produce overly optimistic interpretations [33].

Holding out units does not remove this circularity. Although the records in the test subset are unseen during model fitting, the same deterministic relationship connects the predictors to the labels in both the fitting and test subsets. Target-inclusive classification can nevertheless serve as a positive-control audit. Successful recovery verifies that the analytical pipeline can learn the embedded rule and reveals the performance ceiling created by direct predictor–target overlap. Its value depends on naming the task correctly and preventing the resulting performance from being presented as evidence of advance warning.

This distinction also clarifies the role of multidimensional indicators. Revenue, expenditure, financial ratios, institutional descriptors, and contextual variables may all be relevant to financial monitoring [34,35], but indicator breadth alone does not establish predictive value. Prospective evidence depends on target independence, predictor availability before the forecast origin, and performance in later untouched periods.

2.3. Temporal Persistence and Rolling-Origin Evaluation

One-year-ahead classification depends partly on temporal persistence. When a latent financial state is weakly persistent, measurements from year $t-1$ may contain limited information about a rating in year t. Stronger persistence should increase that information when other elements are held constant. In real data, however, temporal persistence is entangled with measurement error, institutional heterogeneity, policy change, and unobserved shocks. A controlled synthetic experiment can vary the persistence directly, but the resulting scenario ordering constitutes a mechanism check rather than an estimate of real HEI behavior [36].

Validation must respect the same temporal logic. Random splitting may mix earlier and later observations and does not emulate prospective use. Rolling-origin evaluation instead fits models using the historical information available at a given forecast origin, uses a later validation period for configuration selection, and evaluates the selected configuration on the next untouched period [37]. Repeating this process across successive forecast origins can reveal annual instability that a single pooled score may conceal.

Repeated longitudinal predictions also require an appropriate unit of inference. Predictions for the same synthetic units, test years, and common-random-number scenarios are dependent. Record-level tests that treat all saved predictions or probabilities as independent observations can create pseudoreplication. Annual test periods, effect magnitudes, and scenario-consistent patterns therefore provide the defensible basis for inference in the present design [38].

2.4. Model Flexibility, Discrimination, and Probability Quality

Financial-risk research increasingly compares statistical and machine-learning classifiers, because nonlinear learners can represent thresholds and interactions in structured data [39]. Multinomial logistic regression provides an interpretable linear benchmark, a decision tree represents recursive thresholding, and bagging and adaptive boosting provide two established ensemble strategies [40,41]. A majority-class predictor supplies a no-signal reference. This compact comparison is sufficient for testing whether more flexibility is necessary under the disclosed synthetic mechanism; it is not intended to establish a universal ranking of algorithms.

No single metric captures all relevant performance properties. Accuracy can be dominated by the largest class. Macro-F1 and balanced accuracy give more visibility to performance across the Low-, Medium-, and High-Risk classes [42,43,44], while multiclass Macro-AUC evaluates the ranking discrimination [45]. The Brier score and expected calibration error (ECE) assess different aspects of probability quality. The Brier score is a proper scoring rule for probabilistic forecasts [46,47], whereas ECE is a binned summary whose interpretation depends on the calibration definition and binning scheme [48,49,50]. A model can classify nearly all observations correctly while producing poorly calibrated probabilities. Conversely, a model may show an apparently low calibration error while providing little or no discrimination. Classification, discrimination, and calibration must therefore remain conceptually distinct.

Higher complexity is not itself evidence of superiority. Flexible models may closely recover a threshold-based target in a contemporaneous task yet provide little or no advantage once the predictors are lagged. Model selection should occur within each training–validation fold, and any claim of superiority should remain limited to the prespecified tasks, metrics, and synthetic scenario mechanism.

Analytical performance also does not establish the value or safety of downstream workflow automation. Automated data transfer, reconciliation, notification, case routing, and audit recording may support administrative processes [19,20,21], but they cannot establish the target validity, predictive transportability, or decision benefit. Any future automation would require separate source, outcome, temporal, external, calibration, operational, and decision-focused validation. Human actors would need to retain authority for corroboration, interpretation, escalation, corrective action, and closure [22,23]. Workflow automation is therefore outside the present experiment and is considered only as a bounded direction for future prospective evaluation.

2.5. Literature Synthesis, Conceptual Framework, and Prespecified Checks

The literature identifies a recurring chain of validity problems: a current-state rating may be described as early warning; target-defining components may reappear as predictors; non-temporal evaluation may be treated as prospective validation; classification performance may be conflated with probability quality; and analytical outputs may enter operational workflows before their decision value has been established. Table 2 links these problems to the controlled responses and remaining claim boundaries of the present study.

Table 2. Literature problems, controlled study responses, and remaining boundaries.

Literature Problem	Controlled Response	Remaining Boundary
Current-state classification presented as early warning	Separate the Task 1 contemporaneous rule-recovery audit from the Task 2 one-year-ahead evaluation.	Task 2 uses a synthetic rule-derived outcome rather than an independently observed distress event.
Predictor–target overlap	Disclose the complete hierarchical rating rule and use target-inclusive Task 1 as a positive-control audit.	Near-ceiling Task 1 performance cannot support a prospective forecasting claim.
Weak temporal validation	Lag all Task 2 predictors and use eight expanding-window rolling-origin tests.	Eight synthetic test years do not establish external transportability.
Uncontrolled persistence	Reuse common random numbers while varying only $\rho =0.20,0.50,0.80$.	The persistence coefficients are prespecified design constants rather than empirical estimates for HEIs.
Accuracy-centred reporting	Report class-sensitive performance, discrimination, Brier score, calibration, and annual results.	Probability quality within a synthetic experiment does not validate operational thresholds or decision rules.
Prediction–workflow conflation	Treat workflow automation as outside the experiment and subordinate any future use to prior empirical, operational, and decision-focused validation.	Workflow benefit, safety, acceptance, and decision impact require separate prospective testing under accountable human oversight.

The resulting framework has three levels. The first establishes experimental validity through a disclosed generator, a deterministic target, explicit information timing, and pre-modeling integrity gates. The second maps each analytical task to the evidence it produces and the interpretation it permits. The third defines the boundary between internally valid synthetic findings and the additional empirical, external, operational, and decision-focused evidence required for real-world use. Figure 1 visualizes this evidence-to-claim sequence.

Figure 1. Experimental validity foundation and task-specific inferential boundaries of the controlled synthetic-panel study. Task 1 audits contemporaneous recovery of the deterministic rating rule, whereas Task 2 evaluates one-year-ahead predictability under prespecified temporal-persistence conditions.

The framework yields two prespecified checks that serve different analytical purposes. Macro-F1 is the primary class-sensitive outcome for both tasks, whereas Macro-AUC is the co-primary discrimination outcome for Task 2. Balanced accuracy, Brier score, ECE, and annual trajectories provide complementary evidence and are not treated as additional prespecified tests.

Positive-Control Expectation. When the four contemporaneous target-defining components are supplied, each of the four fitted classifiers is expected to substantially outperform the majority-class benchmark and achieve near-ceiling held-out Macro-F1 in recovering the deterministic three-class rating. Confirmation of this expectation would demonstrate rule recoverability under direct predictor–target overlap, not prospective forecasting capability.

Persistence Mechanism Check. Under strict one-year-ahead rolling-origin evaluation, the mean annual Macro-F1 and Macro-AUC are expected to increase from Weak to Moderate and Strong persistence for each of the four fitted classifiers. This ordering is expected because a higher prespecified persistence coefficient, $\rho$, carries more of the preceding latent state into the outcome year. Confirmation of the ordering would verify that model performance responds to the mechanism deliberately embedded in the synthetic generator; it would not establish the same relationship in actual HEIs.

The Positive-Control Expectation addresses RQ1 through Task 1. The Persistence Mechanism Check addresses RQ2 through the three scenario-specific Task 2 analyses. The majority-class benchmark is retained solely as a no-discrimination reference and is excluded from the persistence-ordering check because it does not use the predictors.

3. Data and Methods

This study uses a controlled synthetic-panel experiment to examine two methodological questions in financial-risk modeling: how contemporaneous target-defining information affects classification performance and how one-year-ahead performance varies when only lagged information is available. A simulation design is appropriate for this purpose, because the data-generating mechanism and the factor varied across conditions can be specified explicitly [16,17].

The workflow comprises a common experimental foundation followed by two analytical tasks. Three panels were generated under prespecified Weak, Moderate, and Strong temporal-persistence conditions. A separate verification program then audited the accounting identities, rating labels, temporal pairs, variable ranges, and scenario mechanism before any model was fitted. Task 1 is a contemporaneous rule-recovery audit, whereas Task 2 is a strict one-year-ahead rolling-origin evaluation. The design therefore identifies model behavior within a fully disclosed data-generating mechanism; it does not estimate the financial condition of actual higher education institutions (HEIs).

3.1. Study Design and Evidential Scope

The analytical data consist of three fully synthetic balanced panels. Each panel contains 3000 non-identifiable analytical units observed annually from 2015 through 2025, producing 33,000 unit–year records per scenario and 99,000 records across all three scenarios. Synthetic identifiers range from S0001 to S3000. They do not identify, reproduce, or probabilistically sample actual HEIs. The synthetic type, size, and regional context fields are neutral design variables rather than classifications of named institutions or estimates of a national HEI population.

Publicly available higher-education and financial materials informed the broad selection and interpretation of variables, but no institution-specific record was transcribed into the analytical panels. The analysis contains no identity crosswalk, source-level administrative record, independently observed default, or adjudicated distress outcome. The three-class outcome is generated by a disclosed deterministic rule. Accordingly, the study permits conclusions about target overlap, information timing, and scenario-dependent model behavior, but not about real-world risk prevalence, causal effects, national representativeness, or prospective performance in actual institutions. Table 3 summarizes the experimental scope, analytical role, and permitted interpretation of the controlled synthetic-panel design.

Table 3. Design scope, analytical role, and claim boundary.

Element	Experimental Specification	Permitted Interpretation
Analytical units	3000 neutral synthetic units with 11 annual observations per scenario.	Repeated analytical series, not identifiable HEIs or a population sampling frame.
Scenarios	Weak, Moderate, and Strong persistence, with $\rho =0.20$, $0.50$, and $0.80$.	Prespecified sensitivity conditions, not parameters estimated from real HEI data.
Amounts and ratios	Programmatically generated accounting components, supporting quantities, totals, and derived ratios.	Internally coherent synthetic measurements, not published institutional accounts.
Outcome	Hierarchical deterministic three-class rating.	Embedded synthetic target, not an independently observed financial-distress event.
Task 1	Same-year target components supplied to classifiers.	Diagnostic recovery of the embedded rule.
Task 2	Predictors at $t-1$ used to classify the rating at t.	Temporal predictability within the specified synthetic mechanism.
External validity	No real institutional sample or external outcome.	No claim of deployable HEI early warning or automated institutional action.

3.2. Prespecified Data-Generating Process

All generation parameters, scenario values, rating thresholds, and random seeds were frozen for the reported experiment before Tasks 1 and 2 were executed. The parameters were selected as transparent design constants that produce positive monetary quantities, all three target classes, internally consistent accounting identities, and distinguishable levels of temporal persistence. They are not empirical estimates or calibrated descriptions of the Chinese HEI sector. A Park–Miller generator with seed 20260731 and a Box–Muller transformation for Gaussian innovations supplied the random draws  [51,52].

The three scenarios used the same random-number stream; only $\rho$ changed. This common-random-number design reduces Monte Carlo noise in the scenario contrast and isolates the intended change in persistence [18].

Table 4 records the complete generation order. The order is methodologically important, because the 2020–2021 shifts are applied to component variables before the totals and ratios are recomputed. This dependency order preserves the accounting identities throughout the generated panels.

Table 4. Ordered construction of each synthetic scenario.

Step	Operation	Generated Information	Control Purpose
1	Freeze design	Unit count, years, seed, scenario values, event shifts, thresholds, and audit tolerance.	Prevent result-driven local adjustment.
2	Generate unit heterogeneity	Fixed scale, capacity, debt propensity, research intensity, contextual exposure, and neutral categories.	Create stable between-unit differences.
3	Generate annual latent state	Unit effect, lagged state, common event shift, and annual innovation.	Control within-unit temporal persistence.
4	Generate components	Five revenue components, four expenditure components, assets, liabilities, current accounts, and planned budget.	Create correlated positive quantities from shared and component innovations.
5	Apply event shifts	2020–2021 multipliers applied to tuition, special grants, and operating costs.	Ensure that the event operates before aggregation.
6	Recompute accounts	Round components, sum total revenue and expenditure, and then calculate leverage, budget execution, and liquidity.	Preserve identities and the dependency order.
7	Generate target components	Synthetic credit category and historical default count derived from the annual state.	Complete the disclosed rating inputs.
8	Apply rating rule	Hierarchical High-, Medium-, or Low-Risk assignment.	Create a fully reproducible target.
9	Audit and pair	Recompute identities and labels and construct adjacent-year pairs.	Gate the panels before modeling.

3.2.1. Fixed Unit Heterogeneity and Annual Latent State

Each synthetic unit i received time-invariant continuous attributes for financial capacity, debt propensity, research intensity, contextual exposure, and initial scale. Neutral categorical descriptors comprised four unit types, three size classes, and four regional contexts. Let ${\alpha }_i$ denote the fixed unit-specific stress component, and let $z_{i,t}$ denote the annual latent financial stress. The initial and subsequent states were generated as

$$z_{i,2015}={\alpha }_i+0.65{\epsilon }_{i,2015},$$

(1)

$$z_{i,t}=\rho z_{i,t-1}+(1-\rho ){\alpha }_i+{\gamma }_t+0.65{\epsilon }_{i,t},t=2016,\dots ,2025,$$

(2)

where ${\epsilon }_{i,t}$ is a Gaussian innovation. The prespecified common shift ${\gamma }_t$ equals 0.55 in 2020, 0.30 in 2021, and zero otherwise. The only parameter that differs across scenarios is $\rho$: 0.20 for Weak, 0.50 for Moderate, and 0.80 for Strong persistence. A higher $\rho$ therefore carries more information about the preceding state into the current year. Under the Persistence Mechanism Check, Task 2 performance is expected to increase as $\rho$ increases if the pipeline preserves the deliberately embedded temporal signal. This expected ordering is a check of the disclosed synthetic mechanism, not an empirical proposition about actual HEIs.

3.2.2. Accounting Components, Event Shifts, and Derived Measures

The annual scale followed the deterministic nominal trend $S_{i,t}=S_{i,2015}{\left(1.025\right)}^{t-2015}$. Revenue and expenditure components were generated from unit scale, fixed attributes, the annual latent state, a shared domain innovation, and a component-specific innovation. In general form,

$$Q_{k,i,t}=S_{i,t}{b}_{k,i}exp\left({\lambda }_k{z}_{i,t}+{\eta }_k{u}_{d,i,t}+{\sigma }_k{e}_{k,i,t}\right),$$

(3)

where $b_{k,i}$ is the component’s prespecified base share, $u_{d,i,t}$ is a shared revenue- or expenditure-domain innovation, and $e_{k,i,t}$ is a component-specific innovation. The shared terms induce dependence among related components; the monetary variables were not generated as mutually independent draws. Component-level coefficients and category adjustments are retained in the executable generator and parameter manifest.

For 2020 and 2021, respectively, tuition revenue was multiplied by 0.90 and 0.94, special grants by 1.20 and 1.12, and operating costs by 1.08 and 1.05. After these shifts, the monetary components were rounded to two decimal places. Stored totals were then calculated exactly as

$$\begin{array}{cc}\hfill R_{i,t}=& R_{i,t}^{\mathrm{fiscal}}+R_{i,t}^{\mathrm{tuition}}+R_{i,t}^{\mathrm{research}}+R_{i,t}^{\mathrm{donation}}+R_{i,t}^{\mathrm{grant}},\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill E_{i,t}=& E_{i,t}^{\mathrm{salary}}+E_{i,t}^{\mathrm{research}}+E_{i,t}^{\mathrm{infrastructure}}+E_{i,t}^{\mathrm{operating}}.\hfill \end{array}$$

(5)

The asset–liability ratio, budget-execution rate, and current ratio were computed only after their supporting quantities had been stored:

$$A_{i,t}=100{\displaystyle \frac{L_{i,t}}{T{A}_{i,t}}},B_{i,t}=100{\displaystyle \frac{E_{i,t}}{P{B}_{i,t}}},C{R}_{i,t}={\displaystyle \frac{C{A}_{i,t}}{C{L}_{i,t}}}.$$

(6)

Here, L and $TA$ denote total liabilities and total assets, $PB$ denotes the planned budget, and $CA$ and $CL$ denote current assets and current liabilities. The generation program also maps the annual state to a six-level synthetic credit category (AAA, AA, A, BBB, BB, or B) and a non-negative historical default count. These two quantities are synthetic target components, not observed credit-agency or administrative records. The principal frozen parameters and their analytical roles are summarized in Table 5.

Table 5. Principal frozen parameters of the synthetic experiment.

Parameter	Frozen Value	Analytical Role
Synthetic units; years	3000; 2015–2025	Balanced panel definition.
Generation seed	20260731	Common random draws across scenarios.
Temporal persistence $\rho$	0.20; 0.50; 0.80	Weak, Moderate, and Strong conditions.
Annual-state innovation scale	0.65	Time-varying unit-level disturbance.
Annual nominal growth	2.5%	Common scale trend.
Common state shift	0.55; 0.30	Applied in 2020 and 2021.
Tuition multiplier	0.90; 0.94	Applied in 2020 and 2021 before totals.
Special-grant multiplier	1.20; 1.12	Applied before total revenue.
Operating-cost multiplier	1.08; 1.05	Applied before total expenditure.
Accounting tolerance	${10}^{-6}$	Maximum absolute identity residual.

3.3. Observed Variables and Analytical Roles

The stored panels contain five revenue components, four expenditure components, their totals, accounting-support quantities, three derived ratios, a synthetic credit category, a synthetic default count, an external economic-impact index, three neutral categorical fields, the scenario indicator, and the outcome. Stored variables were assigned to one of four roles: identifier or design field, accounting component or support variable, analytical predictor, or target component. This role assignment was frozen before model fitting.

Task 1 uses only the four contemporaneous variables that define the target: asset–liability ratio, budget-execution rate, historical default count, and credit category. Credit is encoded in the ordered sequence AAA, AA, A, BBB, BB, and B. The resulting model matrix has four columns.

Task 2 uses 12 variables measured at $t-1$. The eight financial predictors are fiscal appropriation, tuition revenue, research funding, total revenue, total expenditure, asset–liability ratio, budget-execution rate, and current ratio. The four non-financial predictors are unit type, size class, regional context, and the external economic-impact index. The rating at t, credit category at t, and historical default count at t are excluded. The three categorical predictors are one-hot encoded using the levels learned in the training period. With all prespecified levels present, the 12 conceptual predictors yield a 20-column encoded matrix comprising nine numeric columns and eleven indicator columns. Distinguishing conceptual predictors from encoded columns prevents the feature count from being misreported.

No statistical imputation was performed because the audited panels contain no missing cells. No target-based encoding, outcome-conditioned feature selection, or class rebalancing was used. Numeric standardization and categorical encoding were performed separately inside the relevant training partition or rolling fold, as detailed below [10,32]. The construction, timing, and analytical use of each variable group are summarized in Table 6.

Table 6. Variable groups, construction, timing, and use in each task.

Group	Variables	Construction/Role	Task 1	Task 2
Revenue and expenditure	Fiscal, tuition, research funding, donations, special grants; salary, research, infrastructure, operating cost; totals.	Generated components; totals are exact sums.	No	Selected components and totals at $t-1$.
Accounting support	Assets, liabilities, current assets, current liabilities, planned budget.	Denominators and supporting quantities for ratios.	No	No.
Financial ratios	Asset–liability, budget execution, current ratio.	Derived after totals and supporting quantities are stored.	First two at t	All three at $t-1$.
Target components	Credit category and historical default count.	Generated from the annual latent state and used by the rating rule.	Both at t	No.
Non-financial design variables	Unit type, size class, regional context, external economic-impact index.	Neutral synthetic descriptors and context index.	No	All at $t-1$.
Outcome	Rule-based three-class rating.	Hierarchical deterministic target.	$Y_{i,t}$	$Y_{i,t}$.

3.4. Deterministic Three-Class Rating

The outcome $Y_{i,t}$ is a rule-based synthetic rating with three classes: Low Risk, Medium Risk, and High Risk. Let $A_{i,t}$ denote the asset–liability ratio in percent, $B_{i,t}$ the budget-execution rate in percent, $D_{i,t}$ the historical default count, and $C_{i,t}$ the credit category. The High-Risk and Medium-Risk indicators are

$$H_{i,t}=\mathbf{1}\left[A_{i,t}>70\vee D_{i,t}\ge 3\vee C_{i,t}\in \{\mathrm{B},\mathrm{BB}\}\right],$$

(7)

$$M_{i,t}=\mathbf{1}\left[H_{i,t}=0\wedge \left(A_{i,t}>50\vee B_{i,t}<70\vee C_{i,t}\in \{\mathrm{A},\mathrm{BBB}\}\right)\right].$$

(8)

The hierarchical assignment is

$$Y_{i,t}=\left\{\begin{array}{cc}\mathrm{High}\mathrm{Risk},\hfill & H_{i,t}=1,\hfill \\ \mathrm{Medium}\mathrm{Risk},\hfill & H_{i,t}=0\wedge M_{i,t}=1,\hfill \\ \mathrm{Low}\mathrm{Risk},\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(9)

High-Risk conditions take precedence over the other classes. The comparisons are exact: $A=70$ does not trigger High Risk, $A=50$ does not trigger Medium Risk, and $B=70$ does not trigger Medium Risk. These edge conditions are coded once in the disclosed rating function and independently verified through complete label recomputation. Because $Y_{i,t}$ is constructed from $(A_{i,t},B_{i,t},D_{i,t},C_{i,t})$, it is a synthetic rating category rather than a realized financial-distress outcome.

3.5. Pre-Modeling Integrity and Mechanism Audit

An audit program separate from the generation routine was executed before model fitting. For each scenario, it verified the following: (i) 33,000 records and 3000 unique units; (ii) 11 observations per unit covering 2015–2025; (iii) no duplicate unit–year keys; (iv) no missing cells; (v) exact revenue and expenditure identities within an absolute tolerance of ${10}^{-6}$; (vi) exact agreement between stored and independently recomputed labels; (vii) admissible ranges and categories; and (viii) 30,000 unique adjacent-year pairs. The audit also calculated lag-one correlations for selected continuous variables and the High-Risk indicator. These correlations are scenario diagnostics: they test whether persistence is ordered as designed, not whether the synthetic values reproduce empirical HEI autocorrelations. Modeling proceeded only after every gate returned PASS.

3.6. Analytical Tasks and Information Boundaries

3.6.1. Task 1 Evaluation Design: Contemporaneous Rule-Recovery Audit

Task 1 supplies the four target-defining variables for the same unit and year to each fitted classifier:

$$\left(A_{i,t},B_{i,t},D_{i,t},C_{i,t}\right)⟶Y_{i,t}.$$

(10)

Synthetic units, rather than individual records, were assigned to a common 70%/15%/ 15% training, validation, and test split using seed 20260801. All 11 annual records of each unit remained in the same partition, preventing repeated observations from that unit from appearing in both the fitting and test data. Each scenario therefore contained 2100 training units (23,100 records), 450 validation units (4950 records), and 450 untouched test units (4950 records). The realized unit assignment was identical across the three scenarios.

Candidate configurations were selected according to validation Macro-F1, after which the selected configuration was evaluated once on the untouched test set. Keeping all repeated observations from a unit in the same partition and fitting preprocessing operations only on the training data reduces cross-partition information leakage [38]. Task 1 is intentionally target-inclusive and serves as a positive-control audit. Its results measure the recoverability of the deterministic rating rule and demonstrate the performance inflation permitted by direct predictor–target overlap. They must therefore not be interpreted as prospective early-warning evidence.

3.6.2. Task 2 Evaluation Design: Strict One-Year-Ahead Rolling-Origin Evaluation

Task 2 pairs the 12 predictors recorded for unit i in year $t-1$ with its rule-based rating in year t:

$${\mathbf{X}}_{i,t-1}⟶Y_{i,t}.$$

(11)

Adjacent-year matching creates exactly 30,000 one-year-ahead pairs per scenario, covering outcome years 2016–2025. The first two outcome years initialize the expanding-window design, and formal testing covers eight outcome years from 2018 through 2025. For a test outcome year $\tau$, the fitting set contains outcome years 2016 through $\tau -2$, the immediately preceding outcome year $\tau -1$ is used only for configuration selection, and outcome year $\tau$ constitutes the untouched test set. Predictors for the test set therefore come exclusively from year $\tau -1$.

This expanding-window rolling-origin structure provides repeated out-of-sample evaluation without using future periods during model development [36]. The annual training, validation, and test partitions are reported in Table 7.

Table 7. Rolling-origin partitions used in Task 2 for every scenario.

Test Outcome	Training Outcomes	Validation Outcome	Test Predictor	${\mathit{N}}_{\mathbf{train}}$	${\mathit{N}}_{\mathbf{val}}={\mathit{N}}_{\mathbf{test}}$
2018	2016	2017	2017	3000	3000
2019	2016–2017	2018	2018	6000	3000
2020	2016–2018	2019	2019	9000	3000
2021	2016–2019	2020	2020	12,000	3000
2022	2016–2020	2021	2021	15,000	3000
2023	2016–2021	2022	2022	18,000	3000
2024	2016–2022	2023	2023	21,000	3000
2025	2016–2023	2024	2024	24,000	3000

Each validation and test year contains all 3000 units. Numeric means, standard deviations, and categorical vocabularies were estimated anew from each rolling training set and then applied unchanged to the corresponding validation and test sets. Any category absent from the training vocabulary but present in a validation or test set was represented by zeros across all training-derived indicator columns.

No test-year outcome contributed to record pairing, preprocessing, feature selection, configuration selection, model fitting, probability adjustment, or threshold selection [32]. The scenarios share the same units, years, random draws, and fold definitions. Their results are therefore paired sensitivity outcomes rather than independent samples. The information boundaries and permitted evidential claims of the two tasks are compared in Table 8.

Table 8. Informationboundary and evidential meaning of the analytical tasks.

Task	Alignment and Partition	Target-Information Control	Permitted Claim
Task 1	Same-year classification using a common unit-level 70%/15%/15% split.	Four direct target components are included.	Recovery of the embedded deterministic rule and diagnosis of performance inflation caused by predictor–target overlap.
Task 2	${\mathbf{X}}_{i,t-1}\to Y_{i,t}$, with an expanding training window, previous-year validation, and annual tests for 2018–2025.	All predictors are lagged; year-t credit, defaults, and rating are excluded.	Scenario-dependent one-year-ahead predictability within the disclosed synthetic mechanism.

3.7. Benchmark and Fitted Classifiers

Both tasks compare a majority-class benchmark with four fitted classifiers: multinomial logistic regression, a decision tree, bagged trees, and adaptive boosting. This compact model set includes a no-signal reference, an interpretable linear classifier, a single nonlinear learner, and two ensemble learners representing bagging and boosting [41]. The purpose is not to establish a universal algorithm ranking, but to compare target-inclusive rule recovery with temporally ordered classification under a common evaluation framework. Neither class weighting nor synthetic resampling was applied.

For each fitted classifier, the candidate configurations were trained using the designated training data and ranked by validation Macro-F1. Ties were resolved using the deterministic ordering implemented in the analysis program: smaller C for multinomial logistic regression; lower maximum depth followed by larger minimum leaf size for the decision-tree and adaptive-boosting grids; and larger minimum leaf size for the bagged-tree grid. These rules favor the simpler candidate when validation performance is identical. The selected configuration was then evaluated once, without further adjustment, on the corresponding untouched test data.

The majority-class benchmark required no configuration selection. It always predicted the most frequent class in the relevant training set, and its probability vector was defined by the corresponding three-class training distribution. The prespecified specifications and candidate configurations for both tasks are reported in Table 9.

Table 9. Prespecified candidate configurations for the official analyses.

Method	Task 1 Specification or Candidates	Task 2 Specification or Candidates
Majority benchmark	Most frequent class in the unit-level training set.	Most frequent class in the rolling training period.
Multinomial logistic regression	$C\in \{0.1,1,10,100\}$; L-BFGS; maximum of 2000 iterations; training-only standardization.	$C\in \{0.1,1,10\}$; L-BFGS; maximum of 2000 iterations; fold-specific training-only standardization and encoding.
Decision tree	Maximum depth $\in \{3,4,5,6,8,10,\mathrm{unrestricted}\}$; minimum leaf size $\in \{1,5,20\}$.	Maximum depth $\in \{3,5,8\}$; minimum leaf size $\in \{10,30\}$.
Bagged trees	200 bootstrap trees; minimum leaf size $\in \{1,5,20\}$.	100 bootstrap trees; minimum leaf size $\in \{10,30\}$.
Adaptive boosting	200 learners; learning rate $=0.08$; weak-tree depth $\in \{1,2,3\}$; minimum leaf size $\in \{1,5,20\}$.	100 learners; learning rate $=0.08$; weak-tree depth $\in \{1,2\}$; minimum leaf size $\in \{10,30\}$.

The official reported results were generated under Python 3.12.13 using scikit-learn 1.8.0 [53], NumPy, and pandas. Task 1 used seed 20260801. Task 2 used base seed 20260802 with deterministic scenario–year offsets recorded in the hyperparameter manifest. The saved run record includes the realized Task 1 unit split, all 24 rolling folds, every selected configuration, class distributions, confusion matrices, record-level predicted classes, and aligned three-class probability vectors. These version-locked outputs, rather than an alternative software translation, define the decimals reported in the Results.

3.8. Evaluation Measures and Aggregation

Macro-F1 was designated the primary class-sensitive performance measure for both tasks, because it assigns equal weight to the Low-, Medium-, and High-Risk classes [44]. For Task 2, Macro-AUC—the macro average of the three one-vs-rest ROC-AUC values—was designated the co-primary discrimination measure [45]. Accuracy, balanced accuracy, and weighted F1 were secondary classification measures. This hierarchy was frozen before analysis and corresponds to the prespecified checks stated in Section 2.5. The class ordering Low Risk, Medium Risk, High Risk was fixed in every confusion matrix and probability file.

The probabilistic performance was evaluated using multiclass Brier score and log loss as proper scoring rules [47], together with a 10-bin confidence-based expected calibration error (ECE) [50]. For observation i and class k, let $p_{ik}$ be the predicted probability and $y_{ik}$ the one-hot outcome. The Brier score was

$$\mathrm{Brier}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\displaystyle \sum _{k=1}^3}{\left(p_{ik}-y_{ik}\right)}^2.$$

(12)

For ECE, observations were grouped into ten equal-width bins according to the maximum predicted class probability. The absolute difference between the mean confidence and observed accuracy in each non-empty bin was weighted by the bin’s sample proportion. This confidence-based measure assesses overall calibration and is not a class-conditional calibration guarantee  [49].

Task 1 metrics were calculated on 4950 held-out records per scenario. The Positive-Control Expectation was assessed by determining whether each of the four fitted classifiers substantially outperformed the majority-class benchmark and achieved near-ceiling held-out Macro-F1 when supplied with the four contemporaneous target-defining components. This assessment concerns the rule recoverability under direct predictor–target overlap and does not constitute a forecasting test.

Task 2 metrics were first calculated separately for each of the eight test years. The principal Task 2 summaries are the mean and sample standard deviation of the eight annual values; so, years—not the 24,000 pooled records—form the repeated evaluation units. Pooled out-of-time metrics across 24,000 test predictions per scenario and method are reported only as complementary descriptive summaries. The 360,000 saved Task 2 prediction rows equal three scenarios multiplied by eight years, 3000 units, and five prediction-producing methods; they do not represent 360,000 independent experimental units.

Scenario contrasts were assessed by their direction, magnitude, and stability across the eight matched test years. The Persistence Mechanism Check was evaluated by determining whether the mean annual Macro-F1 and Macro-AUC followed the prespecified Weak < Moderate < Strong ordering for each of the four fitted classifiers. Because the scenarios share the same random draws, and the outcome is synthetic, conventional population-level p-values were not interpreted as evidence about actual HEIs. The majority-class benchmark was retained solely as a no-discrimination reference and was excluded from the persistence-ordering check, because it does not use the predictors. The contrast between Tasks 1 and 2 distinguishes deterministic contemporaneous rule recovery from temporally ordered classification.

3.9. Reproducibility, Reporting, and Ethical Boundary

The reproducibility archive contains (i) the generation program, (ii) the frozen parameter and seed manifest, (iii) all three synthetic panels, (iv) the data dictionary and feature manifest, (v) independent integrity-audit code and outputs, (vi) the official Task 1 and Task 2 programs, (vii) the realized unit split and rolling-fold manifest, (viii) selected hyperparameters and run logs, and (ix) record-level predictions, probabilities, and confusion matrices. The archive follows the principle that code, data, parameters, and computational steps should be retained together to permit reproducible verification  [54,55].

The Task 1 archive contains 74,250 test-prediction rows ($3\times 4950\times 5$), representing three scenarios, 4950 held-out records, and five prediction-producing methods. The Task 2 archive contains 360,000 rows ($3\times 8\times 3000\times 5$), representing three scenarios, eight test years, 3000 units, and five methods. Each probability vector was verified to be finite, to lie in $[0,1]$, and to sum to one within numerical tolerance.

Any change to the generation parameters, rating thresholds, variable roles, scenario values, seeds, pairing rule, or candidate grids constitutes a new experiment and requires complete regeneration and re-evaluation; local editing of released records is not permitted. Because the panels contain no human participants, identifiable institutions, private records, or linked administrative data, no participant-level consent or privacy inference is involved. The absence of identifiable data does not expand the evidential scope. Any prospective application would still require independently observed post-origin outcomes, verified historical availability dates, external institutional samples, and a separately evaluated human-governed deployment process.

4. Empirical Results

This section follows the analytical sequence specified in Section 3. Section 4.1 first examines whether the generated records satisfy the structural, accounting, label, and temporal conditions required for analysis. Section 4.2 then reports Task 1, the prespecified positive-control audit of same-year deterministic-rule recovery. Section 4.3 reports Task 2, the strict one-year-ahead rolling-origin evaluation of ${\mathbf{X}}_{i,t-1}\to Y_{i,t}$. Finally, Section 4.4 contrasts the conclusions supported by these two non-equivalent tasks.

This sequence addresses the two research questions underlying the study: how much apparent performance is produced by contemporaneous target overlap, and how one-year-ahead performance changes when information is lagged and temporal ordering is enforced. All results concern the disclosed synthetic experiment; none estimates the financial-risk prevalence or predictive performance in actual higher education institutions (HEIs).

4.1. Synthetic-Panel Integrity and Scenario Diagnostics

The independently executed pre-modeling audit returned PASS for every gate in all three scenarios. As shown in Table 10, each panel contains 33,000 records for 3000 synthetic units, with 11 consecutive annual observations per unit. No duplicate unit–year keys or missing required values were detected. Revenue and expenditure totals agreed with their stored components within the prespecified tolerance of ${10}^{-6}$, and independent recomputation reproduced every stored class label. Each scenario also yielded exactly 30,000 unique adjacent-year pairs. Consequently, no record required repair, deletion, or imputation before modeling.

Table 10. Pre-modeling integrity audit across the three synthetic scenarios.

Panel A: Structural Completeness and Temporal Pairing
Scenario	Rows	Units	Years/Unit	Year Range	Lag Pairs	Status
Weak	33,000	3000	11	2015–2025	30,000	PASS
Moderate	33,000	3000	11	2015–2025	30,000	PASS
Strong	33,000	3000	11	2015–2025	30,000	PASS
Total	99,000	3000 a	—	—	90,000	PASS
Panel B: Independent Verification Outcomes
Audit Gate	Scope Checked	Acceptance Criterion		Observed Result		Status
Key uniqueness	99,000 scenario–unit–year keys	No duplicate key within a scenario		99,000 unique		PASS
Required-field completeness	99,000 records	No missing required value		99,000 complete		PASS
Revenue identity	99,000 records	Absolute discrepancy $\le {10}^{-6}$		99,000 matched		PASS
Expenditure identity	99,000 records	Absolute discrepancy $\le {10}^{-6}$		99,000 matched		PASS
Label reconstruction	99,000 records	Exact agreement with Equations (7)–(9)		99,000 matched		PASS
Admissible ranges	99,000 records	Every audited value within its prespecified range		99,000 compliant		PASS

Note: ^a The three scenarios reuse the same 3000 synthetic unit identifiers under alternative persistence conditions. Accordingly, the total reports 3000 shared identifiers rather than 9000 scenario-specific unit trajectories. Panel B aggregates the independently executed checks across the three scenarios. Because no violation was detected, successful verification totals are reported instead of repeated zero failure counts.

Table 11 summarizes the class composition and lag-one mechanism diagnostics. All three classes are represented in every scenario. The High-Risk share increases from 26.05% in the Weak condition to 29.02% in the Moderate condition and 35.53% in the Strong condition. These percentages describe outcomes generated by the experimental mechanism and must not be interpreted as estimates of sectoral prevalence.

Table 11. Outcomecomposition and lag-one diagnostics by persistence scenario.

Panel A: Three-Class Outcome Composition
Scenario	$\mathit{N}$	Low Risk, $\mathit{n}$ (%)	Medium Risk, $\mathit{n}$ (%)	High Risk, $\mathit{n}$ (%)
Weak	33,000	9088 (27.54%)	15,317 (46.42%)	8595 (26.05%)
Moderate	33,000	8894 (26.95%)	14,530 (44.03%)	9576 (29.02%)
Strong	33,000	8987 (27.23%)	12,288 (37.24%)	11,725 (35.53%)
Panel B: Lag-One Correlations across 30,000 Adjacent-Year Pairs
Scenario	Asset–Liability Ratio	Budget Execution	Current Ratio	High-Risk Indicator
Weak	0.497	0.201	0.464	0.307
Moderate	0.610	0.319	0.586	0.412
Strong	0.752	0.514	0.740	0.558

Note: Scenario values are prespecified experimental conditions based on $\rho =0.20$, $0.50$, and $0.80$. The reported correlations are mechanism diagnostics rather than empirical estimates of temporal dependence among HEIs.

The lag-one correlations of the three financial-state indicators and the High-Risk indicator increase monotonically with the prespecified persistence parameter $\rho$. For example, the lag-one correlation of the asset–liability ratio increases from 0.497 in the Weak condition to 0.610 in the Moderate condition and 0.752 in the Strong condition. The corresponding correlation for the High-Risk indicator rises from 0.307 to 0.412 and 0.558. These ordered results confirm that the scenario generator produced the intended temporal-persistence contrast before model fitting.

Figure 2 presents the annual class shares under the three persistence conditions. Before 2020, the class composition is comparatively stable within each scenario. At the onset of the prespecified event period, the Low-Risk share declines and the High-Risk share increases in all three panels. The subsequent paths differ by persistence: the Weak condition returns toward its pre-event composition relatively quickly, whereas the Moderate and especially the Strong conditions retain a larger High-Risk share for longer. The figure therefore confirms both components of the intended mechanism—a common event disturbance and scenario-dependent persistence—without implying that the displayed shares estimate actual HEI prevalence.

Figure 2. Annualclass composition in the controlled synthetic panels under the Weak, Moderate, and Strong temporal-persistence conditions. The shaded interval identifies the prespecified 2020–2021 event period. The series are mechanism diagnostics and must not be interpreted as estimates of financial-risk prevalence among actual HEIs.

4.2. Task 1: Contemporaneous Rule-Recovery Audit

Task 1 is the prespecified positive-control audit. Each fitted classifier receives the four same-year variables used to construct the outcome. The held-out results therefore quantify recovery of the embedded deterministic rule under direct predictor–target overlap, rather than prospective prediction. Each scenario uses the same unit-level split: 2100 units for training, 450 for validation, and 450 for testing, corresponding to 23,100, 4950, and 4950 records, respectively.

Table 12 reports the independent test results. The majority-class benchmark yields a Macro-F1 ranging from 0.1804 to 0.2100 and a balanced accuracy of 0.3333. Its observed accuracy therefore reflects the largest class rather than meaningful multiclass discrimination. In contrast, multinomial logistic regression achieves Macro-F1 values of 0.9968–0.9990, while the three tree-based classifiers recover the rule almost exactly.

Table 12. Task 1 held-out performance in the contemporaneous rule-recovery audit.

Scenario	Model	Balanced Accuracy	Macro-F1	Macro-AUC	Brier	ECE
Weak	Majority-class benchmark	0.3333	0.2100	0.5000	0.6428	0.0026
	Multinomial logistic regression	0.9977	0.9968	0.9996	0.0074	0.0086
	Decision tree	1.0000	1.0000	1.0000	0.0000	0.0002
	Bagged trees	1.0000	1.0000	1.0000	0.0000	0.0000
	Adaptive boosting	1.0000	1.0000	1.0000	0.4553	0.5499
Moderate	Majority-class benchmark	0.3333	0.2036	0.5000	0.6493	0.0021
	Multinomial logistic regression	0.9992	0.9990	0.9999	0.0049	0.0097
	Decision tree	0.9998	0.9998	0.9999	0.0004	0.0000
	Bagged trees	0.9998	0.9998	1.0000	0.0003	0.0003
	Adaptive boosting	0.9998	0.9998	1.0000	0.4571	0.5508
Strong	Majority-class benchmark	0.3333	0.1804	0.5000	0.6604	0.0013
	Multinomial logistic regression	0.9982	0.9978	1.0000	0.0040	0.0064
	Decision tree	1.0000	1.0000	1.0000	0.0000	0.0001
	Bagged trees	1.0000	1.0000	1.0000	0.0001	0.0002
	Adaptive boosting	1.0000	1.0000	1.0000	0.4450	0.5435

Note: $N_{\mathrm{test}}=4950$ in each scenario. Macro-AUC denotes the macro-averaged one-vs-rest ROC-AUC. Lower Brier scores indicate greater probability accuracy. ECE denotes the 10-bin confidence-based expected calibration error. Results are based on the common held-out synthetic-unit split and are rounded to four decimal places; values below 0.00005 are displayed as 0.0000. Task 1 contains direct predictor–target overlap and is not a forecasting evaluation.

Across the Weak, Moderate, and Strong scenarios, multinomial logistic regression produces 16, 5, and 10 test errors, respectively. The decision tree, bagged trees, and adaptive boosting produce no test errors in the Weak and Strong scenarios and one error each in the Moderate scenario.

Classification performance and probability quality should nevertheless be distinguished. Adaptive boosting achieves near-perfect class assignment, but its Brier score remains between 0.4450 and 0.4571 and its ECE between 0.5435 and 0.5508. By comparison, multinomial logistic regression, the decision tree, and bagged trees yield substantially lower probability errors.

The majority-class benchmark further illustrates why ECE should not be interpreted as a measure of discrimination. Its ECE is small because its constant confidence approximately matches the observed frequency of the class that it always predicts, despite a Macro-AUC of 0.5000 and a balanced accuracy of 0.3333. Near-perfect class assignment therefore does not establish that the corresponding scores are well-calibrated risk probabilities. More fundamentally, the near-ceiling Task 1 results are the expected consequence of supplying classifiers with the variables that directly define the target.

4.3. Task 2: Strict One-Year-Ahead Rolling-Origin Evaluation

Task 2 evaluates ${\mathbf{X}}_{i,t-1}\to Y_{i,t}$ over eight untouched test outcome years from 2018 through 2025. For each year, each classifier is trained only on outcomes through $t-2$, configuration selection uses outcome year $t-1$, and the test predictors come exclusively from $t-1$. Each annual test set contains all 3000 synthetic units.

Before performance was summarized, the saved Task 2 outputs were checked against the partition, alignment, and probability requirements specified in Section 3. Table 13 shows that all 24 scenario–year folds satisfy the intended temporal relation, sample-size requirements, and feature roles. The 360,000 prediction rows are model-specific outputs generated from shared test observations; they do not represent 360,000 independent institutional cases.

Table 13. Execution and output verification for Task 2.

Verification Domain	Observed Evidence	Result
Temporal folds	24 scenario–year folds covering every test year from 2018 through 2025	PASS
Temporal alignment	Predictor year $+1=$ outcome year for every test record	PASS
Samples and features	3000 validation and 3000 test units per fold; 12 lagged predictors (8 financial and 4 non-financial)	PASS
Saved outputs and keys	360,000 model-specific probability rows; no duplicate scenario–year–unit–model keys	PASS
Probability and count checks	All probabilities finite and within $[0,1]$; maximum row-sum error $4.44\times {10}^{-16}$; all confusion-matrix totals equal 3000	PASS

Note: SPASS indicates agreement with the prespecified fold, feature, and output-verification rules. Multiple model outputs for the same test record are analytical predictions rather than additional observational units.

Metrics were first calculated separately for each test year. Table 14 reports the mean and sample standard deviation of the eight annual values. Strict temporal ordering produces results far below the near-ceiling Task 1 values. Under Weak persistence, the highest mean annual Macro-F1 is 0.480 and the highest Macro-AUC is 0.696. Under Moderate persistence, these values increase to 0.549 and 0.747, respectively. Under Strong persistence, they reach 0.633 and 0.824. The same monotonic ordering occurs for every fitted classifier. This pattern is consistent with the prespecified mechanism: as more of the preceding latent state is retained, year-$t-1$ measurements contain more information about the year-t rule-based outcome.

Table 14. Strictone-year-ahead Task 2 performance across the three temporal-persistence scenarios.

Scenario	Model	Macro-F1	Macro-AUC	Multiclass Brier
Weak	Majority-class benchmark	0.210 (0.004)	0.500 (0.000)	0.648 (0.016)
	Multinomial logistic regression	0.480 (0.025)	0.696 (0.012)	0.579 (0.025)
	Decision tree	0.458 (0.030)	0.673 (0.012)	0.597 (0.031)
	Bagged trees	0.471 (0.019)	0.685 (0.008)	0.587 (0.022)
	Adaptive boosting	0.476 (0.019)	0.680 (0.008)	0.624 (0.006)
Moderate	Majority-class benchmark	0.201 (0.005)	0.500 (0.000)	0.659 (0.016)
	Multinomial logistic regression	0.549 (0.019)	0.747 (0.007)	0.544 (0.023)
	Decision tree	0.532 (0.014)	0.725 (0.011)	0.568 (0.023)
	Bagged trees	0.540 (0.020)	0.738 (0.005)	0.553 (0.021)
	Adaptive boosting	0.543 (0.021)	0.732 (0.009)	0.613 (0.007)
Strong	Majority-class benchmark	0.173 (0.005)	0.500 (0.000)	0.672 (0.016)
	Multinomial logistic regression	0.633 (0.021)	0.824 (0.012)	0.458 (0.030)
	Decision tree	0.625 (0.017)	0.809 (0.015)	0.475 (0.031)
	Bagged trees	0.629 (0.022)	0.820 (0.011)	0.464 (0.030)
	Adaptive boosting	0.633 (0.020)	0.815 (0.011)	0.583 (0.008)

Note: Values are annual means (sample standard deviations) across eight test years (2018–2025; 3000 units per year). Boldface identifies the classifier with the strongest joint Macro-F1, Macro-AUC, and Brier results within each scenario, based on the unrounded values; it does not imply universal superiority. All predictors are from year $t-1$.

No consistent advantage from greater model complexity is observed in these synthetic scenarios. Multinomial logistic regression provides the strongest joint classification, discrimination, and Brier performance in all three scenarios. Bagged trees produce similar results in the Strong condition, whereas adaptive boosting achieves a nearly identical Macro-F1 but a materially higher Brier score. Within the disclosed generator, the strength of the temporal signal therefore contributes more to the observed performance differences than the move from a linear classifier to a more complex ensemble.

Table 15 presents the annual results for multinomial logistic regression. The scenario ordering is retained in every test year for both Macro-F1 and Macro-AUC. Performance declines during the prespecified 2020 event year in all three scenarios. Relative to 2019, Macro-F1 decreases from 0.503 to 0.432 under Weak persistence, from 0.557 to 0.508 under Moderate persistence, and from 0.646 to 0.588 under Strong persistence. This temporary reduction is consistent with the designed event shift and demonstrates why mean performance alone is insufficient to characterize temporal robustness.

Table 15. AnnualTask 2 results for multinomial logistic regression.

Test Outcome Year	Weak	Moderate	Strong
2018	0.507/0.709	0.566/0.755	0.634/0.812
2019	0.503/0.709	0.557/0.751	0.646/0.828
2020	0.432/0.690	0.508/0.735	0.588/0.804
2021	0.451/0.675	0.531/0.739	0.618/0.816
2022	0.487/0.700	0.553/0.756	0.644/0.839
2023	0.482/0.686	0.554/0.746	0.645/0.831
2024	0.490/0.707	0.561/0.749	0.642/0.835
2025	0.487/0.694	0.560/0.747	0.650/0.831
Mean (SD)	0.480 (0.025)/0.696 (0.012)	0.549 (0.019)/0.747 (0.007)	0.633 (0.021)/0.824 (0.012)

Note: Each cell reports Macro-F1/Macro-AUC. Each year contains 3000 untouched test predictions per scenario. The 2020–2021 rows correspond to the prespecified event period; neither the model nor its decision rule was refitted using the test outcomes.

Because the fitted-classifier trajectories are nearly coincident within several scenario–year combinations, Figure 3 presents the annual Macro-F1 results as annotated performance matrices rather than overlapping line plots. Rows identify prediction-producing methods, and columns identify test outcome years. The four fitted classifiers share a fixed color scale across all three panels, while each cell reports the exact model–year value. The substantially lower majority-class benchmark is retained as a visually neutral reference row so that it does not compress the color differences among the fitted classifiers. The short rules above the 2020 and 2021 columns mark the prespecified event period.

Figure 3. Annual Macro-F1 under strict one-year-ahead rolling-origin evaluation in the controlled synthetic panels. Rows denote prediction-producing methods, and columns denote test outcome years. The four fitted classifiers share a common color scale across all persistence conditions, and the numerical labels give the exact model–year values. The majority-class benchmark is shown as a visually neutral reference row. The short header rules above the 2020 and 2021 columns identify the prespecified event period.

4.4. Task Contrast, Prespecified Checks, and Evidential Boundaries

The central empirical contrast is between a same-year positive-control audit with direct target overlap and a temporally ordered evaluation using lagged predictors. For multinomial logistic regression, Macro-F1 decreases from 0.9968 in Task 1 to a mean annual value of 0.4799 in Task 2 under Weak persistence, from 0.9990 to 0.5488 under Moderate persistence, and from 0.9978 to 0.6332 under Strong persistence. The corresponding absolute reductions are 0.5169, 0.4502, and 0.3647.

Across the three nonlinear classifiers, the Macro-F1 reduction between Tasks 1 and 2 ranges from 0.517 to 0.542 in the Weak condition, from 0.457 to 0.468 in the Moderate condition, and from 0.367 to 0.375 in the Strong condition. By contrast, the majority-class benchmark remains near its no-discrimination level in both tasks.

Table 16 reports the contrast for every fitted classifier. The gap narrows as temporal persistence increases but remains substantial even in the Strong scenario. Because Tasks 1 and 2 differ intentionally in both predictor timing and target overlap, this gap is a diagnostic contrast rather than an estimate of ordinary out-of-sample performance decay.

Table 16. Macro-F1 contrast between Tasks 1 and 2.

Scenario	Model	Task 1	Task 2	Absolute Reduction
Weak	Multinomial logistic regression	0.9968	0.4799	0.5169
	Decision tree	1.0000	0.4576	0.5424
	Bagged trees	1.0000	0.4712	0.5288
	Adaptive boosting	1.0000	0.4761	0.5239
Moderate	Multinomial logistic regression	0.9990	0.5488	0.4502
	Decision tree	0.9998	0.5322	0.4676
	Bagged trees	0.9998	0.5399	0.4599
	Adaptive boosting	0.9998	0.5432	0.4566
Strong	Multinomial logistic regression	0.9978	0.6332	0.3647
	Decision tree	1.0000	0.6254	0.3746
	Bagged trees	1.0000	0.6291	0.3709
	Adaptive boosting	1.0000	0.6330	0.3670

Note: Task 1 is the held-out Macro-F1 obtained from the common unit-level split. Task 2 is the mean annual Macro-F1 across the eight rolling-origin test years. Absolute reduction equals Task 1 minus Task 2. The majority-class benchmark is omitted because it does not use the predictors and serves only as a no-discrimination reference.

Figure 4 visualizes the same contrast as paired horizontal estimates. For each classifier, the square gives the Task 1 held-out Macro-F1, the circle gives the Task 2 mean annual Macro-F1, and the connecting segment represents the absolute reduction. The common zero-to-one scale preserves comparability across scenarios, while the paired design avoids the redundancy of plotting 12 near-ceiling Task 1 bars. The large separation between the endpoints should not be interpreted as conventional performance decay because the two tasks use intentionally different information sets and temporal relations.

Figure 4. Macro-F1 contrast between contemporaneous deterministic-rule recovery (Task 1) and strict one-year-ahead rolling-origin evaluation (Task 2). Squares and circles denote Task 1 and Task 2, respectively, and the connecting segments represent the absolute reductions. Task 1 includes the four target-defining components, whereas Task 2 uses only lagged predictors. The comparison is diagnostic and does not represent two equivalent forecasting tasks.

Both prespecified checks are confirmed within the boundaries of the controlled experiment. The Positive-Control Expectation is confirmed because every fitted classifier recovers the target-inclusive Task 1 relation with a held-out Macro-F1 of at least 0.9968, whereas the majority-class benchmark remains at the no-discrimination level. The Persistence Mechanism Check is also confirmed: for each of the four fitted classifiers, both mean annual Macro-F1 and mean annual Macro-AUC in Task 2 follow the prespecified ordering Weak < Moderate < Strong.

Macro-F1 is the primary class-sensitive outcome for both tasks, and Macro-AUC is the co-primary discrimination outcome for Task 2. The Brier score, ECE, balanced accuracy, and annual model–year results provide complementary evidence and are not treated as additional prespecified checks.

Table 17 summarizes the supported and unsupported interpretations. Three conclusions follow directly from the reported evidence. First, the regenerated panels satisfy the prespecified structural, accounting, label, and temporal-pairing checks. Second, near-perfect same-year classification is explained by deterministic target overlap. Third, strict one-year-ahead performance is lower but increases systematically with the amount of temporal persistence embedded in the generator. The experiment does not establish real-world HEI prediction, external validity, causal risk factors, or an operational early-warning system.

Table 17. Empirical findings and their evidential boundaries.

Result	Observed Evidence	Supported Interpretation	Not Supported
Panel integrity	No duplicate, missing, accounting, range, or label failures across 99,000 scenario records.	The regenerated synthetic panels are internally coherent under the disclosed audit.	Accuracy of external records or national representativeness.
Scenario mechanism	Ordered lag-one correlations and increasing High-Risk persistence from Weak to Strong.	The intended persistence contrast was successfully generated.	Empirical HEI autocorrelation or sectoral prevalence.
Task 1	Fitted-classifier Macro-F1 of 0.9968–1.0000.	Same-year recovery of the deterministic rule and diagnosis of target overlap.	Prospective warning, causal inference, or independent outcome validation.
Task 2	Best mean Macro-F1 of 0.480, 0.549, and 0.633 and Macro-AUC of 0.696, 0.747, and 0.824 across the Weak, Moderate, and Strong conditions.	Temporal predictability increases with prespecified persistence.	Performance in actual HEIs or a universal deployment threshold.
Model comparison	Multinomial logistic regression matches or exceeds the more complex models on the joint Task 2 metrics.	Greater complexity is not required for the strongest results under this synthetic mechanism.	Universal superiority of a specific algorithm.
Probability quality	Adaptive boosting combines near-perfect Task 1 class assignment with high Brier and ECE values.	Classification performance and probability calibration are distinct properties.	Raw scores as calibrated institutional risk probabilities.

No population-level significance test is attached to the scenario contrasts. The three scenarios share the same synthetic units, common random-number stream, years, and outcome mechanism, while the eight annual test sets are repeated evaluations within a designed experiment. Accordingly, the analysis emphasizes the direction, magnitude, annual stability, and two prespecified mechanism checks. Treating the 360,000 saved prediction rows as independent experimental units would constitute pseudoreplication and produce anti-conservative inference.

In summary, the results answer the two research questions without extending beyond the synthetic design. Direct target overlap permits almost complete recovery of the same-year rating rule in Task 1. Once this overlap is removed and all predictors are lagged, Task 2 performance falls markedly and varies with the persistence deliberately embedded in the generator. The results therefore support a methodological conclusion about leakage control, temporal ordering, and reporting discipline; real-world HEI forecasting and deployment remain untested.

5. Discussion

This controlled synthetic-panel experiment distinguishes the contemporaneous recovery of a deterministic rating rule from strict one-year-ahead classification. Section 5.1 interprets the predictor–target overlap examined in Task 1. Section 5.2 discusses the temporal persistence and the prespecified event shift in Task 2. Section 5.3 considers the model complexity, calibration, and model choice. Section 5.4 then answers the two research questions and evaluates the two prespecified checks. Section 5.5 and Section 5.6 conclude by defining the methodological contribution, evidential boundaries, and staged validation requirements.

Three findings guide the discussion. First, the near-ceiling Task 1 results are explained by the inclusion of the same-year variables used to construct the rating. Second, Task 2 performance is substantially lower but increases with the temporal persistence deliberately embedded in the generator. Third, higher model complexity does not consistently improve the temporally ordered results, while accurate class assignment does not necessarily imply reliable probability estimates. These findings concern the methodological validity within the disclosed synthetic experiment. They do not establish financial-risk prevalence, prospective prediction, or operational early-warning performance in actual higher education institutions (HEIs).

5.1. Target Overlap and the Interpretation of Task 1

Task 1 addresses a diagnostic rather than a prospective question. Its four same-year predictors—leverage, budget execution, credit rating, and historical defaults—are the components used by the hierarchical rating rule defined in Section 3.4. The held-out Macro-F1 values of 0.9968–1.0000 in Table 12 therefore demonstrate that the fitted classifiers can recover the disclosed rule on unseen synthetic units. They do not show that the models anticipate a later independently observed adverse outcome.

This distinction is important, because an unseen test set does not by itself eliminate target overlap. When the variables defining an outcome are reused as predictors, a model may learn the target-construction function rather than an independent risk relationship. Task 1 consequently functions as a positive-control audit: its near-ceiling results confirm that the deterministic label is recoverable and illustrate the apparent performance that direct predictor–target overlap can produce.

Differences among the Task 1 classifiers are consistent with the structure of the rating rule. The rule combines threshold conditions, a High-Risk override, and a residual Medium-Risk category. Tree-based classifiers can represent such partitions directly, whereas multinomial logistic regression approximates them through linear decision surfaces. The virtually exact tree-based results and the small residual error of logistic regression are therefore unsurprising. They demonstrate function approximation under the synthetic rule, not the inherent superiority of nonlinear models for real-world HEI forecasting.

The probability results provide a separate warning. Adaptive boosting attains almost exact class assignment but has Brier scores of 0.4450–0.4571 and ECE values of 0.5435–0.5508. Conversely, the majority-class benchmark has a small ECE despite a Macro-AUC of 0.5000 and balanced accuracy of 0.3333. These results show that class assignment, discrimination, and probability calibration measure different properties. Accurate predicted classes do not justify interpreting raw classifier scores as institutional risk probabilities.

5.2. Temporal Persistence and the Interpretation of Task 2

Task 2 imposes the prospective information boundary ${\mathbf{X}}_{i,t-1}\to Y_{i,t}$. All predictors precede the outcome year, the configuration selection uses the immediately preceding validation year, and each test year from 2018 to 2025 remains untouched until final evaluation. The design therefore removes the direct same-year target overlap present in Task 1 and aligns the analysis with a fixed one-year horizon.

The contrast between Tasks 1 and 2 is the principal empirical comparison. Table 16 shows a substantial Macro-F1 reduction for every fitted classifier and persistence condition. This reduction is not a conventional generalization gap, because the two tasks intentionally use different predictor roles and temporal relations. Task 1 assesses the recovery of a known contemporaneous rule, whereas Task 2 assesses next-year classification under a prespecified persistence mechanism. Their difference shows how both performance and evidential meaning change when the information boundary changes; it is not a transportable estimate of prospective performance loss.

Within Task 2, the performance improves monotonically, as the persistence coefficient increases from $\rho =0.20$ to $0.50$ and $0.80$. The strongest mean annual Macro-F1/Macro-AUC results rise from 0.480/0.696 under Weak persistence to 0.549/0.747 under Moderate persistence and 0.633/0.824 under Strong persistence. The same ordering occurs for each of the four fitted classifiers. This pattern is consistent with the generator: higher persistence carries more of the preceding latent state into the next year, making the lagged predictors more informative about the subsequent rule-derived rating. Because this contrast was specified by design, it should not be interpreted as an empirical estimate of persistence in actual HEIs.

The annual results also demonstrate sensitivity to the prespecified event shift. Multinomial-logistic Macro-F1 decreases in the 2020 test year under all three persistence conditions, as shown in Table 15. This designed decline does not estimate the effect of a real-world disruption. It instead shows that averages across years can conceal deterioration when the generating environment changes. Empirical HEI studies would therefore require prespecified stress-period analysis, recalibration checks, and evaluation on later untouched periods.

Although the Task 2 archive contains 360,000 method-specific prediction rows, the repeated evaluation units are the eight test years within each scenario. The methods predict the same unit–year records, and the three scenarios reuse the same synthetic units under a common-random-number design. Treating the prediction rows as independent observations would therefore create pseudoreplication. The appropriate interpretation rests on the effect magnitude, annual variation, and the prespecified scenario ordering rather than inflated record-level precision.

5.3. Model Complexity, Probability Quality, and Model Choice

The temporally ordered results do not show a general advantage for the more complex classifiers. Multinomial logistic regression provides the strongest joint Macro-F1, Macro-AUC, and Brier performance in all three scenarios. Bagged trees approach its performance under Strong persistence, while adaptive boosting attains a nearly identical Macro-F1 but produces a higher Brier error. Within the disclosed mechanism, the strength of the temporal signal has a larger influence on performance than the move from a linear classifier to a more flexible ensemble.

This finding does not establish the universal superiority of logistic regression. The generator contains a limited set of latent processes, fixed functional relations, and a deterministic target rule. Authentic institutional data may contain nonlinear lag structures, measurement error, missingness, policy changes, reporting delays, and more heterogeneity. The defensible conclusion is narrower: model complexity cannot compensate for an invalid information boundary and should be justified through locked temporal evaluation rather than same-year performance alone.

Model selection for early-warning use would also require more than maximizing the Macro-F1. Discrimination, calibration, temporal stability, false-negative costs, review capacity, and alert burden address different aspects of decision use. The present experiment estimates neither an operational threshold nor the consequences of missed and false alerts. None of the Task 2 scores should therefore be converted into an institutional alert without external calibration and decision-focused validation.

Table 18 connects the designed mechanisms, observed results, defensible interpretations, and methodological implications. It synthesizes the evidence reported in Section 4 and does not constitute an additional statistical analysis.

Table 18. Mechanism-to-inference synthesis of the principal findings.

Designed Feature	Observed Evidence	Defensible Interpretation	Methodological Implication
Deterministic target overlap	Task 1 Macro-F1 is 0.9968–1.0000.	The fitted classifiers recover a same-year rating function whose defining components are supplied as predictors.	Target provenance and predictor–target overlap must be audited before high performance is described as forecasting.
Strict one-year ordering	Task 2 Macro-F1 is substantially below Task 1 for every fitted classifier and scenario.	Removing same-year overlap changes both performance and the claim supported by the experiment.	Predictor availability and forecast origin are part of the estimand rather than reporting details.
Prespecified persistence	The strongest Macro-F1 increases from 0.480 to 0.549 and 0.633, while Macro-AUC increases from 0.696 to 0.747 and 0.824.	Lagged predictability increases with the amount of temporal information built into the generator.	Synthetic scenario contrasts verify mechanism sensitivity but do not estimate real-sector persistence.
Prespecified event shift	The logistic-regression Macro-F1 trajectory declines in 2020 under all three persistence conditions.	Average performance can conceal deterioration when the generating environment changes.	Temporal validation should include stress periods and later untouched evaluation windows.
Alternative model flexibility	Logistic regression matches or exceeds the more complex classifiers on the joint Task 2 metrics.	Additional complexity is unnecessary under the disclosed synthetic mechanism.	Algorithm choice should follow temporally valid comparative evidence rather than same-year rankings.
Separate probability metrics	Near-perfect adaptive boosting classification coexists with poor Task 1 Brier and ECE values.	Correct class assignments do not imply reliable probability estimates.	Calibration and operating-threshold validation are required before scores can support decisions.

5.4. Research-Question Answers and Prespecified Checks

The findings provide direct but task-bounded answers to the two research questions. For RQ1, the four fitted classifiers recover the deterministic contemporaneous rating rule with held-out Macro-F1 values of 0.9968–1.0000 when its defining components are supplied as same-year predictors. The result establishes rule recoverability under the disclosed positive-control design; it does not establish independent early-warning validity.

For RQ2, both the mean annual Macro-F1 and Macro-AUC increase from Weak to Moderate and Strong persistence for each of the four fitted classifiers under strict one-year-ahead evaluation. The absolute performance and probability quality remain dependent on the classifier and scenario. The majority-class benchmark is excluded from this ordering assessment, because it does not use the predictors, and its Macro-AUC remains at the no-discrimination value of 0.500.

The Positive-Control Expectation is therefore confirmed: every fitted classifier achieves the prespecified near-ceiling pattern in Task 1 and substantially outperforms the majority-class benchmark. The Persistence Mechanism Check is also confirmed, because each fitted classifier follows

$$\mathrm{Weak}<\mathrm{Moderate}<\mathrm{Strong}$$

for both the mean annual Macro-F1 and Macro-AUC in Task 2. These checks confirm rule recoverability and sensitivity to the persistence mechanism embedded in the generator. They do not establish that actual HEIs exhibit the same persistence levels, performance values, or model ordering. Balanced accuracy, Brier score, ECE, and annual trajectories provide complementary interpretation and are not additional prespecified checks.

5.5. Methodological Contribution and Implications for HEI Research

The contribution of the study is methodological rather than sector-estimating or system-validating. By disclosing the rating rule, persistence parameters, event mechanism, seeds, model configurations, and verification procedures, the experiment demonstrates three related risks of misinterpretation: deterministic rule recovery may be mistaken for prediction; non-temporal evaluation may leave the information boundary unresolved; and accurate class assignment may be mistaken for calibrated probability estimation.

The experiment also clarifies the evidential role of multidimensional indicators. Financial, operational, and contextual variables may all be substantively relevant to HEI monitoring, but their inclusion does not by itself establish predictive value. Predictive evidence depends on when each field becomes available, whether it overlaps with the outcome definition, and whether it improves performance on a later untouched period. The indicator breadth and prospective validity are therefore distinct properties.

For future HEI applications, analytical outputs should remain subordinate to evidence quality and accountable human governance. Automated processes may support authorized data checks, version control, case routing, deadline tracking, and audit recording. However, the present study does not evaluate the workload, institutional acceptance, decision quality, intervention effects, or operational safety. An RPA or alert-routing layer is therefore a future implementation option, not an empirically validated component of the present study.

5.6. Boundary Conditions and Staged Validation Agenda

Six conditions delimit the findings. First, the panels are fully synthetic: no row represents an identifiable HEI, and the scenario parameters are design constants rather than empirical estimates. Second, the outcome is a deterministic three-class rating rather than an independently adjudicated event such as default, liquidity failure, credit deterioration, or formal intervention. Third, the scenarios simplify institutional heterogeneity, measurement error, missingness, policy change, and reporting delay. Fourth, the 2020–2021 disturbance is a prespecified stress mechanism rather than an estimate of an actual event. Fifth, Task 2 contains only eight annual test periods. Sixth, no external cohort, prospective workflow, or decision consequence is evaluated.

These conditions delimit rather than negate the internal computational finding. The audit in Table 10 establishes that the 99,000 generated records satisfy the disclosed structural, accounting, label, range, and temporal checks. It does not establish measurement validity, sectoral prevalence, or model transportability. Internal computational validity and external empirical validity must therefore remain separate claims.

The common-random-number design also means that the three scenarios are controlled paired experimental conditions rather than independent institutional samples. Their monotonic ordering is useful for verifying the sensitivity to the intended persistence mechanism but cannot support a population interpretation.

Table 19 translates these boundaries into a cumulative validation agenda. The sequence matters, because operational piloting cannot substitute for establishing data provenance, outcome independence, temporal validity, external performance, and decision usefulness.

Table 19. Staged agenda for empirical and operational validation.

Stage	Required Validation Design	Minimum Evidence
1. Measurement	Construct a source-indexed institutional panel with dated releases, reconciled accounts, explicit missing-data rules, institutional crosswalks, and a predictor-availability register.	Document traceability, reconciliation rates, annual sample flow, and verified information timing.
2. Outcome	Replace rule-derived ratings with independently adjudicated post-origin events, such as default, material credit deterioration, liquidity failure, persistent budget imbalance, or formal intervention.	Demonstrate outcome reliability, incidence, forecast horizon, temporal ordering, and absence of predictor–target overlap.
3. Model	Freeze preprocessing, model selection, and thresholds before rolling-origin evaluation and testing on a locked external institutional or regional cohort.	Report class-specific performance, Macro-F1, PR-AUC or ROC-AUC, Brier score, calibration, uncertainty, and stress-period stability.
4. Decision	Prespecify review capacity, error costs, escalation rules, and human override, and prospectively compare model-assisted review with the existing process.	Assess net benefit, alert burden, false-negative consequences, subgroup performance, reviewer agreement, and documented overrides.
5. Operations	Pilot authorized routing and audit functions with access control, exception handling, rollback, monitoring, and human closure authority.	Evaluate timeliness, workload, completion, auditability, acceptance, safety events, and performance drift.

These stages are cumulative rather than interchangeable. Successful workflow automation cannot compensate for predictor–target overlap, incorrectly timed information, poor external calibration, or the absence of demonstrated decision benefit. The present experiment therefore supports a methodological distinction between contemporaneous rule-recovery auditing and temporally ordered evaluation. Empirical validity, decision usefulness, and operational performance remain to be established.

6. Conclusions

This study used a controlled synthetic-panel experiment to distinguish contemporaneous recovery of a deterministic HEI financial-risk rating from strict one-year-ahead classification. Task 1 served as a positive-control audit in which the four components defining the same-year rating remained available to the classifiers. Task 2 used lagged predictors and rolling-origin testing under prespecified Weak, Moderate, and Strong persistence conditions. The two tasks therefore addressed different inferential questions rather than successive stages of the same forecasting exercise.

Task 1 produced held-out Macro-F1 values of 0.9968–1.0000 across the four fitted classifiers, confirming that the disclosed deterministic rule is recoverable when its defining information is supplied. Under Task 2, the strongest mean annual Macro-F1/Macro-AUC results increased from 0.480/0.696 under Weak persistence to 0.549/0.747 under Moderate persistence and 0.633/0.824 under Strong persistence. Each fitted classifier followed the same scenario ordering, although higher model complexity provided no consistent advantage. The Positive-Control Expectation and Persistence Mechanism Check were therefore confirmed within the controlled experiment. Neither result establishes equivalent relationships or predictive performance in actual HEIs.

The principal contribution is methodological. The study demonstrates that classification performance must be interpreted in relation to target construction, predictor–target overlap, information availability, forecast origin, temporal evaluation, and probability quality. Near-perfect same-year classification may reflect the recovery of a constructed rule rather than prediction of an independent future outcome. Likewise, temporally ordered performance within a synthetic mechanism does not establish external predictive validity, institutional decision benefit, or operational early-warning effectiveness.

The conclusions remain bounded by the fully synthetic panels, deterministic rule-derived outcome, simplified institutional heterogeneity, prespecified persistence and event mechanisms, eight annual test periods, and the absence of an external institutional cohort or prospective decision evaluation. Future research should construct source-indexed longitudinal HEI data with verified availability dates and independently adjudicated post-origin outcomes. Locked rolling-origin and external validation should then evaluate discrimination, calibration, temporal robustness, and decision usefulness under prespecified review capacities and error costs.

Only after these empirical and decision-validation stages should authorized RPA functions for technical checking, case routing, deadline tracking, and audit recording be piloted prospectively. Interpretation, escalation, corrective action, override, and case closure must remain under accountable human authority. The present study therefore establishes an internally reproducible methodological demonstration, not validated financial-risk prediction, institutional decision benefit, or operational RPA effectiveness in real HEIs.