=============================================================
Significance analysis: Random Forest vs. Single Best Solver
=============================================================
Instances (N): 97

1. Lenient accuracy (credit for any tied-best method; Table 8 criterion)
   Random Forest      : 67.0%
   Single Best Solver : 30.9%

2. Strict single-label accuracy (vs. canonical label only)
   Random Forest      : 57.7%
   Single Best Solver : 30.9%

3. McNemar's test on paired correctness (lenient criterion)
   RF correct, SBS wrong (b): 35
   RF wrong, SBS correct (c): 0
   Both correct              : 30
   Both wrong                : 32
   Continuity-corrected chi^2(1) = 33.029, p = 9.081e-09
   Exact binomial McNemar test   : p = 5.821e-11

4. Balanced accuracy and macro-F1 (vs. canonical label)
   Random Forest      : balanced accuracy = 36.1%, macro-F1 = 34.5%
   Single Best Solver : balanced accuracy = 12.5%, macro-F1 = 9.4%

   Per-class recall, Random Forest:
  class recall
  cmaes  0.000
 ea4eig  0.480
    rde  0.700
     sa  0.000
  sparq  0.625

5. Distribution of the canonical training label (actual_best)

 sparq ea4eig    rde     sa  cmaes 
    48     25     20      3      1 

6. Distribution of Random Forest predictions

 sparq ea4eig    rde 
    41     31     25 

=============================================================
Done. Results saved to: C:/Users/admin/Desktop/article/significance_analysis_results.txt 
=============================================================
