Post Hoc Error Correction for Missing Classes in Deep Neural Networks

Abstract

This paper presents a novel post hoc error correction method that enables deep neural networks to recognize classes that were completely excluded during training. Unlike traditional approaches requiring full model retraining, our method uses hidden layer representations from any pre-trained classifier to detect and correct errors on missing categories. We demonstrate the approach on facial emotion recognition using the RAF-DB dataset, systematically excluding each of the seven emotion classes from training. The results show correction gains of up to 0.811 for excluded classes while maintaining 99% retention on known classes in the best setup. The method provides a computationally efficient alternative to retraining when new categories emerge after deployment.

1. Introduction

Accurate facial emotion recognition has become a cornerstone of the broader domain of affective computing, which aims to develop systems capable of recognizing, interpreting, and responding to human emotions through computational methods [1,2,3]. However, a persistent challenge in facial expression classification is the presence of imbalanced class distributions, where certain emotions are significantly underrepresented [4,5]. Class imbalance often degrades model generalization, leading to biased predictions toward majority classes and poorer recognition of subtle or rare emotions such as fear or disgust [6].

To address such imbalances, researchers have proposed several strategies. Resampling techniques—such as oversampling minority classes or undersampling majority classes—are commonly employed to rebalance datasets, though they risk introducing redundancy or discarding critical information [7,8]. Another approach involves loss reweighting, where algorithms adjust the learning process by assigning greater weight to underrepresented classes. Methods such as class-balanced loss, focal loss, and label-distribution-aware margins exemplify this strategy [9,10,11]. Hybrid approaches combining reweighting and resampling have also been shown to improve robustness in imbalanced emotion datasets [12].

Deep neural networks enhanced with attention mechanisms have demonstrated considerable promise in emotion recognition tasks. For instance, models integrating attention within CNN–LSTM architectures enable the system to focus on discriminative facial regions, thereby improving performance [13,14,15,16]. Furthermore, attention-based frameworks have been explicitly applied to handle uncertainty and class imbalance by emphasizing learning from underrepresented samples [17,18]. Self-attention and Transformer-based architectures have also gained traction, often outperforming traditional CNN-based approaches in scenarios involving noisy or imbalanced data [19,20].

Alongside these developments, a complementary line of research emphasizes AI error correction without retraining legacy systems. This paradigm, pioneered by Gorban and colleagues, introduces external correctors—lightweight modules that detect risky cases and propose alternative decisions. The theoretical foundation rests on stochastic separation theorems, which demonstrate that in high-dimensional spaces, even simple classifiers (such as Fisher’s linear discriminants) can separate error cases from correctly classified ones with high probability [21]. This leads to practical one-shot correction methods capable of improving system reliability without iterative retraining [22,23]. More recent advances generalize these results to fine-grained, clustered data distributions, showing that families of multi-correctors can handle complex structures and even facilitate new class discovery in high-dimensional settings [24,25]. These methods have been validated on benchmarks such as CIFAR-10, where external correctors successfully amended errors of deep convolutional networks and supported incremental learning of novel object classes.

Despite these advances, most studies focus on models trained directly on the full emotion set—with balanced or rebalanced inputs—rather than exploring error correction strategies that recover missing or excluded classes. The present study addresses this gap using an LSTM-based model with an attention mechanism, trained on subsets of emotion classes, followed by error correction applied to the omitted (seventh) class.

In the experiments, the model is trained on all possible combinations of six out of seven emotion classes and then evaluated on its ability to correct predictions for the held-out class. The results show that while the success rate of correction varies across classes, even small or rare emotions benefit, with key metrics—such as precision and recall—improving for minority classes. These findings align with recent evidence suggesting that auxiliary learning and reconstruction-based strategies can significantly enhance minority class recognition in affective computing tasks [26,27].

Recent advances in handling imbalanced datasets for deep learning have explored various innovative approaches. Data augmentation techniques [28] have proven effective for increasing minority class representation, while generative adversarial networks (GANs) [29] can synthesize realistic samples for underrepresented emotions. Metric learning approaches [30] and contrastive learning [31] have shown promise in learning discriminative features for minority classes. Additionally, curriculum learning strategies [32] and self-supervised pre-training [33] have demonstrated improved performance on imbalanced facial expression datasets. Our work complements these approaches by focusing on post-hoc correction rather than rebalancing during training.

Novelty and Contributions: The main contribution of this work is a post-hoc error correction framework that differs from existing approaches in several key aspects:

• It operates after training, requiring no retraining of the base model.
• It can recover completely missing classes excluded from the original training set.
• It is model-agnostic and can work with any pre-trained classifier architecture.
• It maintains performance on original classes while adding capability to recognize new categories.

This finding is particularly relevant for real-world applications where detecting rare or atypical emotional signals is critical—for example, in anti-fraud systems or security contexts. The demonstrated ability to improve recognition of low-frequency classes underscores the practical value of the proposed method in scenarios requiring robust classification under skewed class distributions.

2. Methodology

This section provides the mathematical formulation of our correction approach. The key insight is that hidden representations of neural networks contain sufficient information to detect and correct errors on missing classes.

2.1. The Corrector Model

Let us consider the problem of multiclass classification in the feature space. Let

$$\mathcal{X}\subseteq {\mathbb{R}}^n,\mathcal{Y}=\{1,2,\dots ,K\}.$$

The basic classifier is defined by the mapping

$$f:\mathcal{X}\to \mathcal{Y},x\mapsto f\left(x\right),$$

which assigns a class label $f\left(x\right)$ to each point $x\in \mathcal{X}\in \mathcal{Y}$.

In this case, the model additionally induces the display

$$\mathbf{e}:\mathcal{X}\to {\mathbb{R}}^m,x\mapsto \mathbf{e}\left(x\right),$$

where $\mathbf{e}\left(x\right)$ is interpreted as an embedding vector obtained during the inference process.

To build a correction mechanism, we introduce the binary mapping

$$g:{\mathbb{R}}^m\to \{0,1\},\mathbf{z}\mapsto g\left(\mathbf{z}\right),$$

where $g\left(\mathbf{z}\right)=0$ corresponds to familiar data, and $g\left(\mathbf{z}\right)=1$ corresponds to unfamiliar (out-of-distribution) data.

Based on this, we define a generalized classifier with correction:

$$h:\mathcal{X}\to \mathcal{Y}\cup \left\{\mathrm{new}\mathrm{class}\right\},$$

$$h\left(x\right)=\left\{\begin{array}{cc}f\left(x\right),\hfill & \mathrm{if}g\left(\mathbf{e}\right(x\left)\right)=0,\hfill \\ \mathrm{new}\mathrm{class},\hfill & \mathrm{if}g\left(\mathbf{e}\right(x\left)\right)=1.\hfill \end{array}\right.$$

Thus, the h function combines two mechanisms:

• Standard classification by means of f on a set of known classes $\mathcal{Y}$;
• Identification of new or missing classes using the g corrector.

This separation makes it possible to formalize the error correction task of a pre-trained system as a combination of the classification task and the task of detecting unknown data. In particular, the proposed approach provides

• Resilience to the appearance of classes that were missing at the learning stage;
• The ability to process unbalanced samples by highlighting “rare” or excluded classes;
• The ability to extend the functionality of the original f classifier to the more general h system.

2.2. Metrics for Evaluating the Quality of a Proofreader

Let us give the test set $\mathcal{D}={\left\{(x_j,y_j)\right\}}_{j=1}^N$, where $y_j\in \mathcal{Y}=\{1,\dots ,K\}$. Let us refer to the predictions of the basic (without correction) model f as ${\widehat{y}}_j^{\left(f\right)}$, and the predictions of the system with correction h as ${\widehat{y}}_j^{\left(h\right)}$.

For a fixed class $i\in \mathcal{Y}$, we assume

$$J_i=\{j\in \{1,\dots ,N\}:y_j=i\},N_i=\left|J_i\right|.$$

Let us introduce a set of indexes of class i samples that have been correctly classified by one system or another:

$$A_i=\{j\in J_i:{\widehat{y}}_j^{\left(f\right)}=i\}(\mathrm{correct}\mathrm{by}f),$$

$$B_i=\{j\in J_i:{\widehat{y}}_j^{\left(h\right)}=i\}(\mathrm{correct}\mathrm{by}h).$$

We also denote the error sets (for class i):

$${\overline{A}}_i=J_i\setminus A_i(\mathrm{errors}f\mathrm{on}\mathrm{class}i),$$

$${\overline{B}}_i=J_i\setminus B_i(\mathrm{errors}h\mathrm{on}\mathrm{class}i).$$

2.3. Basic One-Class Metrics

First-class accuracy (synonyms of the True Positive Rate (TPR) for each class):

$${\mathrm{TPR}}_i^{\left(f\right)}={\displaystyle \frac{|A_i|}{N_i}},{\mathrm{TPR}}_i^{\left(h\right)}={\displaystyle \frac{|B_i|}{N_i}}.$$

Relative and absolute increment are determined as follows:

$${\mathrm{\Delta }}_i={\mathrm{TPR}}_i^{\left(h\right)}-{\mathrm{TPR}}_i^{\left(f\right)},R_i={\displaystyle \frac{{\mathrm{TPR}}_i^{\left(h\right)}}{{\mathrm{TPR}}_i^{\left(f\right)}+\epsilon }},$$

where $\epsilon >0$ is a small regularization for stability at zero base.

Conservation and harm metrics for already correctly classified samples.

The characteristic we are interested in is how the corrector affects samples that have already been correctly classified by f. We introduce the following concepts:

Retention—the proportion of samples of class i that were correctly recognized by f, which remained correctly recognized even after applying the corrector:

$${\mathrm{Ret}}_i={\displaystyle \frac{|A_i\cap B_i|}{|A_i|}}(\mathrm{if}\left|A_i\right|>0).$$

Harm—the proportion of i class samples correctly recognized by f that became erroneous after correction:

$${\mathrm{Harm}}_i={\displaystyle \frac{|A_i\setminus B_i|}{|A_i|}}=1-{\mathrm{Ret}}_i.$$

Correction gain—the proportion of previously misclassified Class i samples that became correct after correction:

$${\mathrm{Gain}}_i={\displaystyle \frac{|B_i\setminus A_i|}{|{\overline{A}}_i|}}(\mathrm{if}\left|{\overline{A}}_i\right|>0).$$

2.4. Metrics of “Spillover” and False Positives on Other Classes

To take into account how the corrector affects the incorrect reassignment of the label towards class i, we define

$${\mathrm{Spill}}_i={\displaystyle \frac{\left|\right\{j:y_j\ne i,{\widehat{y}}_j^{\left(h\right)}=i\left\}\right|}{N-N_i}}$$

as the proportion of samples from other classes that were classified as class i by the h system (i.e., “false positive for i”). Accordingly, for f,

$${\mathrm{FPR}}_i^{\left(f\right)}={\displaystyle \frac{\left|\right\{j:y_j\ne i,{\widehat{y}}_j^{\left(f\right)}=i\left\}\right|}{N-N_i}},$$

$${\mathrm{FPR}}_i^{\left(h\right)}={\displaystyle \frac{\left|\right\{j:y_j\ne i,{\widehat{y}}_j^{\left(h\right)}=i\left\}\right|}{N-N_i}}.$$

The false positive percentage change is

$$\mathrm{\Delta }{\mathrm{FPR}}_i={\mathrm{FPR}}_i^{\left(h\right)}-{\mathrm{FPR}}_i^{\left(f\right)}.$$

2.5. Summary Indicators and Standards

The following aggregates are offered for a summary assessment:

$$\overline{\mathrm{Ret}}={\displaystyle \frac{1}{K}}\sum _{i=1}^K{\mathrm{Ret}}_i,\overline{\mathrm{Harm}}={\displaystyle \frac{1}{K}}\sum _{i=1}^K{\mathrm{Harm}}_i,$$

$$\overline{\mathrm{Gain}}={\displaystyle \frac{1}{K}}\sum _{i=1}^K{\mathrm{Gain}}_i,\overline{\mathrm{\Delta }\mathrm{FPR}}={\displaystyle \frac{1}{K}}\sum _{i=1}^K\mathrm{\Delta }{\mathrm{FPR}}_i.$$

One can also weigh the frequencies of the classes $N_i/N$, if desired, to get a “micro” version of the metrics:

$${\mathrm{Ret}}^{\left(\mathrm{w}\right)}=\sum _{i=1}^K{\displaystyle \frac{N_i}{N}}{\mathrm{Ret}}_i,\mathrm{etc}.$$

2.6. Interpretation of Metrics

• ${\mathrm{Ret}}_i$ close to 1 means that the corrector does not spoil the already correctly recognized samples of class i.
• ${\mathrm{Harm}}_i$ demonstrates the share of “collateral damage” for already correct predictions; the lower the value the better.
• ${\mathrm{Gain}}_i$ shows the corrector’s ability to restore previously erroneous examples of this class.
• $\mathrm{\Delta }{\mathrm{FPR}}_i>0$ indicates an increase in false positive assignments to class i (side effect).

3. Data

For analysis, the RAF-DB (Real-world Affective Faces Database) dataset was selected, containing data on the emotions of people of different ages [34]. This dataset was downloaded from the Kaggle website [35].

Examples of images are shown in Figure 1. The dataset contains 15,339 facial images labeled with basic and composite emotions by 40 independent annotators. The label mapper, which shows the face label and the emotion it encodes, is shown in

Table 1. Correspondence between labels and emotions in the RAF-DB dataset (basic emotions).

		Data Split
Label	Emotion	Train	Test	Correct	Sum
1	Surprise	814	379	426	1619
2	Fear	166	88	101	355
3	Disgust	449	220	208	877
4	Happiness	2974	1506	1477	5957
5	Sadness	1216	628	616	2460
6	Anger	435	216	216	867
7	Neutral	1615	798	791	3204
All emotions		7669	3835	3835	15,339

. The images are highly diverse in features such as age, gender, ethnicity, head rotation, lighting conditions, and the presence of partial occlusions (e.g., glasses, beard, or self-occlusion). In addition, the images may contain filters and special effects, which makes the dataset particularly suitable for emotion recognition tasks in near-real-world settings.

Distributions of the proportions of classes in each subset are given in Figure 2.

4. Proposed Correction Method

Our method consists of two independent components that work in tandem to enable post hoc error correction for missing classes shown in Figure 3.

4.1. Base Model

We employ a CNN-LSTM architecture with an attention mechanism for facial emotion recognition. This choice was motivated by our goal to leverage a powerful, state-of-the-art architecture specifically designed for this task. Our implementation is based on the CNN-LSTM facial expression recognition method fused with a two-layer attention mechanism [36], which has demonstrated superior performance.

However, it is crucial to emphasize that any pre-trained classifier can serve as the base model within our framework, highlighting its generalizability. The chosen architecture processes input images through convolutional layers to extract spatial features, which are then reinterpreted as sequential data for the LSTM component. The integrated two-layer attention mechanism dynamically weights the importance of different facial regions across both spatial and temporal dimensions, allowing the model to focus on the most discriminative features for accurate emotion classification.

4.2. Corrector Module

The core innovation of our approach lies in the corrector module, which operates independently of the base model’s architecture:

• Feature Extraction: Collect hidden representations from intermediate layers of the base model, including convolutional feature maps, LSTM hidden states, attention weights, and pre-classification embeddings.
• Corrector Training: Train a gradient boosting classifier (XGBoost) on these extracted features to learn patterns that distinguish between correctly classified samples and potential errors, particularly for classes excluded during base model training.
• Inference Pipeline: During deployment, the corrector analyzes the base model’s hidden representations and can override predictions when it detects high-confidence patterns of excluded classes.

Key Advantage: While we demonstrate the method using CNN-LSTM for emotion recognition, the correction approach is fundamentally model-agnostic. It relies solely on the hidden representations that are available in any deep learning architecture (ResNet, Vision Transformers, etc.), making it applicable across various domains beyond facial emotion recognition.

This separation of concerns allows for the following:

• Computational Efficiency: Lightweight corrector training vs. full model retraining.
• Flexibility: The same correction framework is applicable to different base architectures.
• Incremental Learning: The ability to add new class recognition without modifying deployed models.
• Preservation of Existing Performance: Minimal impact on already learned categories.

The method effectively creates a hybrid system where the base model handles routine classification, while the corrector specializes in identifying and rectifying errors related to previously unseen or underrepresented classes.

5. Technical Implementation of the Experiment

To solve the problem of classifying emotions in facial images, a custom neural network model was developed that combines convolutional layers for feature extraction, a recurrent component (LSTM) for modeling the spatial–sequential structure, and an attention mechanism for identifying the most informative features.

The feature sequence from the convolutional block is processed by an LSTM layer. The LSTM’s role is to model non-linear spatial dependencies and contextual relationships across different facial regions. For instance, it can learn that the combination of features representing an open mouth (from the mouth region) and raised eyebrows (from the eye region) is a strong indicator of “Surprise”.

The output of the LSTM is then passed through a multi-headed attention mechanism. The attention mechanism computes a weighted sum of the entire sequence, allowing the model to focus adaptively on the most salient facial features for the final emotion classification decision. This is particularly useful for emphasizing critical areas like the eyes and mouth while suppressing less informative regions. The architecture diagram is shown in Figure 4 and consists of the main components shown in

Table 2. General architecture of the proposed model and hyperparameters used in the experiments.

Block/Section	Layer/Parameter	Input Dim/Symbol	Output Dim/Value	Description
Conv block	Conv1	$[B,C_{in},H,W]$	$[B,C_1,H/2,W/2]$	Conv(3×3) + BN + ReLU + MaxPool
	Conv2	$[B,C_1,H/2,W/2]$	$[B,C_2,H/4,W/4]$	Conv(3×3) + BN + ReLU + MaxPool
	Conv3	$[B,C_2,H/4,W/4]$	$[B,C_3,H^{\prime },W^{\prime }]$	Conv(3×3) + BN + ReLU + MaxPool
Recurrent block	Reshape	$[B,C_3,H^{\prime },W^{\prime }]$	$[B,T,F]$	$T=H^{\prime }{W}^{\prime }$, $F=C_3$
	LSTM	$[B,T,F]$	$[B,T,H_{lstm}]$	LSTM($F\to H_{lstm}$), batch_first=True
Attention block	MH Attention	$[B,T,H_{lstm}]$	$[B,T,H_{att}]$	Multi-head attention with h heads
	Aggregation	$[B,T,H_{att}]$	$[B,H_{att}]$	Mean/weighted pooling or CLS token
Classifier	FC1	$[B,H_{att}]$	$[B,H_{fc}]$	Linear + ReLU
	Dropout	$[B,H_{fc}]$	$[B,H_{fc}]$	Dropout(p)
	FC2	$[B,H_{fc}]$	$[B,N]$	Output layer
Hyperparameters
Input channels	–	$C_{in}$	3	RGB input
Conv block channels	–	$C_1,C_2,C_3$	32, 64, 128	Number of filters per conv layer
LSTM hidden size	–	$H_{lstm}$	128	Hidden dimension of LSTM
Attention dimension	–	$H_{att}$	128	Attention embedding size
Number of heads	–	h	4	Multi-head attention heads
FC hidden size	–	$H_{fc}$	128	Fully connected hidden units
Dropout rate	–	p	0.2	Dropout probability
Number of classes	–	N	7	Output classes

.

6. Data Augmentation

Before training, all input images are processed using a unified data augmentation pipeline designed to improve the model’s robustness to geometric and photometric variations. The transformations applied are

• Random horizontal flip with a probability of $0.5$;
• Random rotation within $\pm {30}^{\circ }$;
• Random resized cropping to a fixed resolution of $100\times 100$ with a scaling factor sampled from $[0.8,1.0]$;
• Color jitter with brightness and contrast variations up to $0.3$;
• Conversion to tensor and normalization with mean and standard deviation $(0.5,0.5,0.5)$.

This augmentation strategy increases sample diversity and improves generalization by making the model more robust to variations in pose, illumination, scale, and local image statistics.

7. Training Methodology

We address class imbalance using weighted cross-entropy:

$$w_i={\displaystyle \frac{N}{n_i}},\mathcal{L}=-\sum _{i=1}^C{w}_i{y}_{i}log\left({\widehat{y}}_i\right),$$

where N is the dataset size, $n_i$ is the number of samples in class i, and ${\widehat{y}}_i$ are predicted probabilities. Training is performed with Adam, early stopping, and systematic hyperparameter search. Early stopping with patience of 30 epochs prevents overfitting. The dynamics of metrics in the learning process during the implementation of the described scenario is shown in Figure 5.

The explored ranges include the learning rate $\{{10}^{-3}, 5\times {10}^{-4}, {10}^{-4}\}$, batch size $\{32, 64, 128\}$, and dropout $\{0.1, 0.2, 0.3\}$. The optimal configuration uses a learning rate of ${10}^{-3}$, batch size of 64, and dropout of $0.2$. Since we needed to develop a strong and high-quality model to conduct a qualitative correction experiment, hyperparameters were selected based on maximizing classification accuracy in the validation sample. We attribute this relative stability to the model’s capacity and the rich, high-dimensional feature space it generates. This richness provides a robust foundation that is not easily destabilized by small parameter variations. The primary effect of hyperparameter tuning within this stable regime was on the dynamics and speed of convergence rather than on the final performance plateau.

8. Correction Mechanism

After training the base model, we extract multi-level representations (convolutional features, LSTM states, attention outputs, and pre-classification embeddings), forming dense descriptors for each sample. A gradient-boosted corrector (XGBoost, learning rate of $0.1$, max depth of 5, subsample of $0.9$, and 500 estimators) is trained on these descriptors to detect likely errors and adjust predictions when the confidence score $g\left({\mathbf{z}}_i\right)$ falls below threshold $\tau$. At this step, we also maximized the classification accuracy on the hyperparameter plane in order to obtain the strongest corrector. This mechanism acts as a meta-classifier, capturing feature interactions not fully modelled by the base network.

9. Results

This section presents the quantitative results of applying the proposed classification error correction method.

9.1. Metrics

Experimental Protocol: To evaluate our correction method, we employ a systematic class exclusion strategy. For each experiment, one emotion class is completely removed from training data, simulating real-world scenarios where new categories emerge after model deployment. We then measure the corrector’s ability to recover these excluded classes.

Key Evaluation: The primary metrics are Gain (

Table 3. Summary metrics of proofreader quality for each class. $\overline{\mathrm{\Delta }\mathrm{FPR}}$—the average change in the false positive error after applying the corrector. $\overline{\mathrm{Gain}}$—the proportion of previously erroneous examples of the corrected class that were classified correctly.

Label	$\overline{\mathbf{\Delta }\mathbf{FPR}}$	$\overline{\mathbf{Gain}}$
1	0.015	0.517
2	0.001	0.239
3	0.029	0.482
4	0.095	0.811
5	0.023	0.242
6	0.002	0.204
7	0.072	0.590

) (ability to recover missing classes) and Retention (preservation of existing performance in

Table 4. Values of Harm and Retention metrics for all classes. The Harm metric reflects the proportion of examples that were correctly classified by the original model but became erroneous after applying the corrector. The Retention metric shows the proportion of correct predictions that remained after the correction. The values on the main diagonal are underlined and correspond to preserving/distorting predictions within the same class. The Retention and Harm metrics are also highlighted in red, which are higher and lower than the average, respectively.

Retention
	Corrector
Label	1	2	3	4	5	6	7
1	1.000	1.000	1.000	0.963	0.995	0.997	0.971
2	0.898	1.000	0.977	0.989	1.000	0.977	1.000
3	1.000	1.000	1.000	0.977	1.000	1.000	0.945
4	0.999	0.999	0.999	1.000	0.997	0.999	0.975
5	0.992	1.000	0.995	0.876	1.000	1.000	0.881
6	0.981	0.995	1.000	0.903	0.968	1.000	0.977
7	0.982	0.999	0.997	0.955	0.957	1.000	1.000
Average	0.973	0.999	0.996	0.940	0.995	0.996	0.963
Harm
	Corrector
Label	1	2	3	4	5	6	7
1	0.000	0.000	0.000	0.037	0.005	0.003	0.029
2	0.102	0.000	0.023	0.011	0.000	0.023	0.000
3	0.000	0.000	0.000	0.023	0.000	0.000	0.055
4	0.001	0.001	0.001	0.000	0.003	0.001	0.025
5	0.008	0.000	0.005	0.124	0.000	0.000	0.119
6	0.019	0.005	0.000	0.097	0.032	0.000	0.023
7	0.018	0.001	0.003	0.045	0.043	0.000	0.000
Average	0.021	0.001	0.005	0.048	0.012	0.004	0.036

).

The analysis was carried out using the metrics Retention, Harm, changes in Gain, and $\mathrm{\Delta }\mathrm{FPR}$, as well as the integral performance characteristics listed below.

From the results shown in Table 4, it can be seen that the values of Retention remain high (close to unity) in all cases, which indicates that most of the correct predictions of the basic model are preserved. The values of Harm, on the contrary, record cases when the corrector introduces distortions. From Table 4, it can be seen that the corrector causes the most harm to the recognition of Fear(2) when correcting Surprise(1), Sadness(5) and Anger(6) when correcting Happiness(4), and Sadness(5) when correcting Neutral(7).

Summary indicators of the corrector’s effectiveness are given in Table 3. There is a positive trend for all classes Gain, which confirms the corrector’s ability to improve the completeness of the classification of the excluded class. The growth of $\mathrm{\Delta }\mathrm{FPR}$ remains relatively moderate, which makes the increase in Gain statistically significant. The greatest increases in quality are observed for ≪Happiness≫(4), ≪Surprise≫(1), ≪Disgust≫(3), and ≪Neutral≫(7), with greater increases observed for ≪Fear≫(2), ≪Sadness≫(5), and ≪Anger≫(6).

Table 5. Comparison of classification accuracy when using a corrector for different combinations of training classes. Values on the main diagonal represent accuracy for classes excluded from training—without correction, these would be 0, demonstrating the corrector’s ability to recover missing classes. The maximum values by row (the best prediction option for this class) are shown with underscores. The “Model 7 classes” column shows the accuracy of the original model without correction. Column P shows the value characterizing the effectiveness of the retraining correction relative to the case of complete retraining. The color coding reflects the quality of the compensation: green—high efficiency, orange—moderate, red—low.

Predict	Model 6 Classes + Corrector							Model 7 Classes	P
Label	1	2	3	4	5	6	7
1	+0.51	0.77	0.79	0.84	0.75	0.76	0.80	0.78	0.55
2	0.55	+0.24	0.06	0.43	0.47	0.45	0.39	0.43	0.56
3	0.38	0.50	+0.48	0.98	0.98	0.97	0.95	0.74	0.65
4	0.99	0.99	0.99	+0.77	0.86	0.90	0.92	0.90	0.86
5	0.69	0.71	0.72	0.71	+0.24	0.72	0.80	0.78	0.31
6	0.68	0.66	0.63	0.63	0.73	+0.20	0.63	0.67	0.30
7	0.77	0.76	0.73	0.79	0.87	0.71	+0.59	0.90	0.66

shows the results of experiments to exclude classes from the training sample. In different cases, the correction allows for varying degrees of partial compensation for the loss of information about the excluded class.

For a formal assessment of the corrector’s contribution to the final quality of the classification, we introduce the corrector power indicator P, which is defined as the ratio of classification accuracy with correction to accuracy without correction:

$$P={\displaystyle \frac{{\mathrm{Accuracy}}_{\mathrm{with}\mathrm{corr}}}{{\mathrm{Accuracy}}_{\mathrm{without}\mathrm{corr}}}}.$$

Table 5 shows the values of the strength indicator of the corrector P. The best result was recorded for the class ≪Happiness≫ ($P=0.86$), while for the classes ≪Sadness≫ and ≪Anger≫, the values of P are significantly lower ($P\approx 0.30$). This indicates a pronounced class dependence of the method’s effectiveness: for some emotions, correction successfully restores accuracy, while for others, the increase is lower. Thus, the error correction method is a promising tool. It is capable of increasing the stability of multiclass classifiers in conditions of incomplete and unbalanced samples, but its effectiveness depends on the class structure and requires further research.

9.2. Grad-CAM Analysis of LSTM Model

To gain insight into the regions of the face that drive the model’s predictions, we applied the Grad-CAM method to our trained LSTM-based network. The resulting activation maps highlight the facial areas most relevant for each emotion class.

As shown in Figure 6, the network focuses on the following regions:

• Happiness: primarily cheeks and forehead.
• Joy: eyebrows and lips.
• Fear: eyebrows and cheeks.
• Disgust: eyes and mouth.
• Sadness: eyes and mouth.
• Anger: teeth and eyebrows.
• Neutral: distributed across the face; no single region is dominant.

These results are consistent with psychological studies of facial expressions and demonstrate that the model leverages key facial regions corresponding to human-recognizable emotion cues.

10. Computational Efficiency and Resource Requirements

A key advantage of the proposed post hoc correction method is its computational efficiency compared to full model retraining.

Table 6. Computational efficiency and resource usage of the proposed method on NVIDIA RTX 4060 and 64 gb RAM.

Component	Training Time	Inference Time/Image	Memory Usage
Base Model (CNN-LSTM)	∼4 h per model	15–20 ms	∼45 MB
Corrector (XGBoost)	∼10 min per corrector	<1 ms	2–3 MB
Full Model Retraining	4–6 h per model	15–20 ms	∼45 MB

summarizes the main metrics for training time, inference time, and memory usage.

As shown in Table 6, the corrector can be trained in a fraction of the time required for full model retraining while adding minimal inference overhead and negligible memory footprint. This ensures that the method scales efficiently with additional classes and remains suitable for real-time applications.

11. Discussion

Unlike traditional methods for handling class imbalance (e.g., focal loss and SMOTE) that operate during training, our method works in post hoc applications without modifying the base model. Unlike fine-tuning that requires full retraining, we avoid updating model parameters, making our approach suitable for deployed systems. While we demonstrated our method using CNN-LSTM, the correction approach is fundamentally model-agnostic. It relies on hidden representations that are available in any deep learning architecture (ResNet, Transformers, etc.), making it applicable across various domains beyond facial emotion recognition.

However, in the general case, correction turns out to be less effective than complete retraining and does not allow us to reach the level of a fully trained model. In turn, building a corrector is computationally much easier than completely retraining the model, and the proposed method allows one to create many correctors that implement a complex, nonlinear stack model. Further development of the approach may go in the following directions:

• Using more compact embeddings;
• Scaling experiments on large samples to increase stability;
• Developing adaptive correctors that take into account the semantic and statistical relationships of classes.

To better position our contribution within the state of the art, we contrast the proposed method with several alternative strategies. Full model retraining on data that includes the new class represents the most straightforward approach but requires access to the complete original dataset and substantial computational resources, and it risks catastrophic forgetting of previously learned classes. Our method, requiring only the pre-trained model and a small correction dataset, is orders of magnitude more efficient (Table 6) and preserves original class performance by design, as evidenced by high retention metrics.

Fine-tuning only the last layers of a pre-trained model offers greater efficiency than full retraining but still modifies the base model’s parameters through backpropagation and can degrade performance on original tasks. In contrast, our corrector operates as an additive, non-invasive module that leaves the base model unchanged. Post hoc class imbalance techniques such as loss re-weighting (e.g., focal loss) or sampling methods (e.g., SMOTE) are applied during training to improve minority class recognition but remain ineffective when a class is completely absent during training. Our approach specifically addresses this missing class scenario by leveraging the geometric properties of high-dimensional embeddings, as formalized by stochastic separation theorems.

The proposed post hoc error correction method aligns with stochastic separation theorems and multi-correctors from high-dimensional “postclassical” data analysis [21,22,24], enabling error handling on excluded classes via hidden representations without retraining, as demonstrated on RAF-DB with 0.811 gains on unseen emotion and 99% retention on a known emotion in the best setup. Unlike retraining, which demands full data access and epochs-long optimization risking resets on legacy performance, this leverages quasiorthogonality and “blessing of dimensionality” for low-computed, reversible corrections on granular cluster distributions. Fine-tuning offers partial adaptation but introduces forgetting and requires new labels, while class-imbalance techniques like resampling aid minorities yet falter on fully absent categories without separation theorems.

The proposed corrector introduces a trade-off: while it recovers missing classes (quantified by Gain), it can also misclassify previously correct predictions, as captured by the Harm metric (Table 4). A detailed analysis of these errors provides valuable insight into the method’s limitations and the semantic challenges of emotion recognition. The most pronounced Harm occurs in specific, interpretable patterns:

Fear (2) is most often harmed when correcting for Surprise (1). This aligns with psychological and visual similarity: both “Fear” and “Surprise” share wide eyes and raised eyebrows. The corrector, trained to identify patterns of the excluded “Surprise” class, may over-assign this label to ambiguous instances of “Fear” that lie close to the decision boundary in the embedding space. Anger (6) is frequently harmed when correcting for Happiness (Label 4). While these emotions are conceptually opposite, their intense expressions can share similar facial muscle tension and furrowing. The corrector for the highly distinctive “Happiness” class might misinterpret such high-activation “Anger” embeddings as out-of-distribution patterns for the trained set.

These failure modes reveal that the corrector’s effectiveness is constrained by the discriminative quality of the base model’s embeddings. We see the largest problem with emotion pairs that are visually or semantically confusable (e.g., fear/surprise). Future work could focus on making the corrector more conservative for such ambiguous cases or incorporating confidence thresholds to minimize harmful over-corrections.

12. Limitations and Future Work

Limitations:

• Performance depends on the discriminative power of the base model’s hidden representations;
• Storage of intermediate features is required during inference;
• Effectiveness varies with the visual distinctiveness of excluded classes.

Future Work:

• Extension to incremental learning with multiple new classes;
• Investigation of feature compression techniques for efficiency;
• Application to other domains such as medical imaging or security systems.

13. Conclusions

The results show that the proposed error correction method can significantly improve the quality of recognition of excluded classes while maintaining high accuracy on already known data. However, a significant dependence of efficiency on the nature of a particular class has been revealed.

High values of Retention indicate that the basic structure of the model’s predictions is preserved, and the risk of quality deterioration for the initial classes remains low. At the same time, the values Harm and $\mathrm{\Delta }\mathrm{FPR}$ demonstrate that the corrector can introduce undesirable distortions, especially in the case of emotions that are similar in feature space. Thus, the method has a positive effect but requires careful adjustment.

Experiments on class exclusion show that the corrector works especially successfully for classes with pronounced interclass similarity (for example, ≪Happiness≫), whereas for classes less related to others (for example, ≪Sadness≫ and ≪Anger≫), the effectiveness is noticeably lower.