Gradient-Based Multiple Robust Learning Calibration on Data Missing-Not-at-Random via Bi-Level Optimization

Abstract

Recommendation systems (RS) have become integral to numerous digital platforms and applications, ranging from e-commerce to content streaming field. A critical problem in RS is that the ratings are missing not at random (MNAR), which is due to the users always giving feedback on items with self-selection. The biased selection of rating data results in inaccurate rating prediction for all user-item pairs. Doubly robust (DR) learning has been studied in many tasks in RS, which is unbiased when either a single imputation or a single propensity model is accurate. In addition, multiple robust (MR) has been proposed with multiple imputation models and propensity models, and is unbiased when there exists a linear combination of these imputation models and propensity models is correct. However, we claim that the imputed errors and propensity scores are miscalibrated in the MR method. In this paper, we propose a gradient-based calibrated multiple robust learning method to enhance the debiasing performance and reliability of the rating prediction model. Specifically, we propose to use bi-level optimization to solve the weights and model coefficients of each propensity and imputation model in MR framework. Moreover, we adopt the differentiable expected calibration error as part of the objective to optimize the model calibration quality directly. Experiments on three real-world datasets show that our method outperforms the state-of-the-art baselines.

1. Introduction

Recommendation systems (RS) is an effective tool to address the problem of information overload and has been widely used in e-commerce, social media, and entertainment [1]. RS aims to predict user preferences for items based on collected historical interaction data [2,3]. However, the collected data cannot include all ratings from users to items, and the ratings are missing not at random (MNAR) due to users’ self-selection behavior, i.e., users can choose the item to rate freely, which is also known as selection bias problem [4,5]. The MNAR problem indicates that the collected dataset is not representative of the target population of interest (all user-item pairs), and the training distribution differs from the target test distribution. Ignoring such distributional shift will inevitably lead to sub-optimal recommendation performance [2,3,6]. To address the MNAR problem, one line of previous research proposed to use error imputation-based (EIB) methods, which first impute the missing ratings and then train the prediction model based on both observed and imputed ratings [6,7]. Additionally, another category of methods leverages propensity scores, which computes the probability of an event being observed, to reweight the observed ratings and align the distribution of observed data with the target population [3,8]. Furthermore, Doubly Robust (DR) method combines the error imputation and the inverse propensity re-weighting to achieve double robustness, which means the DR estimator achieves unbiasedness if either the imputed errors or the learned propensities are correct [2,9,10]. Furthermore, the Multiple Robust (MR) method is proposed to mitigate inaccuracies in single-model propensity scores or error imputations found in DR method [11]. By considering multiple candidate propensity and imputation models, MR estimator achieves unbiasedness if any of the propensity models, imputation models, or a linear combination of these models accurately estimate the true propensities or prediction errors.

However, we argue that the imputed errors and estimated propensity scores are miscalibrated in the existing MR method, which cannot reflect the ground-truth likelihood of the correctness of the true error or true propensity. For instance, if we have 100 user-item pairs with estimated propensity scores equal to 0.2, there should be exactly 20 ratings being observed and 80 ratings being unobserved. Although previous study has proposed to adopt calibration experts to calibrate the single propensity model and imputation model in DR estimator [12], this approach cannot be directly extended to the MR estimator, as calibrating each model individually is expensive and unreasonable due to the unbiasedness condition of MR in terms of linear combinations is not considered. Furthermore, the calibration metric previously used in [12] is non-differentiable and cannot be directly optimized.

To fill this gap, we propose the calibrated multiple robust learning (Cali-MR) method to calibrate the linear combinations of multiple imputation models and propensity models using bi-level optimization, which aims to learn an ensemble model that simultaneously possesses strong prediction performance and calibration ability. In this bi-level optimization, we adopt differentiable expected calibration errors to quantify the calibration ability that allows it to be directly optimized. The calibrated linear combination of propensity and imputation models is then used to train the prediction model based on a joint learning algorithm. The contributions of this paper are summarized as follows.

• We propose a novel MR calibration method using bi-level optimization via calibrating the ensemble imputation and propensity models and address the non-differentiable issue by adopting differentiable expected calibration errors.
• We further propose a bi-level calibrated multiple robust learning algorithm to update the calibrated imputation models and the prediction model. To the best of our knowledge, this is the first work to perform calibration for the MR estimator.
• We conduct extensive experiments on three real-world datasets, showing the effectiveness of our method compared to the state-of-the-art debiasing methods.

2. Related Works

2.1. Debiased Recommendation

The rating missing not at random (MNAR) problem results in the distribution of the observed population being different from the target population, hindering the prediction model from learning users’ true preferences [4,5,13,14,15]. There are many methods proposed to address this issue [16,17,18,19,20,21].

Specifically, methods including the Inverse Propensity Scoring (IPS), EIB, and DR were proposed to mitigate the MNAR problem in RS [6,22]. EIB methods might produce out-of-bound predictions; while the IPS method may suffer from large variance with small propensities [23]. DR methods combine the advantages of both the EIB and IPS methods, guaranteeing unbiasedness if either the error imputation model or propensity model is correctly specified.

There have been quantities of variants of DR methods to improve the debiasing performance, such as Multi-DR [24], BRD-DR [25], SDR [26], TDR [27], CDR [28], N-DR [29], DT-DR [30], UIDR [31], and OME-DR [32]. Besides, Multiple robust (MR) [11] combines multiple imputation models and propensity models, and is unbiased when there exists a linear combination of them is correct. In addition, Liu et al. [33] use an information bottleneck-based method and Yang et al. [34] and Wang et al. [35] use adversarial learning for debiasing. However, these methods fail to consider model calibration properties.

To mitigate this issue, DCE-DR [12] is proposed to calibrate the propensity and imputation model in the DR method. However, calibrating each imputation model and propensity model in MR is expensive and unreasonable, due to the unbiasedness condition of MR based on linear combinations is not taken into account. In this paper, we propose the Cali-MR method to calibrate multiple propensity and imputation models to further enhance the debiasing performance and reliability of MR method.

2.2. Model Calibration

Calibration means that the probability associated with the predicted class label should reflect its ground truth correctness likelihood [36,37], which plays an important role in building reliable, robust AI systems, especially in safety-critical fields such as medical diagnosis [38,39], self-driving [40,41], and financial decision making [42,43]. Early research demonstrated good calibration performance of simple neural networks on binary classification tasks [44]. However, with the rapid development of deep learning techniques, recent deep and complex neural networks are no longer well-calibrated [36].

Calibration methods can be divided into the following four categories [45]: post-hoc calibration, regularization methods, uncertainty estimation, and hybrid calibration methods. Post-hoc calibration methods aim to calibrate a model after training, including non-parametric calibration histogram binning [46], isotonic regression [47], and parametric methods such as Platt scaling [48]. Regularization methods adopt penalty terms such as the $L_2$ regularization [36], entropy regularization [49], difference between confidence and accuracy [50], and calibration errors [51,52] to ensure the calibration property. Uncertainty Estimation aims to alleviate model miscalibration by injecting randomness using Bayesian neural networks [53], model ensemble [54], Monte Carlo dropout [55], and Gumbel-softmax [56] based approaches. Hybrid calibration methods combine two or more methods to achieve calibration. For example, Zhang et al. [57] combines ensemble and temperature scaling and Laves et al. [58] adopts monte-carlo dropout with temperature scaling. We conclude the categorization of model calibration methods in Figure 1.

In this paper, we adopt a differentiable expected calibration error as part of the objective to ensure the model calibration. Compared to other techniques for calibration, our approach has the following advantages. First, unlike post-hoc methods that adjust calibration after training [47,48], our method based on calibration error regularization explicitly optimizes the calibration metric during the training process. Secondly, uncertainty estimation methods such as Monte Carlo Dropout typically rely on complex sampling processes [55], resulting in high computational costs, while hybrid calibration methods which combine multiple techniques, are similarly complex and challenging to implement [57,58]. By comparison, our proposed approach is both simple and computationally efficient. Furthermore, unlike other indirect regularization methods, such as $L_2$ or entropy regularization [36,49], our method directly incorporates differentiable expected calibration error as a loss function, enabling precise optimization of the calibration objective.

3. Preliminary

3.1. Debiased Recommendation

Let $\mathcal{U}=\{u_1,\cdots ,u_m\}$ be the users set, $\mathcal{I}=\{i_1,\cdots ,i_n\}$ be the item set, and $\mathcal{D}=\mathcal{U}\times \mathcal{I}$ be the set of all user-item pairs. The rating matrix is denoted as $\mathbf{R}\in {\mathbb{R}}^{m\times n}$ with $r_{u,i}$ as element. Let $o_{u,i}\in \{0,1\}$ be the observation indicator, where $o_{u,i}=1$ indicates the rating $r_{u,i}$ is observed, otherwise is not. Define $x_{u,i}$ be the observed features. We denote the prediction model as $f_{\theta }(·)$ parameterized by $\theta$ and the predicted ratings as ${\widehat{r}}_{u,i}=f_{\theta }(x_{u,i})$. The goal is to accurately predict $r_{u,i}$ for all user-item pairs, which can be achieved by minimizing the ideal loss

$${\mathcal{L}}_{\mathrm{ideal}}(\theta )={\displaystyle \frac{1}{\left|\mathcal{D}\right|}}\sum _{(u,i)\in \mathcal{D}}\mathcal{L}(f_{\theta }(x_{u,i}),r_{u,i}):={\displaystyle \frac{1}{\left|\mathcal{D}\right|}}\sum _{(u,i)\in \mathcal{D}}{e}_{u,i},$$

where $\mathcal{L}(·,·)$ is the training loss function such as cross-entropy loss. However, in practice, we cannot obtain the complete rating matrix. We denote the set of user-item pairs with observed ratings as $\mathcal{O}=\{(u,i)\mid o_{u,i}=1\}$. Thus, the naive method optimizes the average loss over the observed user-item pairs

$${\mathcal{L}}_{\mathrm{naive}}(\theta )={\displaystyle \frac{1}{\left|\mathcal{O}\right|}}\sum _{(u,i)\in \mathcal{O}}{e}_{u,i}.$$

Due to the MNAR problem, $\mathbb{E}\left[{\mathcal{L}}_{\mathrm{naive}}(\theta )\right]\ne {\mathcal{L}}_{\mathrm{ideal}}(\theta )$. Several methods were proposed to unbiasedly estimate the ideal loss, including the EIB, IPS, DR, and their variants. Because EIB and IPS can be regarded as special cases of DR, we only introduce the DR methods here. The loss function of the vanilla DR method is formulated as

$${\mathcal{L}}_{\mathrm{DR}}(\theta )={\displaystyle \frac{1}{\left|\mathcal{D}\right|}}\sum _{(u,i)\in \mathcal{D}}\left[{\widehat{e}}_{u,i}+{\displaystyle \frac{o_{u,i}(e_{u,i}-{\widehat{e}}_{u,i})}{{\widehat{p}}_{u,i}}}\right],$$

where ${\widehat{p}}_{u,i}\triangleq \pi (x;\widehat{\alpha })$ is the estimation of propensity score $p_{u,i}=\mathrm{Pr}(o_{u,i}=1\mid x_{u,i})$, and ${\widehat{e}}_{u,i}=\mathcal{L}(m(x_{u,i};\beta ),{\widehat{r}}_{u,i})$ is the imputed error, while the imputation model is denoted as $m(x_{u,i};\widehat{\beta })$. In addition, the multiple robust (MR) considers J propensity models ${\pi }_1(x;{\widehat{\alpha }}_1),\cdots ,{\pi }_J(x;{\widehat{\alpha }}_J)$ and K imputation models $m_1(x;{\widehat{\beta }}_1),\cdots ,m_K(x;{\widehat{\beta }}_K)$. Let ${\widehat{p}}_{u,i}^j\triangleq {\pi }_j(x_{u,i};{\widehat{\alpha }}_j)$, ${\widehat{m}}_{u,i}^k\triangleq m_k(x;{\widehat{\beta }}_k)$, the loss function of MR is shown below:

$${\mathcal{L}}_{\mathrm{MR}}(\theta )={\displaystyle \frac{1}{\left|\mathcal{D}\right|}}\sum _{(u,i)\in \mathcal{D}}{\mathit{u}}^T\left(x_{u,i}\right)·\widehat{\mathit{\eta }}(\theta ),$$

where $\mathit{u}\left(x_{u,i}\right)={\left(1/{\widehat{p}}_{u,i}^1,\cdots ,1/{\widehat{p}}_{u,i}^J,{\widehat{m}}_{u,i}^1,\cdots ,{\widehat{m}}_{u,i}^K\right)}^T$ and $\widehat{\mathit{\eta }}(\theta )$ is the solution by minimizing

$${\displaystyle \frac{1}{\left|\mathcal{D}\right|}}\sum _{(u,i)\in \mathcal{D}}{o}_{u,i}{\left\{e_{u,i}-{\mathit{u}}^T\left(x_{u,i}\right)·\mathit{\eta }\right\}}^2.$$

The MR estimator is unbiased when there exists a weight $W=(w_1,w_2,\dots ,w_J,0,0,\dots ,0)$ satisfying $W\mathit{u}\left(x_{u,i}\right)=1/p_{u,i}$ or $V=(0,\dots ,0,v_1,v_2,\dots ,v_K)$ satisfying $V\mathit{u}\left(x_{u,i}\right)=e_{u,i}$ for all user-item pairs.

3.2. Calibration

A model is calibrated if its output reflects the ground-truth likelihood of correctness [37]. For the propensity model $\pi (x;\widehat{\alpha })$ and the observation indicator o, a formal definition is shown below:

$$\mathbb{E}[o\mid \pi (x;\widehat{\alpha })=\widehat{p}]=\widehat{p}\forall \widehat{p}\in [0,1].$$

For instance, if we have 100 samples with estimated propensity scores equal to 0.2, there should be exactly 20 samples being observed. Similarly, the formal definition for the calibrated imputation model $m(x;\widehat{\beta })$ is formulated below:

$$\mathbb{E}[e\mid m(x;\widehat{\beta })=\widehat{e}]=\widehat{e}\forall \widehat{e}\in \mathbb{R}.$$

To measure the miscalibration of the model, the Expected Calibration Error (ECE) metric is proposed [59]. For a propensity model $\pi (x;\widehat{\alpha })$ and imputation model $m(x;\widehat{\beta })$, the ECE is defined as follows:

$$\begin{array}{cc}\hfill ECE(\widehat{\alpha })& ={\mathbb{E}}_{\widehat{p}}\left[\left|\mathbb{E}[o\mid \pi (x;\widehat{\alpha })=\widehat{p}]-\widehat{p}\right|\right],\hfill \\ \hfill ECE(\widehat{\beta })& ={\mathbb{E}}_{\widehat{e}}\left[\right|\mathbb{E}[e\mid m(x;\widehat{\beta })=\widehat{e}]-\widehat{e}\left|\right].\hfill \end{array}$$

4. Methodology

4.1. Distinctions from Previous Work

Previous studies have proposed to calibrate the single propensity model and imputation model in DR estimator [12], and they propose to use a binning strategy to estimate the ECE metric empirically, for example for the propensity model:

$$\widehat{\mathrm{ECE}}\left(\widehat{p}\right)=\sum _{m=1}^M{\displaystyle \frac{\left|B_m\right|}{N}}\left|{\displaystyle \frac{{\sum }_{(u,i)\in B_m}{o}_{u,i}}{\left|B_m\right|}}-{\displaystyle \frac{{\sum }_{(u,i)\in B_m}{\widehat{p}}_{u,i}}{\left|B_m\right|}}\right|,$$

where $B_m$ is the predefined m-th bin and N is the corresponding number of samples in the bin.

However, how to properly calibrate multiple propensity and imputation models for MR estimators remains unexplored. A naive approach is individually calibrating each propensity and imputation model in the MR estimator. Note that this method is computationally expensive and overlooks the robust property of the MR estimator, that is, the MR estimator achieves unbiased if a linear combination of multiple candidate models is accurate.

Inspired by this, we propose to calibrate the linear combination of multiple models instead of calibrating each model individually. In addition, note that the previously used $\widehat{\mathrm{ECE}}$ involves assigning each sample to a specific hard bin, making it non-differentiable and thus unsuitable for direct incorporation into the training objective. To address this issue, we employ a soft binning strategy to develop the differentiable expected calibration error metric and leverage it to construct a regularization term that constrains the model’s calibration error, which can be used for model training. Next, we will introduce the proposed Cali-MR in detail.

4.2. Differentiable Expected Calibration Error

To address the non-differentiable problem of $\widehat{\mathrm{ECE}}$, we leverage the soft binning strategy [52], using the following differentiable expected calibration error (DECE) that allows directly optimize calibration quality to mitigate the model miscalibration. For example, the DECE for a propensity model $\pi (x;\widehat{\alpha })$ is defined as:

$$DECE\left(\widehat{\alpha }\right)={\displaystyle \frac{1}{\left|\mathcal{D}\right|}}\sum _{m=1}^M\left|\sum _{(u,i)\in \mathcal{D}}{o}_m(x_{u,i};\varphi )(o_{u,i}-{\widehat{p}}_{u,i})\right|,$$

where $o_m(x_{u,i};\varphi )=\mathbb{P}(x_{u,i}\in B_m|{\widehat{p}}_{u,i})$ denotes the probability that how likely it is that ${\widehat{p}}_{u,i}$ belongs to the m-th bin. In practice, the $o_m(x_{u,i};\varphi )$ can be logistic regression or any other model [52].

In our Cali-MR, we adopt DECE as a regularization term and develop a gradient-based learning algorithm model training. Specifically, to calibrate the linear combination of multiple models, we formalize the propensity DECE loss ${\mathcal{L}}_{\mathrm{DECE}}^p$ as follows:

$$\begin{array}{c}{\mathcal{L}}_{\mathrm{DECE}}^p(\mathbf{w};{\varphi }_p;{\alpha }_1,\dots ,{\alpha }_J)={\displaystyle \frac{1}{\left|\mathcal{D}\right|}}\sum _{m=1}^M\left|\sum _{(u,i)\in \mathcal{D}}{o}_m(x_{u,i};{\varphi }_p)(o_{u,i}-\sum _{j=1}^J{w}_j{\widehat{p}}_{u,i}^j)\right|,\end{array}$$

where $\mathbf{w}=(w_1,\cdots ,w_J)$ is a given set of weight coefficients. This loss measures the calibration error of the combination model, and minimizing this loss improves the calibration ability of the current propensity model combination under the current combination coefficients. We use a one-layer neural network with softmax activation function to model the propensity soft binning model $o_m(x_{u,i};{\varphi }_p)$ with parameters ${\varphi }_p$, where m is a pre-defined hyper-parameter.

Similarly, the imputation DECE loss ${\mathcal{L}}_{\mathrm{DECE}}^e$ under weight coefficients $\mathbf{v}=(v_1,\cdots ,v_K)$ is formalized below:

$$\begin{array}{c}{\mathcal{L}}_{\mathrm{DECE}}^e(\mathbf{v};{\varphi }_m;{\beta }_1,\dots ,{\beta }_K)={\displaystyle \frac{1}{\left|\mathcal{D}\right|}}\sum _{m=1}^M\left|\sum _{(u,i)\in \mathcal{D}}{o}_m(x_{u,i};{\varphi }_m)({\displaystyle \frac{o_{u,i}{e}_{u,i}}{{\sum }_{j=1}^J{w}_j{\widehat{p}}_{u,i}^j}}-\sum _{k=1}^K{v}_k{\widehat{e}}_{u,i}^k)\right|,\end{array}$$

where we similarly model the imputation soft binning model $o_m(x_{u,i};{\varphi }_m)$ with parameters ${\varphi }_m$ with a one-layer neural network with softmax activation function. Note that the $e_{u,i}$ is missing for user-item pairs with $o_{u,i}=0$, we reweight the observed $e_{u,i}$ using the inverse of the linear combination of the multiple propensity models.

Based on the DECE loss ${\mathcal{L}}_{\mathrm{DECE}}^p$ or ${\mathcal{L}}_{\mathrm{DECE}}^e$, we can measure the calibration quality of given multiple models and weight coefficients, and further improve the calibration ability of the current combined model using a gradient-based algorithm by minimizing such differentiable loss.

4.3. Calibrated Multiple Robust Learning

Note that in calculating the DECE loss ${\mathcal{L}}_{\mathrm{DECE}}^p$ or ${\mathcal{L}}_{\mathrm{DECE}}^e$, the coefficients $\mathbf{w}$ or $\mathbf{v}$ need to be explicitly specified. However, these coefficients are unknown during the training process of the existing MR method. Therefore, we propose using a gradient-based bi-level optimization method to solve for the optimal coefficients, parameters of the soft binning model, and multiple imputation and propensity models. In addition, we alternatively update the prediction model and the calibrated imputation model based on a joint learning algorithm.

For propensity models, the optimization objective can be formalized as follows:

$$\begin{array}{cc}\hfill & ({\alpha }_1^{*},\dots ,{\alpha }_J^{*})=arg\underset{{\alpha }_1,\dots ,{\alpha }_J}{min}{\displaystyle \frac{1}{J}}\sum _{j=1}^J{\mathcal{L}}_{p_j}({\alpha }_j)+\lambda {\mathcal{L}}_{\mathrm{DECE}}^p({\mathbf{w}}^{*})\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.{\mathbf{w}}^{*}({\alpha }_1,\dots ,{\alpha }_J)=arg\underset{{\mathbf{w}}^{*}}{min}{\mathcal{L}}_p({\mathbf{w}}^{*}({\alpha }_1,\dots ,{\alpha }_J)),\hfill \end{array}$$

(1)

where ${\mathcal{L}}_{\mathrm{DECE}}^p(\mathbf{w})$ loss is the calibration constraints defined in Section 4.2, and ${\mathcal{L}}_{p_j}({\alpha }_j)$ is the training loss for a single propensity model ${\pi }_J(x;{\widehat{\alpha }}_J)$ ensuring the accuracy of each independent propensity model, which is shown below:

$$\begin{array}{c}\hfill {\mathcal{L}}_{p_j}({\alpha }_j)={\displaystyle \frac{1}{\left|\mathcal{D}\right|}}\sum _{(u,i)\in \mathcal{D}}[-o_{u,i}·log{p}_{u,i}^j-(1-o_{u,i})·log(1-p_{u,i}^j)].\end{array}$$

${\mathcal{L}}_p(\mathbf{w}({\alpha }_1,\dots ,{\alpha }_J))$ is the loss for the combination coefficients $\mathbf{w}$, aiming to learn a set of coefficients such that the linear combination ${\sum }_{j=1}^J{w}_j{\widehat{p}}_{u,i}^j$ can accurately predict observation indicator $o_{u,i}$, which is shown below:

$$\begin{array}{c}\hfill {\mathcal{L}}_p(\mathbf{w}({\alpha }_1,\dots ,{\alpha }_J))={\displaystyle \frac{1}{\left|\mathcal{D}\right|}}\sum _{(u,i)\in \mathcal{D}}[-o_{u,i}·log(\sum _{j=1}^J{w}_j{\widehat{p}}_{u,i}^j)-(1-o_{u,i})·log(1-\sum _{j=1}^J{w}_j{\widehat{p}}_{u,i}^j)].\end{array}$$

In this gradient-based bi-level optimization, we aim to train the propensity models such that each model performs well and their linear combination is well calibrated, where the coefficients also ensure the strong prediction performance of the combined model. For practical implementation, we first assumed update coefficients $\mathbf{w}$ through optimizing ${\mathcal{L}}_p(\mathbf{w})$. Using these coefficients, we calculate the DECE loss ${\mathcal{L}}_{\mathrm{DECE}}^p(\mathbf{w})$ and combine it with the base prediction loss of each propensity model denoted as ${\mathcal{L}}_{p_j}({\alpha }_j)$ to form the final loss. This final loss is then used to update the propensity models ${\alpha }_1,\dots ,{\alpha }_J$. After that, the loss ${\mathcal{L}}_p(\mathbf{w})$ and ${\mathcal{L}}_{\mathrm{DECE}}^p(\mathbf{w})$ are used to update the combination coefficients $\mathbf{w}$ and the soft binning model ${\varphi }_p$ sequentially.

Multiple Imputation Calibration

With the calibrated propensity model ${\sum }_{j=1}^J{w}_j{\widehat{p}}_{u,i}^j$ obtained from Equation (1) by the bi-level optimization, we can further calibrate the linear combination of multiple imputation models:

$$\begin{array}{cc}\hfill & ({\beta }_1^{*},\dots ,{\beta }_K^{*})=arg\underset{{\beta }_1,\dots ,{\beta }_K}{min}{\displaystyle \frac{1}{K}}\sum _{k=1}^K{\mathcal{L}}_{e_k}({\beta }_k)+\lambda {\mathcal{L}}_{\mathrm{DECE}}^e({\mathbf{v}}^{*})\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.{\mathbf{v}}^{*}({\beta }_1,\dots ,{\beta }_K)=arg\underset{{\mathbf{v}}^{*}}{min}{\mathcal{L}}_e(v^{*}({\beta }_1,\dots ,{\beta }_K)),\hfill \end{array}$$

(2)

where ${\mathcal{L}}_{\mathrm{DECE}}^e(\mathbf{v})$ is the calibration constraints shown in Section 4.2. The naive training loss for each imputation model $m_K(x;{\widehat{\beta }}_K)$ is expressed as

$$\begin{array}{c}\hfill {\mathcal{L}}_{e_k}({\beta }_k;\theta )={\displaystyle \frac{1}{\left|\mathcal{D}\right|}}\sum _{(u,i)\in \mathcal{D}}{\displaystyle \frac{o_{u,i}{(e_{u,i}-{\widehat{e}}_{u,i}^k)}^2}{{\sum }_{j=1}^J{w}_j{\widehat{p}}_{u,i}^j}},\end{array}$$

and the loss for the combination coefficients $\mathbf{v}$ is

$$\begin{array}{c}\hfill {\mathcal{L}}_e(\mathbf{v}({\beta }_1,\dots ,{\beta }_K))=\sum _{(u,i)\in \mathcal{D}}{\displaystyle \frac{o_{u,i}{(e_{u,i}-{\sum }_{k=1}^K{v}_k{\widehat{e}}_{u,i}^k)}^2}{\left|\mathcal{D}\right|{\sum }_{j=1}^J{w}_j{\widehat{p}}_{u,i}^j}},\end{array}$$

which aims to learn a set of weight coefficients such that the linear combination of the imputation models ${\sum }_{k=1}^K{v}_k{\widehat{e}}_{u,i}^k$ can unbiasedly estimate the prediction error $e_{u,i}$.

Similar to updating the propensity models, we first use ${\mathcal{L}}_e(\mathbf{v})$ to assume update coefficients ${\mathbf{v}}^{\prime }$. Based on that, we compute the DECE loss ${\mathcal{L}}_{\mathrm{DECE}}^e({\mathbf{v}}^{\prime })$, combine it with the base training loss of each imputation model ${\mathcal{L}}_{e_k}({\beta }_k;\theta )$, and use the combined loss to update the imputation models ${\beta }_1,\dots ,{\beta }_K$. Then we adopt loss ${\mathcal{L}}_e(\mathbf{v})$ and ${\mathcal{L}}_{\mathrm{DECE}}^e(\mathbf{v})$ to update the combination coefficients $\mathbf{v}$ and the soft binning model ${\varphi }_e$ sequentially. After obtaining the updated imputation models and coefficients, we jointly train the prediction model using the standard multiple robust learning algorithm. Specifically, we use ridge regression to calculate $\widehat{\eta }$ in the MR estimator and update the prediction model based on the MR loss ${\mathcal{L}}_{\mathrm{MR}}$ using different samples. We summarize the above gradient-based bi-level learning algorithm in Algorithm 1.

Algorithm 1: Gradient-Based Bi-level Calibrated Multiple Robust Learning

5. Experiments

5.1. Datasets

To evaluate the debiasing performance, we conduct experiments on three benchmark datasets Coat (https://www.cs.cornell.edu/~schnabts/mnar/, accessed on 15 January 2025) and Yahoo! R3 (https://webscope.sandbox.yahoo.com), and KuaiRec (https://github.com/chongminggao/KuaiRec, accessed on 15 January 2025) [60], which are widely used in debiased RS with both missing not at random (MNAR) and missing at random (MAR) data. Coat dataset consists of 6960 MNAR training samples and 4640 MAR test samples derived from 290 users rating on 300 items. The Yahoo! R3 dataset includes 311,704 MNAR training samples and 54,000 MAR test samples derived from 15,400 users rating on 1000 items. Both datasets are five-scale, and following previous works [61,62,63], we binarize the ratings greater than three to 1, and others to 0. The KuaiRec dataset is collected from a video-sharing platform and contains 4,676,570 video watching ratios derived from 1411 users evaluating 3327 videos. We binarize the continuous ratios greater than two to 1, otherwise to 0.

5.2. Baselines

We compare our method with the following baselines for comprehensive evaluations:

• Naive method [64] naively optimizes the average loss over the observed user-item pairs.
• IPS method [3] reweights the observed ratings with the inverse propensity scores.
• SNIPS method [65] reweights the observed ratings with self-normalized propensity scores to further reduce the variance.
• ASIPS method [22] generates reliable pseudo-ratings to mitigate propensity estimation bias and high variance problem.
• DR method [10] combines error imputation and inverse propensity reweighting to construct a doubly robust estimator, where imputed errors are typically set based on label prior knowledge, such as the mean value of the labels.
• DR-JL method [2] further proposes modeling error imputation with neural networks and jointly learns the prediction model and imputation model.
• MRDR method [66] enhances the DR-JL method by explicitly controlling the variance of the DR estimator through imputation model learning.
• DR-BIAS method [67] enhances the DR-JL method by further reducing the bias of the DR estimator through imputation model learning.
• DR-MSE method [67] further combines MRDR method and DR-BIAS method to achieve bias-variance trade-off and control the generalization error.
• MR method [11] adopts multiple candidate propensity and imputation models to mitigate inaccuracies in single-model propensity scores or error imputation in DR methods.
• TDR and TDR-JL methods [27] correct the imputed errors with targeted learning to reduce the bias and variance simultaneously for existing DR approaches
• StableDR method [26] constructs a stabilized DR estimator that has a weaker dependence on extrapolation and is robust to small propensities by learning constrained propensity scores.
• IPS-V2 and DR-V2 methods [68] learn the propensity model which can balance some manually selected functions such as the first and second moments of the features.
• KBIPS and KBDR methods [32] further propose to conduct causal balancing in the reproducing kernel Hilbert space (RKHS) and randomly select some kernel functions to balance for propensity model learning.
• AKBIPS and AKBDR methods [32] adaptively select the kernel functions which contribute the most to reducing the estimation bias to balance for propensity model learning.
• DCE-DR and DCE-TDR method [12] propose to calibrate the single propensity model and single imputation model in DR and TDR estimators through Mixture-of-Experts technique.

5.3. Experiment Protocols and Details

We evaluate the prediction performance using three widely adopted evaluation metrics: AUC (Area Under the ROC Curve), NDCG@T (Normalized Discounted Cumulative Gain), and F1@T.

• AUC [69] is a performance metric for classifiers that measures the probability of a randomly chosen positive example being ranked higher than a randomly chosen negative one. A higher AUC score reflects better ranking performance in differentiating positive instances from negative ones.
• NDCG@T [70] evaluates ranking performance by comparing the Discounted Cumulative Gain (DCG) of the top-T results to the Ideal DCG (IDCG), producing a normalized score between 0 and 1. A higher NDCG@T implies that more relevant items are ranked towards the top.

Let $r_i$ be the relevance of the item at rank i. We first compute the DCG at rank T as well as the IDCG@T by placing the most relevant items in the optimal (ideal) order:

$$\begin{array}{c}\hfill \mathrm{DCG}@T=\sum _{i=1}^T{\displaystyle \frac{2^{r_i}-1}{{log}_2(i+1)}},\mathrm{IDCG}@T=\sum _{i=1}^T{\displaystyle \frac{2^{r_i^{*}}-1}{{log}_2(i+1)}}.\end{array}$$

where $r_i$ is the relevance of the i-th item at rank i, and $r_i^{*}$ denotes the relevance of the i-th item in the ideal ranking. NDCG@T is then defined as:

$$\begin{array}{c}\hfill \mathrm{NDCG}@T={\displaystyle \frac{\mathrm{DCG}@T}{\mathrm{IDCG}@T}}.\end{array}$$

• F1@T [71] is the harmonic mean of precision and recall computed over the top-T predictions returned by a model. A higher F1@T indicates a better trade-off between precision and recall in the top-T results.

We set T = 5 on Coat and Yahoo! R3, and T = 20 on KuaiRec. In addition, we tune learning rate in $\{0.01,0.05\}$ and weight decay in $\{1\times {10}^{-6},5\times {10}^{-6},1\times {10}^{-5},\dots ,1\times {10}^{-3},5\times {10}^{-3}\}$.We use the same hyperparameter search space and follow the results in Li et al. [32].

5.4. Performance Analysis

The experimental results are shown in

Table 1. Performance on AUC, NDCG@T, and F1@T on Coat, Yahoo! R3 and KuaiRec. The best and the second best results are bolded and underlined, where * means statistically significant results ($p-\mathrm{value}\le 0.05$) using the paired-t-test.

	Coat			Yahoo! R3			KuaiRec
Method	AUC	NDCG@5	F1@5	AUC	NDCG@5	F1@5	AUC	NDCG@20	F1@20
Naive	0.${703}_{\pm 0.006}$	0.${605}_{\pm 0.012}$	0.${467}_{\pm 0.007}$	0.${673}_{\pm 0.001}$	0.${635}_{\pm 0.002}$	0.${306}_{\pm 0.002}$	0.${753}_{\pm 0.001}$	0.${449}_{\pm 0.002}$	0.${124}_{\pm 0.002}$
IPS	0.${717}_{\pm 0.007}$	0.${617}_{\pm 0.009}$	0.${473}_{\pm 0.008}$	0.${678}_{\pm 0.001}$	0.${638}_{\pm 0.002}$	0.${318}_{\pm 0.002}$	0.${755}_{\pm 0.004}$	0.${452}_{\pm 0.010}$	0.${131}_{\pm 0.004}$
SNIPS	0.${714}_{\pm 0.012}$	0.${614}_{\pm 0.012}$	0.${474}_{\pm 0.009}$	0.${683}_{\pm 0.002}$	0.${639}_{\pm 0.002}$	0.${316}_{\pm 0.002}$	0.${754}_{\pm 0.003}$	0.${453}_{\pm 0.004}$	0.${126}_{\pm 0.003}$
ASIPS	0.${719}_{\pm 0.009}$	0.${618}_{\pm 0.012}$	0.${476}_{\pm 0.009}$	0.${679}_{\pm 0.003}$	0.${640}_{\pm 0.003}$	0.${319}_{\pm 0.003}$	0.${757}_{\pm 0.005}$	0.${474}_{\pm 0.007}$	0.${130}_{\pm 0.005}$
IPS-V2	0.${726}_{\pm 0.005}$	0.${627}_{\pm 0.009}$	0.${479}_{\pm 0.008}$	0.${685}_{\pm 0.002}$	0.${646}_{\pm 0.003}$	0.${320}_{\pm 0.002}$	0.${764}_{\pm 0.001}$	0.${476}_{\pm 0.003}$	0.${135}_{\pm 0.003}$
KBIPS	0.${714}_{\pm 0.003}$	0.${618}_{\pm 0.010}$	0.${474}_{\pm 0.007}$	0.${676}_{\pm 0.002}$	0.${642}_{\pm 0.003}$	0.${318}_{\pm 0.002}$	0.${763}_{\pm 0.001}$	0.${463}_{\pm 0.007}$	0.${134}_{\pm 0.002}$
AKBIPS	0.${732}_{\pm 0.004}$	0.${636}_{\pm 0.006}$	0.${483}_{\pm 0.006}$	0.${689}_{\pm 0.001}$	0.${658}_{\pm 0.002}$	0.${324}_{\pm 0.002}$	0.${766}_{\pm 0.003}$	0.${478}_{\pm 0.009}$	0.${138}_{\pm 0.003}$
DR	0.${718}_{\pm 0.008}$	0.${623}_{\pm 0.009}$	0.${474}_{\pm 0.007}$	0.${684}_{\pm 0.002}$	0.${658}_{\pm 0.003}$	0.${326}_{\pm 0.002}$	0.${755}_{\pm 0.008}$	0.${462}_{\pm 0.010}$	0.${135}_{\pm 0.005}$
DR-JL	0.${723}_{\pm 0.005}$	0.${629}_{\pm 0.007}$	0.${479}_{\pm 0.005}$	0.${685}_{\pm 0.002}$	0.${653}_{\pm 0.002}$	0.${324}_{\pm 0.002}$	0.${766}_{\pm 0.002}$	0.${467}_{\pm 0.005}$	0.${136}_{\pm 0.003}$
MRDR-JL	0.${727}_{\pm 0.005}$	0.${627}_{\pm 0.008}$	0.${480}_{\pm 0.008}$	0.${684}_{\pm 0.002}$	0.${652}_{\pm 0.003}$	0.${325}_{\pm 0.002}$	0.${768}_{\pm 0.005}$	0.${473}_{\pm 0.007}$	0.${139}_{\pm 0.004}$
DR-BIAS	0.${726}_{\pm 0.004}$	0.${629}_{\pm 0.009}$	0.${482}_{\pm 0.007}$	0.${685}_{\pm 0.002}$	0.${653}_{\pm 0.002}$	0.${325}_{\pm 0.003}$	0.${768}_{\pm 0.003}$	0.${477}_{\pm 0.006}$	0.${137}_{\pm 0.004}$
DR-MSE	0.${727}_{\pm 0.007}$	0.${631}_{\pm 0.008}$	0.${484}_{\pm 0.007}$	0.${687}_{\pm 0.002}$	0.${657}_{\pm 0.003}$	0.${327}_{\pm 0.003}$	0.${770}_{\pm 0.003}$	0.${480}_{\pm 0.006}$	0.${140}_{\pm 0.003}$
MR	0.${724}_{\pm 0.004}$	0.${636}_{\pm 0.006}$	0.${481}_{\pm 0.006}$	0.${691}_{\pm 0.002}$	0.${647}_{\pm 0.002}$	0.${316}_{\pm 0.003}$	0.${776}_{\pm 0.005}$	0.${483}_{\pm 0.006}$	0.${142}_{\pm 0.003}$
TDR	0.${714}_{\pm 0.006}$	0.${634}_{\pm 0.011}$	0.${483}_{\pm 0.008}$	0.${688}_{\pm 0.003}$	0.${662}_{\pm 0.002}$	0.${329}_{\pm 0.002}$	0.${772}_{\pm 0.003}$	0.${486}_{\pm 0.005}$	0.${140}_{\pm 0.003}$
TDR-JL	0.${731}_{\pm 0.005}$	0.${639}_{\pm 0.007}$	0.${484}_{\pm 0.007}$	0.${689}_{\pm 0.002}$	0.${656}_{\pm 0.004}$	0.${327}_{\pm 0.003}$	0.${772}_{\pm 0.003}$	0.${489}_{\pm 0.005}$	0.${142}_{\pm 0.003}$
StableDR	0.${735}_{\pm 0.005}$	0.${640}_{\pm 0.007}$	0.${484}_{\pm 0.006}$	0.${688}_{\pm 0.002}$	0.${661}_{\pm 0.003}$	0.${329}_{\pm 0.002}$	0.${773}_{\pm 0.001}$	0.${491}_{\pm 0.003}$	0.${143}_{\pm 0.003}$
DR-V2	0.${734}_{\pm 0.007}$	0.${639}_{\pm 0.009}$	0.${487}_{\pm 0.006}$	0.${690}_{\pm 0.002}$	0.${660}_{\pm 0.005}$	0.${328}_{\pm 0.002}$	0.${773}_{\pm 0.003}$	0.${488}_{\pm 0.006}$	0.${142}_{\pm 0.004}$
KBDR	0.${730}_{\pm 0.003}$	0.${631}_{\pm 0.005}$	0.${482}_{\pm 0.006}$	0.${682}_{\pm 0.002}$	0.${648}_{\pm 0.003}$	0.${323}_{\pm 0.002}$	0.${765}_{\pm 0.004}$	0.${460}_{\pm 0.006}$	0.${138}_{\pm 0.003}$
AKBDR	${\mathbf{0.745}}_{\pm 0.004}$	0.${645}_{\pm 0.008}$	0.493±0.007	0.${692}_{\pm 0.002}$	0.${661}_{\pm 0.002}$	0.${328}_{\pm 0.002}$	0.${782}_{\pm 0.003}$	0.${498}_{\pm 0.008}$	0.${147}_{\pm 0.003}$
DCE-DR	0.${736}_{\pm 0.006}$	0.${648}_{\pm 0.007}$	0.${489}_{\pm 0.005}$	0.${698}_{\pm 0.002}$	0.${670}_{\pm 0.002}$	0.333±0.003	0.${795}_{\pm 0.004}$	0.${512}_{\pm 0.005}$	0.${153}_{\pm 0.002}$
DCE-TDR	0.${740}_{\pm 0.004}$	0.651±0.006	0.${489}_{\pm 0.007}$	0.701±0.002	0.672±0.002	0.${331}_{\pm 0.002}$	${\mathbf{0.798}}_{\pm 0.005}$	0.514±0.006	0.155±0.002
Cali-MR	${0.741}_{\pm 0.002}$	${\mathbf{0.658}}_{\pm 0.004}^{*}$	${\mathbf{0.495}}_{\pm 0.004}$	${\mathbf{0.703}}_{\pm 0.002}^{*}$	${\mathbf{0.678}}_{\pm 0.002}^{*}$	${\mathbf{0.338}}_{\pm 0.004}^{*}$	${\mathbf{0.798}}_{\pm 0.003}$	${\mathbf{0.521}}_{\pm 0.005}^{*}$	${\mathbf{0.158}}_{\pm 0.002}^{*}$

, and we find that all the debiasing methods including both IPS-based and DR-based baselines outperform the Naive method, which demonstrates the importance of debiasing. Besides, among all baselines, AKBDR method introduces balancing kernel functions for propensity model training. This method adaptively identifies the most critical kernel functions to balance by fitting prediction errors, and well-trained propensity model effectively eliminates selection bias, thereby demonstrating strong performance on the Coat dataset. On the other hand, TDR introduces a targeted learning technique, which leverages propensity to enhance the error imputation, achieving reductions in both bias and variance. DCE-TDR further improves upon TDR by calibrating propensity model, utilizing the calibrated propensity for targeted learning, and achieving further performance gains. This calibration strategy enables DCE-TDR to deliver competitive results on the Yahoo! R3 and KuaiRec datasets. Furthermore, the proposed Cali-MR method exhibits superior overall performance and significantly outperforms existing methods on Yahoo! R3 and KuaiRec datasets. This shows that calibrating multiple propensity and imputation models in a Multiple Robust estimator with weight coefficient learned in bi-level optimization further enhances the debiasing performance.

5.5. In-Depth Analysis

We explore the difference between calibrating the ensemble model and calibrating individual models separately, with the number of propensity models J and imputation models K taking values from $\{1,3,5\}$. The results on the KuaiRec and Yahoo! R3 datasets are shown in Figure 2 and Figure 3, where darker colors in the heatmap represent higher values of the corresponding evaluation metrics, indicating stronger model performance. The proposed method for calibrating the linear combination of multiple candidate models is referred to as ‛joint calibration’, while the approach of calibrating each individual candidate propensity model and then randomly selecting one calibrated propensity model to calibrate the candidate imputation models is referred to as ‛individual calibration’. We find that directly calibrating the ensemble model outperforms calibrating a single model individually, especially on the Yahoo! R3 dataset. This demonstrates that the proposed Cali-MR method which takes into account the conditions for achieving unbiasedness in the MR estimator achieves better calibration and debiasing performance. On the other hand, examining the impact of the number of candidate models J and K on prediction performance, we find that when $J=1$ or $K=1$, i.e., when using only one propensity model or one imputation model, the prediction performance is poor, validating the effectiveness of incorporating more candidate models in the MR estimator. Additionally, the optimal $(J,K)$ combinations for Cali-MR on Yahoo! R3 and KuaiRec datasets are $(5,5)$ and $(3,3)$, respectively.

We conduct an ablation study of the proposed Cali-MR on three benchmark datasets, with the experimental results shown in

Table 2. Ablation study of Cali-MR on Coat, Yahoo! R3 and KuaiRec datasets.

Method	Coat			Yahoo! R3			KuaiRec
	AUC	NDCG@5	F1@5	AUC	NDCG@5	F1@5	AUC	NDCG@20	F1@20
Cali-MR	0.741	0.658	0.495	0.703	0.678	0.338	0.798	0.521	0.158
Cali-MR w/o prop	0.730	0.640	0.486	0.700	0.674	0.336	0.787	0.483	0.151
Cali-MR w/o imp	0.736	0.639	0.483	0.703	0.674	0.336	0.793	0.509	0.155
Cali-MR w/o imp & prop	0.727	0.635	0.477	0.698	0.667	0.331	0.783	0.482	0.148

. It can be observed that the prediction performance declines if either the propensity model or the imputation model is not calibrated, demonstrating the necessity of model calibration. Furthermore, the performance drop is more significant when removing the calibration of the propensity model compared to the imputation model, indicating that propensity models are more miscalibrated in the existing MR estimator. This highlights that the current propensity model learning approach, based on simple fitting to the observed variable indicator o with naive binary cross-entropy loss, fails to produce well-calibrated models, emphasizing the need to incorporate calibration loss on top of it.

Figure 4 and Figure 5 investigate the impact of varying model calibration hyper-parameter $\lambda$ in multiple propensity and imputation calibration on prediction performance on Coat and KuaiRec datasets. We record the results for different values of the hyper-parameter $\lambda$ and fit a curve to show its variation trend, with a horizontal line representing the baseline method without calibration. We observe that when the hyper-parameter $\lambda$ falls within an appropriate range, such as $[0.1,100]$, the proposed method stably outperforms the baseline method, with the best performance achieved around moderate values, such as $\lambda =10$. Notably, on the Coat dataset, when the value of hyperparameter $\lambda$ is set too large (e.g., 1000), the model fails to achieve good prediction performance, which indicates the propensity and imputation models overly emphasize calibration at the expense of prediction accuracy, leading to poor performance. This highlights the need to balance the prediction and calibration capabilities of propensity and imputation models during training.

Figure 6 and Figure 7 explore the effect of varying number of bins M in soft binning strategy on prediction performance on Coat and KuaiRec datasets. The scatter points represent the results for different values of M, and a fitted curve illustrates the trend of predictive performance as M changes. The results in the figure indicate that the optimal performance is achieved at moderate values of M, such as around 30 for Coat dataset and 15 for KuaiRec dataset. However, performance declines when M is either too large or too small. A possible reason is that when M is too small, the probability of samples with different prediction values ($\widehat{p}$ or $\widehat{e}$) being assigned to the same bin increases. Conversely, when M is too large, the probability of samples with similar prediction values ($\widehat{p}$ or $\widehat{e}$) being assigned to the same bin decreases. Both situations can lead to inaccurate DECE estimation, thereby reducing the calibration quality of the model.

The results in

Table 3. Comparison of Expected Calibration Error (ECE) on Coat, Yahoo! R3 and KuaiRec datasets, where RD refers to the relative decrease value over the corresponding base model.

	Coat		Yahoo! R3		KuaiRec
Method	ECE	RD	ECE	RD	ECE	RD
DR-JL	0.1626	-	0.0589	-	0.0999	-
DCE-DR	0.1428	0.0198	0.0554	0.0035	0.0451	0.0548
TDR-JL	0.1468	-	0.0481	-	0.0499	-
DCE-TDR	0.1270	0.0198	0.0476	0.0005	0.0488	0.0011
MR	0.1512	-	0.1594	-	0.2367	-
Cali-MR	0.1239	0.0273	0.0349	0.1245	0.0519	0.1848

demonstrate that proposed Cali-MR achieves comparable ECE performance to the current SOTA model, DCE-TDR. However, when evaluating the relative ECE reduction compared to the base models, we observe that DCE-TDR achieves a smaller relative improvement over TDR because TDR itself, leveraging targeted learning, already exhibits a relatively calibrated imputation model. Conversely, the original MR model shows a high ECE, indicating poor calibration, while the proposed Cali-MR significantly reduces the ECE, thereby enhancing calibration quality, which validates the effectiveness of the proposed approach.

We compare the training time and parameter size of different methods on

Table 4. Comparison of training time (seconds) and parameter size (Naive method denoted as 1×) on Coat, Yahoo! R3 and KuaiRec datasets.

Method	Coat		Yahoo! R3		KuaiRec
	Time	Params	Time	Params	Times	Params
Naive	4.04	1×	26.07	1×	11.67	1×
IPS	6.54	2×	32.84	2×	15.14	2×
DR	17.41	3×	43.18	3×	31.20	3×
DR-JL	18.81	3×	166.21	3×	110.01	3×
TDR-JL	21.13	3×	128.88	3×	101.09	3×
MR (J = 2, K = 2)	13.55	5×	127.8	5×	114.78	5×
Cali-DR (J = 1, K = 1)	21.03	3×	132.98	3×	128.31	3×
Cali-MR (J = 2, K = 2)	23.28	5×	144.56	5×	124.58	5×
Cali-MR (J = 1, K = 5)	21.57	5×	139.80	5×	130.34	5×
Cali-MR (J = 1, K = 5)	21.43	7×	136.77	7×	141.46	7×
Cali-MR (J = 3, K = 3)	19.55	5×	127.80	5×	106.66	7×
Cali-MR (J = 3, K = 3)	25.43	7×	151.97	7×	127.47	7×
Cali-MR (J = 3, K = 5)	23.59	9×	166.42	9×	163.02	9×
Cali-MR (J = 5, K = 3)	23.71	9×	143.76	9×	103.60	9×
Cali-MR (J = 5, K = 3)	29.35	9×	132.74	9×	128.03	9×
Cali-MR (J = 5, K = 5)	25.65	11×	153.79	11×	166.73	11×

. The experimental results show that the Naive method, which adopts a single model to fit observed ratings, has a parameter size of $1\times$. The IPS method introduces an additional propensity model, resulting in a parameter size of $2\times$, while the DR method further introduces an imputation model, with a parameter size of $3\times$. In this experiment, the MR method adopts two candidate propensity and imputation models, leading to a parameter size of $5\times$. As an improvement, our proposed Cali-MR (J = 2, K = 2) adds only a soft binning module, which increases the parameter count marginally compared to the original MR method, thus still maintaining a parameter size of $5\times$. In other words, the proposed method achieves significant performance improvements over the existing MR method, while maintaining the same parameter size. Additionally, by comparing the training times for MR and Cali-MR (J = 2, K = 2), we observe that model calibration does not add substantial training time, for example, with times of 127.8 and 144.56 on Yahoo! R3 dataset, respectively. Furthermore, increasing the number of candidate models (J and K) for Cali-MR does not lead to a significant increase in training time. This is because as the number of candidate models increases, the ensemble model tends to overfit, and due to early stopping strategy, the number of epochs required for convergence varies with the number of candidate models. In summary, compared to the MR method, the proposed Cali-MR method improves model calibration and prediction accuracy with almost the same parameter size and acceptable training time.

6. Conclusions

In this paper, we explore how to properly calibrate multiple propensity and imputation models in a Multiple Robust (MR) estimator. First, we argue that calibrating each candidate model individually is too costly and unreasonable due to the unbiasedness condition of MR in terms of linear combinations is not considered. Based on this, we propose using a gradient-based bi-level optimization method to calibrate the linear combination of multiple candidate models. Specifically, in the bi-level optimization, we first assumed to update the combination coefficients to obtain the best-performing combination coefficients under the current candidate model parameters. Then, based on these coefficients, we update the candidate model parameters to ensure that each model maintains good prediction performance, while also ensuring that the combination model achieves strong calibration ability, using the differentiable expected calibration error metric that allows it to be directly optimized. Experimental results on three real-world datasets demonstrate that the proposed multiple propensity and imputation calibration method further enhances the prediction performance.

Regarding the broader impact, this is the first work to address the calibration of MR models, by introducing the differentiable expected calibration error (DECE) to directly optimize the objective, rather than relying on post-hoc adjustments in previous work. This provides a novel perspective on model calibration within the debiasing recommendation field. The main limitation of this work lies in the use of bi-level optimization to update parameters of multiple candidate models and the optimal combination coefficients within the MR method. This results in longer training times for the proposed Cali-MR method compared to the original MR method. A potential future work is to develop an alternative optimization algorithm to reduce the computational cost. Furthermore, in future works, we plan to investigate whether other calibration metrics, aside from ECE, might be more suitable for debiasing recommendation scenarios.