Causally-Informed Instance-Wise Feature Selection for Explaining Visual Classifiers

Abstract

We propose a novel interpretability framework that integrates instance-wise feature selection with causal reasoning to explain decisions made by black-box image classifiers. Instead of relying on feature importance or mutual information, our method identifies input regions that exert the greatest causal influence on model predictions. Causal influence is formalized using a structural causal model and quantified via a conditional mutual information term. To optimize this objective efficiently, we employ continuous subset sampling and the matrix-based Rényi’s $\alpha$-order entropy functional. The resulting explanations are compact, semantically meaningful, and causally grounded. Experiments across multiple vision datasets demonstrate that our method outperforms existing baselines in terms of predictive fidelity.

1. Introduction

As machine learning systems are increasingly deployed in high-stakes domains, the need for transparent and trustworthy model behavior becomes critical, particularly for black-box classifiers with opaque decision-making processes. In the field of computer vision, a common strategy for enhancing interpretability involves generating saliency maps that visually highlight the contribution of each pixel to the model’s prediction. While gradient-based techniques have gained popularity due to their intuitive appeal, they often suffer from instability, lack of robustness, and sensitivity to perturbations in both models and data [1,2]. These limitations undermine their reliability in critical applications. To address these shortcomings, recent research has introduced the paradigm of instance-wise feature selection, which shifts the focus from continuous attribution to the discrete selection of the most informative input features, such as individual pixels or image patches, on a per-instance basis [3,4,5]. This approach not only yields more compact and human-interpretable explanations but also lays the foundation for integrating causal reasoning into model interpretation by emphasizing features that actively influence the prediction, rather than merely correlating with it.

One of the earliest and most influential approaches in the domain of instance-wise feature selection is L2X [6], which formulates the problem as selecting a fixed number of features that maximize mutual information with respect to the model’s output. This method is based on the idea that the most informative features, in terms of their statistical dependence on the prediction, can offer valuable insight into the model’s decision process. However, L2X requires the number of selected features to be predefined, which can limit its adaptability across different datasets or applications where the optimal number of relevant features varies. To address this constraint, INVASE [7] was subsequently introduced as an extension that eliminates the need to specify the subset size in advance. It employs a reinforcement learning-based strategy to dynamically determine the number and identity of important features for each input instance. Despite this architectural enhancement, both L2X and INVASE share a common limitation: they rely on mutual information as their primary objective for optimization.

While mutual information is effective in measuring statistical dependency, it does not necessarily reflect causal relationships. In particular, a feature may exhibit high mutual information with the output variable simply because it is correlated with other influential inputs rather than exerting a direct causal influence on the model’s decision [8,9]. As a result, explanations generated by such methods may emphasize features that are associated with the output but are not causally responsible for the prediction. This can lead to misleading or incomplete interpretations, especially in domains where understanding causal mechanisms is essential for downstream decision-making, such as medicine, finance, or safety-critical systems [10,11].

In this work, we argue that a robust explanation method should prioritize the identification of features that exert a direct causal influence on a model’s prediction. Specifically, we seek to select those input components whose inclusion or removal leads to meaningful changes in the model’s output. Our core hypothesis is that the most causally influential features are typically sparse and class-specific, capturing only the essential parts of the input that truly drive the decision. In contrast to prior methods such as L2X and INVASE, which may conflate statistical association with causation, our approach integrates a principled, information-theoretic measure of causal influence [12,13] into the feature selection process. This yields explanations that are not only compact and interpretable but also more faithful to the model’s underlying decision-making behavior. To formalize this notion, we adopt a structured causal model (SCM) framework [8] and employ an information-theoretic metric [12,13] as a rigorous measure of causal strength.

Although this new measure provides a theoretically sound basis for quantifying causal effects, its direct application in deep learning models can be computationally intensive and difficult to optimize. To make the framework practical, we further introduce the recently proposed matrix-based Renyi’s $\alpha$-order entropy functional [14] to estimate causal strength while enabling efficient training within modern machine learning pipelines. Based on this formulation, we construct an instance-wise feature selector that identifies causally important subsets of input features. To facilitate gradient-based optimization, we adopt a continuous subset sampling strategy using the Gumbel–Softmax technique [15]. This allows the selector network to learn which features are causally influential through standard backpropagation, without resorting to reinforcement learning or combinatorial search.

We validate the proposed approach through a series of experiments on three benchmark vision datasets: MNIST, Fashion-MNIST, and CIFAR-10. For each dataset, we train a black-box classifier and apply our explainer to generate instance-specific explanations. To assess the effectiveness of the selected features, we conduct both qualitative analyses and quantitative evaluations. In our comparisons, we include several popular explanation methods, namely L2X, Grad-CAM, and Saliency maps, which represent diverse families of interpretability techniques. For quantitative comparison, we employ the post-hoc accuracy [5], which measures the predictive consistency of the model when restricted to the selected features. Our experimental findings consistently demonstrate that the proposed method produces more causally meaningful explanations compared to existing approaches. In particular, our method maintains competitive post-hoc accuracy, indicating that the causal features also preserve the classifier’s decision performance. These results highlight the advantages of combining instance-wise selection with causal analysis, and illustrate the potential of our approach for improving the transparency and reliability of black-box decision-making systems.

To summarize, our main contributions are as follows:

• We propose a novel instance-wise feature selection framework that is explicitly grounded in causal inference. By adopting the definition of causality based on the “do”-operator or intervention from [12] and establishing its connection to conventional conditional mutual information in probability theory, our method prioritizes features that have a genuine causal impact on the model’s prediction.
• We develop a practical training strategy by approximating the causal objective using conditional mutual information and enabling reliable estimation with the matrix-based Rényi’s $\alpha$-order entropy functional and differentiable subset selection through the Gumbel-Softmax technique.
• We evaluate our approach on three standard vision datasets. Our method demonstrates improved performance compared to several popular interpretability baselines, especially in capturing causal influence.

2. Related Work

Model interpretability is a critical aspect of trustworthy machine learning, particularly in high-stakes domains such as healthcare, finance, and scientific discovery. A wide array of post-hoc explainability techniques have been developed to shed light on the decision-making process of complex models. These approaches can be broadly categorized into gradient-based methods, Shapley-value-based explanations, instance-wise feature selection, and information bottleneck-based frameworks.

Gradient-based methods aim to identify influential input features by analyzing the gradients of the model’s output with respect to its input. Classic approaches include Saliency Maps [16], which visualize the absolute value of gradients, and Grad-CAM [17], which backpropagates importance scores from intermediate feature maps in convolutional neural networks. While these methods are computationally efficient and provide visual intuitions, they are often criticized for their instability and sensitivity to input perturbations [1]. Furthermore, gradient-based attributions do not offer guarantees of causality, and they may reflect merely local sensitivity rather than true explanatory factors.

Shapley-value-based approaches derive from cooperative game theory, where the goal is to assign each feature an importance value based on its marginal contribution to the prediction. SHAP [18] is one of the most popular techniques in this category, offering a unified framework that satisfies desirable axioms such as local accuracy, missingness, and consistency. Despite its theoretical appeal, SHAP typically relies on strong independence assumptions or background distribution sampling, which can misrepresent causal relationships between features. Additionally, the computational cost of exact Shapley value computation grows exponentially with the number of input features, often requiring sampling-based approximations.

Instance-wise feature selection methods seek to identify a minimal subset of features that are sufficient for the model to make accurate predictions on a per-sample basis. Notable examples include L2X [6], which uses mutual information to guide feature selection through a variational approximation, and INVASE [7], which frames feature selection as a reinforcement learning problem. These methods offer fine-grained interpretability by adapting explanations to individual data points. However, their primary objective is predictive sufficiency rather than uncovering causal mechanisms, and they may capture spurious correlations present in the training data.

In recent years, the Information Bottleneck (IB) principle [19] has emerged as a popular method for instance-wise feature selection in both post-hoc and intrinsically interpretable visual classifiers [3,4]. The general objective is typically formulated as:

$$minI(X;X_S)-\beta I(Y;X_S),$$

(1)

where $X_S$ is a subset of features from X, and Y represents either the ground-truth label [4] or the prediction of the classifier being explained [3,20]. This idea has recently been extended to the Causal Information Bottleneck (CIB) framework [21], which aims to ensure that the selected features are causally related to the prediction. The authors of [21] also introduced causal information gain [22] to quantify causal strength. However, these methods are typically validated only on synthetic data or simple vectorized features.

Our objective, as will be presented in Section 3.3, differs from the standard IB formulation described above. Moreover, we show that causal strength can be effectively measured using a simple conditional mutual information term, rather than relying on causal information gain, which requires the use of the do-operator.

In summary, while existing interpretability methods have made significant strides, they either lack robustness, causal guarantees, or the ability to produce personalized explanations. Our work aims to bridge this gap by introducing an instance-wise explanation framework that explicitly incorporates causal reasoning, providing both accurate and causally meaningful interpretations.

3. Methodology

3.1. Problem Setup

We focus on interpreting a pretrained, non-transparent classifier $F:{\mathbb{R}}^d\to \mathcal{Y}$ by identifying, for each input $x\in {\mathbb{R}}^d$, a subset of k features whose presence most significantly influences the model’s output. To this end, we train an explainer network $E:{\mathbb{R}}^d\to {\mathcal{S}}_k$, where ${\mathcal{S}}_k$ is the set of binary masks with k ones. The true explanation is obtained by inverting this mask, thus highlighting the k most relevant features (see Figure 1).

We adopt a structural causal modeling perspective, where the black-box model is abstracted as a Directed Acyclic Graph (DAG) with unidirectional links from input nodes to output variables. This view supports the interpretation of feature-to-output influence using a formal notion of causality.

To quantify the causal impact of a subset of features $S\subset \{1,\dots ,d\}$, we adopt an information-theoretic measure of causal influence (detailed in the next subsection), which ultimately reduces to the conditional mutual information $I(X_S;Y\mid X_{\overline{S}})$, enabling tractable optimization.

3.2. Quantifying Causal Strength

Definition 1

([12]). Let U, V, and W be disjoint subsets of nodes. The information flow from U to V imposing W, denoted $I(U\to V\mid W)$, is

$${\mathbb{E}}_{w\sim W}\left[{\int }_{U}p(u\mid do\left(w\right)){\int }_{V}p(v\mid do\left(u\right),do\left(w\right))log{\displaystyle \frac{p(v\mid do(u),do(w\left)\right)}{{\int }_{u^{\prime }}p(u^{\prime }\mid do\left(w\right))p(v\mid do\left(u^{\prime }\right),do\left(w\right))}}dVdU\right],$$

(2)

where $do\left(w\right)$ represents an intervention in a model that fixes w to a specified value, regardless of the values of its parents [8].

Despite the elegant and intuitive formulation of causality in Equation (2), it remains challenging to estimate in practice, especially in the context of deep learning. To address this, we first establish its connection to conventional conditional mutual information, as shown in Proposition 1.

Proposition 1.

Information flow in our DAG. The information flow from α to Y imposing β in the DAG of Figure 2 coincides with the mutual information of α and Y conditioned on β,

$$I(\alpha \to Y\mid do(\beta \left)\right)=I(\alpha ;Y\mid \beta ),$$

(3)

where conditional mutual information is defined as $I(X;Y\mid Z)={\mathbb{E}}_{X,Y,Z}\left[log{\displaystyle \frac{p(x,y\mid z)}{p(x\mid z)p(y\mid z)}}\right]$.

Proof. 

The proof follows from the “action/observation exchange” rule of the do-calculus [8] (Thm. 3.4.1). This rule asserts that $p(y\mid do(x),do(z),w)=p(y\mid do(x),z,w)$ if $Y\perp Z\mid X,W$ in ${\mathcal{G}}_{\underline{X}\overline{Z}}$, the causal model modified to remove connections entering X and leaving Z. When applied to our model, it yields:

• $p(Y\mid do(\alpha \left)\right)=p(Y\mid \alpha )$ (because $Y\perp \alpha$ in ${\mathcal{G}}_{\alpha }$);
• $p(\alpha \mid do(\beta \left)\right)=p(\alpha \mid \beta )$ (because $\alpha \perp \beta$ in ${\mathcal{G}}_{\beta }$);
• $p(Y\mid do(\alpha ),do(\beta \left)\right)=p(Y\mid \alpha ,\beta )$ (because $Y\perp (\alpha ,\beta )$ in ${\mathcal{G}}_{\alpha ,\beta }$).

Starting with the definition of the information flow from $\alpha$ to Y imposing $\beta$, we have that $\begin{array}{cc}\hfill I(\alpha \to Y\mid do(\beta \left)\right)& ={\mathbb{E}}_{\beta }\left[{\int }_{\alpha }p(\alpha \mid do\left(\beta \right)){\int }_{Y}p(Y\mid do\left(\alpha \right),do\left(\beta \right))\right.\hfill \\ & \left.\times log{\displaystyle \frac{p(Y\mid do(\alpha ),do(\beta \left)\right)}{{\int }_{{\alpha }^{\prime }}p(\alpha ={\alpha }^{\prime }\mid do\left(\beta \right))p(Y\mid do\left({\alpha }^{\prime }\right),do\left(\beta \right))}}dYd\alpha \right]\hfill \\ & ={\mathbb{E}}_{\beta }\left[{\int }_{\alpha }p(\alpha \mid \beta ){\int }_{Y}p(Y\mid \alpha ,\beta )log{\displaystyle \frac{p(Y\mid \alpha ,\beta )}{{\int }_{{\alpha }^{\prime }}p(\alpha ={\alpha }^{\prime }\mid \beta )p(Y\mid {\alpha }^{\prime },\beta )}}dYd\alpha \right]\hfill \\ & ={\mathbb{E}}_{\beta }\left[{\int }_{\alpha ,Y}p(Y,\alpha \mid \beta )log{\displaystyle \frac{p(Y\mid \alpha ,\beta )}{p(Y\mid \beta )}}dYd\alpha \right]\hfill \\ & ={\int }_{\beta }{\int }_{\alpha ,Y}p(Y,\alpha ,\beta )log{\displaystyle \frac{p(Y\mid \alpha ,\beta )}{p(Y\mid \beta )}}dYd\alpha d\beta \hfill \\ & ={\int }_{\beta }p\left(\beta \right){\int }_{\alpha ,Y}p(Y,\alpha \mid \beta )log{\displaystyle \frac{p(Y\mid \alpha ,\beta )}{p(Y\mid \beta )}}dYd\alpha \hfill \\ & ={\int }_{\beta }p\left(\beta \right){\int }_{\alpha ,Y}p(Y,\alpha \mid \beta )log{\displaystyle \frac{p(Y,\alpha \mid \beta )}{p(Y\mid \beta )p(\alpha \mid \beta )}}dYd\alpha \hfill \\ & =I(\alpha ;Y\mid \beta ).\hfill \end{array}$

   □

3.3. Objective Function and Its Estimation

Given an input vector $X\in {\mathbb{R}}^d$, we aim to find a subset S of size k that maximizes $I(X_S;Y\mid X_{\overline{S}})$.

The most challenging aspect of our framework lies in accurately estimating $I(X_S;Y\mid X_{\overline{S}})$. According to Shannon’s chain rule [23], $I(X_S;Y\mid X_{\overline{S}})$ can be decomposed as:

$$\begin{array}{cc}\hfill I(X_S;Y\mid X_{\overline{S}})=& H\left(Y|X_{\overline{S}}\right)-H\left(Y|X\right)\hfill \\ \hfill =& H\left(Y,X_{\overline{S}}\right)+H\left(X\right)-H\left(X_{\overline{S}}\right)-H\left(Y,X\right),\hfill \end{array}$$

(4)

where H denotes entropy or joint entropy.

In this work, instead of relying on variational approximation or using the popular mutual information neural estimator (MINE) [24], which may make the joint training become unstable or even result in negative mutual information values [25], we use the matrix-based Rényi’s $\alpha$-order entropy functional [14,26] to estimate different entropy terms in Equation (4). This newly proposed estimator can be simply computed (without density estimation or any auxiliary neural network) and is differentiable, which suits deep learning applications well. For brevity, we directly give the definitions.

Definition 2

([26]). Let $\kappa :\chi \times \chi \mapsto \mathbb{R}$ be a real-valued positive definite kernel that is also infinitely divisible [27]. Given ${\left\{{\mathbf{x}}_i\right\}}_{i=1}^n\in \chi$, each ${\mathbf{x}}_i$ can be a real-valued scalar or vector, and the Gram matrix $K\in {\mathbb{R}}^{n\times n}$ computed as $K_{ij}=\kappa ({\mathbf{x}}_i,{\mathbf{x}}_j)$, a matrix-based analog to Rényi’s α-entropy, can be given by the following functional:

$$\begin{array}{cc}\hfill H_{\alpha }\left(A\right)& ={\displaystyle \frac{1}{1-\alpha }}{log}_2\left(tr\left(A^{\alpha }\right)\right)\hfill \\ & ={\displaystyle \frac{1}{1-\alpha }}{log}_2\left(\sum _{i=1}^n{\lambda }_i{\left(A\right)}^{\alpha }\right),\hfill \end{array}$$

(5)

where $\delta \in (0,1)\cup (1,\infty )$. A is the normalized K, i.e., $A=K/tr\left(K\right)$. ${\lambda }_i\left(A\right)$ denotes the i-th eigenvalue of A.

Definition 3

([14]). Given a set of n samples ${\{{\mathbf{x}}_i,{\mathbf{y}}_i,{\mathbf{z}}_i\}}_{i=1}^n$, each sample contains three measurements, $\mathbf{x}\in \chi$, $\mathbf{y}\in \gamma$, and $\mathbf{z}\in ϵ$, obtained from the same realization. Given positive definite kernels ${\kappa }_1:\chi \times \chi \mapsto \mathbb{R}$, ${\kappa }_2:\gamma \times \gamma \mapsto \mathbb{R}$, and ${\kappa }_3:ϵ\times ϵ\mapsto \mathbb{R}$, a matrix-based analog to Rényi’s α-order joint-entropy can be defined as:

$$H_{\alpha }(A,B,C)=H_{\alpha }\left({\displaystyle \frac{A\circ B\circ C}{tr(A\circ B\circ C)}}\right),$$

(6)

where $A_{ij}={\kappa }_1({\mathbf{x}}_i,{\mathbf{x}}_j)$, $B_{ij}={\kappa }_2({\mathbf{y}}_i,{\mathbf{y}}_j)$, $C_{ij}={\kappa }_3({\mathbf{z}}_i,{\mathbf{z}}_j)$, and $A\circ B\circ C$ denotes the Hadamard product between matrices A, B, and C.

Now, given ${\left\{x_{s,i},y_i,x_{\overline{s},i}\right\}}_{i=1}^M$ in a mini-batch of M samples ($x_{s,i}$ and $x_{\overline{s},i}$ refer to, respectively, the selected feature subset and its complement subset for the i-th instance), we first need to evaluate three Gram matrices, $K_s=\kappa (x_{s,i},x_{s,j})\in {\mathbb{R}}^{M\times M}$, $K_y=\kappa (y_i,y_j)\in {\mathbb{R}}^{M\times M}$, and $K_{\overline{s}}=\kappa (x_{\overline{s},i},x_{\overline{s},j})\in {\mathbb{R}}^{M\times M}$, associated with variables $X_S$, Y, and $X_{\overline{S}}$, respectively. Based on Definitions 2 and 3, the entropy and joint entropy terms in Equation (4), all can be simply computed over the eigenspectrum of $K_s$, $K_y$, $K_{\overline{s}}$, or their Hadamard product. Hence, our final estimator is expressed as:

$$\widehat{I}\left(X_S;Y|X_{\overline{S}}\right)=H_{\alpha }\left(K_y,K_{\overline{s}}\right)+H_{\alpha }\left(K_s,K_{\overline{s}}\right)-H_{\alpha }\left(K_{\overline{s}}\right)-H_{\alpha }\left(K_s,K_y,K_{\overline{s}}\right).$$

(7)

Throughout this work, we choose $\alpha =1.01$ [14,26] and use the radial basis function (RBF) kernel $\kappa ({\mathbf{x}}_i,{\mathbf{x}}_j)=exp(-{\displaystyle \frac{\parallel {\mathbf{x}}_i-{\mathbf{x}}_j{\parallel }^2}{2{\sigma }^2}})$ to obtain the Gram matrices. For each sample, we evaluate its k-nearest distances and take the mean. We choose kernel width $\sigma$ as the average of the mean values for all samples.

As can be seen, this new family of estimators avoids explicit estimation of the underlying data distributions, making it particularly attractive for challenging problems involving high-dimensional data. In practice, computing the gradient of $I_{\alpha }(A;C|B)$ is straightforward using any automatic differentiation framework, such as PyTorch (version 1.13.1) [28]. We recommend PyTorch, as its computed gradients are consistent with the analytical ones.

3.4. Continuous Subset Sampling

Exhaustively searching over all possible feature subsets of size k involves evaluating $\left({\displaystyle \genfrac{}{}{0pt}{}{d}{k}}\right)$ combinations, which is computationally intractable for even moderately sized d. To address this challenge, we employ a continuous relaxation strategy based on the Gumbel–Softmax distribution, allowing us to approximate discrete subset sampling in a differentiable manner. Specifically, we independently sample k features from the full set of d candidates using a stochastic softmax parameterization.

We now describe the architecture of our built-in explainer. Given an input instance $x_i\in {\mathbb{R}}^d$ and a predefined subset size k, we denote by ${\wp }_k=\{s_i\subseteq \{1,\dots ,d\}:|s_i|=k\}$ the space of all possible feature subsets of cardinality k. Our goal is to identify the subset $s_i\in {\wp }_k$ that most faithfully accounts for the model’s prediction on $x_i$. To this end, we define a scoring function $S:{\mathbb{R}}^d\to {\mathbb{R}}^d$ that assigns an importance weight to each feature. These scores, produced by a neural selector network (as illustrated in Figure 1), reflect the probability of each feature being included in the selected subset. The final subset is then sampled based on these learned importance scores using the Gumbel–Softmax reparameterization.

However, a direct estimation requires summing over $\left(\genfrac{}{}{0pt}{}{d}{k}\right)$ combinations of feature subsets, which is intractable. Therefore, we employ the Gumbel–Softmax [3,15] to overcome this problem in a differentiable manner. Suppose we aim to approximate a categorical random variable represented as a one-hot vector in ${\mathbb{R}}^d$ with category probabilities $p_1,p_2,\dots ,p_d$. We start by adding a random perturbation to the log probability of each category $log{p}_i$:

$$\begin{array}{cc}\hfill G_i=& -log(-log{u}_i)\mathrm{where}{u}_i\sim \mathrm{Uniform}(0,1),\hfill \\ \hfill & C_i={\displaystyle \frac{\exp \{(G_i+log{p}_i)/\tau \}}{{\sum }_{i=1}^d\exp \{(G_i+log{p}_i)/\tau \}}},\hfill \end{array}$$

(8)

where $\tau$ is a tuning parameter for the temperature of the Gumbel–Softmax distribution (in our paper, we set $\tau =0.1$). Then, we can define a Concrete random vector $C=[C_1,\dots ,C_d]$, which serves as a continuous, differential approximation of a categorical random variable represented as a one-hot vector in ${\mathbb{R}}^d$.

We further define a continuous-relaxed random variable $C^{*}=[C_1^{*},\dots ,C_d^{*}]$ as the element-wise maximum of the independently sampled Concrete vectors $C^{\left(j\right)}$ where $j=1,\dots ,k$:

$$C_i^{*}=\underset{j}{max}{C}_i^{\left(j\right)}\mathrm{i}.\mathrm{i}.\mathrm{d}\mathrm{for}j=1,\dots ,k.$$

(9)

We denote this sampling result $C^{*}$ as the generated explanation $e_i^m\in {\mathbb{R}}^d$, where the top-k features are most informative for the prediction of the model.

4. Experiments and Results

4.1. Experimental Setup

In this section, we conduct both quantitative and qualitative comparisons between our method and four competing approaches: two state-of-the-art instance-wise feature selection approaches, L2X [6] and INVASE [7], as well as two pixel-level attribution methods, GradCAM [17] and Saliency [16]. Experiments are performed on the MNIST, Fashion-MNIST, and CIFAR datasets.

Both L2X and INVASE are information-theoretic approaches. Specifically, L2X aims to maximize the mutual information objective $I(X_S;Y)$, while INVASE minimizes the Kullback–Leibler divergence $D_{\mathrm{KL}}(p\left(Y\right|X)\parallel p\left(Y\right|X_S))$. Our objective, based on the conditional mutual information $I(X_S;Y\mid X_{\overline{S}})$, is closely related to that of L2X. However, we emphasize that L2X does not establish a connection between its objective and causality. That is, there is no theoretical guarantee that it identifies features that are causally responsible for the model’s prediction.

In contrast to L2X and INVASE, which rely on variational approximations to estimate mutual information, our method introduces a matrix-based Rényi’s entropy functional, enabling a more efficient and principled estimation.

Our method operates at the level of superpixels or image patches, selecting informative patches instead of individual pixels. To ensure a fair comparison with the pixel-level attribution baselines, we compute the average attribution score within each patch and then select the top-k patches accordingly. We use post-hoc accuracy [5] for quantitative assessment, which measures predictive consistency using selected features:

$$\mathrm{Accuracy}={\displaystyle \frac{1}{|X_{\mathrm{val}}|}}\sum _{x\in X_{\mathrm{val}}}\mathbb{I}\left[argmaxF\left(x\right)=argmaxF\left(x_S\right)\right].$$

4.2. Quantitative Results

For all datasets, we report the mean and standard deviation of post-hoc accuracy over five independent runs for all competing approaches. In the following tables, k denotes the number of selected $4\times 4$ image patches. GradCAM and Saliency typically do not report variance in this context because they are deterministic methods: given a fixed trained model and a fixed input, they always produce the same output.

4.2.1. MNIST

The MNIST dataset consists of $28\times 28$ grayscale images of handwritten digits. We consider a binary classification task using digits “3” and “8”. In addition, we design four more binary tasks to explore diverse visual distinctions: “1” vs. “7” (vertical vs. angular strokes), “0” vs. “6” (loops with or without tails), “2” vs. “5” (confusingly similar curves), and “4” vs. “9” (sharp angles vs. loops).

A simple convolutional neural network (CNN), composed of two convolutional layers followed by one fully connected layer, is trained as the black-box model and achieves a test accuracy of over 99%. The selector network in L2X, INVASE, and our method is implemented as a three-layer fully convolutional network. For L2X, the variational approximator is parameterized using the same architecture as the black-box model across all experiments.

In detail, the selector network consists of three convolutional blocks designed to process single-channel $28\times 28$ input images. The first block applies a $3\times 3$ convolution with 5 output filters and padding of 2, followed by $2\times 2$ max pooling and a ReLU activation. The second block performs another $3\times 3$ convolution with 16 filters and padding of 2, again followed by $2\times 2$ max pooling and ReLU activation. The final block is a $1\times 1$ convolution that projects the 16-channel feature map to a single channel without any additional non-linearity. The resulting tensor is reshaped to $(\mathrm{batch}\mathrm{size},k^2)$, corresponding to a $k\times k$ grid of logits that can be interpreted as spatial selection scores.

Table 1. Post-hoc accuracy (MNIST, 3 and 8). ** indicates statistically significant improvement over L2X at $p<0.01$. The best performance is in bold.

Method	k = 4	k = 6	k = 8
Ours	$\mathbf{0}.\mathbf{951}\pm \mathbf{0}.\mathbf{007}$ **	$\mathbf{0}.\mathbf{974}\pm \mathbf{0}.\mathbf{004}$ **	$\mathbf{0}.\mathbf{984}\pm \mathbf{0}.\mathbf{004}$ **
INVASE	$0.945\pm 0.006$	$0.969\pm 0.005$	$0.982\pm 0.004$
L2X	$0.940\pm 0.007$	$0.967\pm 0.005$	$0.979\pm 0.003$
GradCAM	$0.802$	$0.829$	$0.842$
Saliency	$0.866$	$0.921$	$0.956$

,

Table 2. Post-hoc accuracy (MNIST, 1 and 7). ** indicates statistically significant improvement over L2X at $p<0.01$. The best performance is in bold.

Method	k = 4	k = 6	k = 8
Ours	$\mathbf{0}.\mathbf{943}\pm \mathbf{0}.\mathbf{006}$ **	$\mathbf{0}.\mathbf{967}\pm \mathbf{0}.\mathbf{004}$ **	$\mathbf{0}.\mathbf{978}\pm \mathbf{0}.\mathbf{005}$ **
INVASE	$0.936\pm 0.005$	$0.961\pm 0.006$	$0.975\pm 0.004$
L2X	$0.929\pm 0.007$	$0.958\pm 0.005$	$0.972\pm 0.003$
GradCAM	$0.794$	$0.821$	$0.838$
Saliency	$0.859$	$0.910$	$0.949$

,

Table 3. Post-hoc accuracy (MNIST, 0 and 6). ** indicates statistically significant improvement over L2X at $p<0.01$. The best performance is in bold.

Method	k = 4	k = 6	k = 8
Ours	$\mathbf{0}.\mathbf{949}\pm \mathbf{0}.\mathbf{006}$ **	$\mathbf{0}.\mathbf{971}\pm \mathbf{0}.\mathbf{004}$ **	$\mathbf{0}.\mathbf{982}\pm \mathbf{0}.\mathbf{005}$ **
INVASE	$0.941\pm 0.007$	$0.965\pm 0.005$	$0.978\pm 0.004$
L2X	$0.934\pm 0.006$	$0.961\pm 0.005$	$0.974\pm 0.003$
GradCAM	$0.790$	$0.818$	$0.835$
Saliency	$0.855$	$0.908$	$0.946$

,

Table 4. Post-hoc accuracy (MNIST, 2 and 5). ** indicates statistically significant improvement over L2X at $p<0.01$. The best performance is in bold.

Method	k = 4	k = 6	k = 8
Ours	$\mathbf{0}.\mathbf{946}\pm \mathbf{0}.\mathbf{007}$ **	$\mathbf{0}.\mathbf{968}\pm \mathbf{0}.\mathbf{004}$ **	$\mathbf{0}.\mathbf{980}\pm \mathbf{0}.\mathbf{004}$ **
INVASE	$0.938\pm 0.006$	$0.962\pm 0.005$	$0.976\pm 0.004$
L2X	$0.931\pm 0.007$	$0.958\pm 0.005$	$0.973\pm 0.003$
GradCAM	$0.788$	$0.815$	$0.831$
Saliency	$0.850$	$0.903$	$0.943$

and

Table 5. Post-hoc accuracy (MNIST, 4 and 9). ** indicates statistically significant improvement over L2X at $p<0.01$. The best performance is in bold.

Method	k = 4	k = 6	k = 8
Ours	$\mathbf{0}.\mathbf{944}\pm \mathbf{0}.\mathbf{006}$ **	$\mathbf{0}.\mathbf{969}\pm \mathbf{0}.\mathbf{004}$ **	$\mathbf{0}.\mathbf{981}\pm \mathbf{0}.\mathbf{004}$ **
INVASE	$0.936\pm 0.007$	$0.964\pm 0.005$	$0.977\pm 0.004$
L2X	$0.928\pm 0.006$	$0.959\pm 0.005$	$0.973\pm 0.003$
GradCAM	$0.786$	$0.812$	$0.830$
Saliency	$0.848$	$0.901$	$0.942$

summarize the post-hoc accuracy results, while Figure 3, Figure 4, Figure 5, Figure 6 and Figure 7 visualize the explanations generated by all competing approaches. Our method consistently achieves the highest accuracy across tasks and produces explanations that align closely with human perceptual understanding.

4.2.2. Fashion-MNIST

The Fashion-MNIST dataset [29] consists of $28\times 28$ grayscale images of fashion items such as T-shirts, shoes, and handbags. For our experiments, we focus on a binary classification task involving class 0 (T-shirts) and class 9 (ankle boots). In addition, we design four supplementary binary classification tasks to explore a range of semantic and structural variations: (1) class 0 vs. class 2 (T-shirt vs. pullover), highlighting subtle differences among upper-body garments; (2) class 5 vs. class 9 (sandal vs. ankle boot), contrasting open and closed footwear styles; (3) class 8 vs. class 4 (bag vs. coat), testing the model’s ability to differentiate between accessories and outerwear; and (4) class 1 vs. class 3 (trouser vs. dress), emphasizing the distinction between symmetric and asymmetric lower-body apparel. These carefully chosen pairs allow us to assess model performance across diverse and challenging visual boundaries.

We use the same convolutional neural network architecture as in the MNIST experiments, achieving a test accuracy over 99%. The selector networks in both L2X and our method are implemented using a three-layer fully convolutional architecture.

Table 6. Post-hoc accuracy (Fashion-MNIST, class 0 (T-shirts) and class 9 (ankle boots)). *, **, and *** indicate statistically significant improvement over L2X at $p<0.05$, $p<0.01$, and $p<0.001$, respectively. The best performance is in bold.

Method	k = 4	k = 6	k = 8
Ours	$\mathbf{0}.\mathbf{914}\pm \mathbf{0}.\mathbf{021}$ ***	$\mathbf{0}.\mathbf{967}\pm \mathbf{0}.\mathbf{010}$ *	$\mathbf{0}.\mathbf{981}\pm \mathbf{0}.\mathbf{004}$ ***
INVASE	$0.902\pm 0.019$ **	$0.962\pm 0.011$	$0.976\pm 0.009$ *
L2X	$0.887\pm 0.025$	$0.960\pm 0.007$	$0.970\pm 0.012$
GradCAM	$0.593$	$0.639$	$0.681$
Saliency	$0.561$	$0.837$	$0.930$

,

Table 7. Post-hoc accuracy (Fashion-MNIST, class 0 (T-shirts) and class 2 (pullovers)). *, **, and *** indicate statistically significant improvement over L2X at $p<0.05$, $p<0.01$, and $p<0.001$, respectively. The best performance is in bold.

Method	k = 4	k = 6	k = 8
Ours	$\mathbf{0}.\mathbf{907}\pm \mathbf{0}.\mathbf{020}$ ***	$\mathbf{0}.\mathbf{958}\pm \mathbf{0}.\mathbf{012}$ **	$\mathbf{0}.\mathbf{976}\pm \mathbf{0}.\mathbf{006}$ ***
INVASE	$0.895\pm 0.019$ **	$0.951\pm 0.013$	$0.971\pm 0.010$ *
L2X	$0.883\pm 0.024$	$0.948\pm 0.010$	$0.967\pm 0.012$
GradCAM	$0.579$	$0.631$	$0.678$
Saliency	$0.549$	$0.829$	$0.921$

,

Table 8. Post-hoc accuracy (Fashion-MNIST, class 5 (sandals) and class 9 (ankle boots)). *, **, and *** indicate statistically significant improvement over L2X at $p<0.05$, $p<0.01$, and $p<0.001$, respectively. The best performance is in bold.

Method	k = 4	k = 6	k = 8
Ours	$\mathbf{0}.\mathbf{919}\pm \mathbf{0}.\mathbf{019}$ ***	$\mathbf{0}.\mathbf{964}\pm \mathbf{0}.\mathbf{011}$ **	$\mathbf{0}.\mathbf{982}\pm \mathbf{0}.\mathbf{005}$ ***
INVASE	$0.905\pm 0.018$ **	$0.957\pm 0.012$	$0.977\pm 0.009$ *
L2X	$0.893\pm 0.023$	$0.953\pm 0.011$	$0.972\pm 0.012$
GradCAM	$0.601$	$0.648$	$0.692$
Saliency	$0.576$	$0.843$	$0.932$

,

Table 9. Post-hoc accuracy (Fashion-MNIST, class 8 (bags) and class 4 (coats)). *, **, and *** indicate statistically significant improvement over L2X at $p<0.05$, $p<0.01$, and $p<0.001$, respectively. The best performance is in bold.

Method	k = 4	k = 6	k = 8
Ours	$\mathbf{0}.\mathbf{926}\pm \mathbf{0}.\mathbf{017}$ ***	$\mathbf{0}.\mathbf{968}\pm \mathbf{0}.\mathbf{009}$ **	$\mathbf{0}.\mathbf{984}\pm \mathbf{0}.\mathbf{004}$ ***
INVASE	$0.912\pm 0.018$ **	$0.962\pm 0.010$	$0.978\pm 0.008$ *
L2X	$0.898\pm 0.022$	$0.957\pm 0.009$	$0.974\pm 0.011$
GradCAM	$0.606$	$0.654$	$0.699$
Saliency	$0.583$	$0.849$	$0.936$

and

Table 10. Post-hoc accuracy (Fashion-MNIST, class 1 (trousers) and class 3 (dresses)). *, **, and *** indicate statistically significant improvement over L2X at $p<0.05$, $p<0.01$, and $p<0.001$, respectively. The best performance is in bold.

Method	k = 4	k = 6	k = 8
Ours	$\mathbf{0}.\mathbf{931}\pm \mathbf{0}.\mathbf{015}$ ***	$\mathbf{0}.\mathbf{971}\pm \mathbf{0}.\mathbf{008}$ **	$\mathbf{0}.\mathbf{985}\pm \mathbf{0}.\mathbf{004}$ ***
INVASE	$0.918\pm 0.017$ **	$0.964\pm 0.010$	$0.980\pm 0.007$ *
L2X	$0.904\pm 0.021$	$0.958\pm 0.009$	$0.975\pm 0.010$
GradCAM	$0.612$	$0.662$	$0.703$
Saliency	$0.586$	$0.855$	$0.938$

summarize the post-hoc accuracy results, while Figure 8, Figure 9, Figure 10, Figure 11 and Figure 12 visualize the explanations generated by all competing approaches. Our method consistently achieves the highest accuracy across tasks and produces explanations that align closely with human perceptual understanding.

4.2.3. CIFAR-10

The CIFAR-10 dataset [30] consists of 60,000 $32\times 32$ RGB images spanning 10 object categories. For our experiments, we focus on a binary classification task involving class 2 (birds) and class 9 (trucks), which represent a high-level semantic contrast between natural and man-made objects. Additionally, we design three supplementary binary tasks to capture various types of visual and semantic diversity: (1) class 0 vs. class 1 (airplane vs. automobile), contrasting aerial and ground transportation modes; (2) class 4 vs. class 7 (deer vs. horse), comparing quadrupedal mammals with differing body shapes and textures; and (3) class 6 vs. class 8 (frog vs. ship), contrasting natural amphibians with large-scale artificial structures. These pairings are carefully selected to challenge the model across a spectrum of low-level (e.g., color, shape) and high-level (e.g., semantics, context) visual cues.

We train a three-layer convolutional neural network as the black-box model, achieving 94% test accuracy. The selector networks used in both L2X and our method are implemented as three-layer fully convolutional networks.

Table 11. Post-hoc accuracy (CIFAR-10, class 2 (birds) and class 9 (trucks)). *, **, and *** indicate statistically significant improvement over L2X at $p<0.05$, $p<0.01$, and $p<0.001$, respectively. The best performance is in bold.

Method	20% Pixels	30% Pixels	40% Pixels
Ours	$\mathbf{0}.\mathbf{608}\pm \mathbf{0}.\mathbf{055}$ **	$\mathbf{0}.\mathbf{728}\pm \mathbf{0}.\mathbf{047}$ ***	$\mathbf{0}.\mathbf{785}\pm \mathbf{0}.\mathbf{029}$ ***
INVASE	$0.582\pm 0.042$ *	$0.693\pm 0.032$ **	$0.752\pm 0.028$ **
L2X	$0.513\pm 0.125$	$0.605\pm 0.012$	$0.661\pm 0.011$
GradCAM	$0.576$	$0.658$	$0.703$
Saliency	$0.549$	$0.574$	$0.615$

,

Table 12. Post-hoc accuracy (CIFAR-10, class 0 (airplanes) and class 1 (automobiles)). *, **, and *** indicate statistically significant improvement over L2X at $p<0.05$, $p<0.01$, and $p<0.001$, respectively. The best performance is in bold.

Method	20% Pixels	30% Pixels	40% Pixels
Ours	$\mathbf{0}.\mathbf{641}\pm \mathbf{0}.\mathbf{048}$ **	$\mathbf{0}.\mathbf{755}\pm \mathbf{0}.\mathbf{036}$ ***	$\mathbf{0}.\mathbf{801}\pm \mathbf{0}.\mathbf{027}$ ***
INVASE	$0.624\pm 0.041$ *	$0.727\pm 0.034$ **	$0.778\pm 0.028$ **
L2X	$0.576\pm 0.096$	$0.684\pm 0.015$	$0.730\pm 0.014$
GradCAM	$0.591$	$0.662$	$0.703$
Saliency	$0.563$	$0.601$	$0.648$

,

Table 13. Post-hoc accuracy (CIFAR-10, class 4 (deer) and class 7 (horses)). *, **, and *** indicate statistically significant improvement over L2X at $p<0.05$, $p<0.01$, and $p<0.001$, respectively. The best performance is in bold.

Method	20% Pixels	30% Pixels	40% Pixels
Ours	$\mathbf{0}.\mathbf{615}\pm \mathbf{0}.\mathbf{046}$ **	$\mathbf{0}.\mathbf{736}\pm \mathbf{0}.\mathbf{033}$ ***	$\mathbf{0}.\mathbf{789}\pm \mathbf{0}.\mathbf{027}$ ***
INVASE	$0.598\pm 0.041$ *	$0.711\pm 0.032$ **	$0.763\pm 0.028$ **
L2X	$0.554\pm 0.078$	$0.662\pm 0.017$	$0.716\pm 0.015$
GradCAM	$0.567$	$0.641$	$0.687$
Saliency	$0.547$	$0.596$	$0.642$

and

Table 14. Post-hoc accuracy (CIFAR-10, class 6 (frogs) and class 8 (ships)). *, **, and *** indicate statistically significant improvement over L2X at $p<0.05$, $p<0.01$, and $p<0.001$, respectively. The best performance is in bold.

Method	20% Pixels	30% Pixels	40% Pixels
Ours	$\mathbf{0}.\mathbf{633}\pm \mathbf{0}.\mathbf{051}$ **	$\mathbf{0}.\mathbf{752}\pm \mathbf{0}.\mathbf{039}$ ***	$\mathbf{0}.\mathbf{801}\pm \mathbf{0}.\mathbf{027}$ ***
INVASE	$0.615\pm 0.047$ *	$0.723\pm 0.036$ **	$0.774\pm 0.028$ **
L2X	$0.562\pm 0.091$	$0.672\pm 0.019$	$0.719\pm 0.017$
GradCAM	$0.579$	$0.644$	$0.693$
Saliency	$0.552$	$0.593$	$0.641$

summarize the post-hoc accuracy results, while Figure 13, Figure 14, Figure 15 and Figure 16 visualize the explanations generated by all competing approaches. Our method consistently achieves the highest accuracy across tasks and produces explanations that align closely with human perceptual understanding.

4.3. Ablation Study

We finally provide an ablation study to evaluate the sensitivity of our method with respect to the entropy order $\alpha$ used in the matrix-based Rényi’s $\alpha$-order entropy functional. Specifically, we varied $\alpha$ from 0.5 to 5 in increments of 0.5 to perform binary classification between digits 3 and 8 in the MNIST dataset. The results, summarized in

Table 15. Post-hoc accuracy (MNIST 3 vs. 8) for different values of $\alpha$ in our method.

Method	k = 4	k = 6	k = 8
Ours ($\alpha =0.5$)	0.947 ± 0.006	0.971 ± 0.006	0.981 ± 0.006
Ours ($\alpha =1.01$)	0.951 ± 0.007	0.974 ± 0.004	0.984 ± 0.004
Ours ($\alpha =1.5$)	0.948 ± 0.006	0.972 ± 0.006	0.982 ± 0.006
Ours ($\alpha =2.0$)	0.945 ± 0.006	0.970 ± 0.006	0.978 ± 0.006
Ours ($\alpha =2.5$)	0.942 ± 0.007	0.968 ± 0.007	0.975 ± 0.007
Ours ($\alpha =3.0$)	0.939 ± 0.007	0.962 ± 0.007	0.972 ± 0.007
Ours ($\alpha =3.5$)	0.936 ± 0.008	0.959 ± 0.008	0.969 ± 0.008
Ours ($\alpha =4.0$)	0.934 ± 0.009	0.956 ± 0.009	0.967 ± 0.009
Ours ($\alpha =4.5$)	0.931 ± 0.010	0.953 ± 0.010	0.964 ± 0.010
Ours ($\alpha =5.0$)	0.929 ± 0.011	0.950 ± 0.011	0.962 ± 0.011
INVASE	0.945 ± 0.006	0.969 ± 0.005	0.982 ± 0.004

, show that our method consistently outperforms INVASE (the second-best method) across a wide range of $\alpha$ values. This indicates that our framework is robust to the choice of $\alpha$, and maintains strong performance even as the entropy order varies, especially in the range $\alpha \in (0.5,2)$.

5. Conclusions and Future Work

We introduce a new approach for interpreting deep vision models via causally grounded instance-wise feature selection. By aligning feature subsets with maximal conditional influence on outputs, our method produces compact and informative explanations. Experimental results demonstrate superior performance in causal effect and predictive alignment, showing promise for real-world deployment.

Future work is twofold. On the one hand, although this paper employs the matrix-based Rényi’s $\alpha$-order entropy functional to effectively estimate the conditional mutual information term without requiring density estimation or variational approximation, the computational complexity remains $\mathcal{O}\left(M^3\right)$, where M is the mini-batch size. In future work, we plan to adopt a fast computation scheme for the matrix-based Rényi’s $\alpha$-order entropy functional [31], which leverages randomized numerical linear algebra techniques and reduces the complexity to below $\mathcal{O}\left(M^2\right)$.

On the other hand, while our method is primarily demonstrated on visual classification tasks, its core framework maximizes a differentiable estimate of the conditional mutual information (CMI) between selected features and model outputs, is modality-agnostic, and can be conceptually extended to other domains such as text or tabular data. The matrix-based Rényi entropy estimator used for CMI depends only on positive-definite kernels and does not rely on domain-specific assumptions, which makes it applicable beyond image data.

For text data, individual tokens can be treated as features [3], with masking performed by replacing tokens with a special [MASK] token or by dropping their embeddings. For tabular data, features correspond to columns or column groups, and masking can be implemented by substituting values with their mean or a designated sentinel value. In both cases, the explainer network and kernel function can be adapted to the data modality, while the overall training and optimization pipeline remains unchanged.