Information Theory Meets Quantum Chemistry: A Review and Perspective

Abstract

In this survey, we begin with a concise introduction to information theory within Shannon’s framework, focusing on the key concept of Shannon entropy and its related quantities: relative entropy, joint entropy, conditional entropy, and mutual information. We then demonstrate how to apply these information-theoretic tools in quantum chemistry, adopting either classical or quantum formalisms based on the choice of information carrier involved.

1. Motivation

Information theory was first established in the 1920s through the works of Harry Nyquist and Ralph Hartley [1,2] and propelled to prominence in the 1940s by Claude Shannon [3]. The concept of information theory, which encompasses the quantification, storage, and communication of information [4,5], is too broad to be simply described. Today, information theory serves as a versatile tool in statistics, natural sciences, machine learning, quantum computing, and numerous other fields.

In molecular electronic structure theory, we routinely encounter sets of non-negative values that sum to unity, corresponding to valid probability distributions. This includes properly normalized electron density distributions, the eigenvalues of the reduced density matrix, the squared modulus of wave function coefficients in an orthonormal basis, and more [6,7]. Information theory can be used to analyze these probability distributions, an approach that has been actively pursued by researchers with remarkable success since the 1970s [8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34]. Incorporating concepts from information theory into quantum chemistry has provided valuable insights into the nature and behavior of electronic systems.

The two most popular approaches to electronic structure theory are quantum many-body theory [35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50] and density functional theory (DFT) [51,52,53,54]. Integrating these strategies with classical information theory (CIT) leads to what is normally called the information-theoretic approach (ITA) [8,9,10,11,12,13,14,25,26,27,28,29,30,31,32,33,34]; integrating these strategies with quantum information theory (QIT) [15,16,17,18,19,20,21,22,23,24] leads to a different set of tools, with different interpretations and utility. Some key concepts of classical and quantum information theory in quantum chemistry that we will discuss in this article are summarized in Figure 1. Although information theory extends far beyond Shannon’s work, other frameworks such as Rényi and Fisher information have their own application domains [55,56,57,58]. Due to space limitations, our discussion in this paper will be confined to Shannon’s framework, although we note that the Rényi formulation and Tsallis formulations [55,57], in particular, reduce to Shannon’s in appropriate limits and, as such, the following analysis can be generalized to these quantities.

This review concisely introduces some fundamental aspects of information theory and its applications in quantum chemistry. Section 2 discusses the basic concepts of Shannon entropy and its related quantities: relative entropy, joint entropy, conditional entropy, and mutual information. To integrate quantum information theory into quantum chemistry, we introduce the reduced density matrix (RDM) in Section 3, which serves as a powerful tool to simplify the complexity of quantum states while retaining essential information about the state of a subsystem. Section 4 introduces the application of information theory in quantum chemistry; for the classical approach, we convert the reduced density matrix in the position representation to derive the electron density and pair density, which act as information carriers for classical information theory. We also explore the corresponding quantum concepts in Section 5, such as von Neumann entropy and quantum mutual information, which are used to analyze the entanglement in quantum many-body systems.

2. Brief Introduction to Information Theory

2.1. Shannon Entropy

One of the core concepts in information theory is the Shannon entropy, named after Claude Shannon, the founder of information theory [3]. Let X be a discrete random variable with alphabet $\mathcal{X}$ and probability distribution function $p\left(x\right)=Pr\{X=x\}$ for $x\in \mathcal{X}$. The Shannon entropy is defined as:

$$\begin{array}{cc}\hfill H_{\left(X\right)}& =-\sum _{x\in \mathcal{X}}p\left(x\right)logp\left(x\right)\hfill \\ \hfill & ={\mathbb{E}}_p\left[log{\displaystyle \frac{1}{p\left(X\right)}}\right]\hfill \end{array}$$

(1)

where the expectation is denoted by $\mathbb{E}$. Thus, if $X\sim p\left(x\right)$, the expected value of the random variable $g\left(X\right)$ is written as:

$${\mathbb{E}}_p\left[g\left(X\right)\right]=\sum _{x\in \mathcal{X}}g\left(x\right)p\left(x\right)$$

(2)

The Shannon entropy admits multiple interpretations, one of which states that it provides a mathematical framework to measure the uncertainty of a random event. The term entropy originates from the Greek word trope (meaning change) and was first introduced by Clausius in 1854 in the context of the second law of thermodynamics [59]. Shannon himself explained his rationale for adopting this term in a disarming way:

My greatest concern was what to call it. I thought of calling it ‘information’, but the word was overly used, so I decided to call it ‘uncertainty’. When I discussed it with John von Neumann, he had a better idea. Von Neumann told me, “You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, nobody knows what entropy really is, so in a debate you will always have an advantage [60].

2.2. Relative Entropy

To quantify how distinguishable two probability distributions are, by introducing a second probability distribution $q\left(x\right)=Pr\{X=x\}$, where $x\in \mathcal{X}$, we now look for a distinguishability measure of those two probability distributions [61].

Analogous to the concept of distance (norm) in Euclidean space, information theory defines the relative entropy (information divergence) $D_{\left(P\right|\left|Q\right)}$ between two probability distributions $p\left(x\right)$ and $q\left(x\right)$. Among the most popular divergence measures include the Bregman divergences [62,63] and f-divergences [64,65,66]. The f-divergence is defined as follows:

$$D_{f\left(P\right|\left|Q\right)}=\sum _{x\in \mathcal{X}}f\left({\displaystyle \frac{p\left(x\right)}{q\left(x\right)}}\right)q\left(x\right)$$

(3)

where $f\left(x\right)$ is a convex function satisfying $f\left(1\right)=0$. The Bregman divergence is defined as follows:

$$D_{F\left(P\right|\left|Q\right)}=F\left(P\right)-F\left(Q\right)-\sum _{x\in \mathcal{X}}{\displaystyle \frac{\delta F\left[p\right(x\left)\right]}{\delta q\left(x\right)}}\left(p\left(x\right)-q\left(x\right)\right)$$

(4)

where $F\left[p\right]$ is a convex functional.

Most measures of relative entropy fit these frameworks, but the Kullback–Leibler (KL) divergence [67] is both an f-divergence and a Bregman divergence. Therefore, the Kullback–Leibler divergence is the most widely used definition of relative entropy, with its expression given by the following:

$$\begin{array}{cc}\hfill D_{KL\left(P\right|\left|Q\right)}& =\sum _{x\in \mathcal{X}}p\left(x\right)log\left({\displaystyle \frac{\begin{array}{c}\hfill p\left(x\right)\end{array}}{\begin{array}{c}\hfill q\left(x\right)\end{array}}}\right)\hfill \\ \hfill & ={\mathbb{E}}_p\left[log{\displaystyle \frac{p_{\left(X\right)}}{q_{\left(X\right)}}}\right]\hfill \end{array}$$

(5)

Formally, a metric (such as the Euclidean norm) must satisfy four axioms for all distributions P, Q, and R: [68]

• Non-negativity: $d(P,Q)\ge 0$
• Identity of indiscernibles: $d(P,Q)=0\iff P=Q.$
• Symmetry: $d(P,Q)=d(Q,P)$
• Triangle inequality: $d(P,Q)\le d(P,R)+d(R,Q)$

Figure 2 provides an intuitive illustration of the non-symmetric nature of the Kullback–Leibler divergence. Additionally, counterexamples exist where

$$D_{KL\left(P\right|\left|Q\right)}>D_{KL\left(P\right|\left|R\right)}+D_{KL\left(R\right|\left|Q\right)}$$

(6)

demonstrating that the Kullback–Leibler divergence violates the triangle inequality. Thus, the Kullback–Leibler divergence is considered a premetric but not a full metric.

2.3. Bivariate Entropy

To study correlations between variables, it is useful to extend the Shannon entropy to the multivariate case. For simplicity, consider a pair of variables $(X,Y)$. Then, bivariate information-theoretic quantities such as joint entropy, conditional entropy, and mutual information can be defined, and their relationships are illustrated in Figure 3.

The most straightforward two-variable extension of Shannon entropy is the joint entropy $H_{(X,Y)}$, which measures the total uncertainty when considering a pair of variables together, and its corresponding expression is as follows:

$$\begin{array}{cc}\hfill H_{(X,Y)}& =-\sum _{x\in \mathcal{X}}\sum _{y\in \mathcal{Y}}p(x,y)logp(x,y)\hfill \\ \hfill & ={\mathbb{E}}_p\left[log{\displaystyle \frac{1}{p_{(X,Y)}}}\right]\hfill \end{array}$$

(7)

where $p(x,y)$ is the joint probability distribution function. Since a pair of variables can be treated as a single vector of length two, this extension does not introduce fundamentally new concepts.

Other bivariate entropy measures include conditional entropy and mutual information, which can be expressed in the framework of the Kullback–Leibler divergence we introduced in Equation (5). The conditional entropy of one variable, given another, is defined as the expected value of the entropy of the conditional distributions, averaged over the conditioning variable.

$$\begin{array}{cc}\hfill H_{\left(Y\right|X)}& =\sum _{x\in \mathcal{X}}p\left(x\right)H_{\left(Y\right|X=x)}\hfill \\ \hfill & =-\sum _{x\in \mathcal{X}}p\left(x\right)\sum _{y\in \mathcal{Y}}p\left(y\right|x)logp\left(y\right|x)\hfill \\ \hfill & =-\sum _{x\in \mathcal{X}}\sum _{y\in \mathcal{Y}}p(x,y)logp\left(y\right|x)\hfill \\ \hfill & ={\mathbb{E}}_p\left[log{\displaystyle \frac{1}{p_{\left(Y\right|X)}}}\right]\hfill \end{array}$$

(8)

Compared to the concept of entropy, which is a probabilistic measure of uncertainty; information is a measure of a reduction in that uncertainty. The mutual information is a measure of the amount of information that one variable contains about another. It is defined as follows:

$$\begin{array}{cc}\hfill I_{(X;Y)}& =D_{KL\left(p\right(x,y\left)\right|\left|p\right(x\left)p\right(y\left)\right)}\hfill \\ \hfill & =\sum _{x\in \mathcal{X}}\sum _{y\in \mathcal{Y}}p(x,y)log{\displaystyle \frac{p(x,y)}{p\left(x\right)p\left(y\right)}}\hfill \\ \hfill & ={\mathbb{E}}_p\left[log{\displaystyle \frac{p_{(X,Y)}}{p_{\left(X\right)}{p}_{\left(Y\right)}}}\right]\hfill \end{array}$$

(9)

The conditional and mutual information, defined using the Kullback–Leibler divergence, exhibit several important properties, enabling a clear and rigorous analysis.

• The chain rule.

  $$H_{(X,Y)}=H_{\left(X\right)}+H_{\left(Y\right|X)}$$

  (10)
• Subadditivity.

  $$H_{(X,Y)}\le H_{\left(X\right)}+H_{\left(Y\right)}$$

  (11)
• The relationship between different bivariate entropy,

  $$\begin{array}{cc}\hfill I_{(X;X)}& =H_{\left(X\right)}\hfill \end{array}$$

  (12)

  $$\begin{array}{cc}\hfill I_{(X;Y)}& =I_{(Y;X)}\hfill \end{array}$$

  (13)

  $$\begin{array}{cc}\hfill I_{(X;Y)}& =H_{\left(X\right)}-H_{\left(X\right|Y)}\hfill \end{array}$$

  (14)

  $$\begin{array}{cc}\hfill I_{(X;Y)}& =H_{\left(Y\right)}-H_{\left(Y\right|X)}\hfill \end{array}$$

  (15)

  $$\begin{array}{cc}\hfill I_{(X;Y)}& =H_{\left(X\right)}+H_{\left(Y\right)}-H_{(X,Y)}\hfill \end{array}$$

  (16)

  $$\begin{array}{cc}\hfill I_{(X;Y)}& =H_{(X,Y)}-H_{\left(X\right|Y)}-H_{(Y,X)}\hfill \end{array}$$

  (17)

3. Basic Ingredients of Information Theory in Quantum Chemistry: Reduced Density Matrix

First proposed by Paul Dirac in 1930 [69], the reduced density matrix (RDM) becomes a fundamental tool in quantum chemistry and many-body physics, enabling the analysis of subsystems within larger quantum systems [70,71,72,73,74]. Because electron–electron repulsion is a two-body operator, the one- and two-electron reduced density matrices (1-RDM and 2-RDM) are the quantities of greatest interest in quantum chemistry, as it suffices to determine most molecular properties, including the electronic energy [75,76,77]. Similarly, for the application of information theory in quantum chemistry, the reduced density matrix emerges as the natural mathematical framework for elucidating the correlations and entanglement [78,79].

3.1. Density Matrix and Reduced Density Matrix

For quantum many-body systems, other than the usual state vectors $|\Psi \rangle$ (mathematically described as rays in a projective Hilbert space), quantum states can also be represented by the density matrix $\mathbf{D}$ (density operator $\widehat{D}$), which unifies the description of pure and mixed states. If a density matrix is obtained as

$$\mathbf{D}=|\Psi \rangle \langle \Psi |$$

(18)

then the quantum state is a pure state. A mixed state cannot be expressed this way and instead requires a statistical mixture,

$$\mathbf{D}=\sum _n{\omega }_n|{\Psi }_n\rangle \langle {\Psi }_n|({\omega }_n\ge 0,\sum _n{\omega }_n=1)$$

(19)

where $\left\{\right|{\Psi }_n\rangle \}$ is an ensemble of pure states with weights (probability) ${\omega }_n\ge 0$.

To further elucidate the concepts of pure states and mixed states, we employ a two-state quantum system as an illustrative example. It is often implemented using a spin ${\displaystyle \frac{1}{2}}$ physical particle, with two basis states $\left\{\right|0\rangle ,|1\rangle \}$. A pure state of such a two-state quantum system is represented by a state vector:

$$|\psi \rangle =\alpha |0\rangle +{\beta |1\rangle |\alpha |}^2+{\left|\beta \right|}^2=1\alpha ,\beta \in \mathbb{C}$$

(20)

The constraint conditions in Equation (20) indicate that a pure state of a two-state quantum system can be equivalently expressed as

$$|\psi \rangle =cos\left({\displaystyle \frac{\theta }{2}}\right)|0\rangle +e^{i\phi }sin\left({\displaystyle \frac{\theta }{2}}\right)|1\rangle \theta \in [0,\pi ],\phi \in [0,2\pi ]$$

(21)

Thus, the pure state of a two-state quantum system can be uniquely determined by two parameters $\theta$ and $\phi$. Here, we can establish a geometric representation for this two-state quantum system: every pure state $|\psi \rangle$ defined in Equation (21) uniquely corresponds to a point (cos$\phi$sin$\theta$, sin$\phi$sin$\theta$, cos$\theta$) on the surface of the Bloch sphere as shown in Figure 4.

For the mixed states (the same applies to pure states) of a two-state quantum system, it is evident that the density matrix can be represented by a 2 × 2 matrix, since

$$\begin{array}{cc}\hfill \mathbf{D}& =\sum _n{\omega }_n|{\psi }_n\rangle \langle {\psi }_n|\hfill \\ \hfill & =\sum _n{\omega }_n\left(\right|{\alpha }_n{|}^2|0\rangle \langle 0|+{\alpha }_n{\beta }_n^{*}|0\rangle \langle 1|+{\alpha }_n^{*}{\beta }_n|1\rangle \langle 0|+|{\beta }_n{|}^2|1\rangle \langle 1\left|\right)\hfill \\ \hfill & =\left[\begin{array}{cc}\sum _n{\omega }_n{\left|{\alpha }_n\right|}^2& \sum _n{\omega }_n{\alpha }_n{\beta }_n^{*}\\ \sum _n{\omega }_n{\alpha }_n^{*}{\beta }_n& \sum _n{\omega }_n{\left|{\beta }_n\right|}^2\end{array}\right]\hfill \end{array}$$

(22)

It can be rigorously proven that every point within the Bloch sphere corresponds to a mixed state. In particular, the center point represents the maximally mixed state, as established in quantum information theory [80].

For conciseness, we will henceforth assume that the density matrix (DM) and the reduced density matrix (RDM) correspond to pure states, but most of the analysis extends directly to mixed states. The k-electron reduced density matrix (k-RDM) is defined as the partial trace, over $N-k$ dimensions, of an N-electron density matrix; it is usually expanded on the basis of single-particle states (i.e., spin orbitals). Specifically,

$${}^k\overline{\mathbf{D}}=\sum _{p_1,\dots ,p_k,q_1,\dots ,q_k}|p_1,\dots ,p_k\rangle \langle q_1,\dots ,p_k|\left({}^k\overline{D}_{q_1,\dots ,q_k}^{p_1,\dots ,p_k}\right)$$

(23)

where

$${}^k\overline{D}_{q_1,\dots ,q_k}^{p_1,\dots ,p_k}=\langle \Psi |a_{p_1}^{†}\dots ,a_{p_k}^{†}{a}_{q_k},\dots ,a_{q_1}|\Psi \rangle$$

(24)

are elements of the k-RDM, which is Hermitian and positive semidefinite because the density matrix is Hermitian and positive semidefinite. We use the overbar above symbols when we need to explicitly distinguish quantities expressed in the spin-orbital, rather than the spatial orbital, basis.

3.2. 1-RDM and 2-RDM

The quantities of greatest interest to chemists are the one- and two-electron reduced density matrices. The 1-RDM is defined as follows:

$$\begin{array}{cc}\hfill {}^1\overline{\mathbf{D}}& =\sum _{pq}|p\rangle \langle q|\langle \Psi |a_p^{†}{a}_q|\Psi \rangle \hfill \\ \hfill & =\sum _{pq}{}^1\overline{D}_q^p|p\rangle \langle q|\hfill \end{array}$$

(25)

and the 2-RDM is defined as follows:

$$\begin{array}{cc}\hfill {}^2\overline{\mathbf{D}}& =\sum _{pqrs}|pq\rangle \langle rs|\langle \Psi |a_p^{†}{a}_q^{†}{a}_s{a}_r|\Psi \rangle \hfill \\ \hfill & =\sum _{pqrs}{}^2\overline{D}_{rs}^{pq}|pq\rangle \langle rs|\hfill \end{array}$$

(26)

In our convention, the trace of the 1-RDM is the number of electrons and the trace of the 2-RDM is the number of electron pairs:

$$\begin{array}{cc}\hfill \mathrm{Tr}\left[{}^{\mathbf{1}}\overline{\mathbf{D}}\right]& =N\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill \mathrm{Tr}\left[{}^{\mathbf{2}}\overline{\mathbf{D}}\right]& =\left({\displaystyle \genfrac{}{}{0pt}{}{N}{2}}\right)={\displaystyle \frac{N(N-1)}{2}}\hfill \end{array}$$

(28)

The reader is cautioned that some authors use different normalization conventions (e.g., unit normalization or the number of non-unique electron pairs, $\mathrm{Tr}\left[{}^{\mathbf{2}}\overline{\mathbf{D}}\right]=N(N-1)$).

The reduced density matrix (RDM) can be represented in a spin block format because the projection of the spin vector onto a specified axis, ${\widehat{S}}^z$, commutes with the molecular Hamiltonian.

$$[\widehat{H},{\widehat{S}}^z]=\widehat{H}{\widehat{S}}^z-{\widehat{S}}^z\widehat{H}=0$$

(29)

Thus, the one-particle reduced density matrix (1-RDM) will have a block-diagonal form.

$${}^1\overline{\mathbf{D}}=\left[\begin{array}{cc}{}^1\overline{\mathbf{D}}_{\alpha }^{\alpha }& 0\\ 0& {}^1\overline{\mathbf{D}}_{\beta }^{\beta }\end{array}\right]$$

(30)

Similarly, the spin-block format of the two-particle reduced density matrix (2-RDM) is as follows:

$${}^2\overline{\mathbf{D}}=\left[\begin{array}{cccc}{}^2\overline{\mathbf{D}}_{\alpha \alpha }^{\alpha \alpha }& 0& 0& 0\\ 0& {}^2\overline{\mathbf{D}}_{\alpha \beta }^{\alpha \beta }& {}^2\overline{\mathbf{D}}_{\beta \alpha }^{\alpha \beta }& 0\\ 0& {}^2\overline{\mathbf{D}}_{\alpha \beta }^{\beta \alpha }& {}^2\overline{\mathbf{D}}_{\beta \alpha }^{\beta \alpha }& 0\\ 0& 0& 0& {}^2\overline{\mathbf{D}}_{\beta \beta }^{\beta \beta }\end{array}\right]$$

(31)

Note that knowledge of any one of the four opposite-spin blocks, e.g., ${}^2\overline{\mathbf{D}}_{\beta \alpha }^{\alpha \beta }$, suffices to determine the others using (anti)symmetry.

3.3. 3-RDM and 4-RDM

In general, reduced density matrices of order greater than two are essentially redundant for most quantum chemistry problems, as electrons interact only pairwise. However, for certain niche quantum chemistry applications, the information provided by the 3-RDM and 4-RDM is still required. The three-electron reduced density matrix (3-RDM) can be explicitly defined as follows:

$$\begin{array}{cc}\hfill {}^3\overline{\mathbf{D}}& =\sum _{pqrstu}|pqr\rangle \langle stu|\langle \Psi |a_p^{†}{a}_q^{†}{a}_r^{†}{a}_u{a}_t{a}_s|\Psi \rangle \hfill \\ \hfill & =\sum _{pqrstu}{}^3\overline{D}_{stu}^{pqr}|pqr\rangle \langle stu|\hfill \end{array}$$

(32)

and, similarly, the four-electron reduced density matrix (4-RDM) is defined as follows:

$$\begin{array}{cc}\hfill {}^4\overline{\mathbf{D}}& =\sum _{pqrstuvw}|pqrs\rangle \langle tuvw|\langle \Psi |a_p^{†}{a}_q^{†}{a}_r^{†}{a}_s^{†}{a}_w{a}_v{a}_u{a}_t|\Psi \rangle \hfill \\ \hfill & =\sum _{pqrstuvw}{}^4\overline{D}_{tuvw}^{pqrs}|pqrs\rangle \langle tuvw|\hfill \end{array}$$

(33)

We can easily determine lower-order reduced density matrices (RDM) by taking the partial trace of a higher-order one. However, using higher-order reduced density matrices than required is undesirable, as the computational cost and memory requirements of high-order reduced density matrices are prohibitively large. For example, the storage required to store the complete 3-RDM and 4-RDM scales as $O\left(n^6\right)$ and $O\left(n^8\right)$, respectively, where n is the number of basis functions. Higher-order reduced density matrices can be systematically, but approximately, expressed in terms of lower-order reduced density matrices using diagrammatic and statistical techniques [81,82,83,84,85,86,87,88,89,90,91].

Specifically, using the cumulant expansion one can decompose higher-order RDM into sums of products of lower-order quantities and nonreducible k-order cumulants ${}^k\Delta$ [81,82,88,92]. As a starting point, the 1-RDM can be expressed as the sum of a mean-field term and a correlated cumulant term:

$${}^{1}D_q^p={\left({}^{1}D_0\right)}_q^p+{}^1\Delta _q^p$$

(34)

where ${}^{1}D_0$ is the known mean-field 1-RDM, the two-particle reduced density matrix (2-RDM) can then be expressed as the wedge (or Grassmann) product of two one-particle reduced density matrices (1-RDM) and a cumulant 2-RDM.

$${}^{2}D_{rs}^{pq}=2{}^{1}D_r^p\wedge {}^{1}D_s^q+{}^2\Delta _{rs}^{pq}$$

(35)

where the wedge product, denoted as ∧, is defined as an antisymmetric tensor product. Given two reduced density matrices of orders k and l, denoted ${}^{k}D$ and ${}^{l}D$ respectively, their wedge product yields a (k + l)-order RDM satisfying

$$\begin{array}{c}\hfill {}^{k}D\wedge {}^{l}D={\displaystyle \frac{1}{(k+l)!}}\sum _{\pi }\mathrm{sgn}\left(\pi \right){\mathcal{P}}_{\pi }({}^{k}D\otimes {}^{l}D)\end{array}$$

(36)

where ${\mathcal{P}}_{\pi }$ permutes the upper and lower indices according to the permutation $\pi$. $\mathrm{sgn}\left(\pi \right)$ is the parity of the permutation (+1 for even, −1 for odd). As the most elementary example, the wedge product of two 1-RDMs is given by

$${}^{1}D_r^p\wedge {}^{1}D_s^q={}^{1}D_r^p{}^{1}D_s^q-{}^{1}D_s^p{}^{1}D_r^q$$

(37)

The cumulant expansions of the three-particle and four-particle reduced density matrices (3-RDM and 4-RDM) are given by

$${}^{3}D_{stu}^{pqr}=6{}^{1}D_s^p\wedge {}^{1}D_t^q\wedge {}^{1}D_u^r+9{}^{2}D_{st}^{pq}\wedge {}^{1}D_u^r+{}^3\Delta _{stu}^{pqr}$$

(38)

and

$$\begin{array}{cc}\hfill {}^{4}D_{tuvw}^{pqrs}& =24{}^{1}D_t^p\wedge {}^{1}D_u^q\wedge {}^{1}D_v^r\wedge {}^{1}D_w^s+72{}^{2}D_{tu}^{pq}\wedge {}^{1}D_v^r\wedge {}^{1}D_w^s\hfill \\ \hfill & \begin{array}{c}\hfill +\end{array}24{}^{2}D_{tu}^{pq}\wedge {}^{2}D_{vw}^{rs}+16{}^{3}D_{tuv}^{pqr}\wedge {}^{1}D_w^s+{}^4\Delta _{tuvw}^{pqrs}\hfill \end{array}$$

(39)

The traces of the density matrices in these expressions give the number of distinguishable pairs, triples, and quartets of electrons, respectively. Dividing by $k!$, where k is the order of the reduced density matrix, recovers the definition we’ve used elsewhere.

4. Classical Information Theory in Quantum Chemistry

The Hohenberg–Kohn theorems [51,52,93,94,95] imply that all ground state properties are functionals of the electron density; this establishes the theoretical basis for using classical information theory that extracts chemical insights by treating electron densities as probability distributions.

4.1. Electron Density in Position Space

In Section 3, we establish reduced density matrices as the basic ingredients for information-theoretic analysis in quantum chemistry. For molecular systems, the concept of probability distribution $p\left(x\right)$ in general information theory is specialized as an analytical functional of the electron density $\rho \left(\mathbf{r}\right)$, which depends solely on the spatial coordinates, $\mathbf{r}\in {\mathbb{R}}^3$:

$$\rho \left(\mathbf{r}\right)=N\int |\Psi ({\mathbf{x}}_{\mathbf{1}},{\mathbf{x}}_{\mathbf{2}},\dots ,{\mathbf{x}}_{\mathbf{N}}){|}^{2}d{s}_{1}d{\mathbf{x}}_{\mathbf{2}}\dots d{\mathbf{x}}_{\mathbf{N}}$$

(40)

By construction,

$$\rho \left(\mathbf{r}\right)\ge 0,\int \rho \left(\mathbf{r}\right)d\mathbf{r}=N$$

(41)

Another fundamental information carrier in quantum chemistry is the two-electron distribution function, or pair density, defined as

$$\Gamma ({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})={\displaystyle \frac{N(N-1)}{2}}\int {|\Psi ({\mathbf{x}}_{\mathbf{1}},{\mathbf{x}}_{\mathbf{2}},\dots ,{\mathbf{x}}_{\mathbf{N}})|}^{2}d{s}_{1}d{s}_{2}d{\mathbf{x}}_{\mathbf{3}}\dots d{\mathbf{x}}_{\mathbf{N}}$$

(42)

By definition,

$$\Gamma ({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})\ge 0,\int \int \Gamma ({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})d{\mathbf{r}}_{\mathbf{1}}d{\mathbf{r}}_{\mathbf{2}}={\displaystyle \frac{N(N-1)}{2}}$$

(43)

Analogous to density-functional theory, a complete description of electronic structure can be constructed based on the pair density (and also higher-order electron distribution functions) using appropriate generalizations of the Hohenberg–Kohn theorem [96,97,98,99,100,101,102,103,104].

The electron density and pair density can be computed from the reduced density matrices (RDMs) we introduced in Section 3. However, in that section, we considered the spin-resolved reduced density matrices, and it is more convenient in this context to trace out the spin coordinates and obtain a representation of the reduced density matrices in terms of spatial orbitals, i.e.,

$$\begin{array}{cc}\hfill {}^1\mathbf{D}& ={}^1\overline{\mathbf{D}}_{\alpha }^{\alpha }+{}^1\overline{\mathbf{D}}_{\beta }^{\beta }\hfill \end{array}$$

(44)

$$\begin{array}{cc}\hfill {}^2\mathbf{D}& ={}^2\overline{\mathbf{D}}_{\alpha \alpha }^{\alpha \alpha }+{}^2\overline{\mathbf{D}}_{\alpha \beta }^{\alpha \beta }+{}^2\overline{\mathbf{D}}_{\beta \alpha }^{\beta \alpha }+{}^2\overline{\mathbf{D}}_{\beta \beta }^{\beta \beta }\hfill \end{array}$$

(45)

Next, we transform the one- and two-electron reduced density matrices from the (second-quantized) spatial-orbital representation into the (first-quantized) position representation,

$$\begin{array}{cc}\hfill \rho (\mathbf{r};{\mathbf{r}}^{\prime })& ={}^{1}D_{\nu }^{\mu }{\varphi }_{\mu }\left(\mathbf{r}\right){\varphi }_{\nu }\left({\mathbf{r}}^{\prime }\right)\hfill \end{array}$$

(46)

$$\begin{array}{cc}\hfill \Gamma ({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}};{\mathbf{r}}_{\mathbf{1}}^{\prime },{\mathbf{r}}_{\mathbf{2}}^{\prime })& ={}^{2}D_{\kappa \lambda }^{\mu \nu }{\varphi }_{\mu }\left({\mathbf{r}}_{\mathbf{1}}\right){\varphi }_{\nu }\left({\mathbf{r}}_{\mathbf{2}}\right){\varphi }_{\kappa }\left({\mathbf{r}}_{\mathbf{1}}^{\prime }\right){\varphi }_{\lambda }\left({\mathbf{r}}_{\mathbf{2}}^{\prime }\right)\hfill \end{array}$$

(47)

We use the indices $\mu$, $\nu$, $\kappa$, and $\lambda$ to label atomic orbitals. The one-electron density $\rho \left(\mathbf{r}\right)$ (Equation (40)) and the pair-electron density $\Gamma ({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})$ (Equation (42)) represent the diagonal components of the spinless one- and pair-electron reduced density matrix in position space, respectively, which is obtained by setting the un-primed spatial variables equal to the primed spatial variables, ${\mathbf{r}}_i={\mathbf{r}}_i^{\prime }$. Note that off-diagonal elements of the orbital representation of the RDM contribute to diagonal elements of the spatial representation of the RDM and vice versa. This has significant implications for the N-representable of electron distribution functions [96,105].

4.2. Information-Theoretic Approach Chemical Descriptors

Taking electron density as the information carrier, classical information theory has been applied to DFT since the mid-20th century to study atoms and molecular systems within the framework of the information-theoretic approach (ITA) [8,9,10,11,12,13,14,25,26,27,28,29,30,31,32,33,34].

The Shannon entropy, with $\rho \left(\mathbf{r}\right)$ as its information carrier, measures the spatial delocalization of the electron density and is defined as follows:

$$S_S\equiv S_{S\left(\mathbf{X}\right)}=-\int \rho \left(\mathbf{r}\right)log\rho \left(\mathbf{r}\right)d\mathbf{r}$$

(48)

In accordance with the traditions of the information-theoretic approach, the base-10 logarithm is used throughout Section 4.2. In the information-theoretic approach, we employ a broader set of formulas beyond Shannon entropy. To maintain notational clarity, we adopt a systematic symbolic convention where Shannon’s formula is specifically denoted by the symbol $S_S$. When only the one-electron density is considered, the subscript_(X) in Equation (48) is typically omitted.

By introducing the reference density ${\rho }^0\left(\mathbf{r}\right)$, which corresponds to the electron density of the promolecule constructed under the assumption that each atom retains its density as if it were isolated [106,107,108,109,110,111], we can define the relative Shannon entropy $S_S^r$. This quantity, also referred to as the Kullback–Leibler divergence, information gain, or information divergence [14,26,27,111,112,113,114,115], is defined by

$$S_S^r\equiv D_{KL\left(\mathbf{X}\right|\left|{\mathbf{X}}^0\right)}=\int \rho \left(\mathbf{r}\right)log{\displaystyle \frac{\rho \left(\mathbf{r}\right)}{{\rho }^0\left(\mathbf{r}\right)}}d\mathbf{r}$$

(49)

The joint entropy $S_{S(\mathbf{X},\mathbf{Y})}$, which measures the localization of the pair of electrons in their respective spaces, is defined as follows:

$$S_{S(\mathbf{X},\mathbf{Y})}=-\int \int \Gamma ({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})log\Gamma ({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})d{\mathbf{r}}_{\mathbf{1}}d{\mathbf{r}}_{\mathbf{2}}$$

(50)

In addition to joint entropy, other bivariate entropy measures need to maintain the same normalization condition for the state density and the reference state density. Thus, the normalized one- and pair-electron densities ${\rho }^{\sigma }\left(\mathbf{r}\right)$ and ${\Gamma }^{\sigma }({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})$ are defined as

$${\rho }^{\sigma }\left(\mathbf{r}\right)\equiv \sigma \left(\mathbf{r}\right)=\rho \left(\mathbf{r}\right)/N$$

(51)

and

$${\Gamma }^{\sigma }({\mathbf{r}}_1,{\mathbf{r}}_2)=\Gamma ({\mathbf{r}}_1,{\mathbf{r}}_2)/\left({\displaystyle \genfrac{}{}{0pt}{}{N}{2}}\right)$$

(52)

where N is the number of electrons. Following the definition of Parr and Bartolotti in 1983 [116], these unit-normalized densities are also referred to as shape functions $\sigma \left(\mathbf{r}\right)$ [116,117,118,119]. They exhibit the obvious non-negative properties ${\rho }^{\sigma }\left(\mathbf{r}\right)\ge 0\forall \mathbf{r}$ and ${\Gamma }^{\sigma }({\mathbf{r}}_1,{\mathbf{r}}_2)\ge 0\forall \{{\mathbf{r}}_1,{\mathbf{r}}_2\}$.

Using the normalized pair electron density as the distribution functions, the joint entropy $S_{S(\mathbf{X},\mathbf{Y})}^{\sigma }$ is defined as

$$S_{S(\mathbf{X},\mathbf{Y})}^{\sigma }=-\int \int {\Gamma }^{\sigma }({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})log{\Gamma }^{\sigma }({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})d{\mathbf{r}}_{\mathbf{1}}d{\mathbf{r}}_{\mathbf{2}}$$

(53)

The conditional entropy is defined as the Kullback–Leibler divergence of the unit-normalized pair electron density from the unit-normalized electron densities.

$$\begin{array}{cc}\hfill S_{S\left(\mathbf{X}\right|\mathbf{Y})}^{\sigma }& =-D_{KL\left({\Gamma }^{\sigma }({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})\right|\left|{\rho }^{\sigma }\left({\mathbf{r}}_{\mathbf{2}}\right)\right)}\hfill \\ \hfill & =-\int {\rho }^{\sigma }\left({\mathbf{r}}_{\mathbf{2}}\right)d{\mathbf{r}}_{\mathbf{2}}\int {\Gamma }^{\sigma }\left({\mathbf{r}}_{\mathbf{1}}\right|{\mathbf{r}}_{\mathbf{2}})log{\Gamma }^{\sigma }\left({\mathbf{r}}_{\mathbf{1}}\right|{\mathbf{r}}_{\mathbf{2}})d{\mathbf{r}}_{\mathbf{1}}\hfill \\ \hfill & =-\int \int {\Gamma }^{\sigma }({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})log{\displaystyle \frac{{\Gamma }^{\sigma }({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})}{{\rho }^{\sigma }\left({\mathbf{r}}_{\mathbf{2}}\right)}}d{\mathbf{r}}_{\mathbf{1}}d{\mathbf{r}}_{\mathbf{2}}\hfill \end{array}$$

(54)

The mutual information $I_{(\mathbf{X};\mathbf{Y})}$, is defined as the Kullback–Leibler divergence of the unit-normalized pair electron density from the product of two unit-normalized electron densities. The mutual information then measures the divergence of the pair electron density from the value it would have if the electrons moved entirely independently.

$$\begin{array}{cc}\hfill S_{S(\mathbf{X};\mathbf{Y})}^{\sigma }\equiv I_{(\mathbf{X};\mathbf{Y})}& =D_{KL\left({\Gamma }^{\sigma }({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})\right|\left|{\rho }^{\sigma }\left({\mathbf{r}}_{\mathbf{1}}\right){\rho }^{\sigma }\left({\mathbf{r}}_{\mathbf{2}}\right)\right)}\hfill \\ \hfill & =\int \int {\Gamma }^{\sigma }({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})log{\displaystyle \frac{{\Gamma }^{\sigma }({\mathbf{r}}_{\mathbf{1}},{\mathbf{r}}_{\mathbf{2}})}{{\rho }^{\sigma }\left({\mathbf{r}}_{\mathbf{1}}\right){\rho }^{\sigma }\left({\mathbf{r}}_{\mathbf{2}}\right)}}d{\mathbf{r}}_{\mathbf{1}}d{\mathbf{r}}_{\mathbf{2}}\hfill \end{array}$$

(55)

Within the information theory framework introduced in Section 2, which utilizes the one- and pair-electron densities as fundamental information carriers, all the classical information-theoretic quantities we defined in this subsection preserve the essential mathematical properties we introduced in Section 2.3.

Examples and Illustrations

Using the information-theoretic approach, we can systematically interpret chemical concepts such as chemical bonds, chemical reactivity, electron shells, lone electron pairs, and more [8,9,10,12,13,14,120,121,122,123,124,125,126,127,128,129,130]. The information-theoretic approach descriptors can be categorized based on the scope of molecular features they capture:

• Global Descriptors: Assign a value to the entire system.
• Local Descriptors: Assign a value to each position in the system.
• Non-local Descriptors: Assign a value to each pair of positions in the system.

The global ITA descriptors describe the overall properties of the system as a whole, enabling prediction of properties such as polarizability, aromaticity, acidity/basicity, and reactivity [9,10,12,120,121,122,123,131,132]. Figure 5 plots the correlation between Shannon entropy aromaticity ($\Delta S_S$) [120] and several established aromaticity indices: the harmonic oscillator model of aromaticity (HOMA) [133,134], the aromatic fluctuation index (FLU) [135], and nucleus-independent chemical shifts (NICS) [136,137]. The strong correlations demonstrate that Shannon entropy effectively characterizes aromaticity.

The local ITA descriptors can serve as regioselectivity indicators, through a coarse-graining process, which involves integrating their values over atoms, functional groups, or fragment regions; condensed descriptors are obtained [131,132,133]. These descriptors help identify the most reactive atoms, functional groups, or bonds. Figure 6 provides an example of how local Shannon entropy is applied to reveal electron shell structures in noble gas atoms [125]. The radial distribution functions of the Shannon entropy densities are defined as

$$\begin{array}{cc}\hfill {\overline{h}}_{\left(\mathbf{X}\right)}\left(r\right)& =\int r^2{h}_{\left(\mathbf{X}\right)}(r,\theta ,\varphi )sin\left(\theta \right)d\theta d\varphi \hfill \\ \hfill & =4\pi r^2{h}_{\left(\mathbf{X}\right)}\left(r\right)\hfill \end{array}$$

(56)

It shows step-like increases in entropy at shell boundaries.

Non-local ITA descriptors quantify how the properties of a molecule at one location respond to changes at another distant point within the same molecule. These descriptors can also be condensed into response matrices, to quantify the correlation between fragments of the system [139]. Figure 7 illustrates the application of joint entropy, conditional entropy, and the mutual information kernel to analyze electron correlation in the krypton atom.

5. Quantum Information Theory in Quantum Chemistry

In this section, we transition our discussion of information theory to the quantum realm [21,23,24,47,78,79,80,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155]. As we delve deeper, it will become evident that the quantum case holds far richer possibilities, primarily due to the superposition principle.

5.1. Bipartite Entanglement

In quantum chemistry, for a system of basis set $\left\{\right|n_p\rangle \}$ with finite cardinality L, each basis $|n_p\rangle$ associates with a local Hilbert space ${\mathcal{H}}_i$. Then, the total Hilbert space $\mathcal{H}$ spanned by the entire basis set is the tensor product of all local Hilbert spaces $\mathcal{H}={\otimes }_{p=1}^L{\mathcal{H}}_p$ [24,142,143,144]. For an arbitrary state $|\Psi \rangle \in \mathcal{H}$

$$\begin{array}{cc}\hfill |\Psi \rangle & =\sum _{n_1,\dots ,n_L}{C}_{n_1,\dots ,n_L}|n_1,\dots ,n_L\rangle \hfill \\ \hfill & =\sum _{n_1,\dots ,n_L}{C}_{n_1,\dots ,n_L}|n_1\rangle \otimes \dots \otimes |n_L\rangle \hfill \end{array}$$

(57)

When the system is divided into two parts A and B, the composite Hilbert space $\mathcal{H}\equiv {\mathcal{H}}_{AB}$ of the whole system is given as follows:

$$\mathcal{H}\equiv {\mathcal{H}}_{\mathcal{AB}}={\mathcal{H}}_A\otimes {\mathcal{H}}_B$$

(58)

where ${\mathcal{H}}_A$ and ${\mathcal{H}}_B$ are the Hilbert spaces of subsystems A and B, respectively. Only when there is no entanglement between the systems can the state $|{\Psi }_{AB}\rangle$ be represented as a tensor product of the states of the subsystems $|{\Psi }_A\rangle$ and $|{\Psi }_B\rangle$, as shown in Figure 8a.

$$|\Psi \rangle \equiv |{\Psi }_{AB}\rangle =|{\Psi }_A\rangle \otimes |{\Psi }_B\rangle$$

(59)

As a consequence of quantum entanglement among arbitrary subsystems, as shown in Figure 8b, for a generic $|\Psi \rangle \in \mathcal{H}$, it should be represented as a series of tensor products of basis states of subsystems A and B,

$$|\Psi \rangle \equiv |{\Psi }_{AB}\rangle =\sum _{pq}{C}_{pq}|{\Psi }_A^p\rangle \otimes |{\Psi }_B^q\rangle$$

(60)

The strategy for measuring bipartite entanglement emerges from the concept of partial measurements [145]. In Section 3.1, we have an introduction of the density matrix for both pure and mixed states; for the sake of brevity, here we only consider the case of pure states. A bipartite reduced density matrix of a pure state can be approached by tracing out (“averaging over”) one of the subsystems:

$${}^A\mathbf{D}={\mathrm{Tr}}_B|\Psi \rangle \langle \Psi |{}^B\mathbf{D}={\mathrm{Tr}}_A|\Psi \rangle \langle \Psi |$$

(61)

thus, ${}^A\mathbf{D}$ and ${}^B\mathbf{D}$ are the reduced density matrices of each subsystem.

The quantification of bipartite entanglement can be approached through quantum information theory with the von Neumann entropy [141], which quantifies the degree of mixedness of a quantum state, measuring the departure of a given density operator from a pure state (for which $S=0$, representing complete knowledge of the system) [80]. In the specific case of pure states in bipartite quantum systems, it quantifies the entanglement of the quantum system as

$$S\left(\mathbf{D}\right)=-\mathrm{Tr}(\mathbf{D}ln\mathbf{D})$$

(62)

where $\mathbf{D}$ is the density matrix. As the quantum counterpart of Shannon entropy we introduced in Equation (1), the von Neumann entropy can be expressed as the Shannon entropy of the eigenvalues ${\lambda }_p$ for the density matrix:

$$S\left(\mathbf{D}\right)=-\sum _p{\lambda }_{p}ln{\lambda }_p$$

(63)

To measure the entanglement between the bipartite subsystems, the entanglement entropy $S_A$ and $S_B$ are defined as follows:

$$\begin{array}{cc}\hfill S_A& =-\mathrm{Tr}({}^A\mathbf{D}ln{}^A\mathbf{D})\hfill \end{array}$$

(64)

$$\begin{array}{cc}\hfill S_B& =-\mathrm{Tr}({}^B\mathbf{D}ln{}^B\mathbf{D})\hfill \end{array}$$

(65)

For pure states, the relation $S_A=S_B$ holds, as can be proven via Schmidt decomposition [80].

5.2. Orbital Reduced Density Matrix

For a wavefunction expressed in terms of spatial orbitals, each orbital holds four possible occupations, which can be empty $|0\rangle$, singly occupied with a spin-up electron $|\uparrow \rangle$, singly occupied with a spin-down electron $|\uparrow \rangle$, and doubly occupied with an electron pair $|\uparrow \downarrow \rangle$, i.e., the possible orbital occupations are

$$\begin{array}{c}\hfill \{\end{array}|n_i\rangle \begin{array}{c}\hfill \}\end{array}=\left\{\right|0\rangle ,|\uparrow \rangle ,|\downarrow \rangle ,|\uparrow \downarrow \rangle \}$$

(66)

As shown in Figure 9, if the system is divided into subsystem A composed of p-orbitals and complement subsystem B (orbital bath) composed of the remaining $L-p$ orbitals, the RDM of subsystem A is then called the p-orbital reduced density matrix (p-orbital RDM), which can be defined in terms of the full, N-electron RDM or, equivalently, the N-electron wavefunction.

The 1-orbital RDM ${}_o{}^1\mathbf{D}_p$ corresponding to the one orbital partition in Figure 9a is expressed in the basis {-, ↑, ↓, ↑↓}, as shown in

Table 1. The matrix elements of 1-orbital RDM ${}_o{}^1\mathbf{D}_p$ expressed in terms of 1- and 2-electron RDM.

	-	↑	↓	↑↓
-	$1-{}^1\overline{D}_p^p-{}^1\overline{D}_{\overline{p}}^{\overline{p}}+{}^2\overline{D}_{p\overline{p}}^{p\overline{p}}$	0	0	0
↑	0	${}^1\overline{D}_p^p-{}^2\overline{D}_{p\overline{p}}^{p\overline{p}}$	0	0
↓	0	0	${}^1\overline{D}_{\overline{p}}^{\overline{p}}-{}^2\overline{D}_{p\overline{p}}^{p\overline{p}}$	0
↑↓	0	0	0	${}^2\overline{D}_{p\overline{p}}^{p\overline{p}}$

[146,147,148,149], the elements of the matrix can be represented in terms of the spin-dependent 1-electron RDM ${}^1\overline{\mathbf{D}}$ and 2-electron RDM ${}^2\overline{\mathbf{D}}$, where the indices p and $\overline{p}$ indicate spin-up and spin-down electrons of p-th orbital respectively.

The two orbital partition is shown in Figure 9b, elements of the 2-orbital RDM ${}_o{}^2\mathbf{D}_{pq}$ are summarized in

Table 2. The matrix elements of 2-orbital RDM ${}_o{}^2\mathbf{D}_{pq}$ expressed in terms of 1-, 2-, 3-, and 4-electron RDM.

	-	↑	↑	↓	↓	↑↑	↓↓	↑↓	↑↓	↓↑	↑↓	↑↑↓	↑↓↑	↓↑↓	↑↓↓	↑↓↑↓
-	1,1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
↑	0	2,2	2,3	0	0	0	0	0	0	0	0	0	0	0	0	0
↑	0	3,2	3,3	0	0	0	0	0	0	0	0	0	0	0	0	0
↓	0	0	0	4,4	4,5	0	0	0	0	0	0	0	0	0	0	0
↓	0	0	0	5,4	5,5	0	0	0	0	0	0	0	0	0	0	0
↑↑	0	0	0	0	0	6,6	0	0	0	0	0	0	0	0	0	0
↓↓	0	0	0	0	0	0	7,7	0	0	0	0	0	0	0	0	0
↑↓	0	0	0	0	0	0	0	8,8	8,9	8,10	8,11	0	0	0	0	0
↑↓	0	0	0	0	0	0	0	9,8	9,9	9,10	9,11	0	0	0	0	0
↓↑	0	0	0	0	0	0	0	10,8	10,9	10,10	10,11	0	0	0	0	0
↑↓	0	0	0	0	0	0	0	11,8	11,9	11,10	11,11	0	0	0	0	0
↑↑↓	0	0	0	0	0	0	0	0	0	0	0	12,12	12,13	0	0	0
↑↓↑	0	0	0	0	0	0	0	0	0	0	0	13,12	13,13	0	0	0
↓↑↓	0	0	0	0	0	0	0	0	0	0	0	0	0	14,14	14,15	0
↑↓↓	0	0	0	0	0	0	0	0	0	0	0	0	0	15,14	15,15	0
↑↓↑↓	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	16,16

$(1,1)=1-{}^{1}D_p^p-{}^{1}D_{\overline{p}}^{\overline{p}}-{}^{1}D_q^q-{}^{1}D_{\overline{q}}^{\overline{q}}+{}^{2}D_{p\overline{p}}^{p\overline{p}}+{}^{2}D_{q\overline{q}}^{q\overline{q}}+{}^{2}D_{pq}^{pq}+{}^{2}D_{p\overline{q}}^{p\overline{q}}+{}^{2}D_{\overline{p}q}^{\overline{p}q}+{}^{2}D_{\overline{p}\overline{q}}^{\overline{p}\overline{q}}-{}^{3}D_{pq\overline{q}}^{pq\overline{q}}-{}^{3}D_{\overline{p}q\overline{q}}^{\overline{p}q\overline{q}}-{}^{3}D_{p\overline{p}q}^{p\overline{p}q}-{}^{3}D_{p\overline{p}\overline{q}}^{p\overline{p}\overline{q}}$$+{}^{4}D_{p\overline{p}q\overline{q}}^{p\overline{p}q\overline{q}}(2,2)={}^{1}D_q^q-{}^{2}D_{pq}^{pq}-{}^{2}D_{\overline{p}q}^{\overline{p}q}-{}^{2}D_{q\overline{q}}^{q\overline{q}}+{}^{3}D_{p\overline{q}q}^{p\overline{q}q}+{}^{3}D_{p\overline{p}q}^{p\overline{p}q}+{}^{3}D_{\overline{p}q\overline{q}}^{\overline{p}q\overline{q}}-{}^{4}D_{p\overline{p}q\overline{q}}^{p\overline{p}q\overline{q}}(2,3)={(3,2)}^{†}={}^{1}D_p^q-{}^{2}D_{p\overline{p}}^{q\overline{p}}-{}^{2}D_{p\overline{q}}^{q\overline{q}}$$+{}^{3}D_{p\overline{p}\overline{q}}^{q\overline{p}\overline{q}}(3,3)={}^{1}D_p^p-{}^{2}D_{p\overline{p}}^{p\overline{p}}-{}^{2}D_{pq}^{pq}-{}^{2}D_{p\overline{q}}^{p\overline{q}}+{}^{3}D_{pq\overline{q}}^{pq\overline{q}}+{}^{3}D_{p\overline{p}q}^{p\overline{p}q}+{}^{3}D_{p\overline{p}\overline{q}}^{p\overline{p}\overline{q}}-{}^{4}D_{p\overline{p}q\overline{q}}^{p\overline{p}q\overline{q}}(4,4)={}^{1}D_{\overline{q}}^{\overline{q}}-{}^{2}D_{p\overline{q}}^{p\overline{q}}-{}^{2}D_{\overline{p}\overline{q}}^{\overline{p}\overline{q}}-{}^{2}D_{q\overline{q}}^{q\overline{q}}+$${}^{3}D_{p\overline{p}\overline{q}}^{p\overline{p}\overline{q}}+{}^{3}D_{pq\overline{q}}^{pq\overline{q}}+{}^{3}D_{\overline{p}q\overline{q}}^{\overline{p}q\overline{q}}-{}^{4}D_{p\overline{p}q\overline{q}}^{p\overline{p}q\overline{q}}(4,5)={(5,4)}^{†}={}^{1}D_{\overline{p}}^{\overline{q}}-{}^{2}D_{p\overline{p}}^{p\overline{q}}-{}^{2}D_{q\overline{p}}^{q\overline{q}}+{}^{3}D_{pq\overline{p}}^{pq\overline{q}}(5,5)={}^{1}D_{\overline{p}}^{\overline{p}}-{}^{2}D_{\overline{p}q}^{\overline{p}q}-{}^{2}D_{\overline{p}\overline{q}}^{\overline{p}\overline{q}}-{}^{2}D_{p\overline{p}}^{p\overline{p}}+$${}^{3}D_{\overline{p}q\overline{q}}^{\overline{p}q\overline{q}}+{}^{3}D_{p\overline{p}q}^{p\overline{p}q}+{}^{3}D_{p\overline{p}\overline{q}}^{p\overline{p}\overline{q}}-{}^{4}D_{p\overline{p}q\overline{q}}^{p\overline{p}q\overline{q}}(6,6)={}^{2}D_{pq}^{pq}-{}^{3}D_{p\overline{p}q}^{p\overline{p}q}-{}^{3}D_{pq\overline{q}}^{pq\overline{q}}+{}^{4}D_{p\overline{p}q\overline{q}}^{p\overline{p}q\overline{q}}(7,7)={}^{2}D_{\overline{p}\overline{q}}^{\overline{p}\overline{q}}-{}^{3}D_{p\overline{p}\overline{q}}^{p\overline{p}\overline{q}}-{}^{3}D_{\overline{p}q\overline{q}}^{\overline{p}q\overline{q}}+{}^{4}D_{p\overline{p}q\overline{q}}^{p\overline{p}q\overline{q}}(8,8)$$={}^{2}D_{q\overline{q}}^{q\overline{q}}-{}^{3}D_{pq\overline{q}}^{pq\overline{q}}-{}^{3}D_{p\overline{q}q}^{p\overline{q}q}+{}^{4}D_{p\overline{p}q\overline{q}}^{p\overline{p}q\overline{q}}(8,9)={(9,8)}^{†}={}^{2}D_{p\overline{q}}^{q\overline{q}}-{}^{3}D_{p\overline{q}\overline{p}}^{q\overline{q}\overline{p}}(8,10)={(10,8)}^{†}=-{}^{2}D_{q\overline{p}}^{q\overline{q}}+{}^{3}D_{pq\overline{p}}^{pq\overline{q}}(8,11)={(11,8)}^{†}$$={}^{2}D_{p\overline{p}}^{q\overline{q}}(9,9)={}^{2}D_{p\overline{q}}^{p\overline{q}}-{}^{3}D_{p\overline{p}\overline{q}}^{p\overline{p}\overline{q}}-{}^{3}D_{pq\overline{q}}^{pq\overline{q}}+{}^{4}D_{p\overline{p}q\overline{q}}^{p\overline{p}q\overline{q}}(9,10)={(10,9)}^{†}=-{}^{2}D_{q\overline{p}}^{p\overline{q}}(9,11)={(11,9)}^{†}={}^{2}D_{p\overline{p}}^{p\overline{q}}-{}^{3}D_{pq\overline{p}}^{pq\overline{q}}(10,10)$$={}^{2}D_{\overline{p}q}^{\overline{p}q}-{}^{3}D_{p\overline{p}q}^{p\overline{p}q}-{}^{3}D_{\overline{p}q\overline{q}}^{\overline{p}q\overline{q}}+{}^{4}D_{p\overline{p}q\overline{q}}^{p\overline{p}q\overline{q}}(10,11)={(11,10)}^{†}=-{}^{2}D_{p\overline{p}}^{q\overline{p}}+{}^{3}D_{p\overline{p}\overline{p}}^{q\overline{p}\overline{q}}(11,11)={}^{2}D_{p\overline{p}}^{p\overline{p}}-{}^{3}D_{p\overline{p}q}^{p\overline{p}q}-{}^{3}D_{p\overline{p}\overline{q}}^{p\overline{p}\overline{q}}+{}^{4}D_{p\overline{p}q\overline{q}}^{p\overline{p}q\overline{q}}(12,12)$$={}^{3}D_{pq\overline{q}}^{pq\overline{q}}-{}^{4}D_{p\overline{p}q\overline{q}}^{p\overline{p}q\overline{q}}(12,13)={(13,12)}^{†}=-{}^{3}D_{pq\overline{q}}^{pq\overline{q}}(13,13)={}^{3}D_{p\overline{p}q}^{p\overline{p}q}-{}^{4}D_{p\overline{p}q\overline{q}}^{p\overline{p}q\overline{q}}(14,14)={}^{3}D_{\overline{p}q\overline{q}}^{\overline{p}q\overline{q}}-{}^{4}D_{p\overline{p}q\overline{q}}^{p\overline{p}q\overline{q}}(14,15)={(15,14)}^{†}$$=-{}^{3}D_{\overline{p}p\overline{q}}^{\overline{p}q\overline{q}}(15,15)={}^{3}D_{p\overline{p}\overline{q}}^{p\overline{p}\overline{q}}-{}^{4}D_{p\overline{p}q\overline{q}}^{p\overline{p}q\overline{q}}(16,16)={}^{4}D_{p\overline{p}q\overline{q}}^{p\overline{p}q\overline{q}}$.

[146,147,148,149]. Note that ${}_o{}^2\mathbf{D}_{pq}$ requires only some diagonal elements of the 3- and 4-electron reduced density matrices, as well as a few off-diagonal elements of the 1-, 2-, and 3-electron reduced density matrices.

We should note that the 1-orbital RDM ${}_o{}^1\mathbf{D}_p$ can be further simplified for seniority-zero state. Since it excludes singly occupied orbitals and we have the relations ${}^1\overline{D}_p^p={}^1\overline{D}_{\overline{p}}^{\overline{p}}$, the corresponding basis {-, ↑↓} holds a 2 cardinality, thus the 1-orbital RDM of a seniority-zero state is simplified as a $2\times 2$ matrix:

$${}_o{}^1\mathbf{D}_p=\left[\begin{array}{cc}1-{}^{1}D_p^p& 0\\ 0& {}^{1}D_p^p\end{array}\right]$$

(67)

Following the same simplification method, the 2-orbital RDM ${}_o{}^2\mathbf{D}_{pq}$ of a seniority-zero state expressed in the basis {−, ↑↓, ↑↓, ↑↓↑↓} is reduced to a $4\times 4$ matrix:

$${}_o{}^2\mathbf{D}_{pq}=\left[\begin{array}{cccc}1-{}^{1}D_p^p-{}^{1}D_q^q+{}^{2}D_{p\overline{q}}^{p\overline{q}}& 0& 0& 0\\ 0& {}^{1}D_p^p-{}^{2}D_{p\overline{q}}^{p\overline{q}}& {}^{2}D_{q\overline{q}}^{p\overline{p}}& 0\\ 0& {}^{2}D_{p\overline{p}}^{q\overline{q}}& {}^{1}D_q^q-{}^{2}D_{q\overline{p}}^{q\overline{p}}& 0\\ 0& 0& 0& {}^{2}D_{p\overline{q}}^{p\overline{q}}\end{array}\right]$$

(68)

5.3. Orbital Entanglement

With the preliminary knowledge of quantum information theory and orbital reduced density matrix, we can define the single-orbital entropy and mutual information from 1- and 2-orbital RDM. The first quantity, single–orbital entropy $s{\left(1\right)}_p$, measures the entanglement between a given orbital p and the complementary orbital bath, using the eigenvalues of the one–orbital reduced density matrix as information carriers. The single–orbital entropy $s{\left(1\right)}_p$ is defined as follows:

$$\begin{array}{cc}\hfill s{\left(1\right)}_p& =-\mathrm{Tr}({}_o{}^1\mathbf{D}_{p}ln{}_o{}^1\mathbf{D}_p)\hfill \\ \hfill & =-\sum _{\alpha }^M{\lambda }_{\alpha ,p}ln{\lambda }_{\alpha ,p}\hfill \end{array}$$

(69)

where ${\lambda }_{\alpha ,p}$ and M are the eigenvalues and dimension of the p-th one-orbital reduced density matrix respectively, with $M=2$ for seniority-zero state and $M=4$ for other states. The total quantum information encoded in the system is given by the sum of single-orbital entropy:

$$I_{tot}=\sum _p^{L}s{\left(1\right)}_p$$

(70)

Given two states described by the one-orbital reduced density matrices ${}_o{}^1\mathbf{D}_p$ and ${}_o{}^1\mathbf{D}_q$, one can define the relative orbital entropy by the KL divergence:

$$\begin{array}{cc}\hfill s{\left(1\right)}_{\left(p\right|\left|q\right)}& =D_{KL\left({}_o{}^1\mathbf{D}_p\right|\left|{}_o{}^1\mathbf{D}_q\right)}\hfill \\ \hfill & =\mathrm{Tr}({}_o{}^1\mathbf{D}_{p}ln{\displaystyle \frac{{}_o{}^1\mathbf{D}_p}{{}_o{}^1\mathbf{D}_q}})\hfill \end{array}$$

(71)

It measures how much the entanglement of orbital p deviates from that of orbital q.

If the system is divided into two orbitals $(p,q)$ and the remaining $L-2$ orbital bath as shown in Figure 9b, the entanglement between them is quantified by the two-orbital entropy $s{\left(2\right)}_{(p,q)}$ defined as

$$\begin{array}{cc}\hfill s{\left(2\right)}_{(p,q)}& =-\mathrm{Tr}({}_o{}^2\mathbf{D}_{pq}ln{}_o{}^2\mathbf{D}_{pq})\hfill \\ \hfill & =-\sum _{\alpha }^M{\lambda }_{\alpha ,pq}ln{\lambda }_{\alpha ,pq}\hfill \end{array}$$

(72)

where ${\lambda }_{\alpha ,pq}$ and M are the eigenvalues and dimension of the two-orbital reduced density matrix respectively, with $M=4$ for seniority-zero state and $M=16$ for other states. The conditional orbital entropy can be written by the KL divergence as follows:

$$\begin{array}{cc}\hfill s{\left(2\right)}_{\left(p\right|q)}& =-D_{KL\left({}_o{}^2\mathbf{D}_{pq}\right||{}_o{}^1\mathbf{I}_p\otimes {}_o{}^1\mathbf{D}_q)}\hfill \\ \hfill & =-\mathrm{Tr}({}_o{}^2\mathbf{D}_{pq}ln{\displaystyle \frac{{}_o{}^2\mathbf{D}_{pq}}{{}_o{}^1\mathbf{I}_p\otimes {}_o{}^1\mathbf{D}_q}})\hfill \\ \hfill & =s{\left(2\right)}_{(p,q)}-s{\left(1\right)}_q\hfill \end{array}$$

(73)

where ${}_o{}^1\mathbf{I}_p$ is the identity matrix with the same dimensions as ${}_o{}^1\mathbf{D}_p$. The total amount of entanglement between two orbitals p and q can be measured by the orbital pair mutual information, written by the KL divergence as

$$\begin{array}{cc}\hfill I_{(p;q)}& =D_{KL\left({}_o{}^2\mathbf{D}_{pq}\right||{}_o{}^1\mathbf{D}_p\otimes {}_o{}^1\mathbf{D}_q)}\hfill \\ \hfill & =\mathrm{Tr}({}_o{}^2\mathbf{D}_{pq}ln{\displaystyle \frac{{}_o{}^2\mathbf{D}_{pq}}{{}_o{}^1\mathbf{D}_p\otimes {}_o{}^1\mathbf{D}_q}})\hfill \\ \hfill & =s{\left(1\right)}_p+s{\left(1\right)}_q-s{\left(2\right)}_{(p,q)}\hfill \end{array}$$

(74)

As an application of the information theory framework introduced in Section 2 where the orbital reduced density matrix serves as the information carrier, those quantum information theory quantities naturally inherit key properties we introduced in Section 2.3, including but not limited to the chain rule and subadditivity.

Examples and Illustrations

The concept of orbital entanglement serves as a complementary tool for interpreting electronic structures, proving particularly valuable in strongly correlated systems. In this subsection, we will present several representative application examples.

Electron correlation effects are conventionally categorized as dynamic and nondynamic (also termed static), where in this classification static is used synonymously with nondynamic [156]. The dynamic correlation effect, though large, arise from relatively small contributions from a large number of configurations. In contrast, the nondynamic/static effect involving large contributions arising from a few configurations, which collectively address orbital (near-) degeneracy. In contexts demanding more nuanced differentiation, static electron correlation (strict degeneracy) and nondynamic correlation (near-degeneracy) are considered separate effects in our formalism [148]. As presented in Refs. [47,148], the strength of single-orbital entropy $s{\left(1\right)}_p$ to measure orbital entanglement and the orbital-pair mutual information $I_{(p;q)}$ can be associated with different types of electron correlation effects. In

Table 3. Relation between the strength of orbital entanglement and electron correlation effects.

Correlation Effects	Intensity	$\mathit{s}{\left(1\right)}_{\mathit{p}}$	${\mathit{I}}_{(\mathit{p};\mathit{q})}$a
Nondynamic	Strong	>0.5	≈10−1
Static	Medium	0.5–0.1	≈10−1
Dynamic	Weak	<0.1	≈10−1

^a An alternative definition of orbital mutual information from Ref. [148] is employed: $I_{(p;q)}={\displaystyle \frac{1}{2}}\left[s{\left(2\right)}_{pq}-s{\left(1\right)}_p-s{\left(1\right)}_q\right](1-{\delta }_{pq})$. ${\delta }_{pq}$ is the Kronecker delta to ensure $I_{(p;q)}=0$ when $p=q$.

, we map the strength of orbital interactions onto a certain type of correlation effects. Computational investigations demonstrate that orbitals with nondynamic/static electron correlation effects signify substantial multireference character in the system. In contrast, weakly entangled orbitals predominantly exhibit dynamic correlation effects; systems where all orbitals are weakly entangled can usually be adequately treated by single-reference approaches.

Quantitative visualization of $s{\left(1\right)}_p$ and $I_{(p;q)}$ enables a more intuitive analysis. As shown in Figure 10, the strength of the orbital-pair mutual information classified in Table 3 is color-coded,

• Blue lines: Nondynamic correlated orbital pairs.
• Red lines: Static correlated orbitals.
• Green: Dynamic correlated orbitals.

An important issue for the density matrix renormalization group (DMRG) method [157,158,159] is the order of orbitals in the one-dimensional matrix product state (MPS) wavefunction ansatz; an optimal ordering of orbitals corresponding to maximum entanglement will produce the most efficient results [21,160,161]. Since strong (nondynamic/static) electron correlation is essential for proper molecular dissociation into fragments, orbital entanglement provides both a fundamental framework for understanding bond formation/breaking processes [150] and a practical tool for analyzing chemical reactivity.

For many strongly correlated calculation methods such as complete active space self-consistent field (CASSCF) [41,162], the selection of the complete active space (CAS space) is a crucial prerequisite step to keep computational costs within feasible limits. As shown in Figure 11, for the CAS methodology, all the molecular orbitals are classified into three spaces:

• Inactive space: Always doubly occupied.
• Active space: All the possible configurations are allowed.
• Virtual space: Always empty.

Specifically, the active space, usually denoted as CAS$(n,m)$ where n and m are the number of electrons and orbitals respectively, should encompass orbitals and electrons essential for capturing strong electron correlation effects. Orbital entanglement serve as powerful tools for identifying critical orbitals in active space [163,164]. Comparing the entanglement diagrams as shown in Figure 10, we can evaluate the quality and convergence behavior of those active-space calculations.

6. Summary and Outlook

This review presents a unified perspective on information theory and its applications in quantum chemistry, integrating both classical and quantum frameworks. Beginning with fundamental concepts like Shannon entropy and its related concepts such as joint entropy, relative entropy, conditional entropy, and mutual information, we demonstrate how information theory can be applied to molecular systems through information carriers such as the electron density and orbital reduced density matrix. The discussion bridges classical concepts with their quantum counterparts, including classical Shannon entropy and quantum correlations to the quantum von Neumann entropy and entanglement. By tracing the historical development from early works by Shannon’s foundational contributions, we highlight how information theory has evolved into a versatile framework with broad applications in quantum chemistry, particularly for analyzing electronic structure and quantum phenomena in chemical systems.

This article only scratches the surface of the vast scope for existing and future applications of information theory in quantum chemistry. Notably, for the application of information theory in quantum chemistry, Shannon’s framework is not the only reasonable formula; the Rényi entropy, the Fisher information, and other f- and Bregman divergences can also be used as a measure of correlation and entanglement. Importantly, these concepts extend far beyond simple pairwise interactions: from single- and pair-electron densities to many-body electron distributions, and from bipartite systems to complex multipartite quantum entanglement. The physical manifestations of information carriers are likewise diverse, ranging from atoms in molecules (AIM) to localized molecular orbitals. By creatively combining the above extended concepts as well as other potential extensions, we can even derive additional novel concepts and tools for advancing our understanding of quantum chemistry.

Furthermore, other domains such as machine learning and quantum computing that represent one of the most active research frontiers in science, also possess profound foundations in information theory. The integration of quantum chemistry with information theory, machine learning, and quantum computing is establishing a transformative new paradigm for quantum chemical research.