Deep Learning of the Biswas–Chatterjee–Sen Model

Abstract

We investigate the critical properties of kinetic continuous opinion dynamics using deep learning techniques. The system consists of N continuous spin variables in the interval $[-1,1]$. Dense neural networks are trained on spin configuration data generated via kinetic Monte Carlo simulations, accurately identifying the critical point on both square and triangular lattices. Classical unsupervised learning with principal component analysis reproduces the magnetization and allows estimation of critical exponents. Additionally, variational autoencoders are implemented to study the phase transition through the loss function, which behaves as an order parameter. A correlation function between real and reconstructed data is defined and found to be universal at the critical point.

1. Introduction

Machine learning (ML) comprises a set of techniques for analyzing large volumes of data and is now a key tool in diverse fields, including statistics, condensed matter, high energy physics, astrophysics, cosmology, and quantum computing. ML encompasses three main approaches: supervised learning (using labeled data to learn mappings and make predictions), unsupervised learning (discovering patterns without labels, such as clustering and dimensionality reduction), and reinforcement learning (where an agent learns optimal policies through rewards and penalties).

In this work, we apply supervised and unsupervised deep learning methods to investigate disorder-induced phase transitions in the Biswas–Chatterjee–Sen (BChS) model. In this model, opinions are continuous, $s_i\in [-1,1]$, and pairwise interactions can be cooperative (+) or antagonistic (−). The fraction q of antagonistic interactions controls the disorder. On fully connected networks, the model exhibits a continuous phase transition with Ising mean-field exponents [1].

Various geometries have been explored in the literature, motivating our analysis. On regular lattices, as for example square and cubic, the continuous version exhibits a second-order transition and belongs to the Ising universality class in the corresponding dimensions [2]. On Solomon networks (two coupled networks), both discrete and continuous versions show a continuous transition. In Solomon networks, the exponents depend on dimensionality and may differ from those of the Ising model [3,4,5].

On Barabási–Albert networks, the discrete version exhibits a second-order transition and universality class differences compared to other topologies [6]. On random and complex graphs, extensions with memory and bias confirm the occurrence of transitions and discuss changes in universality class [7]. Modular structures with two groups reveal, in the mean-field regime, a stable antisymmetric ordered state in addition to symmetric ordered and disordered states [8].

The aim of this work is to employ deep learning methods to investigate the continuous phase transition in the BChS model, demonstrating the applicability of these techniques to various network geometries. We generate data using kinetic Monte Carlo dynamics and analyze the resulting configurations with dense neural network classifiers, principal component analysis (PCA), and variational autoencoders (VAE) to accurately identify the critical point and characterize the critical behavior, even in the presence of disorder.

In the following sections, we apply machine learning techniques to study the continuous phase transition of the BChS model on both square and triangular lattices. We begin by presenting our results using supervised learning with dense neural networks, followed by unsupervised learning with PCA and dense neural networks.

2. Data Generation

We generate spin configuration data for the BChS model using the following kinetic rules [1,2]:

• Assign to each of the N nodes in the network an opinion variable $s_i$ in the continuous interval $[-1,1]$. The network state is

  $$\mathit{s}=\left(s_1,s_2,\dots ,s_N\right).$$

  (1)
  The initial configuration is generated by randomly sampling each $s_i$ from a uniform distribution in $[-1,1]$.
• At each step (discrete time), randomly select a node i for update.
• Randomly select a neighbor j of node i. The affinity parameter ${\mu }_{i,j}$ for the link between i and j is chosen randomly: ${\mu }_{i,j}$ is an annealed random variable in $[-1,1]$, negative with probability q (antagonistic interaction), positive with probability $1-q$ (cooperative interaction).
• Update both nodes i and j according to

  $$\left\{\begin{array}{c}{s}_i(t+1)=s_i\left(t\right)+{\mu }_{i,j}{s}_j\left(t\right),\hfill \\ s_j(t+1)=s_j\left(t\right)+{\mu }_{i,j}{s}_i\left(t\right),\hfill \end{array}\right.$$

  (2)

  where $s_i\left(t\right)$ and $s_j\left(t\right)$ are the states before the update, and $s_i(t+1)$, $s_j(t+1)$ are the updated states. One Monte Carlo step (MCS) consists of N such updates.
• If any updated state $s_{i,j}(t+1)$ exceeds 1, set $s_{i,j}(t+1)=1$; if $s_{i,j}(t+1)<-1$, set $s_{i,j}(t+1)=-1$. This enforces the bounds and introduces nonlinearity.

The BChS model exhibits a continuous phase transition at a critical noise $q_c$ between a ferromagnetic phase ($q<q_c$) with nonzero average opinion and a paramagnetic phase ($q>q_c$) with zero average opinion.

To collect stationary configurations, we discard the first $N_{\mathrm{term}}={10}^4$ Monte Carlo steps. After the dynamics become stationary, we sample $N_t$ configurations ${\mathit{s}}_{\mathcal{l}}$ ($\mathcal{l}=0,1,\dots ,N_t$), discarding additional steps between samples to reduce correlations [9]. These stationary configurations are then used for deep learning analysis of the BChS model on square and triangular lattices.

3. Supervised Learning

Neural networks have been widely applied to study second-order phase transitions in models such as the Ising model [10,11,12], directed percolation [13], the pair contact process with diffusion [14], and quantum phase transitions [15]. In this work, we extend these methods to the BChS model on square and triangular lattices. Dense neural networks are trained to classify configurations as ferromagnetic ($q<q_c$) or paramagnetic ($q>q_c$). Training is performed on square lattice data, and inference is carried out on stationary configurations from both square and triangular lattices, allowing us to assess the network’s ability to identify the critical point in a nonequilibrium system with continuous states.

To account for the ${\mathbb{Z}}_2$ symmetry, each configuration generated during simulation is paired with its inverted counterpart. The resulting training dataset contains $N_{\mathcal{D}}=4\times {10}^6$ configurations, comprising ${10}^4$ stationary samples for each of 200 noise values in the range $0.5q_c^s$ to $1.5q_c^s$ on the square lattice. The critical noise for the BChS model on the square lattice is $q_c^s\approx 0.2266$ [2]. Of the total dataset, $20\%$ is reserved for validation.

During Monte Carlo simulations, the first ${10}^4$ steps are discarded to ensure that the dynamics become stationary, and an additional ${10}^3$ steps are omitted between stored configurations to reduce correlations. One Monte Carlo step corresponds to updating all $N=L^2$ spins. Lattice sizes used are $L=16$, 20, 24, 32, and 40. Configurations sampled at $q<q_c^s$ are labeled as ferromagnetic, while those at $q>q_c^s$ are labeled as paramagnetic.

The neural network architecture is as follows:

• Input layer of size $L^2$, with each input representing a continuous spin value $s_i\in [-1,1]$;
• First hidden layer with 128 neurons, ReLU activation, ${\mathcal{l}}_2$ regularization, batch normalization, and dropout rate $0.2$;
• Second hidden layer with 64 neurons, ReLU activation, ${\mathcal{l}}_2$ regularization, batch normalization, and dropout rate $0.2$;
• Output layer with two neurons (${\rho }_1$, ${\rho }_2$) and softmax activation.

The output ${\rho }_1$ represents the score for a pure ferromagnetic state ($q=0$), while ${\rho }_2$ is the complementary score for a paramagnetic state ($q\to \infty$). Configuration labels are set as $y_i=1$ for the ferromagnetic phase and $y_i=0$ for the paramagnetic phase. The point of maximum confusion, corresponding to the transition threshold, occurs when ${\rho }_1={\rho }_2=0.5$. We chose a dense neural network architecture for this task, as dense neural networks are well suited for classification problems on arbitrary geometries since they do not rely on spatial structure. The neural network is implemented and trained using the Keras 3.11.3 and Tensorflow 2.20.0 libraries in Python 3.13.5.

The neural network was trained for at least ${10}^3$ epochs with a batch size of 128, using the ADAM 3.11.3 optimizer with a learning rate $\eta ={10}^{-4}$, and the chosen loss function is the sparse categorical cross-entropy

$${\mathcal{l}}_{\mathrm{SCE}}=-{\displaystyle \frac{1}{N_{\mathcal{D}}}}\sum _{i=1}^{N_{\mathcal{D}}}{y}_{i}ln{y}_i^{\prime }\left(\mathit{\theta }\right),$$

(3)

where $N_{\mathcal{D}}$ is the size of the dataset, $y_i$ is the true label of the configuration, $y_i^{\prime }\left(\mathit{\theta }\right)$ is the predicted label by the neural network, and $\mathit{\theta }$ represents the neural network parameters (weights and biases), which are optimized during training. The categorical cross-entropy loss function measures the dissimilarity between the true and predicted labels, encouraging the neural network to make accurate classifications.

Figure 1 shows the classification results for the BChS model on the square lattice. In panel (a), the crossing point ${\rho }_1={\rho }_2=0.5$ closely matches the critical noise $q_c^s$, indicated by the dashed vertical line. Panel (b) demonstrates that the outputs collapse according to the finite-size scaling relation

$${\rho }_{1,2}\propto f_{{\rho }_{1,2}}\left(N^{1/\nu }\left(q-q_c^{\prime }\right)\right),$$

(4)

where $\nu =1$ is the correlation length exponent for the Ising universality class in two dimensions, and $q_c^{\prime }$ denotes the crossing abscissas. The scaling functions $f_{{\rho }_1}$ and $f_{{\rho }_2}$ are universal up to a rescaling of the argument. These results confirm that the neural network accurately identifies the critical point $q_c^s$ of the BChS model on the square lattice.

Next, we generated an inference dataset for the triangular lattice using the same parameters as for the square lattice. The neural network trained on square lattice data was then used to infer on triangular lattice configurations. The results are shown in Figure 2. In panel (a), the crossing point ${\rho }_1={\rho }_2=0.5$ is close to the critical noise $q_c^t$, indicated by the dashed vertical line. Panel (b) shows that the outputs collapse according to Equation (4) with $\nu =1$.

An extrapolation of the crossing points $q_c^{\prime }$ seen in Figure 2 as a function of $1/L$ is shown in Figure 3. The extrapolation is performed according to the linear regression

$$q_c^{\prime }=q_c-{\displaystyle \frac{a}{L}},$$

(5)

where a is a constant. The extrapolation yields an estimate for the critical noise, $q_c\approx 0.2397\pm 0.0002$, which provides an estimate of the critical noise of the BChS model on the triangular lattice.

Results for the BChS model on the triangular lattice are not available in the literature. To compare the classification neural network results with standard methods, we simulated the model on triangular lattices using the kinetic Monte Carlo method. The fundamental observable is the average opinion balance (magnetization) per spin:

$$m=\left|{\displaystyle \frac{1}{N}}\sum _{i=1}^N{s}_i\right|.$$

(6)

The order parameter is the time average of m in the stationary regime, and its fluctuation defines the susceptibility. The order parameter M, susceptibility $\chi$, and Binder cumulant U are defined as [16]

$$\begin{array}{ccc}\hfill M\left(q\right)& =& \langle m\rangle ,\hfill \\ \hfill \chi \left(q\right)& =& N\left(\langle m^2\rangle -{\langle m\rangle }^2\right),\hfill \\ \hfill U\left(q\right)& =& 1-{\displaystyle \frac{\langle m^4\rangle }{3{\langle m^2\rangle }^2}},\hfill \end{array}$$

(7)

where $\langle \cdots \rangle$ denotes the time average over the Markov chain. All observables depend on the noise parameter q, so independent simulations are performed for each value of q.

We performed simulations on triangular lattices of sizes $L=50$, 60, 70, 80, 90, and 100. For each noise value, we discarded the first $2\times {10}^5$ Monte Carlo steps to ensure stationarity, then collected ${10}^7$ samples, omitting 10 steps between samples to reduce correlations. The results are shown in Figure 4. We estimated the critical noise $q_c^t\approx 0.240$ using Binder’s cumulant method [17], which is close to the extrapolation estimate shown in Figure 3. The critical behavior matches that of the square lattice, as expected, except for the non-universal value of the critical noise. The agreement between the critical noise estimated by the neural network and that obtained via Monte Carlo simulations confirms the effectiveness of supervised learning in identifying phase transitions in nonequilibrium systems with continuous states.

4. Unsupervised Learning

Unsupervised learning methods have also been applied to study phase transitions in the Ising model [18,19,20]. Here, we extend both supervised and unsupervised learning approaches to the BChS model on square and triangular lattices. We first employ PCA, a classical unsupervised technique, followed by VAEs, a deep learning method.

PCA identifies the directions (principal components) along which the variance in the data is maximized. The first principal component captures the largest variance, the second captures the next largest, and so on. These components correspond to the eigenvectors of the covariance matrix, with their associated eigenvalues indicating the amount of variance explained. PCA is commonly used for dimensionality reduction, visualization, and feature extraction.

Figure 5 shows the results of PCA applied to BChS model training data. For low noise values, the principal components form two clusters centered at $(0,-L)$ and $(0,L)$. For higher noise values, a single cluster appears at $(0,0)$. The cluster plot provides a clear visualization of the phase transition in the principal component space.

We further analyzed the principal component data using finite-size scaling techniques. Specifically, we considered the ratio of the two largest eigenvalues ${\lambda }_2/{\lambda }_1$ of the covariance matrix, as well as the averages of the absolute values of the first and second principal components, denoted $P_1$ and $P_2$, respectively. The finite-size scaling relations for these observables are

$$\begin{array}{ccc}\hfill {\lambda }_2/{\lambda }_1& \propto & f_{\lambda }\left(N^{1/\nu }(q-q_c)\right),\hfill \\ \hfill P_1/L& \propto & N^{-\beta /\nu }{f}_{P_1}\left(N^{1/\nu }(q-q_c)\right),\hfill \\ \hfill L{P}_2& \propto & N^{\gamma /\nu }{f}_{P_2}\left(N^{1/\nu }(q-q_c)\right),\hfill \end{array}$$

(8)

where $f_{\lambda }$, $f_{P_1}$, and $f_{P_2}$ are universal scaling functions.

Figure 6 summarizes the results for these PCA observables. Panel (a) shows that the ratio ${\lambda }_2/{\lambda }_1$ is universal at the critical noise $q_c^s$ for the square lattice. Panel (b) demonstrates the scaling collapse, allowing estimation of the critical exponent $\nu =1$. Panel (c) presents $P_1/L$ as a function of noise, which coincides with the average magnetization per spin and scales as $L^{\beta /\nu }$ with $\beta /\nu =1/8$, as confirmed by the collapse in panel (d). Panel (e) displays $L{P}_2$, whose maximum increases with $L^{\gamma /\nu }$ at the critical point, where $\gamma /\nu =7/4$, as shown in panel (f).

We also investigated the continuous phase transition using VAEs, which are generative models combining autoencoder architectures with variational inference. A VAE consists of an encoder that maps input data to a latent space and a decoder that reconstructs the input from the latent representation. The encoder learns a probabilistic mapping, enabling the generation of new samples by sampling from the latent space.

The encoder architecture is as follows:

• Input layer of size $L^2$, with each input corresponding to a continuous spin variable $s_i\in [-1,1]$;
• First hidden layer with 625 neurons, ReLU activation, ${\mathcal{l}}_1$ regularization, batch normalization, and dropout rate $0.2$;
• Second hidden layer with 256 neurons, ReLU activation, ${\mathcal{l}}_1$ regularization, batch normalization, and dropout rate $0.2$;
• Third hidden layer with 64 neurons, ReLU activation, ${\mathcal{l}}_1$ regularization, batch normalization, and dropout rate $0.2$;
• Output layer with two neurons (linear activation): one outputs the mean $\mu$ and the other outputs the logarithm of the variance $\sigma$ of the latent variable Z.

The decoder mirrors the encoder structure and receives the latent encoding Z as input. Additionally, it includes an extra input neuron for the normalized noise of the configuration, making the neural network a conditional VAE.

The VAE was trained for at least ${10}^3$ epochs with a batch size of 128, using the RMSprop optimizer with a learning rate $\eta ={10}^{-3}$. The loss function is the sum of the mean squared error and the Kullback–Leibler loss

$${\mathcal{l}}_{\mathrm{VAE}}={\mathcal{l}}_{\mathrm{MSE}}+{\mathcal{l}}_{\mathrm{KL}},$$

(9)

where ${\mathcal{l}}_{\mathrm{MSE}}$ is the mean squared error between the input and reconstructed configurations,

$${\mathcal{l}}_{\mathrm{MSE}}={\displaystyle \frac{1}{N_{\mathcal{D}}}}\sum _{i=1}^{N_{\mathcal{D}}}{\left[y_i-y_i^{\prime }\left(\mathit{\theta }\right)\right]}^2,$$

(10)

with $N_{\mathcal{D}}$ the dataset size, $y_i$ the input configuration, $y_i^{\prime }\left(\mathit{\theta }\right)$ the reconstructed configuration, and $\mathit{\theta }$ the network parameters. The Kullback–Leibler loss [21] is

$${\mathcal{l}}_{\mathrm{KL}}=-{\displaystyle \frac{1}{2}}\sum _{i=1}^d\left[1+log{\sigma }_i^2-{\mu }_i^2-{\sigma }_i^2\right],$$

(11)

where d is the dimension of the latent space, and ${\mu }_i$, ${\sigma }_i$ are the mean and standard deviation of the latent variable. The Kullback–Leibler loss regularizes the latent space by encouraging the learned distribution to approximate a standard normal distribution, preventing overfitting and enabling sampling of new artificial configurations.

The latent space consists of a single statistical variable Z, sampled from a normal distribution with mean $\mu$ and variance $\sigma$ provided by the encoder. This minimal latent space encourages the encoder to capture only the most relevant features of the data and prevents trivial reproduction of the input configurations. The VAE was implemented and trained using the Keras and Tensorflow libraries in Python.

Figure 7 shows the latent encoding Z of the input data as a function of magnetization and normalized noise. In panel (a), a clear separation between positive and negative magnetizations is observed according to the sign of Z, reflecting the ${\mathbb{Z}}_2$ symmetry. Panel (b) demonstrates that the relationship learned by the neural network between magnetization m and latent encoding Z is approximately linear. At low noise values, two clusters centered at $(-2,-1)$ and $(2,1)$ appear, while at higher noise values, a single cluster at $(0,0)$ emerges, indicating the phase transition. Panel (c) further confirms that the phase transition is evident from the latent encoding.

We define a correlation function between the real data configurations ${\mathit{s}}_{\mathrm{real}}$ and those reconstructed by the VAE, ${\mathit{s}}_{\mathrm{recon}}$, as

$$C\left({\mathit{s}}_{\mathrm{real}}\mid {\mathit{s}}_{\mathrm{recon}}\right)\equiv {\displaystyle \frac{1}{L^2}}{\displaystyle \frac{〈\left|{\mathit{s}}_{\mathrm{real}}·{\mathit{s}}_{\mathrm{recon}}\right|〉}{m_{\mathrm{real}}{m}_{\mathrm{recon}}}},$$

(12)

where $\langle \cdots \rangle$ denotes an average over the dataset, $m_{\mathrm{real}}$ is the average magnetization of the real data, and $m_{\mathrm{recon}}$ is the average magnetization of the reconstructed data. The correlation function is universal at the critical point, enabling estimation of the transition threshold in the same way as the Binder cumulant in Monte Carlo simulations. Therefore one can expect the following scaling dependence

$$C\left({\mathit{s}}_{\mathrm{real}}\mid {\mathit{s}}_{\mathrm{recon}}\right)\propto f_C\left(N^{1/\nu }\left(q-q_c\right)\right),$$

(13)

which allows estimation of the correlation length exponent $\nu$.

We also calculate the binary cross-entropy loss function ${\mathcal{l}}_{\mathrm{BCE}}$,

$${\mathcal{l}}_{\mathrm{BCE}}=-\sum _{i=1}^{N_{\mathcal{D}}}{y}_{i}ln{y}_i^{\prime }\left(\mathit{\theta }\right)-(1-y_i)ln\left[1-y_i^{\prime }\left(\mathit{\theta }\right)\right],$$

(14)

by renormalizing the input configurations to the interval $[0,1]$. The loss functions ${\mathcal{l}}_{\mathrm{MSE}}$ and ${\mathcal{l}}_{\mathrm{BCE}}$ between the input and reconstructed output configurations serve as indicators of the phase transition. In the paramagnetic regime ($T\to \infty$), the input and reconstructed outputs behave as two effectively random configurations, yielding limiting values ${\mathcal{l}}_{\mathrm{MSE}}\to 3/2$ and ${\mathcal{l}}_{\mathrm{BCE}}\to ln2$ for random uniform data. Consequently, the quantities $1-3{\mathcal{l}}_{\mathrm{MSE}}/2$ and $1-{\mathcal{l}}_{\mathrm{BCE}}/ln2$ act as order parameters and obey the following scaling relations:

$$\begin{array}{ccc}\hfill 1-3{\mathcal{l}}_{\mathrm{MSE}}/2& \propto & L^{2\beta /\nu }{f}_{\mathrm{MSE}}\left(N^{1/\nu }(q-q_c)\right),\hfill \\ \hfill 1-{\mathcal{l}}_{\mathrm{BCE}}/ln2& \propto & L^{2\beta /\nu }{f}_{\mathrm{BCE}}\left(N^{1/\nu }(q-q_c)\right),\hfill \end{array}$$

(15)

where the loss functions scale with system size as $2\beta /\nu =1/4$, consistent with the universality class of the two-dimensional Ising model.

Figure 8 presents the VAE observables for the BChS model. Panel (a) shows the correlation function defined in Equation (12), which is universal at the critical noise $q_c^s$ for the square lattice. The scaling collapse in panel (b) confirms the finite-size scaling relation for the correlation function with the Ising critical exponent $\nu =1$. Panels (c) and (e) display $3{\mathcal{l}}_{\mathrm{MSE}}/2$ and ${\mathcal{l}}_{\mathrm{BCE}}/ln2$, respectively, both serving as order parameters that vanish at the transition. The scaling collapses in panels (d) and (f) confirm the expected scaling relations for these quantities with the Ising exponent $2\beta /\nu =1/4$.

5. Conclusions

In this work, we applied supervised and unsupervised deep learning techniques to study the continuous phase transition of the BChS model on square and triangular lattices. We generated spin configuration data using kinetic Monte Carlo simulations and trained dense neural networks to classify configurations into ferromagnetic and paramagnetic phases. The networks accurately identified the critical points, with outputs collapsing according to finite-size scaling relations.

Also, we employed PCA to analyze the data, revealing clustering behavior that visualizes the phase transition. The ratio of the two largest eigenvalues of the covariance matrix was found to be universal at the critical point, and we estimated critical exponents consistent with the Ising universality class. Furthermore, we implemented VAEs to study the phase transition through the loss function, which behaved as an order parameter. We defined a correlation function between the input and reconstructed configurations, finding it to be universal at the critical point. The scaling collapses of the correlation function and loss functions confirmed the critical exponents of the Ising universality class.