Thermal Analysis and Evaluation of Memristor-Based Compute-in-Memory Chips

Abstract

The rapid advancement of artificial intelligence (AI) technologies has significantly increased the demand for high-performance computational hardware. Memristor-based compute-in-memory (CIM) technology, also known as resistive random-access memory (RRAM)-based CIM technology, shows great potential for addressing the data transfer bottleneck and supporting high-performance computing (HPC). In this paper, a multi-scale thermal model is developed to evaluate the temperature distribution in RRAM-based CIM chips and the influence of various factors on thermal behavior. The results indicate that hotspot temperatures can be mitigated by reducing the epoxy molding compound (EMC) thickness, increasing the substrate thickness, and lowering boundary thermal resistance. Moreover, optimizing the layout of analog computing circuits and digital circuits can reduce the maximum temperature by up to 4.04 °C. Furthermore, the impact of temperature on the conductance of RRAM devices and the inference accuracy of RRAM-based CIM chips is analyzed. Simulation results reveal that thermal-induced accuracy loss in CIM chips is significant, but the computation correction method effectively reduces the accuracy loss from 66.4% to 1.4% at 85 °C.

1. Introduction

AI technologies, for example autonomous driving, large language models (LLMs), and cloud computing, have garnered significant attention and widespread application in recent years [1,2,3]. However, the rapidly increasing demand for computational hardware has become a critical challenge [4]. The memory wall has emerged as a bottleneck for improving the performance of integrated circuits [5]. Significant energy and latency are consumed during frequent data transfers between physically separated memory and computing units in the conventional Von Neumann architecture [6,7]. CIM technology based on emerging non-volatile memory (eNVM) devices, such as RRAM, can perform both storage and computation functions within a single device [8,9,10]. This technology demonstrates significant potential for overcoming the bottleneck of the memory wall, and has become a promising choice for AI applications, including image classification, human–machine interfaces, and image reconstruction [11,12,13].

RRAM devices and RRAM-based CIM chips have gained widespread attention and achieved rapid development in recent years [14,15,16]. However, the power density of CIM chips has seen an increasing trend, and addressing thermal issues has become increasingly urgent [17]. Recent studies indicate that RRAM-based CIM chips are capable of performing matrix-vector multiplication [18,19,20,21]. During the computing process, the parallel operation of RRAM devices results in high local power consumption and high local temperatures [22]. Changes in external factors, such as environmental temperature and the power consumption of other chips in 3D integration, can cause temperature variations in RRAM-based CIM chips [23]. Furthermore, both the characteristics of RRAM devices and overall chip performance are highly temperature-sensitive [24]. For example, the conductance of analog RRAM devices drifts with temperature. In particular, when analog devices are used to represent multiple bits, the conductance drift leads to significant degradation in the computing accuracy of the chip. Without effective thermal management methods, the performance of CIM chips can decline significantly [25]. Some existing works primarily focus on the RRAM device characteristics, for example, the compact model of the temperature coefficient, and retention characteristics [24,26]. However, these models fail to account for the impact of temperature distribution on the neural network from a system-level perspective. Chip-level thermal simulations often treat the entire chip as a uniform heat source, disregarding the detailed circuit structure [23]. Some works have explored multi-scale thermal simulations [22], but comprehensive analyses of temperature distribution and thermal effects in CIM chips remain incomplete. While the work in [22] mainly focused on the impact of power consumption variations on temperature, this study expands the evaluation to include the substrate thickness, the EMC thickness, and the layout of digital circuits and analog computing module tiles, which can directly guide the design and packaging of CIM chips.

In this work, a multi-scale thermal model is developed to evaluate the temperature distribution in RRAM-based CIM chips. This model is further employed to analyze the influence of boundary thermal resistance, the layout of digital circuits and analog computing module tiles, the thickness of the EMC, and the thickness of the substrate. Furthermore, the impact of temperature on RRAM device conductance and the inference accuracy of RRAM-based CIM chips is investigated. Simulation results reveal that the computing accuracy loss of CIM chips due to rising temperatures is unacceptable, and the computation correction method can reduce the accuracy loss from 66.4% to 1.4% at 85 °C.

2. Thermal Modeling

The thermal modeling and analysis framework for RRAM-based CIM chips is shown in Figure 1. First, a multi-scale thermal model is constructed for CIM chips with parameters extracted from a real 28 nm CIM chip, and device characteristics as well as packaging factors are also considered. Second, based on Fourier’s law and the finite element method (FEM), the equations are formulated and solved under steady-state conditions in this work. Third, we evaluate the temperature distribution of the CIM chips and the impact of various factors on temperature, including process, circuit architecture, and cooling methods. Fourth, the thermal effects on RRAM devices and CIM chips will be evaluated. The conductance range of RRAM devices in this work is from 2 to 20 μS. Within this range, the conductance of the RRAM devices exhibits semiconductor behavior with a positive temperature coefficient. A compact model of RRAM conductance is used to evaluate the impact of temperature on the conductance of the RRAM devices. Next, based on our previous work [22], a typical deep neural network, the 18-layer ResNet neural network (ResNet-18) [27], is used to classify images from the CIFAR-10 [28] dataset to evaluate the thermal effect on the computing accuracy of RRAM-based CIM chips. Finally, based on the simulation results, several design guidelines for optimizing the temperature distribution and thermal-induced performance degradation are provided.

The architecture of the RRAM-based CIM chip, from the RRAM device to the full CIM chip used in the modeling, is shown in Figure 2. The RRAM device can change its state through ion movement and process analog signals. The RRAM device used is the TiN/HfOx/TaOy/TiN memristor. Based on Ohm’s law and Kirchhoff’s law, the RRAM array performs matrix-vector multiplication.

One RRAM array and the corresponding peripheral circuits, such as analog–digital converters (ADCs) and digital–analog converters (DACs), form one processing element (PE). Each tile consists of four PEs and the corresponding peripheral circuits, such as the tile controller, shift, and adder, activation function, and pooling modules. Nine tiles and their corresponding peripheral circuits constitute the RRAM-based CIM chip. The convolutional parameters of ResNet-18 are mapped onto the RRAM arrays, while certain specialized computations, such as pooling, are handled by the digital circuit section.

The parameters used in this work are listed in

Table 1. Parameters used in this work.

Parameters	Value
RRAM array	1152 × 1024
Condutance	2~20 μS
Set voltage	2.5 V
Reset voltage	2 V
Read voltage	≤0.5 V
Technology	28 nm
ADCs per PE	128
DACs per PE	1152
Digital power per Tile	200 mW
Digital power except Tiles	200 mW
ADCs power per PE	80 mW
DACs power per PE	60 mW
Thermal conductivity of Si	149 W/m/K
Thermal conductivity of EMC	1.4 W/m/K

[22,29]. The typical conductance range of a RRAM device is between 2 and 20 μS, and each RRAM device has 16 independent conductance states.

The flip-chip packaging method is adopted for the CIM chip in this work. Joule heat is primarily released into the environment from the upper surface. The heat dissipation from the four sides and the bottom side is minimal, with their thermal resistances approximated as adiabatic. The ambient temperature is assumed to be 26.85 °C.

3. Results of Thermal Effect Evaluation

Based on the multi-scale thermal model, the temperature distribution of the RRAM-based CIM chip is obtained. The chip size is 10 mm × 10 mm. The cross-sectional view of the temperature map obtained from the thermal simulation of the CIM chip is shown in Figure 3a, where the power of each RRAM array is 40 mW, with an upper thermal resistance of 7 cm²·K/W. As illustrated in the figure, there are hotspots in the PE areas with higher power density, while the temperature in the digital circuit areas remains relatively low. The temperature difference across the chip reaches approximately 10 °C. The impact of boundary thermal resistance on the temperature distribution in CIM chips is evaluated. Figure 3b shows that with an increase in boundary thermal resistance, both the average temperature and maximum temperature of the CIM chip increase, and the temperature difference also increases significantly. Furthermore, the difference between the maximum and average temperatures grows as the boundary thermal resistance increases. Therefore, reducing boundary thermal resistance can effectively lower the chip’s hotspot temperature. If the temperature limit is set to 85 °C, the boundary thermal resistance should not exceed 6.8 cm²·K/W to ensure the safe operation of the chip.

The thermal conductivity of the EMC material is low, so the thickness of the EMC layer significantly impacts the maximum temperature of the chip. As shown in Figure 4a, the maximum temperature differs by 1.6 °C between a 50 μm thick EMC cap and no EMC layer. The substrate material of the chip is monocrystalline silicon, which has a high thermal conductivity of 148 W/m/K. Figure 4b evaluates the impact of substrate thickness on both the maximum and average temperatures in the chip. As shown in Figure 4b, when the substrate thickness is less than 300 μm, the hotspot temperature decreases rapidly as the substrate thickness increases. However, when the substrate thickness exceeds 300 μm, the temperature reduction effect becomes less significant as the thickness increases further. As the substrate thickness increases, the average temperature of the chip shows a slow increasing trend.

To reduce the hotspot temperature, chip design optimization can be applied. As shown in Figure 3a, the temperatures in the digital circuit area are lower than that in the tile area with high power. By adjusting the position of the tiles and placing the digital circuits around the tiles, the distribution of temperature can be improved. The optimized result, shown in Figure 5a, indicates a more uniform temperature distribution. The temperature difference between the maximum and minimum temperatures on the chip is only about 2 °C. Compared with the original configuration shown in Figure 5b, the maximum temperature has decreased by 4.04 °C after optimization.

Based on the evaluations above, hotspot temperatures on the chip can be reduced by optimizing the layout of tiles and digital circuits, decreasing the EMC thickness, appropriately increasing the substrate thickness, and reducing boundary thermal resistance. Additionally, lowering the power consumption of various on-chip modules can further help reduce temperature. While optimizing circuit design can eliminate hotspots, it may impact chip performance. Advanced thermal management technologies can reduce boundary thermal resistance but come with higher costs. The selection of EMC and substrate thickness is relatively straightforward, though it offers limited flexibility for adjustment. These methods are not entirely independent; instead, they can be combined to optimize the chip’s temperature distribution. A balance must be struck between cost, performance, and other factors.

The drift of RRAM conductance with temperature is a basic characteristic of semiconductor materials. The conductance of a RRAM device drifts when temperatures increase. Four-bit analog RRAM devices are used in the simulation, and the conductance difference between two adjacent states is only about 1 μS. As shown in Figure 6a, the conductance of the RRAM device drifts with temperature based on the compact model of RRAM conductance [19]. The conductance drift significantly exceeds the difference between two states, resulting in data drift of the CIM. As a result, the mapped weight of DNN on CIM chips will drift when temperature increases, which leads to accuracy loss in the CIM chip. The deep neural network ResNet-18 is used to classify images from the CIFAR-10 dataset to evaluate the thermal effect on the computing accuracy of RRAM-based CIM chips. The baseline recognition accuracy of the trained ResNet-18 is 90.64%. The computing accuracy loss in the CIM chip is shown in Figure 6b, where the accuracy loss is found to be significant and unacceptable. As the temperature increases, the accuracy decreases more. Optimizing the temperature coefficient of RRAM materials can reduce conductivity drift and computing accuracy loss in CIM chips. For the current RRAM devices, we explore optimization at the circuit design level. The temperatures within a single RRAM array are almost uniform, leading to a similar drift ration of RRAM conductance. The ADC quantization process must be scaled to correct the conductance change with temperature. According to the simulation results in Figure 6b, ADC correction can decrease the accuracy loss from 66.4% to 1.4% at 85 °C.

The impact of temperature on the inference accuracy of RRAM-based CIM chips is influenced by various factors.

Firstly, the extent of temperature variation directly influences the degree of accuracy drift in RRAM-based CIM chips. Currently, these chips are still in the laboratory research phase. During testing, factors such as power consumption, environmental temperature, packaging, and cooling methods remain relatively constant, leading to stable operating temperatures. However, once RRAM-based CIM chips are mass-produced and integrated into everyday devices, such as smartphones and smartwatches, environmental temperature fluctuations will become more significant. In this case, effective thermal management will be crucial to mitigating the accuracy loss caused by temperature variations.

Second, key neural network parameters, such as the precision of the parameters and the number of layers, should be considered. Higher precision increases the sensitivity of RRAM devices to conductance drift, while deeper networks lead to greater error accumulation. Additionally, introducing noise during network training can help mitigate accuracy loss.

Finally, the impact of temperature on RRAM devices varies across different material systems and designs. Correcting for this temperature-induced accuracy loss introduces additional circuit overhead. Therefore, for each chip, it is essential to evaluate the extent of temperature’s impact on accuracy and consider the trade-off between accuracy loss and the overhead required for correction.

4. Conclusions

A multi-scale thermal simulation model for RRAM-based CIM chips is developed in this paper. Then, the temperature distribution characteristics of CIM chips and the impact of boundary thermal resistance are evaluated. A temperature reduction of 4.04 °C is achieved by optimizing the layout of digital circuits and tiles. Finally, the effect of temperature on RRAM conductance and CIM computing accuracy is analyzed, and optimized solutions are proposed. Based on the evaluation results, the ADC correction method is shown to reduce the accuracy loss from 66.4% to 1.4% at 85 °C.