A New Low-Power Circuit Design Optimization for Image Processing

Abstract

A simple-to-implement and easy-to-integrate strategy for image processing is proposed in this paper, which effectively and efficiently optimizes the power consumption of both DRAM and SRAM. Since the power consumption of DRAM is proportional to the number of bit-‘1’s and the power consumption of SRAM is linear relative to the flip probability, the proposed strategy first drops and encodes the image to minimize the number of bit-‘1’s per pixel. The processed data are then decoded, with the flag bit set to “1” to reduce the flip probability. In the experimental simulations, the power consumption of DRAM was reduced by up to 64.88%, while that of SRAM was reduced by up to 62.01%, with negligible circuit costs. In image display applications, the proposed strategy effectively compensates for certain errors in the JPEG system. In image classification tasks, there was only a 1–2% reduction in test set accuracy, demonstrating the superiority of truncation compensation. Additionally, the performance of the robot was negligibly affected by the approximate strategy, which shows the significant potential of the proposed strategy in the field of artificial intelligence and robotics.

1. Introduction

With the increasingly widespread adoption of new computer vision and artificial intelligence technologies, user demands for image and video applications have become more diverse, necessitating high definition, real-time processing, and virtual reality. As a result, deep learning algorithms for image applications have made significant advancements, leading to increasingly complex image and video algorithms and expanding datasets. This suggests that the current research focus for embedded image applications should be on effectively handling large-scale data storage.

In typical embedded image processing systems, power consumption from storage mainly originates from off-chip memory (typically using DRAM) and on-chip cache (typically using SRAM). For instance, in video processing such as MPEG or H.264, on-chip SRAM operations can consume a significant amount of energy, accounting for approximately 75% of the entire motion estimation process [1]. Simultaneously, off-chip DRAM operations account for about 30% of the total energy cost in the entire mobile circuit system [2]. Therefore, the optimization focus for power consumption generally centers on research concerning DRAM and SRAM.

After the advent of Moore’s Law [3], CMOS sizes gradually decreased, leading to increased circuit integration. Traditionally, power optimization could be achieved to the same extent by decreasing supply voltage [4]. However, in the post-Moore era, despite the ongoing increase in circuit integration, the threshold voltage of transistors in nanoscale CMOS has experienced only a marginal decrease. When the supply voltage approaches the threshold voltage, the charging and discharging capabilities of transistors diminish [5], resulting in a reduction in the operational frequency of the circuit. Additionally, in on-chip high-speed storage, lowering the supply voltage can elevate the probability of flip errors in SRAM storage cells [6]. This implies that design margins need to be provided for the supply voltage [7]. As the power consumption of on-chip storage is proportional to the square of the supply voltage, failure to reduce the supply voltage further requires exploring new storage solutions to optimize on-chip storage power consumption [8].

In this context, this paper proposes an approximate storage strategy that is simple to implement and easy to integrate. In the experimental simulations, the power consumption of DRAM was reduced by up to 64.88%, while that of SRAM was reduced by up to 62.01%. Furthermore, the proposed optimization, which sets the flag position to “1”, achieved an additional 10% reduction in power consumption for SRAM. In image display applications, it is surprising to find that the proposed strategy in this paper effectively compensates for certain errors in the JPEG system. In image classification tasks, there was only a 1–2% loss in test set accuracy, with hardware costs at the $\mathsf{\mu }$W and $\mathsf{\mu }$m² level. This paper introduces an innovative concept for low-power designs in AI and robotics at the storage level. The paper shows that on a robotic platform, the performance of the robot is almost negligibly affected by the approximate storage strategy of this paper.

Section 2 of this paper reviews related work, followed by the presentation of the proposed method and the simulation of relevant experiments in Section 3 and Section 4. Section 5 details experiments conducted on the robotic platform, with a conclusion summarizing the key points of the paper in the final section.

2. Related Work

Reducing storage power consumption typically involves power supplies or materials. For energy supply, the power sources can be explored in depth first. Exploring various power sources such as graphene batteries, solar energy, tidal energy, and other energy harvesting technologies is key. Furthermore, in terms of materials, in contrast to the traditional DRAM and SRAM storage structures under CMOS technology, various new storage materials like memristors and non-volatile ferroelectric memories [9] have garnered widespread attention in recent years. However, the development of new power supply technologies currently lags behind processor computing speeds. The standardization of graphene battery production is in its initial phases, and energy harvesting technologies like solar and tidal energy can only generate minimal power output, ranging from hundreds of microwatts to several milliwatts [10]. The advancement of power supply technologies still faces significant challenges. Resistive random access memory (RRAM) enhances high-performance large-scale computing by utilizing its crossbar switch matrix for approximate convolutional neural network computations, significantly improving energy efficiency [11]. Yet, the extensive signal analog-to-digital interface brings hefty overhead. Moreover, since this technology is still immature, significant variations occur, and wide drift in the threshold voltage range hampers practical performance. Hence, the focus is shifting towards neural network computing applications.

An alternative approach to reducing storage power consumption involves tackling the data themselves, aiming to shrink the stored amount. Recently, there has been a surge in interest in implementing and optimizing lossless compression algorithms for various embedded video image systems. In alignment with this approach, researchers focus on trimming data needs for storage by using efficient compression algorithms before saving image or video data to off-chip DRAM storage. But despite these efforts showing some gains, lossless compression algorithms for image and video still grapple with two key drawbacks. The first drawback is implementing the compression algorithms themselves. Developing hardware circuits that align with these requirements is a daunting task in that their implementation would strain the processor and raise power consumption, with the additional expense outweighing the advantages. The second issue arises from system integration difficulties. Adding compression algorithms to existing image and video systems, like MPEG and H.264 processing, entails significant system modifications and incurs considerable overhead. Given these challenges, some researchers have suggested a fresh idea: ditching the decoding system. The majority of computer vision neural networks operate with RGB images for inference. However, RGB data entail a substantial volume, leading to considerable additional storage overhead. During the CVPR 2023, a paper proposing a novel idea captured people’s attention: skipping the RGB decoding step and employing Transformer directly to extract features for training visual algorithms [12]. Yet, integrating Transformer into embedded systems poses challenges. Unlike traditional JPEG decoding, Transformer networks demand more computational resources and memory. Currently, JPEG decoding, optimized with NEON instruction sets, stands as a notably efficient option.

Contrasting with precise computation circuits, embedded image processing systems exploit the intrinsic fault tolerance of images, i.e., the human visual perception prioritizes low-frequency components over high-frequency ones [13]. Common JPEG compression algorithms capitalize on this, extracting frequency-domain information, then reducing stored data via quantization and encoding. Essentially, in storage, image pixel information is categorized into vital and less vital based on the sensitivity of human vision. Notably, high-bit information outweighs low-bit information for a pixel byte. Based on this characteristic, scholars both domestically and internationally have made considerable attempts.

As shown in Figure 1 [14], the power consumption of DRAM stems from refresh operations. When data are stored as bit-‘1’s, the capacitor charge decreases over time, requiring dynamic refresh to offset the leaked charge and preserve data integrity. In other words, off-chip DRAM power is directly linked to the refresh frequency and the number of high bits stored. Leveraging this, designs aiming to reduce off-chip DRAM power via approximate computing can adopt simple data truncation methods, reducing the bit-‘1’ count and hence power. Yet, data truncation poses a risk of losing original data information; despite that, it is convenient to implement and embed in preprocessing circuits. Real-time monitoring is crucial to assess output quality and ensure image standards, which also adds extra overhead. Dr. Song and his team adopted a strategy using hierarchically tiered refresh processes for off-chip DRAM [15]. They gradually reduced the refresh frequency for lower-bit data while maintaining standard refresh frequencies for higher-bit data of image pixels, aiming to lower power consumption to some extent, albeit at the expense of sacrificing some output quality. Yet, altering the entire refresh system could introduce extra overhead, potentially negating the power reduction benefits achieved in practical applications.

As shown in Figure 2, the power consumption of SRAM involves three parts: write power, read power, and leakage power. Notably, the power consumption of the write operation, which is proportional to the data flip probability and supply voltage, dominates. Dr. Mohapatra used these traits to develop energy-efficient on-chip storage for tasks like image processing by replacing the standard 6T SRAM structure with a more complex 8T design [16,17]. This 8T structure design boasts lower read operation errors when the voltage is reduced. To minimize power consumption, the author opted for a hierarchical setup, storing high-bit image data in the 8T structure and low-bit pixel data in the 6T structure. This 8T structure has a lower read operation error probability than the 6T structure during voltage reduction. To reduce power usage, the author employed a hierarchical design, storing high-bit image data in the 8T structure and low-bit pixel data in the 6T structure. Through a unified voltage reduction strategy, a successful low-power design was achieved. Since the high-bit 8T structure exhibits a low error probability under low voltage, overall image quality reduction is minimal. However, the additional area and power overhead brought about by this method can significantly outweigh the benefits in practical applications. Furthermore, redesigning the original storage cells is necessary. It should be noted that although the output quality stays at an acceptable level, significant voltage reduction decreases the operational frequency, which could lead to timing errors.

In short, new power sources and storage materials are still in development, implementing lossless compression algorithms is challenging, and current image codec systems are well established. So the focus should be on offering hardware designers storage solutions with minimal extra overhead in current embedded image codec setups, ultimately reducing storage power consumption.

Compared to previous approaches, the method proposed in this paper offers significant advantages in terms of hardware implementation. It can be seamlessly integrated into existing image codec systems without any modifications to the current circuit layout, with negligible additional hardware overhead. More importantly, it effectively reduces the power consumption of both DRAM and SRAM. Surprisingly, the proposed method also compensates for the errors introduced during the JPEG decoding process.

3. Proposed Method

Dr. Yang et al. leveraged DRAM and SRAM storage traits, along with image fault tolerance, to devise a low-power, high-performance storage solution for embedded image applications [18]. Building upon this, this paper has made further optimizations and applied them in actual JPEG decoding systems for experimental simulations.

As shown in Figure 3, the image data retrieved from the hard disk or terminal sensor are initially stored in off-chip DRAM before being transferred to on-chip SRAM for additional processing. In some cases, a portion of the computation results is written back to the off-chip DRAM. The power consumption of DRAM is directly proportional to the number of high bits stored. Therefore, reducing the number of bit-‘1’s before sending the data to off-chip storage can lower the power consumption of DRAM. Notably, if an image predominantly contains bit-‘0’s, the flip probability when transferring the data to on-chip SRAM is greatly reduced. This, in turn, optimizes the power consumption of SRAM. The strategy based on the error-compensated dropping method and encoding for joint optimization proposed by Dr. Yang et al. capitalizes on this feature, with which they partitioned each pixel byte into “important” and “unimportant” parts based on the image’s fault-tolerant characteristics. For the high-weight “important part”, they implemented flip encoding, designating the last bit as the flag bit. For the low-weight “unimportant part”, they used an error-compensated dropping method instead of a complete dropping approach to avoid excessive image loss.

While Dr. Yang’s strategy demonstrated impressive power consumption outcomes with original RGB data, it has not been applied in real image encoding and decoding systems for simulation yet. In this paper, the original RGB image is processed by the JPEG core algorithm, i.e., discrete cosine, quantization, inverse quantization, inverse discrete cosine, to obtain the decoded RGB image and the original RGB image for comparison. In the analysis of the JPEG-decoded image errors in this system, depicted in Figure 4, it is noted that most errors fall within the range of −5 to 0, with approximately 80% of the errors being negative. This indicates a bias in the JPEG decoding system’s errors. By setting flag bit, i.e., the least significant bit, to “1” before decoding into on-chip SRAM, this bias can be compensated for to some extent. After compensation, a significant improvement in error bias is observed in Figure 5. Moreover, when the last bit of every pixel byte is always “1”, the probability of bit flips in SRAM is significantly lowered. This optimization not only compensates for the image loss due to the JPEG compression algorithm but also further reduces the power consumption of SRAM.

Guided by the pseudocode of the proposed strategy in Algorithm 1, the flow of the error-compensated dropping method and encoding is illustrated in the top half of Figure 6. The raw 8-bit pixel information is divided into the “important part” (high part) and “unimportant part” (low part), and the variable “K” denotes the number of bits in the “important part” of the 8-bit pixel, where K can be any value from 1 to 7. If the count of bit-‘1’s in this part is greater than K/2, a flip operation is performed, and the last bit is set to “0”. If it is less than K/2, only the flag bit is set to “1”, without any other changes. For the “unimportant part”, the error-compensated dropping method is applied. This involves scanning the bits from high to low, and when encountering the first “1”, retaining that bit and setting all subsequent bits to “0”. In the decoding process, as can be seen in the lower part of Figure 6 and Algorithm 2, if the flag bit is “0”, the “important part” is flipped back to its original state. If the flag bit is “1”, no action is taken. Finally, the flag bit is uniformly set to “1”.

Algorithm 1 Data Encoder

1:   function DataEncoder(PixelByteStream) 2:       Initialize EncodedDataStream as empty 3:       for all Pixel in PixelByteStream do 4:           $(HighPart,LowPart)$ ← SplitPixel(Pixel)     ▹ Split into high and low parts 5:           $DroppedLowPart$ ← ErrorCompensatedDropping(LowPart) 6:           $HighPartOnesCount$ ← CountOnes(HighPart) 7:           if $HighPartOnesCount>{\displaystyle \frac{h}{2}}$ then 8:              $HighPart$ ← Invert(HighPart) 9:              $FlagBit\leftarrow 0$ 10:         else 11:            $FlagBit\leftarrow 1$ 12:         end if 13:         Add$(EncodedDataStream,(FlagBit,HighPart,DroppedLowPart\left)\right)$ 14:     end for 15:     return EncodedDataStream 16: end function

Algorithm 2 Data Decoder

1:   function DataDecoder(EncodedDataStream) 2:       Initialize DecodedDataStream as empty 3:       for all EncodedPixel in EncodedDataStream do 4:           $(FlagBit,HighPart,DroppedLowPart)$ ← SplitEncodedPixel(EncodedPixel) 5:           if $FlagBit==0$ then 6:              $HighPart$ ← Invert(HighPart) 7:           end if 8:           $ReconstructedPixel$ ← CombineParts(HighPart, DroppedLowPart) 9:           Add$(DecodedDataStream,ReconstructedPixel)$ 10:     end for 11:     return DecodedDataStream 12: end function

Taking an example, when K = 3, as shown in Figure 7, it can be seen that if the count of bit-‘1’s in the three high-order bits exceeds 1, a flip operation is performed, and simultaneously the least significant bit is set to “0”. For the four low-order bits, scanning from high to low, upon encountering the first “1”, all subsequent bits are truncated to “0”. During the decoding process, the flag bit determines whether the high-order bits should be reversed, after which the flag bit is consistently set to “1”. Subsequent experiments demonstrate the compensatory advantages of the error-compensated dropping method and the optimization of SRAM power consumption achieved by setting the least significant bit to “1”.

As can be seen, the lower the value of K in the strategy, indicating more bits in the low segment, the greater the truncation of information, resulting in more loss of image data. In practical image applications, the specific value of K depends on the requirements of the specific application for the output quality of image; experimental simulations can be carried out before the design of the system to dynamically adjust the value of K, to assess the output quality of image, and to take the smallest value of K under the condition of satisfying the output quality of image in order to achieve the optimization of the storage power consumption.

4. Simulation Experiment

In system design for embedded image processing applications, sensors like industrial cameras at terminals often come with integrated JPEG and MPEG encoders capable of outputting the respective encoded data. Hence, hardware designers should focus on decoder research to minimize storage power consumption. The strategy proposed in this paper, indeed, is also applicable to the design of terminal systems, which can be deployed to the front-end of the image encoder of the terminal sensors, leading to a significant reduction in storage power consumption for these systems.

As depicted in Figure 8, the proposed strategy is applied to the JPEG decoding system, which includes core JPEG decoding processes. After decoding to an RGB image, the image is passed to hardware modules for the error-compensated dropping method and encoding before being written to off-chip storage DRAM. For further common image processing tasks, such as display, classification, segmentation, and object detection, the operations are performed on the on-chip storage SRAM. This occurs after the data have been decoded and processed by the hardware that sets the flag bit to “1”.

The experiments in this paper were conducted within this framework, using C++ for low-level image operations and leveraging the OpenCV library for compilation. We simulated 1 million JPEG images altogether. To eliminate the impact of image resolution, the resolutions of these images were randomly distributed. After decoding, we performed the error-compensated dropping method and encoding operations, calculating the count of bit-‘1’s as the metric for the DRAM power consumption simulation. Following decoding and flagging the last bit with “1”, we varied the cache sizes to calculate the flip probability as the SRAM power consumption metric. Additionally, we simulated extra hardware modules to determine additional costs. Finally, given the lossy nature of the error-compensated dropping method, the paper focuses on specific image processing applications. In this research, the accuracy of the test set for image classification tasks is utilized to gauge the output quality of images subjected to lossy processing, the results of which are also compared with the results using the complete dropping method.

4.1. Storage Power Consumption Simulation

As shown in Figure 9, the different modules of the storage power simulation were compiled and implemented by calling the OpenCV library via C++ and have been released to the open-source community, which makes it convenient for hardware designers to call the relevant interfaces directly for simulation to determine the optimal K value.

The 1,000,000 JPEG images were first processed with the error-compensated dropping method and encoding operations (with varying K values). These processed images were then fed into a DRAM simulation model to count the number of bit-‘1’s, providing a metric of DRAM power consumption. Analyzing the data from

Table 1. DRAM power consumption simulation results.

K	With No Processing	1	2	3	4	5	6	7
Navg	3.955	1.389	1.835	2.321	2.581	2.969	3.043	2.970

Note: Navg represents the average number of logical “1” in a single pixel.

, with no processing, the average count of bit-‘1’s is obtained. With this baseline, we computed the percentage decrease in the number of bit-‘1’s after the error-compensated dropping method and encoding operations with different K values, which reflects the average decrease in DRAM power consumption. In Figure 10, with K = 7, losing only the flag bit (the last bit) results in a notable 24.91% average decrease in power consumption, demonstrating the efficacy of the encoding technique. For cases where high image output quality is not necessary, selecting smaller K values with the error-compensated dropping method will further reduce DRAM power consumption. Specifically, when K = 1, DRAM power consumption can drop significantly, by an average of 64.88%.

   It should be noted that there is not much difference in the power consumption of DRAM when K = 5, K = 6, or K = 7. This phenomenon occurs when the low segment contains very few bits, minimizing the impact of the error-compensated dropping method on optimization due to its inherent error-compensation properties. Especially when K = 6, there is only 1 bit in the low part, the data of this bit will not be changed, and the error-compensated dropping method fails. At this time, comparing with the case of K = 7, what affects the DRAM power consumption is rather the number of bits of encoding, and the number of bits of encoding is more in K = 7, so the DRAM power optimization of K = 7 has a better performance than that of K = 6.

After decoding and setting the flag bit to “1”, the image data are moved to cache areas of 512 KB and 128 KB, respectively. The process includes determining the likelihood of bit flips during read/write operations. Subsequently, the average bit-flip probability is calculated across the 1 million images to serve as a metric for simulating the SRAM power consumption. In order to compare the benefits of the operation of having the flag set to “1”, this experiment also counts the results of the experiment with the flag not set to “1”. As shown in

Table 2. SRAM power consumption simulation results.

K	With No Processing	1	2	3	4	5	6	7
Filp probability/cache areas of 128 KB	0.49	0.20	0.27	0.32	0.37	0.41	0.43	0.43
Filp probability/cache areas of 128 KB (Flag bit is not set to 1)	0.49	0.26	0.29	0.38	0.42	0.47	0.47	0.48
Filp probability/cache areas of 512 KB	0.44	0.17	0.23	0.28	0.34	0.37	0.39	0.39
Filp probability/cache areas of 128 KB (Flag bit is not set to 1)	0.44	0.20	0.24	0.33	0.37	0.42	0.43	0.44

, the average bit-flip probability of the unprocessed image is around 50%, consistent with the expected distribution.Comparing this baseline with different K values reveals the reduction in bit-flip probability, indicating a reduction in SRAM power consumption.

As shown in Figure 11, generally, a 512 KB cache size shows better power consumption than 128 KB. While selecting a larger cache size can successfully cut down SRAM power consumption, it is crucial to take into account the associated hardware resource expenses. At K = 7 (512 KB), setting only the flag bit of each pixel (the last bit) to “1” significantly reduces SRAM power consumption, by 12.71%. This highlights the optimization effect of setting the flag bit to “1” for SRAM. Similar to the simulation results of DRAM, as K decreases, the number of “0”s per pixel increases, leading to a significant reduction in bit flips. At K = 1 (512 KB), SRAM power consumption even drops to an average of 62.01%.

To better illustrate the optimization effect of setting the flag bits to “1” on SRAM, we conducted a comparison between the outcomes with and without implementing this setting. As shown in Figure 12 and Figure 13, across different K values, the power consumption of SRAM can be further reduced by an additional 10% without the flag bit being set to “1”. When the flag bit is “1”, one-eighth of the bits will never be flipped, so the results of this comparison experiment are also consistent with the probability distribution.

4.2. Circuit Overhead Simulation

This section focuses on conducting a circuit simulation on the hardware modules for the error-compensated dropping method and encoding, as well as hardware modules for decoding and setting the flag bit to “1”.

In the case of K = 3, the proposed strategy’s logical structure is outlined in Algorithm 3. It takes 8-bit pixel input, and a three-bit adder cnt is introduced to perform a counting operation on the high part. If the number of bit-’1’s in the top three bits exceeds 1, the encoding operation is performed, setting the flag bit to “0”. If not (≤1), only setting the flag bit to “1”. For the lower four bits (from the fifth to the second bit), if the fifth bit is “1”, the subsequent bits are truncated to “0”. Otherwise (the fifth bit is “0”), the fourth bit is checked. If the fourth bit is “1”, subsequent bits are also truncated to “0”, and so on.

Algorithm 3 Circuit logic structure

INPUT: $P_{\mathrm{input}}[7:0];K=3$ OUTPUT: $P_{\mathrm{output}}[7:0];K=3$ 1:   $\mathrm{cnt}\leftarrow 0$ 2:   $P_{\mathrm{output}}\left[1\right]\leftarrow \overline{P_{\mathrm{input}}\left[4\right]}\cap \overline{P_{\mathrm{input}}\left[3\right]}\cap \overline{P_{\mathrm{input}}\left[2\right]}\cap \overline{P_{\mathrm{input}}\left[1\right]}$ 3:   $P_{\mathrm{output}}\left[2\right]\leftarrow \overline{P_{\mathrm{input}}\left[4\right]}\cap \overline{P_{\mathrm{input}}\left[3\right]}\cap \overline{P_{\mathrm{input}}\left[2\right]}$ 4:   $P_{\mathrm{output}}\left[3\right]\leftarrow \overline{P_{\mathrm{input}}\left[4\right]}\cap \overline{P_{\mathrm{input}}\left[3\right]}$ 5:   $P_{\mathrm{output}}\left[4\right]\leftarrow \overline{P_{\mathrm{input}}\left[4\right]}$ 6:   for $i=7$  to  $i>8$  step  $i=i-1$  do 7:       if $P_{\mathrm{input}}\left[i\right]$ then 8:           $\mathrm{cnt}\leftarrow \mathrm{cnt}+1$ 9:       else 10:         $\mathrm{cnt}\leftarrow \mathrm{cnt}$ 11:     end if 12: end for 13: if $\mathrm{cnt}>1$  then 14:     $P_{\mathrm{output}}[7:5]\leftarrow P_{\mathrm{input}}[7:5]$ 15:     $P_{\mathrm{output}}\left[0\right]\leftarrow 0$ 16: else 17:     $P_{\mathrm{output}}[7:5]\leftarrow P_{\mathrm{input}}[7:5]$ 18:     $P_{\mathrm{output}}\left[0\right]\leftarrow 1$ 19: end if 20: RETURN:  $P_{\mathrm{output}}[7:0]$

For the decoding module, according to the flag bit, judge whether to carry out the flip operation or not for the high part, and finally unify the flag bit as “1”. The logic computation structure in this paper is implemented in the Verilog language, utilizing the SMIC 55 nm process. It is synthesized with Synopsys’ Design-Compiler tool and analyzed using the PrimeTime Tool to assess the power and area metrics, representing the hardware overhead discussed. Power consumption and area are in the microwatt ($\mathsf{\mu }$W) and square micrometer ($\mathsf{\mu }$m²) range, respectively, as shown in

Table 3. Power and area overhead.

K Value	1	2	3	4	5	6	7
Power ($\mathsf{\mu }$W)	77.48	61.83	27.55	18.64	18.22	20.75	25.47
Area ($\mathsf{\mu }$m2)	241.84	171.92	120.44	84.61	91.64	71.40	89.32

.

For JPEG decoder hardware with millions of gates, the additional circuit cost and its impact on timing are negligible. The primary advantage of the proposed method in this paper over related works lies in its ease of hardware implementation. As illustrated in Figure 14, this flowchart represents the typical data flow in image processing. The method proposed in this paper can be seamlessly integrated into on-chip hardware systems for image processing: the peripheral interface captures data from vision sensors, which are then processed by image/video decoding hardware and transferred to the SDRAM for storage via the error-compensated dropping method and encoding hardware under the control of an off-chip storage controller. When image display or network inference is required, the data undergo further processing through the decoding and flag-setting hardware modules. This hardware optimization enhances the feasibility and scalability of the proposed method for practical applications.

4.3. Output Quality Simulation for Image Application

As a result of the error-compensated dropping method’s lossy nature in handling pixel information, this section assesses the output quality of the image. Notably, image display is pivotal in embedded image applications.

For specific embedded image applications in resource-constrained terminals, the GUI display consumes significant resources. Since human vision is less sensitive to images with an output peak signal-to-noise ratio (PSNR) exceeding 30 dB [19], the method proposed in this paper can be applied to a wide range of image display applications. The JPEG core algorithm, which includes DCT, quantization, inverse quantization, and IDCT, was applied to one million raw images with varying resolutions. The proposed method was then applied to these images, and the resulting output images were compared with the original raw images. The average output PSNRs for the one million images, corresponding to different K values, were calculated and are shown in Figure 15. Additionally, experiments were conducted on images that were not processed using the proposed method, and the PSNR benchmark was obtained. To highlight the contribution of this work, Figure 15 also compares the results with those from processing without setting the flag bit to ‘1’. It can be observed that the impact of setting or not setting the flag bit to ‘1’ on the image output is minimal. Notably, the PSNR of the images with the flag bit set to ‘1’ is slightly higher than that without it, further confirming the optimization effect of the proposed method on the JPEG decoding system. For specific applications, the case with K = 4 was selected, as it meets most display requirements.

However, with the increasing popularity of artificial intelligence, embedded image applications have evolved beyond mere display functions. To conserve terminal resources, common applications now skip GUI interfaces and directly perform tasks like image classification, segmentation, and object detection after acquiring image data. Therefore, it is essential to explore the approximate storage’s impact on common image processing tasks, ensuring it optimizes storage consumption while maintaining excellent performance in image processing. However, it should be noted that the method proposed in this paper for AI applications is only applicable to the storage management process before the image is sent to CPU for network inference. In subsequent network inference computations, accurate storage is required, and no further approximate storage operations can be performed. This limitation of the proposed method highlights the need for additional strategies to optimize both computing and storage power consumption across multiple network layers.

In embedded systems, artificial neural networks (ANNs), particularly deep convolutional networks, are frequently employed for tasks like image classification, object detection, and semantic segmentation. The error-compensated dropping method, as a lossy processing method, may impact the accuracy and precision of ANN inference tasks. As shown in Figure 16, when deploying the AlexNet network to MCUs [20], the TensorFlow framework was used to convert the model files to special files for memory mapping analysis. It was found that the flash memory required was as high as 208 MB, which exceeds the capacity of most MCUs currently on the market. This highlights the fact that embedded devices typically have limited computational power and storage resources, necessitating lean and efficient models. Simpler models, such as lightweight neural networks or support vector machines, are sufficient to validate the impact of the proposed method on model performance, without the need for complex network architectures. Therefore, this paper selects common datasets, including Fashion MNIST, MNIST handwritten digits, and the Iris flower dataset. Additionally, to assess the method’s impact on the performance of slightly more complex models, the paper also incorporates the CIFAR-10 dataset.

Table 4. Network structure and hyperparameters.

Dataset	Network Structure	Initial Learning Rate	Lr_Decay (0.9)	Batch_Size	Epochs
Fashion	MLP 128-MLP 4	0.001	/	32	20
MNIST handwritten digits	MLP 128-MLP 4	0.001	/	32	20
Iris flower	MLP 16-MLP 8-MLP 3	0.001	/	32	20
CIFAR-10	ResNet18	0.01	2500	128	40

presents the network structure and some hyperparameter settings used for training on each dataset. Common popular neural network models for image classification were used to validate the accuracy of the test set after approximate storage processing, serving as a research metric for image output quality.

As an example, the Fashion dataset and the CIFAR-10 dataset both have 60,000 JPEG images in the training set and 10,000 JPEG images in the test set. By using a fully connected neural network with 128 neurons in the hidden layer for the Fashion dataset, we output 10 classifications through softmax. As indicated in

Table 5. Fashion dataset training results with different storage strategies.

Epoch	1	2	3	...	15	16	17	18	19	20
Accuracy (Accurate storage)	84.9%	85.4%	85.9%	…	88.1%	87.9%	88.9%	88.0%	88.6%	88.5%
Accuracy (K = 7)	82.9%	85.9%	85.7%	…	88.4%	88.6%	88.2%	89.3%	88.5%	88.9%
Accuracy (K = 6)	84.2%	86.2%	86.4%	…	87.6%	88.9%	88.4%	88.8%	88.7%	89.2%
…	…	…	…	…	…	…	…	…	…	…
Accuracy (K = 1)	82.7%	85.3%	86.2%	…	87.4%	88.1%	87.9%	87.8%	88.1%	88.3%

, without approximate storage processing, the test set accuracy stabilizes between 88% and 89% after 20 training epochs. Notably, using different K values for the approximate storage strategies has minimal impact on the test set accuracy. This suggests that for simple image classification tasks, the neural network learns the most crucial features during training. Despite significant quality loss in input images, the network can still extract crucial features for accurate classification. As shown in

Table 6. CIFAR-10 dataset training results with different storage strategies.

Epoch	1	2	3	...	35	36	37	38	39	40
Accuracy (Accurate storage)	64.9%	67.7%	70.3%	…	89.3%	88.9%	89.7%	90.4%	89.6%	89.8%
Accuracy (K = 7)	61.2%	69.9%	71.7%	…	89.4%	88.7%	90.2%	89.3%	90.4%	89.9%
Accuracy (K = 6)	65.7%	69.3%	69.9%	…	89.7%	89.8%	89.4%	90.6%	89.7%	89.8%
…	…	…	…	…	…	…	…	…	…	…
Accuracy (K = 1)	61.8%	66.5%	70.2%	…	90.3%	89.1%	89.5%	88.8%	89.5%	89.9%

, for the slightly more complex CIFAR-10 dataset, even with the ResNet18 network [21], the results remain similar to those of the Fashion dataset. This indicates that the method proposed in this paper is also applicable to more complex networks. Similar results are observed with the MNIST handwritten digits and Iris flower datasets. Hence, for such image classification tasks, hardware designers can confidently adopt approximate storage strategies with K = 1. As shown in Figure 17, the green curve depicts the test set accuracy trend during training without any processing, while the blue curve illustrates the trend after implementing an approximate storage strategy (K = 1), resulting in only approximately a 1% decrease in accuracy. Yet, applying the complete dropping method (K = 1) results in a 9–10 percentage point decrease in accuracy (see the purple curve), deemed unacceptable for classification tasks. This suggests the advantage of error compensation in our proposed approximate storage strategy.

4.4. Experiment on a Robotic Platform

With the rapid advancements in AI, machine learning, and sensing technologies, robots have gained enhanced capabilities in perception, decision making, and task execution across various fields. This paper exactly focuses on designing a low-power storage system in an actual robot platform.

As shown in Figure 18, this is a simple tracking application. The tracked robot on the left is equipped with a Richbeam single-line lidar and uses a differential drive model for motion control at the lower level. Data fusion from gyroscopes and motor encoders through extended Kalman filtering generates odometry [22]. The Cartographer algorithm constructs a global map by incorporating lidar point cloud data [23]. The RRT* path planning algorithm directs the robot for patrol in specified scenarios [24]. The tracked robot also features a controllable temperature ceramic heater on top, while the tracking robot is fitted with an infrared camera that captures infrared images. It runs an object detection model to identify the coordinates of the ceramic heater. The network structure and hyperparameter settings for target detection are shown in

Table 7. Network hyperparameters of object detection.

Network Structure	Initial Learning Rate	Lr_Decay (0.9)	Batch_Size	Epochs
Conv(1,16)-Conv(16,32) -Conv(32,64)-MLP 128-MLP 4	0.01	2500	400	60

. Once the coordinates are recognized, it issues motion control commands to track the robot.

In this paper, the tracking robot is equipped with a low-power storage system design. An infrared camera captures infrared images and outputs a stream of images, and this image stream undergoes the error-compensated dropping method and encoding operations. Subsequently, the decoding process and setting the flag bit to “1” are carried out at the cache’s front end. The results of inference are sent to the robot’s underlying control board via the TCP communication protocol to control motion and achieve tracking.

As shown in

Table 8. Comparison of power consumption and performance of different deployment platforms.

Storage Method	Accurate Storage	Approximate Storage
Time taken to reach destination (s)	12.17	12.56

, we compare the approximate image stream with the unprocessed image stream. The actual robot performance is hardly affected at all, and it can be seen that the time taken to reach the final destination is almost the same.

5. Conclusions

In this paper, we explore the features of image coding and decoding systems, introduce a novel optimization of the approximate storage strategy, and implement it in the JPEG decoding system. In the JPEG decoding process, we perform the operation of the error-compensated dropping method and encoding on the decoded image data, which reduces the number of high bits to be stored before sending to DRAM, and greatly reduces the storage power consumption of DRAM. Before sending to SRAM for further computation, we perform the decoding and then set the flag “1”. Experimental verification confirms that the optimization strategy of setting “1” not only compensates for the JPEG decoding error but also leads to a reduction in SRAM power consumption. The method in this paper can achieve the joint optimization of DRAM and SRAM at the same time, while the hardware cost and the modifications of the system are almost negligible, and the significant advantage of this method in error compensation can be seen in the evaluation of the image output quality. After the validation of the final robotics platform, we believe that this strategy has unlimited potential in the future era of embodied AI.