Adaptive Color Selection to Limit Power Consumption for Multi-Object GUI Applications in OLED-Based Mobile Devices

: Recently, as OLED displays are widely used in smartphones, their power management has become an important issue. Since OLED display devices show a large difference in power consumption depending on the colors used, proper color selection is important to reduce power consumption, but this may affect quality-of-experience (QoE) perceived by users. To address this tradeoff, this paper proposes a new power management scheme for graphical-user-interface (GUI) applications in OLED-based mobile devices. First, we examine the relationship between power consumption and color usage based on actual measurements and derive a model to express color preferences. Next, we present a dynamic programming algorithm that adaptively changes the color of the GUI objects in the applications to limit the total power consumption while effectively reﬂecting the color preferences of the user. The proposed scheme was implemented on a commercial smartphone, and real power measurements were conducted to examine whether it uses the given energy budget accurately under various color preference conﬁgurations. Experimental results show that color preferences signiﬁcantly affect both power consumption and QoE, thereby demonstrating that personalized color selection plays an important role in reducing total power consumption of an OLED smartphone. Author Contributions: Conceptualization, M.S. and Y.L.; methodology, M.S. and Y.L.; software, Y.L.; supervision, M.S.; validation, M.S. and Y.L.; writing-original draft, M.S. and Y.L.; Writing-review and editing, M.S.; All authors read and agreed to the published version of the manuscript.


Introduction
Recent advances in mobile technology have enabled people to use their mobile devices such as smartphones and tablet PCs anytime, anywhere. Organic light-emitting diode (OLED) displays become widely used as their display devices because they have many advantages compared with liquid crystal display (LCD) displays such as wide viewing angles, high picture quality, thin form factor and so on [1].
Display is known to be one of the biggest energy consumers in mobile devices. Although OLED has no backlight, consuming relatively lower power than LCD with backlight, it still consumes a high proportion of power in mobile devices [1]. For example, it is known to account for as much as two-thirds of the power used by the device, which makes its effective power management essential. In addition, the tendency to use smartphones with large displays makes this power problem worse.
Each pixel in OLED is composed of light emitting diodes with three primary colors (red, green, and blue), and a combination of these colors allows the representation of a specific color. The amount of power consumed by the OLED display is then highly dependent on the intensity of these colors; thus, a pixel with white color with the highest intensity consumes the highest power, whereas that with black color consumes the lowest power [1,2]. Therefore, reducing their intensity is known as an effective way to reduce OLED power consumption and most of the OLED power management schemes

Related Work
As OLED displays in smartphones are gaining popularity, many studies have been conducted to reduce their power consumption. For example, several color-conversion techniques have been presented using the relationship between color usage and OLED power consumption. Dong and Zhong [2] developed a web browser that allows color transformation based on a color-mapping table that reflects user preferences. To avoid high computational overhead on smartphones, this mapping table is constructed offline on a cloud server. Li et al. [6] developed an automatic way to rewrite a web application that produces energy-efficient web pages based on a program analysis of the structure of the web-application implementation. Linares-Vasquez et al. [9] also developed a color-conversion technique that generates color palettes using a multi-objective optimization technique. This technique produces color maps by optimizing the contrast while using consistent colors with respect to the original color palette. Chang et al. [10] presented an OLED power-saving scheme under scrolling operations for web browsers based on the generation of power-saving color-transformation maps for web pages. Most of the studies mentioned above, however, were developed specifically for web applications.
Some previous studies made use of partial dimming techniques in which a portion of the display is darkened to reduce power consumption. Tan et al. [3] proposed a technique that darkens relatively non-focused display areas to take account of user usage patterns. Chen et al. [4] presented a method for darkening the touched part by tracking the touch of the finger on the smartphone. Ginny et al. [11] proposed a dynamic local dimming scheme that dims the display region of non-critical application windows by making use of context switching information among applications. Lin et al. [12] developed an app called ShiftMask that enhances the display region of users' interests, while dimming the remainder of the screen. To find the regions of interests, it kept track of visual attention, changes in view focus, screen scrolling, etc. Yeh et al. [13] presented a visual-attention-based pixel dimming technique to achieve power saving for real-time video playback. For this purpose, they extracted a motion vector from each frame to partially dim unnoticeable parts of the video frames.
Alternatively, some previous works have attempted to minimize image quality degradation during the process of color conversion. Li et al. [14] presented a scheme to change the display color of regions that are not classified as regions of interest by making use of edge-detection algorithms. Lin et al. [15] proposed a power saving scheme that minimizes image distortion incurred during the dimming process by analyzing humans' focus of attention. Yan et al. [16] presented a scheme called ShutPix, which is a power-saving display system for OLED smartphones that can optimally shut off the redundant subpixels before the content is displayed. Pagliari et al. [17] focused on reducing overheads required for image quality enhancement during the process of color conversion, which is necessary for OLED power minimization.
Parameters used in video processing affect OLED power consumption. By exploiting this property, Lin et al. [15] adjusted the degree of brightness in GUI applications by reducing the brightness of display areas that are not attracting the interest of the user at a given moment. Chen et al. [18] developed a video-recording system called MORPh that optimizes the parameters of video clips during the video-recording phase to reduce OLED energy consumption in video playback. Park and Song [19] presented an alpha-blending scheme specifically for video playback, wherein the brightness factors are dynamically adjusted by considering QoE. Refresh rates in display systems also have a significant impact on OLED power consumption. Kim et al. [20] presented a scheme that optimally adjusts the refresh rates according to the content frame rate. This allows for the elimination of redundant display updates, thereby minimizing OLED power consumption.
A QoE metric is the result of personal preferences, which need to be reflected in power optimization. For example, Yan et al. [21] examined how the use of dynamic backlight scaling policies employed in LCD power optimization affects QoE and proposed the combination of different backlight scaling techniques. However, to the best of our knowledge, no OLED power management scheme has been reported that limits power consumption based on personal color preferences.

Basic Idea
GUI applications are typically composed of multiple GUI objects, each with its own color. The amount of OLED power consumption can be effectively reduced by converting GUI colors to less power-consuming colors, but this may affect the QoE perceived by users. In particular, people have strong preferences concerning the perception of colors. Such preferences must be reflected for color selection [7,8]. For example, some users prefer dark colors over bright ones, leaving room for more power reduction without deteriorating the QoE. To address this tradeoff, the objective of this paper is to take account of such color preferences, subject to a prescribed energy budget, in GUI-based applications. Accordingly, our objectives can be concisely explained as follows: 1.
Color preference score modeling: each color is assumed to be associated with a score. The most preferred color may be found through a survey [7,8]. For example, users can be shown colors and be asked to rate their preferences for at an absolute score or they can be asked to indicate which of two simultaneously presented colors they prefer for each possible pair of colors in the sample. We assume that the most preferred color is known in advance. The color preference score for a specific color is then modeled by its relative color difference from the most preferred color. To express the color difference, we used the Euclidean distance model based on the CIE76 formula, one of the most famous color difference models [7,8]. We also used a 5-point scale metric that is commonly used to express QoE: the most preferred color is 5 points, and the color with the maximum difference from the most preferred color is 0 points, thereby following the ITU-T guidelines for QoE scores [22]. The minimum unit of score is set to 0.01.

2.
Power modeling: the color of the pixel in the screen determines the amount of power consumed by the OLED display [2]. Given that each smartphone has different OLED power characteristics depending on the color, we made real power measurements for various colors on a prescribed smartphone. We then derived the amount of power per pixel for each color.

3.
Problem formulation and algorithm development: the goal of our study is to accurately limit the power consumption while taking account of personal color preferences. For this purpose, we define an optimization problem through which the color of each GUI object is determined with the aim of maximizing the overall color preference score subject to a power limit. We then propose an optimal algorithm to solve such a problem using a dynamic programming technique.

Color Preference Model
Consider a GUI application with N object GUI objects and N pixel pixels ( Table 1 summarizes the symbols used in this paper). Let N color be the number of colors that can be configured for each GUI object in the application. Let N pixel i be the number of pixels for the i-th GUI object, (i = 1, ..., N object ). Assume that the background image in the GUI application is fixed. Then, the relative color preference score compared to the background image is expressed as a value between 0 (worst) and 5 (best) with a minimum unit of 0.01 [22]. We consider 29 colors with their corresponding indices as tabulated in Table 2. There are seven basic colors (i.e., red, green, blue, cyan, magenta, yellow, and white). Based on this, the intensity of each color is reduced to three levels from the original color that, in turn, reduces its brightness. For example, the intensity of red, {R, G, B}, is (255,0,0) and then the three colors with reduced brightness are (191,0,0), (127,0,0), and (63,0,0) as shown in Table 2. A black color is also added to express the darkest color. We define the color preference score for the j-th color of the i-th GUI object as follows: • We assume that the QoE score based on color preference for the j-th color of the i-th GUI object in the background image of the game is fixed. Then, the relative-color-preference score in comparison with the background image is expressed as a value between 0 (worst) and 5 (best) with a minimum unit of 0.01. These scores can be derived using a survey to reflect personalized preferences [7,8].
Let Q color j be the color preference score for color j compared to the background image.

•
The color of large GUI objects has a greater impact on the user QoE than smaller objects. To reflect this, we calculate the score for the j-th color of the i-th GUI object as follows: To derive the color difference, the CIE76 formula is typically used in which the color difference compared with the preferred color can be objectively calculated by using the Euclidean distance in the CIELAB color space [7,23]. We assume that the most preferred color is known a priori. Let I prefer be the index of the color that a particular user prefers the most for a specific background color image. The values of Q i,j can then be calculated as follows: 1.
Suppose that (L j , A j , B j ) respectively represents the values of (lightness, red/green, blue/yellow) for color j in the CIELAB color space. These values can be easily calculated from the intensity values of red, green, and blue, {R, G, B} [7,19]. Based on the CIE76 formula, we can calculate the color difference, D m,j , between color j and m as follows [7,19]:

2.
Let I max be the index of the color that shows the maximum value of color difference compared with color I prefer . It can be expressed as follows:

3.
A color preference score of 0 is given to color I max , which represents the maximum color difference compared to the most preferred color. Accordingly, the color preference score for color j, Q j can be calculated to derive the relative color difference as follows: In this equation, the value of Q color I prefer is set to 5, the highest value on the 5-point scale which is widely used for the QoE score. Thus, the highest score of 5 is given to color I prefer , and the score decreases as the difference in color increases.

4.
Then, a color preference score of the j-th color of the i-th GUI object, Q i,j can be easily derived using Equation (1).
A user survey may be required to find the most preferred color. There are various methods for the survey [7,8]. One simple method to derive the color index I prefer is to ask the user to score each color when the background image is fixed and then the color with the highest score can be selected as the most preferred color [8]. These kinds of methods typically takes a little time [7,8].

Oled Power Modeling
Each OLED-display device has its own power consumption characteristics depending on color, which is typically represented by red, green, and blue intensity values, denoted as {R, G, B}, in an sRGB color space, where (0 ≤ R, G, B ≤ 255). To model OLED power consumption, we used a Monsoon power measurement tool [24] to measure the power consumed by the entire smartphone (Samsung Galaxy Note Edge) according to the following procedure:

1.
Power was measured when the entire display was filled with one of the 29 colors and black. Let P color j be the full-screen power for the j-th color, (j = 1, ..., N color ).

2.
We then derived the power consumption per pixel for the j-th color, P pixel j , by dividing P color j by the total number of pixels in the screen, (j = 1, ..., N color ). The values of P pixel j are described in Table 2. 3.
Let N pixel i (i = 1, ..., N object ) be the number of pixels of the i-th GUI object. Then, the power consumed by the object i for the j-th color, P object i,j , can be calculated as follows: Table 2 tabulates the power consumption per pixel for 29 different colors, from which the following conclusions can be drawn:

1.
As expected, for each basic color, the amount of power is reduced as the color intensity decreases.

2.
Power consumption varies notably depending on the color. For example, white requires 2.6 times higher power than green but consumes only 1.03 times more energy than cyan. 3.
The power consumption of each color can be sorted in an ascending order as follows: black, green, red, blue, yellow, magenta, cyan, and white. As expected, black consumes the least power, whereas white consumes the greatest power. Green and red are relatively power-efficient colors compared to magenta and cyan.

Problem Formulation
Based on the color preferences and power model in the previous section, we here formulate an optimization problem that selects the color of each GUI object to maximize the overall color preference score while limiting the overall smartphone power consumption. Let X i be the selection parameter indicating that the X i -th color is selected for the i-th GUI object, (X i = 1, ..., N color ). Then, the overall color preference score for all the GUI objects can be calculated as The objective of the problem is to limit the smartphone power consumption by P limit . We can then formulate the color selection problem CSP that determines the value X i , (X i = 1, ..., N color ) for GUI object i, (i = 1, ..., N object ), as follows: The CSP is NP-hard because it may correspond to a multiple-choice knapsack problem [25]. In a multiple-choice knapsack problem, there is a limited-weight knapsack; each object has different items, each of which has a weight and a profit. This problem then determines the item of each object to be included into the knapsack to maximize the total profit of the objects in the knapsack, subject to the limited knapsack weight. The CSP has a finite number of GUI objects that can be filled with one of the N color different colors, where each color j for each GUI object i also has Q i,j and P object i,j values. Then, the CSP determines the color of each object to maximize the overall color preference score, subject to the power budget P limit .

Color Selection Algorithm
Here, we present a dynamic-programming algorithm called color selection algorithm (CSA) that provides an optimal solution for the CSP. The CSA basically builds a table that contains a tuple composed of a maximum color preference score and its color index for each GUI object i and power value, m, (i = 1, ..., N object and m = 1, ..., P limit ) (here, the minimum unit of power value is set to 1 mW). Let M i,m be the maximum color preference score that can be achieved for the GUI objects from 1 to i when the power value is m, (i = 1, ..., N object and m = 1, ..., P limit ). It then uses a recurrence relationship between two consecutive GUI objects, i and i + 1, to find the values of M i,m , (i = 1, ..., N object − 1) so as to fill the entire table. Given that M i,m represents the maximum color preference score, M N object ,P limit represents the optimal color preference score achieved over all the GUI objects when the power budget is P limit .
The CSA also records the color index for each object i that achieves the maximum color preference score of M i,m so that the algorithm can find the values of X i after all the columns in the table are filled using recurrence. Let I i,m be the color index for object i that should be selected to produce the value of M i,m when the power value is m. Therefore, the CSA is composed of three phases, namely initialization, recurrence establishment, and color determination:

1.
Initialization: To fill the initial values for the entire table that stores tuples of (M i,m ,I i,m ), the values of M i,m , I i,m are all initialized to 0 and 1, respectively, (i = 1, ..., N object and m = 1, ..., P limit ). Next, for all the values of m, (m = 1, ..., P limit ), the values of M 1,m and I 1,m for the first GUI object are initialized according to the following steps: (a) It first finds the color indices of the first object that satisfies P Color determination: After completing the recurrence process, we can find the value of M N object ,P limit that contains the maximum color preference score achieved over all the GUI objects when the power is limited by P limit . Given that I N object ,P limit contains the color index for the N object th GUI object to achieve the maximum color preference score, the value of X N object is set to I N object ,P limit . Next, the backtracking phase starts to find the color indices of the previous GUI objects, from object N object − 1 to 1, step by step. For example, suppose that the backtracking step chooses I i+1,m as the color index of GUI object i + 1. Then, the color index of the i-th GUI object can be set to I i,m−P object i,I i+1,m because the value of P object i,I i+1,m is allocated for the power of the GUI object i + 1. This backtracking step needs to be run until the color index of the first GUI object can be found.

Algorithm 1 Color Selection Algorithm (CSA)
1: Temporary variables: i, j and m; 2: for i = 1 to N object do 3: for m = 1 to P limit do 4: M i,m ← 0; 5: I i,m ← 1; 6: end for 7: end for 8: for m = 1 to P limit do 9: Find a set S for all the color indices of the first object satisfying P end if 14: end for 15: for i = 2 to N object do 16: for m = 1 to P limit do 17:  Figure 1 shows two display images before and after the CSA is applied when P ratio = 50%. Table 3 shows the color and CIE color coordinate values selected for each GUI object, which indicates that the CSA aims at minimizing color differences.
Some existing mobile applications have multi-color objects. Since our color preference model is mainly developed for single-color GUI objects, it cannot be applied directly to the multi-color objects. One way to extend this model to support multicolored objects is to divide each GUI object into multiple segments of a single color, so that each segment can be an individual object of a single color. Since the pixel-wise power value for each color is already available, the power values of the segments can be easily derived regardless of the shape of each segment.

Experimental Environments
We developed a simple GUI application as shown in Figure 2a. The goal of the application is to find a specific GUI object by moving the icon object. The acceleration sensor data are used to detect this motion. One out of two colors (black or white) was used as a background color, which cannot be then used as the color of a GUI object. Six different GUI objects, each with different sizes, can be configured with up to 29 colors tabulated in Table 2.
To verify the effectiveness of the proposed method, experiments were conducted on a Samsung Galaxy Note-edge smartphone equipped with an OLED display. A Monsoon digital multimeter [24] was used to measure the power consumed by the smartphone, as shown in Figure 2b. In the following subsections, we examine the efficacy of the proposed scheme in terms of the following three aspects: (1) power comparison depending on the color preferences, (2) overall color preference score comparison, and (3) effectiveness of the power capping algorithm, i.e., CSA.
Suppose that a particular user prefers one among seven basic colors (red, green, blue, cyan, magenta, yellow, and black or white) in Table 2. Let P max and P min respectively be the maximum and minimum power that can be consumed by the color selection of each GUI object for each background color. For example, if the background color is white, P max is used when all objects are cyan, whereas P min power is used when all objects are black, because the cyan color consumes the greatest power whereas black consumes the least, as described in Table 2. We then introduce a new variable, P ratio , to effectively represent the power limit value, P limit , as follows: P limit = P min + P ratio P max P min , so that P limit increases with P ratio .

Power Comparison against Color Preferences
We first examined how the values of P ratio and the most preferred color affect total power consumption of an OLED smartphone when the aforementioned simple GUI application is running. The results are shown in Figure 3. The most preferred color is chosen from seven basic colors (red, green, blue, cyan, magenta, yellow, and black or white). Tables 4 and 5 tabulate the color combinations selected for each object for different values of P ratio when white and black were selected as background colors, respectively.  From Figure 3 and Tables 4 and 5, the following observations can be drawn: 1.
The total power consumption of an OLED smartphone increases with the value of P ratio , but it gradually tails off.

2.
A white background consumes more power than a black background.

3.
If black is the most preferred color, black is always selected for the color of each object, so the OLED smartphone consumes the same system-wide power regardless of the value of P ratio . Similarly, if the most preferred color consumes low power per pixel, the preferred color is usually chosen even at low values of P ratio . For example, when the most preferred color is red, green, or blue, these colors or their dark versions are usually chosen regardless of P ratio , as tabulated in Table 4 and 5. 4.
The difference in total power consumption of an OLED smartphone between colors tends to be high at higher values of P ratio . However, as P ratio decreases, this power gap narrows. For example, when P ratio is 90% and the background color is white, if the most preferred color is cyan, 1865 mW is consumed, but if the most preferred color is red, 1514 mW is consumed. By contrast, when P ratio is 10%, if the most preferred color is cyan, 1447 mW is consumed, but if the most preferred color is red, 1442 mW is consumed. The CSA attempts to select colors with as little difference as possible compared to color I prefer . Therefore, if the most preferred color already consumes small power, the actual power consumption is low, even at high values of P ratio . However, if the most preferred color consumes high power, the colors of high power consumption are usually selected to increase the overall color preference score.

5.
Brightness of colors also affects power consumption. When P ratio is low, dark colors with low color intensity tend to be selected as described in Tables 4 and 5. For example, when P ratio = 10%, 19% of colors are set at 25% color intensity whereas 85.7% of colors are set at 75% intensity or below compared to the original basic colors. Figure 4 shows overall color preference scores when one out of seven basic colors was chosen as the most preferred color when the background color is white or black. The following observations can be drawn:

1.
As expected, the overall score increases with P ratio . For example, when the background is white, on average, the score is 3.62 when P ratio = 10% and 4.97 when P ratio = 90%.

2.
The most preferred color considerably affects the scores. For example, when the background is white, if the most preferred color is black, then the score is always 5 because black, which is the color achieving the lowest power consumption, is always chosen. However, if the most preferred color is magenta, the score can be decreased to 2.71, and if the most preferred color is cyan, the score can be decreased to 3.45.

3.
The worst scores for each preferred color range between 2.71 and 5 when the background is white and between 2.89 and 5 when the background is black. The worst score is not considered low (it exceeds 2.71 even when P ratio = 10%) because the CSA algorithm attempts to find the optimal color configuration to meet the power limit. 4.
If the most preferred color is white when the background is black, then a darker version of white (gray) is usually chosen, thereby indicating a score higher than 4 because the Euclidean color difference between white and gray is not high, rendering high scores. This also argues when color preference score degradation is allowed by less than 10%, the proposed method can reduce the total power consumption by 16.1% as described in previous section.

Effectiveness of Power Capping
We performed real power measurements to evaluate how accurately the CSA algorithm predicts the power consumption. Tables 6 and 7 tabulate the percentage difference (minimum, maximum, and average) between the predicted and actual power measured for different values of P ratio . From these tables, we point out the following observations: 1.
If the background color is white, the largest difference between expected and actual power was 2.55% whereas the smallest difference was 0.91%, but for the black background color, they are 4.33% and 0.15%, respectively. 2.
The difference between real and predicted power consumption is not affected by the values of P ratio .

3.
The maximum average difference is 2.15%, which indicates that the power model is accurate. This can be effectively used for power capping of smartphones.

Comparison Results with a Brightness Reduction Scheme
Power can be reduced by lowering the brightness on the OLED smartphone; however, this may result in QoE degradation because personal color preference is not taken into account. To verify this, we consider two GUI object configurations when the black color is chosen for the background with the following (object number, color) pairs: • GUI configuration 1: ((1, Yellow (100%)), (2, Cyan (100%)), (3, Red (100%)), (4, Green (100%)), (5, Blue (100%)), (6, White (100%))) • GUI configuration 2: ((1, Red (100%)), (2, Green (100%)), (3, Blue (100%)), (4, Cyan (100%)), (5, Magenta (100%)), (6, Yellow (100%))) The color intensity determines the brightness level of the OLED display, so the brightness reduction (BR) method uniformly reduces the color intensity to 25%, 50% and 75%. The power and color preference scores were then calculated for each brightness level. Tables 8 and 9 show the color preference scores for the CSA and BR schemes for GUI configurations 1 and 2, respectively when P limit was set to the power value obtained at each brightness level. The BR scheme can also reduce total power consumption by lowering the brightness level, but the overall preference score is significantly reduced compared to CSA. For example, when the same power is consumed, the proposed technique has a color preference score of 23% to 66% higher than the BR scheme because it takes into account personal color preferences.

Effects of Color Difference Model
To calculate the color difference in our color preference model, we used the CIE76 formula, the most widely used model [7,23]. We applied another color difference model based on the CIEDE2000 formula [26,27] to examine the effect of the color difference model on the score. The proposed algorithm, CSA, can be used for any kind of color difference model by modifying the value of D m,j to match the model.
To examine the effect of this perceptual uniformity, the results produced by the CIEDE2000 and CIE76 formulas were compared when the same algorithm, CSA is used. Tables 10 and 11 tabulate the difference values of color preference score and power between them when the background colors are white and black, respectively. There is no color preference score difference for 54.3% of the total case, which shows that the two color distance models mostly give similar results. However, the maximum color preference score difference is 0.7, when the most preferred color is magenta and P ratio is 10%. The color distance model may affect power consumption, but its impact is almost negligible. For example, the maximum percentage difference of indicates only 1.13%, and the selected colors of both models are the same for 63% of the whole case.

Conclusions
We proposed a method that considers the color preferences of the users to limit the power consumption of applications with multiple GUI objects. First, we developed a color preference model to derive a color preference score for each color based on a color difference model in the CIELAB color space and an OLED power model to derive the power consumption of each color based on real power measurements. A color selection problem was formulated to maximize the overall color preference score while limiting the power consumption of the smartphone, and a dynamic programming algorithm was presented to derive an optimal combination of colors for each GUI object.
Experimental results show that the CSA can reduce the total power consumption of an OLED smartphone by up to 27.3% when less than 10% in color preference score degradation is allowed. They also demonstrated that our scheme is accurate in terms of power limitations, achieving a maximum average difference of 2.15% compared to real measurements. The scheme can also maximize the overall color preference score by choosing the colors that achieve the highest score at the lowest power consumption. As our future works, we plan to apply machine learning techniques to derive a color preference model based on measured preference data.