A Framework to Optimize Energy Efficiency in Data Centers Based on Certified KPIs

Abstract

Both electricity consumption and CO₂ emission from Germany’s 50,000 data centers have grown continuously in recent years. However, projects that can be used to evaluate the energy efficiency of data centers are no longer available to companies. Thus, there is a need for solid solutions that are going to help data center operators to assess and improve a data center’s energy efficiency. A particular approach in the form of an evolutionary prototype, which is described in this article, is designed to serve as a basis for the development of the software, which will support companies in addressing their personalized energy efficiency needs.

1. Introduction

The energy requirements of German data centers have grown steadily in recent years [1]. Due to the development towards centralized infrastructures (e.g., “cloud computing”) and progressive digitalization initiatives, both private households and companies are generating more and more data, which should be accessed remotely (e.g., via the Internet) and should be stored centrally. Whereas in 2006 the consumption rate was around 8.7 billion kWh per year [2], by 2016 the 50,000 data centers in Germany already required a total of 12.4 billion kWh, which corresponds to the annual production volume of four medium-sized coal-fired power plants, or about two percent of the annual electricity demand in overall Germany [1]. According to the Federal Environment Agency, energy savings of up to 50 percent [3] could be achieved by increasing efficiency in data centers. Whether energy costs, in turn, account for half of the total costs of a data center operation, which offers great potential for savings [4]. Based on the initial situation described above, the aim of the Germany-wide research project called TEMPRO (“Total Energy Management for Professional Data Centers”) is to develop solutions together with companies and research institutes to reduce an energy consumption of data centers. In the TEMPRO project, a complete life cycle of all equipment and systems of a data center is considered. It includes the energy consumption during the extraction of raw materials, as well as transportation, production and recycling of the systems involved. This means that, in addition to consumption energy, so called “grey energy” is also taken into account [5]. In order to reduce the total energy consumption of all data centers in Germany, a holistic view of data centers is necessary. This perspective makes it possible to identify relevant indicators for increasing efficiency. Since the respective efficiencies must be determined by companies themselves, each individual data center bears a large responsibility. Since heterogeneous system landscapes and infrastructures are often used in data centers (e.g., servers, network components, UPS (Uninterruptible Power Supply) systems, air conditioning systems), an overview of consumers and optimization possibilities in companies could be very diverse. Often companies have only marginal contact points with the energy efficiency of devices, e.g., when purchasing a new server. Certifications, such as “Der Blaue Engel” (http://www.blauer-engel.de/), are intended to raise awareness for a holistic view and provide generic conditions and measurements support to increase efficiency. However, “Der Blaue Engel” certificate represents a major challenge for companies. This can be seen from the fact that only four certified data centers currently exist in Germany as of the date of preparation of this work [6]. With its introduction in 1978, “Der Blaue Engel” is considered to be the oldest environmental label and is also a pioneer in the development of eco-labelling systems. It was originally developed by the Federal Ministry for the Environment, Nature Conservation and Nuclear Safety (http://www.bmub.bund.de/) of Germany. Usually, defined standards are intended to give manufacturers an orientation to the respective Life Cycle Assessments of their products and services. In order to be awarded the “Der Blaue Engel” certificate, a product or a service should fulfill a number of criteria. In general, products and services organized in a way to protect the environment and the health of users. Among other things, resources should be saved during a production phase, as well as sustainable resources should be preferred for production, and a product at the end of a cycle should say durable and be repairable [7]. The certificate has been developed for the operation of data centers. The aim of this certification is the energy-efficient and resource-saving operation of data centers. All requirements must be documented in an annual energy efficiency report and submitted for the review. The costs for a certification consist out of the one-off costs for the application and the annual fees, which, depending on the annual turnover of the company, are divided into ten different fee classes. Annual costs of 320 euros for a turnover of 250,000 euros to 10,500 euros for a turnover of over 40 million euros must be planned [8]. In the diversity of key figures and metrics it is relevant for companies to know, which exact metrics have to be recorded and to what extent statements can be derived from recorded data. At this point there is a demand for solutions and consulting assistance in a form of applications, which could support companies. The research project “TEMPRO” described above is divided into five different work packages, while this contribution is included in the third package. In particular, the current work conducted under the “Information and evaluation models for energy efficiency in data centers” and is coordinated by the Very Large Business Applications Department (VLBA) at the University of Oldenburg [9]. This work shows a possible approach to the aforementioned problems. The approach includes a creation of an evolutionary prototype that presents measured values from the data centers of the companies and compares them with the relevant target values of the efficiency indicators by using defined key figures. The functional and non-functional requirements for the prototype were determined from literature research and expert interviews. The research question addressed in this paper is: “To what extent can an interactive system support the calculation, evaluation and presentation of the energy efficiency of data centers and be conducive to the efficiency development of data centers?”.

2. Energy Efficiency and KPIs (Key Performance Indicators)

The energy efficiency of a data center is difficult to summarize or present in the form of a key figure. In the literature, there are only approaches that can evaluate sub-areas of a data center [10]. In the past, many different KPIs for different areas were developed by the non-profit organization “The Green Grid” [11] and proposed to ISO for the creation of standards. One of the key figures is PUE (Power Usage Effectiveness). It is defined in the second part of the ISO standard and describes the ratio of the total power consumption of the data center to the power consumption of the IT equipment. The optimum value is 1.0, which means that 100% of the energy consumption is used by computers [12]. An excellent value is 1.3, Google gives an average value of 1.12 for its data centers [13]. A holistic view of the data center is made more difficult by the use of PUE. as the evaluation basis for efficiency, since only energy consumption is used to evaluate efficiency, but not the utilization of the systems in the data center. To illustrate the efficiency of individual sub-systems in relation to the energy consumption of the IT equipment, partial PUE is available for the sub-systems “power supply”, “air conditioning” and “cooling systems”. This is determined by adding the respective power consumption of the sub-system to the total power consumption of the IT equipment and dividing it by the value of the IT equipment. This makes it possible to determine the proportion of the sub-systems within the PUE key figure. The REF (Renewable Energy Factor) is another indicator of sustainability that can be used to evaluate the data center’s energy consumption. It describes the ratio of used renewable energies to the total power consumption of the data center. The ideal value is 1.0. The key figure CUE (Carbon Usage Effectiveness) supplements the metric of PUE by CO₂ emission and thus describes the sustainability of data center operations. It is calculated by multiplying the emission factor of the electricity mix, which is published annually by the Federal Environment Agency or by a local energy supplier, and the PUE value. In order to be able to include the continued use of energy resources used in the data center (e.g., through the further use of waste heat from the data center) in the overall efficiency, ERE (Energy Reuse Effectiveness) is used to offset the recovered energy by subtracting it from the total energy consumption of the data center into the PUE [12]. More and more data centers also use water for cooling, either as direct water cooling of the computer systems or when using adiabatic cooling (in which heat exchangers are sprayed with water to generate cooling). WUE (Water Usage Effectiveness) was developed to calculate the efficiency of the use of water resources. It is calculated by dividing the amount of annually water used by the total electricity consumption of the IT equipment. It indicates how many liters of water were consumed per kilowatt hour of IT equipment. The ideal value is 0 [14]. Due to the fact that metrics do not have natural thresholds, certifications define standards for evaluating the efficiency of data centers. One of the best-known certifications is the environmental certificate “Der Blaue Engel—Energieeffizienter Rechenzentrenbetrieb”. In addition to compliance with limit values (CO₂ emissions, EUE (PUE with annual values)) and the achievement of minimum values (annual performance factor of the cooling system, efficiency of the power supply), these are the criteria for an energy-efficient data center in accordance with the certificate. In addition, energy efficiency reports must be prepared and submitted in regular cycles [15]. The fact that such certification goes hand in hand with great expenditure and represents a great challenge for a company is shown by the fact that there are currently only four certified with the “Der Blaue Engel—Energieeffizienter Rechenzentrenbetrieb” certificate data centers [6].

3. Implementation

Based on the motivation behind key figures and the certification “Der Blaue Engel” described above, the idea to produce a software artifact that enables companies to evaluate and improve their energy efficiency in data centers was developed. It should be implemented in form of an evolutionary prototype. Currently in Germany, there are methods that use similar indicators to determine or evaluate the energy efficiency of data centers. The well-known examples include the GreenIT RZ Benchmark and the ServerraumCheck. The GreenIT RZ Benchmark is a tool that provides 30 additional indicators for evaluation of a data center’s energy efficiency by justifying data with a central value of the PUE [16]. The ServerraumCheck project offers consulting services for data centers and server rooms with a maximum of 50 servers in order to demonstrate savings potentials and suitable measures for more optimal utilization of own potentials [17]. The current work is reinforced by the fact that there are currently no freely available software solutions for companies to evaluate the energy efficiency of data centers. For the implementation of the prototype, an iteration of the “Human-Centered Design Process” was used, which not only provides an analysis of an utilization context, a development of a draft and an implementation of the draft, but also an evaluation of created prototype [18]. The context of use consisted of users in the form of data center operators with expertise in the operation of IT systems, operating facilities of the infrastructure as well as measurement procedures and facilities. From the conducted expert interviews, it could be determined that many measured values, which are usually used for the evaluation of various key figures, have already been collected and stored in the company (mostly for monetary aspects of cost distribution). These existing measured values mainly relate to the energy consumption of various infrastructure components and the computer systems (server and storage systems) of a data center. A suitable evaluation horizon for the efficiency of a data center could be created for the prototype by combining the key figures mentioned in the literature and the criteria of the environmental certificate “Der Blaue Engel”. In order to be able to record the correct input, which is essential for proper evaluation, an input mask has been used. This should include all measured values and information necessary for calculating various key figures and for evaluating the certification criteria. The prototype should then calculate key figures from the data entered and return the evaluation to the user in a visual form.

3.1. Architecture

A classic client-server architecture was chosen for the prototype. Of particular importance was addressing the interchangeability of the components in order to be able to implement later the end product, which could include other components as well. The graphical user interface has been implemented in form of a web application to ensure that the prototype can be used by users regardless of device and location. Another decisive factor in the decision was that changes (such as software updates that add further functions) could be incorporated centrally in one place and made directly available to all users. The general architecture can be visualized by the following diagram, where inputs and outputs are bidirectional from sub-system to sub-system, which is shown in Figure 1:

3.2. Frontend

The mockup tool “Balsamiq” (https://balsamiq.com/) was used to create the design of the prototype’s graphical interface. This enables the creation of low fidelity prototypes by simple “drag & drop” of prefabricated elements. The aim was to create a sequential prototype showing as many elements of the prototype as possible, but without functionality and further details. The design of the input mask looks as follows in Figure 2:

At the time of this article, standard web technologies are HTML (version 5), CSS (version 3) and JavaScript. They were selected to implement the prototype. The CSS framework “Materialize” (https://materializecss.com/) was used for a simplified implementation of designed elements. This offers a multitude of ready-made components for the interface design and function of a modern graphical user interface. A major advantage of “Materialize” is that developed by Google “Google Material Design” includes the large number of design rules for interactive and intuitive user interfaces and automatically inherits provided within the standard elements. Materialize also makes the graphical user interface automatically “responsive” so that devices with different screen sizes and resolutions can be used without any source code adjustments. The chart framework “ChartJS” was used to display the calculated and entered information. This is published under the “MIT” license [19] and thus allows free use for the prototype. It offers a wide variety of different diagrams that can be filled with data via an API and dynamically generated. At the same time, this framework supports HTML5, which embeds the technology in the basic web technology used and makes it directly usable. For the evaluation of the calculated efficiencies symbols (UTF-8) were used, which are set via JavaScript and CSS depending on the evaluation. A distinction is made between the evaluation of key figures and the review of the certification criteria. If the criteria are positive, the user is shown a green check mark, in the negative case a red cross. Efficiency is evaluated using the key figures using a three-step scale, which was implemented using “emoticons”. Overall, the front end of the prototype looks like Figure 3 and Figure 4:

The evaluation of the key figures takes place using the provided table. The table shows the respective key figures with a reference text, the pertinent optimal value, the value computed by the server, the respective average value of all data sets in the database and the pertinent efficiency evaluation in the form of an “emoticon”, which is shown in Figure 5.

The evaluation of the individual criteria of “Der Blaue Engel” certification is also presented as an analogy to the estimated key figure evaluation value. Furthermore, the individual non-binary criteria above the table are displayed in form of a network diagram, where targeted values and the actual values are compared respectively. The individual key figures within the table can be explored and contain more detailed information about the evaluation. It shows which exactly measured values were used to calculate the respective key figure and which target values are necessary (in a case of “Der Blaue Engel” certification) and were achieved. In addition, when you call up a subpage, two diagrams are generated via JavaScript. You can switch between both diagrams using “radio buttons”. One diagram represents the optimal value, the other visualizes the differences with the average value of all companies. The diagrams are filled out dynamically based on a data in the JSON responses returned from the server. If data is transferred to the server, a dynamic loading bar is displayed. Buttons can be used to navigate between the individual input steps. A user can also switch between the evaluation screens and return to the input view. It is also possible to resend modified data and thus repeat the efficiency evaluation process.

3.3. Backend

A relational database (MariaDB) was chosen to persistently store the measured values entered via the graphical user interface and calculated values. A persisting data inside the database allows to process inquiries more efficiently as well as trace back all applied changes. MariaDB (https://mariadb.org/) is an open source fork of the well-known database “MySQL” and can be freely used. The database contains tables for the data of the individual company, the data of the data center and the measured values of the respective infrastructure (energy consumption, water consumption, data on air conditioning and UPS). These are linked together via foreign keys to the IDs. To transfer the data and measured values entered in the user interface, the standard format JSON was selected and a communication interface was created between the client and the server in combination with the web service framework “Spring Boot” (http://spring.io/projects/spring-boot). The programming language in the backend is Java. If the input mask filled out with data is sent, a “POST request” is initiate towards the web service, which saves the individual parameters on the server side and makes them available for further processing. Then, all parameters are persisted inside the relevant database tables and linked to the data entered for the organization or data center. Next, all key figures that can be used to evaluate energy efficiency on the basis of the data entered are calculated and saved again in a linked table. On the basis of these data, average values of the achieved efficiencies are calculated and used to evaluate the company values. Due to the fact that metrics do not have natural thresholds for evaluating efficiency, it makes it difficult to evaluate absolute efficiency (which values are good and which values are bad), the average of all records in the database was used in conjunction with the standard deviation of all records to overcome this problem. It creates an area of a normally distributed data, which allows assessment of an efficiency measurement. Once all necessary calculations have been performed, several JSON objects are created. Created objects contain all elements and information that the graphical user interface should display. In the current implementation of the prototype, the graphical user interface does not contain any “business logic” and only the server is able to perform relevant calculations and return estimated values to the graphical user interface via the web service. Calls to the web client and the return of the calculated information are processed asynchronously.

4. Evaluation

The evaluation was carried out through an expert interview with a partner of the research project “TEMPRO”. The aim was to determine, which exact requirements could be met and whether the prototype would be of beneficial for a company. As a result, it was possible to determine that all functional requirements were met. In addition, it was found that companies are very interested in such type of software artifact. With the prototype created, key figures can be calculated and graphically displayed on the basis of provided measured values. The surveyed company sees a great advantage in the possibility of conducting benchmarks with respect to other companies on the basis of the averaged values and the possibility to test certification requirements. Changing efficiency requirements of certifications can be managed centrally and updated for all companies at once. This facilitates access to certification requirements without external consulting entities and supports the actual certification preparations at best.

5. Conclusions and Outlook

The implemented prototype was designed to be flexible in a way that some new key figures and certifications for efficiency assessment could be implemented as well. This can make the efficiency assessment of a data center more detailed and holistic. The widely adapted process known as “Human-Centered Design Process” was used to implement the demonstrated within the work prototype. In addition, expert interviews were conducted to compare impressions from the carried-out literature research with the practical experience of companies and to establish requirements for the prototype. The prototype can persistently record measured values and company data and perform calculations to evaluate the efficiency of a data center. A graphical user interface in the form of a web application is used as the representation layer. By using the standard deviation to calculate an average range in relation to individual metrics values, natural data center efficiency thresholds that dynamically adjust to the data inventory are created. These could be used as a basis for a meaningful valuation between companies. The research question defined at the beginning could be answered by the creation of the prototype. A possible extension of the prototype is a user administration, where companies can register and access the platform, which is represented by the prototype. This would allow access to historical data, as the user can be identified and assigned to data already entered and calculated information. Measures taken to improve the efficiency of data center operations can be evaluated using progress data or the efficiency potential of measures to be implemented can be checked before they are implemented. Trends in the individual efficiency indicators can also be identified and evaluated. In addition, in the event of further developments or maintenance in the future, if there are new guidelines for the certificate, these also could be taken into consideration and adjusted accordingly. It would also be possible to carry out simulations of future efficiency developments using collected data. Companies can compare themselves with other companies (even anonymously) and draw conclusions about potential investment gaps in their data center infrastructure. An incentive system for effective advertising awards for particularly efficient data centers is also conceivable. The knowledge and results gained through this contribution flow into the further development of “TEMPRO” project and actively support associated research activities.