The quality of indoor environments has significant effects on the health and productivity of occupants [1
], and the desire of occupants for a pleasant indoor environment is continuously increasing. Thus, improvement of indoor environment quality through management is gaining importance. The indoor environment quality is determined by factors such as thermal comfort, indoor air quality, acoustic quality, and light quality. Thermal comfort felt by the occupants considers factors such as the temperature, relative humidity, air velocity, and mean radiant temperature.
The predicted mean vote (PMV) proposed by Fanger [3
] represents the human sensations of thermal comfort in an integrated way including both environmental and personal factors; it is one of the most widely known indices. The PMV sets the thermal neutrality to zero and presents the thermal comfort of occupants quantitatively on a seven-stage numerical scale ranging from −3 (cold) to 3 (hot). The satisfaction range of thermal comfort based on the American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) standard 55 [4
] is −0.5 < PMV < 0.5. The PMV considers six main factors that include both environmental and personal factors. The environmental factors are the air temperature, relative humidity, air velocity, and mean radiant temperature, and the personal factors are the metabolic rate and clothing insulation. Here, 1 met = 58 W/m2
and 1 clo = 0.155 m2
The metabolic rate is the rate of heat generation from the body surface of an individual due to activities maintaining heat balance. Figure 1
presents the effect of metabolic rate on the PMV calculation by comparing the difference of PMV according to activity. The environmental factors are assumed in the general range to satisfy thermal comfort as an air temperature of 25 °C, a relative humidity of 30%, and mean radiant temperature of 25. In ISO 7730 of the International Organization for Standardization and a number of studies [5
], the relative air velocity was taken into account for accurate PMV calculations. Therefore, the value of air velocity which is assumed as 0.1 m/s is adjusted to relative air velocity by referring to the equation of d’Ambrosio Alfano [6
According to ISO 9920 [8
] and Havenith et al. [9
], clothing insulation is affected by the relative air velocity such as an occupant’s movement or wind speed. However, the purpose of Figure 1
is only intended to identify the impact of the metabolic rate on the PMV calculation. To clarify the effect of metabolic rate on PMV calculations, fixed values of clothing insulation were applied. The clothing insulations are 0.5 clo and 1.0 clo, respectively, which assumes two different conditions of the occupant clothing situations.
In accordance with the ASHRAE 55 standard, four types of general activities were selected: “Sleeping” (0.7 met), “Seated.quiet” (1.0 met), “Standing.relaxed” (1.2 met), and “Walking about” (1.7 met). As the metabolic rate changed from 0.7 to 1.7 in 0.5 clo and 1.0 clo, the maximum PMV difference was as large as 3.64 and 2.82, respectively. In addition, although the indoor environment factors such as temperature and humidity satisfied the thermal comfort conditions, the PMV may have been outside the comfort range (−0.5 < PMV < 0.5) according to the metabolic rate. This indicates that the metabolic rate has a significant influence on the thermal comfort in an actual indoor environment and that an accurate method for measuring the metabolic rate is necessary.
Metabolic rate is measured based on various methods such as activity classification, heart rate, acceleration, and so forth. A number of studies have been conducted to measure human metabolic rates using an accelerometer [10
]. Particularly, Kozey et al. [11
] measured physical activities of 277 participants by an accelerometer according to various physical and gender conditions. However, metabolic rate calculation using an accelerometer is only possible when the occupant is moving; there are limitations in distinguishing sedentary activities such as writing and typing.
International standard ISO 8996 [13
] classifies the measurement methods for the metabolic rate into four levels: screening (level 1), observation (level 2), analysis (level 3), and expertise (level 4). The accuracy of each level was presented at at 20% for level 2, at 10% for level 3, and at 20% for level 4. Higher levels such as analysis (level 3) and expertise (level 4) correspond to more accurate methods for measuring the metabolic rate. They use information directly measured from the body such as heart rate and oxygen consumption.
Studies have been conducted to develop heart-rate-based methods (level 3) for calculating the metabolic rate [14
]. Hasan et al. [14
] measured heart rate by using a wearable device and compared the measurements with the constant metabolic rate value. In the best case, 80% accuracy was achieved. In a similar study, Lee et al. [15
] compared the calculated metabolic rate based on the heart rate from a sensor that was obtained via a location-based method. Calvaresi et al. [10
] calculated the metabolic rate by wearing a chest strap with a multi-parametric device that measures heart rate, breathing rate, vector magnitude units, and acceleration. In spite of the accuracy of the method using the heart rate, individual variations occurred as the intensity of the activity increased [16
] and the accuracy of the calculated metabolic rate depended on emotional conditions and stress levels [17
For more precise metabolic rate calculation, studies on the expertise (level 4) method were conducted using oxygen consumption and doubly labeled water. Usually, the occupant respiration was measured using a wearable device such as a face mask covering the mouth and nose to collect exhaled gas [18
]. Ji et al. [19
] measured the metabolic rate according to the CO2
concentration changes of the subject in an airtight chamber and compared the accuracy with that of the heart-rate-based method (level 3). It was confirmed that the CO2
production method provides easier and more accurate measurements of the metabolic rate than those provided by the heart-rate-based method.
To acquire occupant information for levels 3 and 4, it is essential to attach additional instruments to the human body that can be sometimes inconvenient and cumbersome. For example, devices such as a smart wrist ring (level 3) and a mask (level 4) must always be attached to the body for collecting data. Data for calculating the metabolic rate can be missing owing to improper device usage such as desorption. In particular, level 4, which is the most accurate method, is more applicable to experimental settings than to actual building environments.
For applicability to actual buildings, there is a need for the estimation of the metabolic rate of occupants without complex systems or devices. Observation (level 2) is a simpler method that employs the occupant activity for determining the metabolic rate [20
]. Typically, a specific activity type in a space (e.g., “seated with typing” in the office) is determined by observation; then, the tabulated value for the activity is determined as the metabolic rate of the occupant. Despite its simplicity, the observation method has not been successfully applied for metabolic rate. Because the observation is judged by experts, it is difficult to be accomplished automatically without experts. Therefore, the activity in a space is assumed as a specific behavior and the metabolic rate is employed as a fixed value corresponding to the assumed activity. However, if there is an accurate and simple device that can recognize the actual activity of the occupants, the possibility of simply applying the observation method can be secured.
Recently, intelligent systems recognizing human faces or gestures by training images has been developing [21
], and the performance of classifiers adopting machine learning techniques such as deep learning has been improved [22
]. With an intelligent system that can automatically classify the actual occupant activity in real time, the observation method can be a practical and convenient solution. The intelligent system would provide the actual occupant activity, and then, the actual activity would be compared with the tabulated values for determining the metabolic rate value.
The objective of the present study is to develop a method that can automatically estimate the actual activity of the occupant for application to the calculation of metabolic rate. For this, a pose-categorization model and an activity-decision algorithm are developed. The pose-categorization model employs a deep neural network (DNN) for classifying the occupant poses using indoor images, and an activity-decision algorithm is designed to determine the representative activity based on the categorized real-time poses. This intelligent and automated method for estimating occupant activity provides a simple and practical solution that can be a basis for determining the metabolic rate of the occupant.
An intelligent system that can automatically classify the activity of an occupant in real time was developed. The system includes a pose-categorization model for classifying occupant poses and an activity-decision algorithm for determining the representative activity based on the categorized real-time poses. In contrast to conventionally applied methods with complicated systems or devices, this is a simple and convenient system that can be applied to an actual building. The conclusions of this study are as follows:
The pose-categorization model was trained with indoor images of home and office environments. The optimized structure of the DNN comprised one input layer, four hidden layers, and one output layer. The trained pose-categorization model had 100% accuracy for the training and valid datasets.
A real-time dataset consisting of 720 images for each activity was used for testing the pose-categorization model. For home and office activities, respectively, the pose-categorization model exhibited classification accuracies that were 98.9% and 88.2% and average F1 scores that were 0.99 and 0.89. The average AUC of the ROC curve was close to 1 for both environments.
The activity-decision algorithm is designed to determine the representative activity for 1 min. An accuracy of representative activity decision was compared using frequency and average methods based on the real-time poses output from the pose-categorization model. As a result, the frequency method decided the representative activity more accurately than the average method by 4.58% for home and 7.22% for office, determined to be applied to the activity-decision algorithm.
Thus, the developed model and algorithm confirmed that it is possible to identify the activities of occupants using indoor images. In addition, the development of the activity-decision algorithm is expected to decide more accurate metabolic rates considering the actual activity of indoor occupant’s compared with a constant metabolic rate by assuming a specific activity.
The developed intelligent model does not require the direct intervention of the occupants, and the experiments confirmed that real-time metabolic rate can be measured automatically when the model was applied to the actual building. In the actual environment, it is possible to develop an adaptive model that is customized to the occupant by continuously training the data of occupants. In addition, if a model for measuring clothing insulation is also developed in a further study, it can be applied to calculate an accurate PMV with the model and algorithm developed in this study.
To increase the accuracy of the activity estimation, the accuracy of the pose-categorization model must be enhanced. Although the pose-categorization model and the activity-decision algorithm cover some of the major indoor activities that occur in home and office environments, it is necessary to expand the scope to various activities and multiple occupants. Additionally, there were errors due to the absence of object detection in the images (e.g., “Typing” was classified as “Seated.quiet”). Therefore, to expand the range of activities and to classify various indoor poses, an indoor-object-detection model must be developed. In the future, to determine individual metabolic rates according to the activity, models for estimating other personal factors such as gender, age, sex, and body mass index must be developed. Furthermore, studies must be conducted to compare with the performances of the heart-rate-based and oxygen-consumption-based methods.