Evaluation of a Mixed Method Approach for Studying User Interaction with Novel Building Control Technology
1.2. Methodologies for Visual Environment Research in Real-World Settings
2.1. Case Study Context and Focus
- How effectively does the EC glazing system control the internal luminous environment?
- What luminous environment conditions are perceived as comfortable or uncomfortable by building occupants?
- What visual comfort actions (VCAs) do occupants take to control/maintain comfort?
- Which contextual parameters affect occupants’ use of the EC glazing system?
2.1.1. Window Arrangements and EC Glazing Controls
2.1.2. Office Layout and Participants
2.2. Case Study Methodology
2.2.1. Data Collection Tools
2.2.2. Data Analysis
3. Results: Data from One Day
3.1. EC Glazing Control Log
- There is an early peak in the illuminance values, at approximately 10:30; the upper glazing zones (1, 3, 5, 7 and 8) were automatically tinted to the intermediate level and then untinted again. There were no manual overrides at that time.
- The external illuminance peaked again, shortly after 11:00; the upper zones were tinted in both rooms to the darkest level and low zones to the intermediate level. At this time, a manual override was used in Room A to additionally tint part of the lower zone to the darkest level (Zone 2). No manual overrides were recorded in Room B.
- The manual override for Zone 2 remained active until the default time of 2 h had expired. All other zones were controlled automatically in response to the varying illuminance values.
- A peak in external illuminance values can be observed in the afternoon that can be linked to a further manual override in Room A (13:43). The manual controls were used to tint some of the lower panes (Zone 2) to the darkest level. No manual override was recorded in Room B at that time, and no further automatic control action occurred in either room.
3.2. Blinds Diary and Facade Photo
3.3. Luminance Images
3.4. Daily Experience Sheet
3.5. Questionnaire Responses
3.6. Interview Data
3.7. Observations from Combined Data
4.1. Critique of Data Sources and Tools
4.1.1. Measured Data
4.1.2. Observed Data
4.1.3. Self-Reported Data
4.2. Recommendations for Methodology Design in Future Studies
4.2.1. Design for Participant Engagement
4.2.2. The Research Team
4.2.3. Choosing and Adapting Data Collection Methods
- The most useful tool was the interview. This facilitated a detailed exploration of occupants’ perception and experience. If used in combination with tools that can log VCAs (e.g., EC glazing log, blinds diary), a valuable and complementary dataset can be derived that captures both actual occupant behaviour, as well as qualitative insight that helps to interpret the observed behaviour.
- To extend the dataset, e.g., to investigate the physical conditions and occupants’ perception of those conditions in more detail, luminance maps and questionnaire data can be useful additional tools. Adapting their design and deployment (i.e., to ensure simultaneous capture) is recommended in order to increase their effectiveness as data collection tools for visual comfort research. Significant effort may be required to achieve automated data collection and processing of the derived dataset.
- A potentially useful intermediate step could be to occasionally collect physical measurements simultaneously with specific ad hoc qualitative data from occupants. This could be targeted at certain seasons, weather conditions, etc., in order to capture the range of conditions that occupants experience, but would not require the automated monitoring and frequent deployment of questionnaires of the advanced approach outlined in Point 2.
- Using an interview approach allows a deeper exploration of the participants’ experiences and motivations than questionnaires; thus, it may be preferable in studies with small sample sizes where statistical analysis is limited. It may be helpful to occasionally use a short questionnaire to capture current or emerging issues to feed into the interview questions in order to ensure that the interview effort achieves its full potential.
- To facilitate investigation of uncomfortable vs. comfortable conditions, physical measurements should be taken as closely as possible in time to VCAs, i.e., to be able to compare “before” and “after” conditions.
- Observational data may be derived from measured data (e.g., control logs) and self-reported data (e.g., blinds diary), which can reduce the need for active participant engagement. However, custom processing procedures may be required to extract relevant data.
- Self-reported data collection tools demand careful design in order to justify the time that can be involved for both participants (to complete) and researchers (to analyse). For questionnaire design, a strict focus on the most important aspects is crucial in order to manage participant burden.
- It is useful to allow participants to enter free-text comments for selected parameters in questionnaires, as this can provide useful qualitative data that can help to interpret the quantitative responses.
- To be able to link self-reported data to physical measurements, ensure simultaneous capture of both data sources, and focus the questionnaire on a very small number of parameters.
- Ad hoc data from opportunistic interaction with the participants can provide additional information during the periods between interviews; this may be one way to reduce participant burden, for example if it is used instead of questionnaires.
- Simple tools, such as a daily experience sheet and a blinds diary, are useful for gathering coarse data. Whilst they may have limitations in terms of completeness and detail, they can provide continuous feedback whilst requiring small time investment from participants.
4.3. Further Work in Regards to EC Glazing Research
Conflicts of Interest
- Galasiu, A.D.; Veitch, J.A. Occupant preferences and satisfaction with the luminous environment and control systems in daylit offices: A literature review. Energy Build. 2006, 38, 728–742. [Google Scholar] [CrossRef]
- Yan, D.; O’Brien, W.; Hong, T.; Feng, X.; Gunay, H.B.; Tahmasebi, F.; Mahdavi, A. Occupant behavior modeling for building performance simulation: Current state and future challenges. Energy Build. 2015, 107, 264–278. [Google Scholar] [CrossRef]
- O’Brien, W.; Gunay, H.B. The contextual factors contributing to occupants’ adaptive comfort behaviors in offices—A review and proposed modeling framework. Build. Environ. 2014, 77, 77–87. [Google Scholar] [CrossRef]
- Hoes, P.; Hensen, J.; Loomans, M.; de Vries, B.; Bourgeois, D. User behavior in whole building simulation. Energy Build. 2009, 41, 295–302. [Google Scholar] [CrossRef]
- O’Brien, W.; Gunay, H. Mitigating office performance uncertainty of occupant use of window blinds and lighting using robust design. Build. Simul. 2015, 8, 621–636. [Google Scholar] [CrossRef]
- Wienold, J.; Christoffersen, J. Evaluation methods and development of a new glare prediction model for daylight environments with the use of CCD cameras. Energy Build. 2006, 38, 743–757. [Google Scholar] [CrossRef]
- Konis, K. Evaluating daylighting effectiveness and occupant visual comfort in a side-lit open-plan office building in San Francisco, California. Build. Environ. 2013, 59, 662–677. [Google Scholar] [CrossRef]
- Van Den Wymelenberg, K.; Inanici, M. A Critical Investigation of Common Lighting Design Metrics for Predicting Human Visual Comfort in Offices with Daylight. Leukos 2014, 10, 145–164. [Google Scholar] [CrossRef]
- Konis, K. Predicting visual comfort in side-lit open-plan core zones: Results of a field study pairing high dynamic range images with subjective responses. Energy Build. 2014, 77, 67–79. [Google Scholar] [CrossRef]
- Jakubiec, J.A.; Reinhart, C.F. The “adaptive zone”—A concept for assessing discomfort glare throughout daylit spaces. Light. Res. Technol. 2012, 44, 149–170. [Google Scholar] [CrossRef]
- Sutter, Y.; Dumortier, D.; Fontoynont, M. The use of shading systems in VDU task offices: A pilot study. Energy Build. 2006, 38, 780–789. [Google Scholar] [CrossRef]
- Heschong, L. Daylight Metrics: PIER Daylighting Plus Research Program. In Final project report for Public Interest Research (PIER) Program; California Energy Commission: Sacramento, CA, USA, 2012. [Google Scholar]
- Mardaljevic, J.; Heschong, L.; Lee, E. Daylight metrics and energy savings. Light. Res. Technol. 2009, 41, 261–283. [Google Scholar] [CrossRef]
- Clear, R.D.; Inkarojrit, V.; Lee, E.S. Subject responses to electrochromic windows. Energy Build. 2006, 38, 758–779. [Google Scholar] [CrossRef]
- Robson, C. Real World Research; John Wiley & Sons, Inc.: Chichester, UK, 2011. [Google Scholar]
- Creswell, J.; Clark, V. Designing and Conducting Mixed Methods Research; SAGE Publications: Los Angeles, CA, USA, 2011. [Google Scholar]
- Krosnick, J.A. Survey research. Annu. Rev. Psychol. 1999, 50, 537–567. [Google Scholar] [CrossRef] [PubMed]
- Painter, B.; Fan, D.; Mardaljevic, J. Evidence-Based Daylight Research: Development of a New Visual Comfort Monitoring Method; Lux Europa: Istanbul, Turkey, 2009; pp. 953–960. [Google Scholar]
- Hirning, M.; Isoardi, G.; Coyne, S.; Hansen, V.G.; Cowling, I. Post occupancy evaluations relating to discomfort glare: A study of green buildings in Brisbane. Build. Environ. 2013, 59, 349–357. [Google Scholar] [CrossRef][Green Version]
- Moosmann, C. Visual Comfort and Daylight in Offices (Visueller Komfort und Tageslicht am Büroarbeitsplatz. Eine Felduntersuchung in Neun Gebäuden). Ph.D. Thesis, Karlsruher Institut für Technologie (KIT), Karlsruhe, Germany, 2014. [Google Scholar]
- Rubin, A.I.; Collins, B.L.; Tibbott, R.L. Window blinds as a potential energy saver—A case study. In NSB Building Science Series 112; National Bureau of Standards: Washington, DC, USA, 1978. [Google Scholar]
- Meerbeek, B.; te Kulve, M.; Gritti, T.; Aarts, M.; van Loenen, E.; Aarts, E. Building automation and perceived control: A field study on motorized exterior blinds in Dutch offices. Build. Environ. 2014, 79, 66–77. [Google Scholar] [CrossRef]
- Correia da Silva, P.; Leal, V.; Andersen, M. Occupants interaction with electric lighting and shading systems in real single-occupied offices: Results from a monitoring campaign. Build. Environ. 2013, 64, 152–168. [Google Scholar] [CrossRef]
- Lee, E.S.; Claybaugh, E.S.; LaFrance, M. End user impacts of automated electrochromic windows in a pilot retrofit application. Energy Build. 2012, 47, 267–284. [Google Scholar] [CrossRef]
- Inoue, T.; Kawase, T.; Ibamoto, T.; Takakusa, S.; Matsuo, Y. The development of an optimal control system for window shading devices based on investigations in office buildings. ASHRAE Trans. 1988, 104, 1034–1049. [Google Scholar]
- Painter, B.; Mardaljevic, J.; Fan, D. Monitoring daylight provision and glare perception in office environments. In Proceedings of the CIB World Congress 2010, Manchester, UK, 11–13 May 2010.
- Fernandes, L.; Lee, E.; Ward, G. Lighting energy savings potential of split-pane electrochromic windows controlled for daylighting with visual comfort. Energy Build. 2013, 61, 8–20. [Google Scholar] [CrossRef]
- Piccolo, A.; Pennisi, A.; Simone, F. Daylighting performance of an electrochromic window in a small scale test-cell. Sol. Energy 2009, 83, 832–844. [Google Scholar] [CrossRef]
- Mardaljevic, J.; Kelly Waskett, R.; Painter, B. Neutral Daylight Illumination with Variable Transmission Glass: Theory and Validation. Light. Res. Technol. 2015. [Google Scholar] [CrossRef][Green Version]
- Kelly, R.; Painter, B.; Mardaljevic, J.; Irvine, K. Capturing the User Experience of Electrochromic Glazing in an Open Plan Office. In Proceedings of the CIE Midterm Conference—Towards a New Century of Light, Paris, France, 12–19 April 2013.
- Brown, N. Image processing for overnight lighting quantification in buildings. In Proceedings of IEECB 2010, Improving Energy Efficiency in Commercial Buildings, Frankfurt, Germany, 13–14 April 2010.
- Aries, M.B.; Veitch, J.A.; Newsham, G.R. Windows, view, and office characteristics predict physical and psychological discomfort. J. Environ. Psychol. 2010, 30, 533–541. [Google Scholar] [CrossRef]
- Xu, J.; Le, K.; Deitermann, A.; Montague, E. How different types of users develop trust in technology: A qualitative analysis of the antecedents of active and passive user trust in a shared technology. Appl. Ergon. 2014, 45, 1495–1503. [Google Scholar] [CrossRef] [PubMed]
- Hoff, K.; Bashir, M. Trust in automation: Integrating empirical evidence on factors that influence trust. Hum. Factors 2015, 57, 407–434. [Google Scholar] [CrossRef] [PubMed]
- Rogers, E.M. Diffusion of Innovations; Simon and Schuster: New York, NY, USA, 2010. [Google Scholar]
- Venkatesh, V.; Morris, M.G.; Davis, G.B.; Davis, F.D. User Acceptance of Information Technology: Toward a Unified View. MIS Q. 2003, 27, 425–478. [Google Scholar]
|Data Type||Data Collection Method||Examples of Real-World Studies|
|Measured||Visual environment measurements, e.g., illuminance, luminance images||[7,11,18,19,20]|
|Observed||Occupant actions and behaviour logging, e.g., blind use, manual overrides of control system, control settings and switch events, system behaviour||[7,11,20,21,22,23,24]|
|Self-reported||Questionnaire, interview, ad hoc feedback||[7,18,19,20,21,23]|
|Type||Method Used in Our Study||Collection Interval|
|Measured||External illuminance||Continuous (1/min; part of control system log)|
|Luminance images||30 min|
|(Desk illuminance)||(Equipment was error prone, thus disregarded)|
|Observed||Blinds diary *||When applicable (can be more than once a day)|
|Facade photo||Opportunistic, approximately twice weekly|
|EC glazing control log (tint settings||Continuous (1/min)|
|and manual overrides)|
|Self-reported||Blinds diary *||When applicable (can be more than once a day)|
|Daily experience sheet||Once per day|
|Questionnaire||Monthly (initially twice weekly until July 2013)|
|Interview||Every 3 months (+ pre- and post-installation)|
|Ad hoc feedback||Opportunistic (varying from weekly to monthly)|
|Question||Participant B1 Response||Participant B2 Response|
|Q3: How would you describe the total light level (from the windows and overhead lighting together) in your office at the moment?||Slightly too bright||Just right|
|Q4: How would you describe the distribution of light in your office at the moment?||Quite nicely distributed||Nicely distributed|
|Q7: At the moment, are you experiencing any reflections on your computer screen? If yes, whatdo you think is the source of the reflections?||Yes—windows||No|
|Q12: Please indicate how tired or alert you currently feel.||Alert||Sleepy, but no difficulty staying awake|
|Q14: How satisfied are you with the clarity of the view through the windows at the moment?||Very satisfied||Very dissatisfied|
|Q15: At this moment, how easy is it to gauge the weather outside by looking through the window?||Quite easy||Very difficult|
|Q20: During the past two weeks, please indicate to what extent your work was hindered by (i)direct sunlight on your desk, (ii)reflectionsonyourscreen.||(i) Quite a lot||(ii) Quite a lot|
|(i) Not at all||(ii) Not at all|
|EC Glazing Control Log||Luminance Maps (HDR)||Daily Experience Sheet||Blinds Diary and Facade Photos||Questionnaire||Interviews and Ad-Hoc Data|
|How effectively does the EC glazing system control the internal luminous environment?||✓||✓**||✓✓||✓||✓||✓✓✓|
|Which luminous environment conditions are perceived as comfortable or uncomfortable?||✓*||✓*||✓*||✓||✓✓|
|Which visual comfort actions (VCAs) do occupants take to control/maintain comfort?||✓✓✓||✓||✓||✓✓✓||✓✓✓|
|Which contextual parameters inform occupants’ use of the EC glazing system?||✓✓||✓✓✓|
© 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons by Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).
Painter, B.; Irvine, K.N.; Kelly Waskett, R.; Mardaljevic, J. Evaluation of a Mixed Method Approach for Studying User Interaction with Novel Building Control Technology. Energies 2016, 9, 215. https://doi.org/10.3390/en9030215
Painter B, Irvine KN, Kelly Waskett R, Mardaljevic J. Evaluation of a Mixed Method Approach for Studying User Interaction with Novel Building Control Technology. Energies. 2016; 9(3):215. https://doi.org/10.3390/en9030215Chicago/Turabian Style
Painter, Birgit, Katherine N. Irvine, Ruth Kelly Waskett, and John Mardaljevic. 2016. "Evaluation of a Mixed Method Approach for Studying User Interaction with Novel Building Control Technology" Energies 9, no. 3: 215. https://doi.org/10.3390/en9030215