In this chapter, we present the interdisciplinary methodology approach, comment on participants, and present the procedure and instruments used for the main study.
In regard to the positivist (or postpositivist) paradigm, we would like to address requirements validation through an interdisciplinary interpretive approach that would provide adequate external validity through a logical-deductive premise to conclude whether this kind of qualitative insight can offer benefits to distinct design approaches towards creating new (digital) services.
As blurring of disciplinary boundaries, in the milieu of scientific and commercial research, is becoming more and more evident, calls for mixed methodology are now common. A (mono)disciplinary approach can be limited to capture challenges of modern (multidimensional and complex) phenomena. Scientific disciplines (e.g., ecology, chemistry, biology, psychology, sociology, economy, philosophy, linguistics, etc.) are categorized into distinct scientific cultures: the natural sciences, the social sciences and the humanities. Interdisciplinary research may involve different disciplines within a single scientific culture [4
Interdisciplinary research design starts with the “conceptual design” which addresses the ‘why’ and ‘what’ of a research project at a conceptual level to ascertain the common goals pivotal to interdisciplinary collaboration [5
]—then moving to a paradigm or perspective and after that to interpretive practices (or methods) that represent the empirical world [6
Diverse definitions and labels try to establish this growing research position: multistrategy, multimethods, mixed methodology or mixed methods [7
]. Use of triangulation in research has been the subject of different considerations. Some explain it as a contribution to a deeper understanding of the study phenomenon, others try to establish an argument that triangulation is one of the validity measures to increase the study accuracy [8
]. In essence, triangulation is the use of at least two methodological approaches, theoretical perspectives, data sources, researchers or analysis methods in studying the same phenomenon in order to increase study credibility.
Throughout literature research, we encountered several types of triangulation: methodological triangulation, investigator triangulation, theoretical triangulation, analysis triangulation and data triangulation [8
]. To narrow the choice of the theoretical basis according to our capabilities, in terms of the method and variety of data used, our estimate is that our study can benefit from focus on the theoretical and investigator triangulation approach.
Theoretical triangulation is defined as the use of multiple theories in the same study for the purpose of strengthening or refuting findings through a variety of examining same-dimensional factors from different theoretical perspectives [8
]. That approach alone implies the involvement of experts from various fields in order to assure a broad and thorough enough overview of the research. Therefore, investigator triangulation will also be employed in this study. Investigator triangulation can be employed in a variety of ways, for example, interviews in a case study research can be conducted by employing more than one interviewer [9
]. Investigator triangulation involves the use of multiple observers, interviewers and data analysts in the same study for better completeness purposes.
Unlike triangulation for confirmation purposes, with classical benefit for validation in the form of quantitative results by qualitative studies, the use of triangulation for completeness seems a better fit for interpreting findings of this study as it is mainly used in researching less explored or complex research problems [7
]. One of the advantages of the qualitative research paradigm is generating a rich amount of data that further the range of concept investigation in order to help researchers developing hypotheses for future quantitative studies. After a large amount of data has been generated by a qualitative research method, a researcher has to employ quantitative research methods in the form of data collection methods and analysis to get a deeper and more comprehensive picture of the phenomenon under investigation [7
In terms of the requirements of the engineering process, this would be a stage of requirements validation—checking that the documented requirements and models are consistent and meet the needs of the stakeholder.
Our conceptual design entails focus on important life events, goals and constraints in the developmental stages of an individual who functions in a network of long-lasting meaningful relationships that have extensive impact on their behaviour patterns in order to extract some valuable information regarding the conception of functional features used in meaningful digital services use case scenarios.
In regard to the technical design of the research, input data will not be operationalised in quantitative measures; rather, we will use a “soft”, qualitative approach. Semistructured interviews on the topic of users’ experience with an advanced web application that tries to address particular needs of a modern family will be held after a certain testing time frame. After collecting and recording interviews with participants in accordance with the designated pluralistic approach, a team of experts from different fields will separately review the content and make remarks that can be interpreted with their distinct expertise, although with an overall focus on important life events, goals and traits of the participants.
To analyse these interviews, we employ thematic analysis—a structured approach to discover themes and concepts embedded throughout interviews. Thematic analysis is a method for identifying, analysing, and reporting patterns (themes) within data [10
]. The thematic analytic method and procedure is thoroughly described by Braun & Clark [10
], and their article serves us as a manual on how to address the interview analysis in a flexible yet structured manner.
Our vision is that knowledge from the fields: developmental psychology, anthropology, sociology, user experience design and computer application programming will provide us with enough coverage to detect possible valuable themes regarding features (users’ needs) that were not (properly) addressed in previous iterative stages of the EkoSmart prototype development.
A more detailed process of our inquiry is described in adjacent chapters “Participants”, ”Instruments” and “Procedure”.
Initially, 8 families with more than 30 individuals, family members, responded to our call for testing the MyFamily interactive prototype for a period of approximately two weeks. Five families with 20 participants were genuinely active during the testing period, and they took part in the initial and subsequent user experience surveys and semistructured interviews that we included in our data sets.
Four of the active families were from urban or suburban areas, and one family was from a predominantly rural environment. One member did not have any in-person contact with their family during the testing period.
Four members of families were children (up to 18 years old), 12 were adults (18 to 65 years old), and 4 were seniors (more than 65 years old). Ten men and ten women were included in testing and interviewing. All adults were employed or self-employed. All seniors were in retirement. The mean age of the participants was 40.65 years.
Semistructured interviews were conducted after a two-week period using the MyFamily progressive web application within the family. Five interviews with families were recorded between August and October 2018. Interviewers asked the participants a series of open-ended questions regarding the ease and frequency of use, technical and usability problems they may have encountered, their behaviour regarding the use of ICT in family communication and organisation as well as visual and overall experience of the app. Interviews lasted from 35 min to 1 h and 15 min. Interviews with families were transcribed verbatim. Participants were also asked to keep a diary of subjective remarks regarding usage during the testing period. Those diaries were analysed in combination with user interviews.
In addition, standard usability testing forms were distributed among participants to acquire an insight into the perceived usability and user experience of the prototype before and after the usage. These testing forms included: a User Experience Questionnaire (UEQ) to measure 6 dimensions of user experience with the product, Standard Usability Scale (SUS) for subjective assessments of usability, and the NASA-TLX questionnaire for cognitive workload measurements. These questionnaires were filled in before and after actual prototype testing in order to validate and assess previous efforts in the iterative design approach of an interactive prototype.
Our main goal was to improve and broaden the understanding of family needs and attitudes towards the specific usage of certain, also nonimplemented, features. This would be conducted outside of the user experience and prototype usability testing scope with an analysis of themes that were most common among different families. The thematic analysis procedure [10
] was chosen as the analytic method. In our analysis, we followed the stages of thematic analysis as described by Smahelova et al. [11
Familiarising yourself with your data and transcription of verbal data.
Generating initial codes and grouping them into topics.
Searching for themes.
Defining and naming themes.
Producing a report.
The corpus of our data comprised all interview transcriptions and user diary entries. Sets that were used consisted of clear and meaningful answers from participants. We generated these sets with the triangulation method approach—every researcher created their own set via listening and reading transcripts with making remarks for relevant topics (Figure A4
). Designated codes created upon initial readings of sets and discovering related topics were grouped into themes. Themes that would emerge from these codes were then grouped into three main categories for additional clarity: confirmation of functionalities, problems and criticism of functionalities, and suggestions for improvement. That enabled us to extract meaningful interpretations of the results regarding the prototype user experience, user needs, behaviour patterns and expectations towards this kind of digital services and products.