Proper requirements elicitation in a software project is considered one of the most important and difficult tasks during the software process. Shortcomings in the treatment of requirements has been identified as the main cause of failure of software projects [1
]. The negative effects of a misconducted software requirements elicitation are well known: project delays, cancellations and deliveries of incomplete work due to the bureaucratization of the development of software established in the mid-2000s. Aware of this, a group of influencers of the Extreme Programming community gathered and discussed these problems. The agile methodology has been popular since 2001 due to the valorization of teams members and the intense interaction among them, constant software deliveries, more effective stakeholder collaboration and an openness to receive changes from the users [3
The increasing stakeholders involvement in the project has helped obtain the requirements that satisfy the project needs. However, in practice, the relationship among stakeholders is more or less unrealistic [4
]. A systematic review of 21 studies showed that software requirements engineering practices helped to solve some challenges in the traditional requirements engineering. Nevertheless, it also introduced a number of constraints that showed limitations in the achievement of the appropriate balance between agility and the stability pursued by agile methodologies [5
An empirical study based on the analysis of the data collected in 16 software development organizations in the U.S.A revealed seven challenges in the agile methodologies: (1) problems with cost estimation and schedule; (2) inadequate or inappropriate architecture; (3) neglecting non-functional requirements; (4) stakeholder access and participation; (5) prioritization in a single dimension, compromising scalability; (6) inadequate verification of requirements; and (7) minimal documentation [4
]. Among these seven challenges, some are directly related to the lack of stakeholder involvement in the software requirements elicitation process, namely Items 1, 3, and 4.
Out of these seven challenges, three are directly related to the lack of stakeholder involvement in the software requirements elicitation process, namely Items 1, 3, and 4. This problem confirms the idea defended by Pressman [6
]: he explained the importance of customer involvement in the specification, towards the success of any project. Linked to this and taking into account the current high competitiveness, the goal of the software industry is to increase productivity and quality of the end-products for its customers, thus more and more software has been experimented to develop different techniques and to elicit requirements in order to bypass this bottleneck.
Design Thinking (DT) emerges as a methodology that provides design-inspired practices for resolving and developing projects, using empathy, encouragement of creativity and rationality to meet the needs of users and converge into innovative solutions [7
]. The mission of Design Thinking is to translate observations into insights and to generate products and services with the objective of improving people’s life quality. It is based on how the user would like to be treated: with empathy, encouragement of collaboration and participation in the application of generated results. DT-related literature considers that when initiating a project the most important task is to understand and to question what people’s needs are and to understand what impacts the product may cause. Software project teams involved delves into the user’s universe in order to extract all the information [7
]. Thus, DT methodology seems to be a mechanism that can help requirements elicitation in agile methodology-based software projects [8
Adikari et al. [9
] defined Design Thinking as: (1) an approach to creating new or better solutions based on customer needs that adds value to business; and (2) a design approach. The Design Thinking process is basically divided into three activities: (i) identification of the real problem (immersion) that an organization/customer faces; (ii) ideation run-through, which is the moment when ideas and concepts are generated; and (iii) prototyping, in order to generate innovations about problems identified in the initial phase [7
Due to the importance of the requirement’s elicitation, and with the aim of achieving success in the software projects, this paper intends to respond the following research questions: What are the Design Thinking techniques used? what are the challenges faced in the requirements elicitation of agile software development projects?
The scientific methodology adopted in this research helped to understand how design thinking can contribute to the requirements elicitation in agile software development. We also classify our research method as descriptive and focused on the subjective character of the question under analysis and its quantitative changes. To gather information, we used methods such as bibliographical research, based on the Systematic Literature Review (SLR) method, to answer research question [13
]. In addition, we conducted a case study, a questionnaire application and interviews with the software requirements elicitation teams involved in the project, in order to answer RQ.2 and RQ.3. Such methods were used together according Trivinos approach [14
] as means to making a triangulation of qualitative data.
The main contribution of this work is to present an analysis of the use of Design Thinking to verify if it fits properly to be used as a means of solving the challenges of software requirements elicitation when using the agile methodology, as well as new profiles of users demanding an increased level of value in their acquired products.
Agile methodologies emerged as an innovative proposal so that companies could deliver products expected by users on time and efficiently for the development team, due to the fact that software development processes should be extremely organized, wasting only a minimum of resources [14
]. Agile methodologies for software development have emerged as an answer to the challenges faced by traditional methodologies in the document-oriented software development [3
]. Even though agile methodologies have broken the paradigm of cascade development and other models, they do not replace the processes that already exist, but present an alternative to existing paradigms [16
One of the main objectives of agile software development is to develop software more quickly by means of a series of iterations (short periods of time) that are feasible in relation to cost and time. Each iteration produces a version of the software for the client/users in a manner that ensures that the defined requirements were implemented [9
]. According to Fadel and Teixeira [14
], in agile methodologies, at each iteration, a sub-project (components, functionalities or modules of the software) is created, and all steps of the software development process are realized: planning, requirements, coding and testing. The iterations last for a few weeks, which leads to quick results, mainly for users. The stakeholder and part of the project development team make suggestions and improvements, participate in the scope planning of each iteration and approval of each delivery. In addition, agile development teams are generally small, which facilitates communication.
2.1. Requirements Elicitation
Software requirements reflect needs expressed by users and written in sentences that impose conditions to software quality. They describe which services, constraints and features a system must provide, must be compliant and represent, and also to specify the knowledge necessary for its development [17
]. They are obtained by means of a systematic process that involves five activities: (1) elicitation; (2) analysis and negotiation; (3) documentation; (4) verification and validation; and (5) management. This process is known as Requirements Engineering (RE) [18
]. A complete understanding of software requirements is fundamental to a software project, no matter how the project will be done. A poorly analyzed and specified software will disappoint the user and will cause annoyance to the developer [6
In the requirements elicitation activity, teams work with stakeholders to learn more about the domain of the application, system services to be provided, the constraints of the operation and the required performance of the system (non-functional requirements) [20
]. During agile software development, requirements elicitation demands an interactive face-to-face communication with users. Agile methodology recognizes that requirements change constantly, evolve over time and are discovered throughout the software development process [4
In the context of agile methodologies, requirements engineering is performed iteratively throughout the development process in instead of a single phase at the beginning of the project. Due to this iterative cycle, a just-in-time model is often used to refine the high-level requirements and derive them into low-level tasks that can be implemented by developers [21
]. There are several techniques for elicitation by using agile methods: (1) interviews; (2) brainstorming; (3) observation; and (4) case analysis [20
2.2. Design Thinking
Design Thinking (DT) is an approach to creating new solutions based on the needs of users and that adds value to the business, considered an approach towards conception [9
]. The main idea of Design Thinking is to see how designers progress during the design process, with creative ideas and solutions designed to discover new opportunities [9
]. DT is user-centered innovation which requires collaboration, interaction and practical approaches to find the most appropriate ideas and, consequently, coherent final solutions [11
The Design Thinking process is basically defined by three phases: immersion; ideation; and prototyping [7
: It can be divided into two stages: preliminary and in-depth. The preliminary immersion’s objective is the re-framing and initial understanding of the problem. It aims to define the purpose of the design and its limitations, in addition to identifying the user and other key authors who should be considered. At this stage, it is also possible to raise issues which can be explored in the in-depth immersion. The stage of the in-depth immersion starts with the elaboration of a research plan with the protocol of the first research and the list of user’s profiles and key authors who assist in the mapping of the investigated context. The communication with these users can take place with the help of certain techniques, such as interviews, generative sessions and awareness raising. These techniques can help understand the context related to the product and/or the service explored during the project [25
: The intention here is to generate new ideas for the project with the use of tools and people to stimulate creativity and generate solutions within the context of the project. Generally, the ideation phase begins with a brainstorming around the subject in order to collect ideas, which are discussed, documented and validated constantly in meetings with the client during the prototyping phase [25
: It has the function of assisting in the validation of ideas. Even though presented as the ultimate stage of the DT process, it can be carried out in parallel with the immersion and ideation phases throughout the project [25
]. The main outcome of this process is to learn about strengths and weaknesses of the idea as well as identifying new directions for the prototype, a reverse way of the traditional creative thinking, with the aim to visualize and imagine new alternatives and solutions. After the solutions are well defined and inspired by the user’s needs, which is in fact the focus of the whole analysis process, the solution will then be implemented [11
It is also important to emphasize that DT is compliant with the initial elicitation practices of requirements engineering, and that the use of rapid prototyping and customer involvement is consistent with the agile development methods. This provides a consistent methodology for both documentation, and team management, which is one of the main focuses of the agile development methods [26
]. Activities such as brainstorming with multidisciplinary teams is an example that can be used to elicit the best ideas in order to do a team’s evaluation. The ideas that are approved take shape with the rapid prototyping to generate data that will be useful for the continuity of ideas [11
2.3. Related Works
Integrating Design Thinking into the Agile system development methodology means customers take part in every sprint, and more focus is put in the beginning to determine the customer needs, requirements, and environment [27
]. Design Thinking would enable clearer focus of customers requirements—affecting the product vision and product backlog. There is less rework during the sprints, since there is a clearer vision of requirements and customer expectations at the start [27
Schon et al. [21
] presented a review with the aim of capturing the current state of the art literature related to agile requirements, focusing on stakeholder’s involvement. They investigated which methodologies are often used that present the user’s perspective on managing project requirements.
Higuchi et al. [12
] developed a project management model that covers the whole process of digital game development and proposes Design Thinking approaches combined with agile methodology. The model has three phases: (i) project definition; (ii) the connection; and (iii) the scrum.
Inayat et al. [5
] presented practices and challenges encountered during the requirements elicitation phase when using agile methodologies. Adikari et al. [9
] developed a framework based on the Design Thinking methodology in order to support the activities of an agile software project in a real context.
Hehn and Uebernickel [28
] conducted an exploratory case study to identify how DT and Requirements Engineering can work together in an agile development environment, covering since the conception until its implementation. These authors defined seven requirements for engineering challenges in agile methodologies: minimal documentation, customer or user issues, non-functional requirements neglected or improper architecture, knowledge of tacit requirements, inaccurate effort estimates, and difficulty in prioritizing requirements. The findings of their case study suggest that DT has the potential to support RE practices and vice versa.
Correa et al. [29
] presented a case study report aiming to verify if the use of DT and its techniques allow a proper identification of a software solution understanding. It used the double diamond model and the Hasso Plattner model [30
] with a set of 13 DT techniques and, based on the opinion of the stakeholders and other participants, the understanding was adequate and the proposed solution near the stakeholders, and suggests that the techniques are appropriate to support software development.
Carroll and Richardson [31
] showed the importance of using Design Thinking to perform requirements elicitation process within software engineering and presented a case study that analyzed how Design Thinking can assist in requesting software requirements for an application in the context of health systems. Carroll and Richardson [31
] claimed that Design Thinking complements traditional requirements engineering techniques in prototype-oriented requirements analysis by encouraging the generation of innovative results through the creative and exploratory interaction of people, processes, technology and business practices. In addition, Carroll and Richardson [31
] provided an overview of Design Thinking phases (empathy, definition, idea, prototype, and testing) that promote a learning lifecycle that helps in understanding health issues and the software design process.
Corral and Fronza [32
] discussed a methodology strategy that integrates Design Thinking into the software development process, in order to meet undergraduate software engineering principles by comparing experience in software engineering courses in two educational environments, one that meets the principles of agile and one that meets the principles of Design Thinking. In addition, they also compared the experience of teaching software engineering to educational environments with a similar work environment. Corral and Fronza [32
] based their studies on the fact that software is a product that has a mission to meet user requirements and, therefore, in the software process, analysis–design–implementation–test can be reinterpreted as empathy–define–idea–prototype–test.
In this study, we used data triangulation technique. According to Denzin [33
], methodological triangulation involves multiple qualitative and/or quantitative methods for investigating a certain issue, while data triangulation uses dissimilar sources of data or different data from the same source to examine the same object. Triangulation is a process in which several methods are used in the study of one phenomenon, and might be used in four basic ways: (1) data triangulation; (2) researcher triangulation; (3) theory triangulation; and (4) methodological triangulation.
According to Trivinos [34
], data triangulation technique is presented in three different aspects: (1) subject-centered product processes; (2) elements produced by the subject environment; and (3) processes and products originated by the socioeconomic and social structure of the social macro-organism of the subject [34
]. Figure 1
shows the aspects of Data Triangulation we used.
In this research, data analysis was made by triangulation based on four sources: (i) bibliographic research; (ii) observation; (iii) questionnaire; and (iv) interview. They are contained in the first aspect (process and products centered on research subject). This aspect emphasizes the products prepared by the research, ascertaining the perceptions of the subject through interviews and questionnaires, and the behavior and actions of the subject by means of observations (free or directed) of the behavior and the processes and products built by the subject (autobiographies, daily intimate, confessions, etc). Thus, the main data source was bibliographical research, observation, questionnaire and interviews which work together in a linear way, namely the outcome of one is an input for the others [34
], as shown in Figure 2
Seeking to reach the objective of this study, three research questions (RQ) were defined:
RQ.1. What are the Design Thinking techniques used and what are the challenges faced in the requirements elicitation of agile software development projects?
RQ.2. How do Design Thinking techniques work as a method of eliciting requirements in a real agile software development project?
RQ.3. What are the results obtained in the use of Design Thinking as requirements elicitation method in agile software development project and was there evidence of the use of Design Thinking regarding the identified challenges?
To answer RQ.1 we performed a systematic literature review. RQ.2 and RQ.3 were answered using case study, observation, questionnaire and interview.
4. Systematic Literature Review
We present here our a Systematic Literature Review (SLR) of existing works that approach the challenges of using Design Thinking techniques faced by the software developing community, regarding requirements elicitation in the context of agile methodologies. The SLR contributed to answer the first research question, as mentioned in the Section 3
, as means of primary studies [35
]. During the SLR, the Planning, Conduct and Publication of Results were followed, as defined in the work presented by Kitchenham [35
], together with the Manual Search and the Snowballing [36
] in order to answer RQ.1.
4.1. SLR Process Method
Our SLR used Automatic Search [37
], which consists of applying the search string in the electronic databases, followed by the manual search [37
], through which searches were performed for works in conferences, newspapers or magazines. The manual search activity was performed in the annals of the conferences and periodicals specific to the Software Engineering area. The automatic search was performed in the following databases:
4.1.1. Search String
The definition of a search string
was according to the set of criteria PICO [38
]: P (population) corresponds to a specific role within the lifecycle of the research focus, an area of application, or even a specific group of the industry; I (intervention) delimits the research focus within a broader scope; C (comparison) identifies alternatives and compares with the delimitation performed in the intervention; and O (outcomes) lists what is intended to be achieved, measured, improved or affected in relation to the population.
By defining the criteria, the following search string was created: (“agile software development” OR “agile methodologies”) AND (“design thinking”) AND (requirements elicitation OR requirements elicitation OR requirements analysis).
4.1.2. Selection Criteria (Inclusion and Exclusion)
The following selection criteria were defined for the selection of primary studies:
As criterion of exclusion of the studies was considered as the non-fulfillment of some of the inclusion criteria, as well as:
4.1.3. SLR Quality Criteria
The evaluation of the quality of the studies identified after the execution of the search process allowed the selection of the most relevant articles to compose the SLR, executed through the three stages defined by Kitchenham [35
] and by Kitchenham et al. [13
], which are presented in Figure 3
The stages adopted in the search strategy were:
Execution of search strategy involving automatic and manual searches.
Identification of potentially relevant studies, based on reading the title, abstract and keywords. In this stage, it is possible to discard studies that are clearly irrelevant to the research. In case of doubt about the permanence of any study in the SLR, the next stage assists in this definition.
Reading the introduction, methodology and conclusion of the pre-selected works, again applying the inclusion and exclusion criteria.
The papers that were selected in Stage 3 were read in full and the volume of articles resulting from this stage were used to compose the SLR and support the answers to the research questions.
4.2. Selection of Primary Studies
The SLR was conducted by using a search string, applied in some electronic databases. The outcome was a total of 24 publications which where filtered by the read of its tittle, abstracts and keywords. The application of this filter led to five publications being selected. Another filter was applied according to the inclusion and exclusion criteria, as defined in the research. After the application of these filters, four publications were classified to data extraction. Beyond this, the search strategy involved the use of manual search found that generated more 162 publications and, after resulting applying the previous filters including a complete reading, had as its outcomes 16 publications selected. Thus, after executing an automatic and manual search (Snowballing), 20 primary studies
we selected to answer the research questions. Figure 4
resumes the results obtained during data collection. The primary studies selected are presented in Table 1
4.3. SLR Results
RQ.1. What Are the Challenges Faced in the Requirements Elicitation of Agile Software Development Projects and What Are the Design Thinking Techniques Used?
SLR allowed answering the first research question RQ.1.
A case study carried out in 16 software development organizations in the United States revealed seven challenges found in the Requirements Elicitation (RE) by using software methodologies. Among them, three are directly related to the requirement elicitation activity [4
Estimation of costs and schedule:
It is more difficult to develop accurate estimates of costs and schedule during the initial stages of an agile project than those found in traditional ones. In traditional methods, the definition of the problem lies in the initial phase of the project and, in agile approaches, the definition of the problem is unstable in these stages, and the aim of the project is subject to constant changes [4
]. However, we noticed that short iterations and frequent feedback of the stakeholders help agile development teams to create better estimates of time and individual costs for each iteration [4
Attention to non-functional requirements:
A large concern of the ER in agile software development lies in the lack of attention regarding non-functional requirements, because they are poorly defined or ignored in the early stages of the project development. Stakeholders often focus on the main functionality and ignore technical issues related to the scalability, maintenance, portability, safety and performance [4
Communication with stakeholders will depend on many factors such as availability, consensus and confidence, especially in the early stages of the project. An American health software developer who participated in [4
] empirical studies reported that the best users will always be occupied with their activities in the organization or their boss will not allow them to join the development team in full time. “In fact, in my experience, even part-time is a problem. We were fortunate to have about 3 to 5 hours per week with the product manager. When a system involves more than one group of stakeholders and each one is focused on different system features, reaching consensus or compromise in the iterations can be a challenge. In addition, each stakeholder may not have a complete understanding that his needs were fully understood by the development team because of the iterative nature of the process which are often not fully understood by the users” [4
Software developers do not usually read all the artifacts produced by the project [21
]. To solve this problem, it is important to find the right combination of artifacts which best fit into the context of the project, so that members can work better. In addition, users stories are not sufficient to define the question of usability, since they do not describe these aspects. To work better with the usability, even though user stories have enough focus on technique, usability needs to be treated on a higher level than that of the development of stories [21
A systematic review of the practices and challenges encountered in the requirements engineering in agile software methodologies presented by Inayat et al. [5
] reports that the practices of requirements elicitation used in the agile methodologies for software development can compensate some of the deficiencies found in traditional approaches. However, this flexible approach still faces some challenges such as lack of user involvement, negligence of non-functional requirements and problems with the estimation of cost and schedule, which are directly related to the phase of requirements elicitation.
Salah et al. [46
] presented seven challenges that are integrated in the use of the user-centered design framework and agile software development, and two of these challenges are directly related to the activity of requirements elicitation.
Planning of initial activities: The initial phase is a pre-development stage divided into agile projects with the aim of assessing the initial requirements and understanding the users needs and goals. In the agile approach, the team focuses on results only in terms of functionality. Thus, the agile methods discourage the developer community to make better plans for the initial project activities, because they are sensitive to a change of requirements.
Short amount of time in the iteration to realize usability tests: Agile teams report a lack of time in the iteration for performing usability tests during development, leaving aside requirements verification and validation.
Soares et al. [47
] indicated nine challenges met during software requirements elicitation in agile methodologies, which are presented in Table 2
presents the results of all the challenges met during software requirements elicitation by using agile methodologies and some referenced primary studies.
5. Design Thinking Techniques Used in Requirements Elicitation of Agile Software Development
Requirements engineering involves the stages of data acquisition, analysis and negotiation, specification and validation, and it is not usually enough to satisfy this demand, except by immersing into the user’s context [26
There is a certain conflict in the way that requirements engineering and agile development define user’s participation. The agile development tends to have a greater effort designing the code itself, rather than a more rigorous documentation, and it requires greater involvement of the user during the development process. The requirements engineering activities tend to reduce user’s participation after the initial mapping. However, even though agile development is guided by user descriptions, its outputs are obtained by functional and non-functional requirements’ perspective, with guidance towards user and a good professional team, it is still complicated to capture what is necessary in a whole. However, it became necessary to search for a way to bring the agile characteristics into the realm of requirements engineering, which includes the elicitation phase as well as the management of the challenges in the method [26
Some works indicate a solution to this problem of using Design Thinking approach as an initial practice of requirements engineering, elicitation and prototyping and support user involvement, as defended by the agile methodology [26
]. The relationship between DT and agile methodologies is so obvious that the literature points out the similarity between them, as presented in Table 4
A project management model covers the whole process of developing digital games through the combination of two approaches to Design Thinking and agile methodology. This model can be used as a method for requirements elicitation in agile methodology, mainly in applications which require creativity [12
]. Throughout the history of creative solutions for solving problems, designers have developed a set of techniques to help them control the three phases of Design Thinking: immersion, ideation and prototyping [7
]. Design Thinking as well as the agile software development methodologies offer competitive advantages, in the differentiation of the product and in the efficiency of cost estimation [12
]. Additionally, both approaches have many similarities, which strengthen their joint use [12
shows the DT techniques used together with the agile methodologies brought up in research performed by the authors of [9
Queiros et al. [45
] presented a study on how the use of technology in education has attracted the attention of researchers in the last few years. It shows that brainstorming is mentioned as well as three other techniques that can be used in the process of DT: introspection cards, conceptual maps and mental maps. Design Thinking can be used from the view of strategic management of innovation using techniques such as: (1) ethnography, a practice of observing how people behave in their day-to-day life or how they exercise a particular activity in organizations; (2) storytelling, a technique which helps create a common understanding of the challenge that is being explored; and (3) brainstorming, i.e., obtaining and evaluating ideas [11
]. DT can transform organizations and inspire innovation through the use of mental map, prototypes of paper, scenarios and high fidelity prototypes which are DT techniques that can help stakeholders improve and discover new requirements [10
In our systematic literature review, we identified 12 challenges that organizations confronted within the elicitation of requirements in agile projects and 31 Design Thinking techniques: 13 techniques are related to the immersion phase, 10 techniques are related to the ideation phase and 8 techniques are related to the implementation phase.
6. Requirements Elicitation with the Use Design Thinking: A Project Case Study at the Brazilian Federal Court of Accounts (TCU)
To answer RQ.2 and RQ.3, a practical case study was carried out in an institution that used requirements elicitation by using Design Thinking. This study was conducted from December 2017 to August 2018. The implementation stages of the case study were guided by a timeline organized by the research team.
The institution is the Federal Court of Accounts (TCU)
, the Brazilian authority responsible for the evaluation of accounts of public administrators and is also responsible for money, federal public funds as well as the accounts of any person who causes loss, theft, corruption or other irregularities that results in damage to the national treasury. It is a federal autonomous and independent body whose main basic functions can be grouped as follows: judicial, advisory, informative, sanctioning, corrective, normative and as ombudsman.
One of the main projects executed by the Information Technology Secretary (STI) covers the appreciation of legality of granting pensions, reforms and admission of staff of the federal public service. The motivation for choosing this project came up by verdicts which have identified structural and operational difficulties in the determination and instruction of these acts. Among them, a low quality of information reaching the TCU by means of filled out forms, with errors and omissions, which, even with great effort for clarification, created issues for more precise automated analysis. This raised the amount of acts for all those involved in the process, by consequently ignoring deadlines and complicating the detection of acts which were not forwarded to the TCU.
In addition, there was a need to review all the legislation that conducted the acts subject to registration.Thus, a presentation of a detailed schedule for the implementation of improvements in the treatment of the acts was determined to facilitate the execution of the activities with greater efficiency, extending the automated analysis for information and, consequently, reducing the need for allocating providers for the manual and individual analysis of acts, optimizing the time needed to analyze these documents.
TCU/STI expected a system that could focus on the quality of the data with automatic verifications of the data input with a differentiated treatment of the roles played by the manager of staff. The roles are as issuer and as registrar of the act, allowing also that the personnel manager and the internal controller to organize their work by creating internal divisions as well as providing a user-friendly and adjustable interface within the context of each type of act. Due to its size and importance, this project took more than two years to be delivered.
shows the levels of the system and how the actors play a role in each of them. Each unit level has a profile associated with their respective functions.
6.1. RQ.2. Do Design Thinking Techniques Contribute to Overcome Challenges of Requirements Elicitation in a Real Agile Software Development Project and What is the Input DT Gives to the Challenges Identified in the Case Study?
The project was led by two development teams: a team responsible for the elicitation of requirements and another with 6 stakeholders from different institutions. It took a 4-h requirements gathering workshop involving a total of 40 participants. Its objective was to identify the needs, wishes and difficulties of the users concerning the information about the acts of the staff, as shown in Table 6
. The first activity performed in the workshop aimed at integrating users, because many of them were from different institutions. Interaction was necessary for a good development of the workshop as a whole. After the interaction was presented to the conception of Design Thinking, a method that would be adopted for the requirements elicitation, by making the collaborative aspect of the approach evident. The idea was to work together with the users and not for them, by seeking solutions based on understanding and experience of those who work with the system on a daily basis.
The requirements elicitation workshop obtained information about the desires and needs of the users of the e-staff system, which was compiled and subdivided into groups, in order to facilitate the work of the team in defining the scope of the project. The workshop results were used as an input for the definition of the project scope, after a preliminary evaluation of all the data collected with the aim of grouping them according to their characteristics. Thus, items that dealt with the same subject or could in some way contribute to the subject by a defined need were gathered in order to organize the information in a better way and consequently provide a better treatment. During the workshop, Design Thinking techniques were used, as shown in Table 7
After the scope definition, the software development was initiated. In this first moment, several functionalities that had to be implemented were defined, a task which would take an estimated four months to be completed. The development of these functionalities was based on the generated prototypes. The solution was made available for all users compliant to the planning; some adjustments were carried out after delivery, in accordance with the users validation. Thus, the use of Design Thinking techniques was compliant to the development of the actual project adopted in this case study.
As mentioned above, we performed requirements elicitation in a software project of the Brazilian Federal Court of Accounts (TCU) by using Design Thinking techniques. The objective was to observe how the practitioners applied the techniques of DT in their daily activities. During the case study, nine DT techniques were observed by TCU practitioners: strategic challenge, challenge selection, re-framing, generative sessions, co-creation Workshop, brainstorms, ideas menu, hypotheses and tests, and paper prototyping.
6.2. RQ.3. What Were the Results Obtained in the Use of Design Thinking in an Agile Software Development Project and Was there Evidence of the Use of Design Thinking Regarding the Identified Challenges?
We proposed an approach with the intention of evaluating the use of Design Thinking in the requirements elicitation via agile methodologies. The approach was developed in parallel with the execution of the case study, identifying potential evaluation criteria, the potential interviewees and how to make assessment. The resulting evaluation process applied in the case study was subsequently generalized for use in similar contexts. The goal of the evaluation approach was to evaluate whether the use of Design Thinking techniques can contribute positively to the challenges identified in the requirements elicitation, in projects developed with agile methodologies.
DT is focused on the user/stakeholder to obtain collaboration, interaction and practical approaches in order to find the most appropriate ideas and, consequently, coherent final solutions [11
]. In addition, as the level of satisfaction with the product is obtained by the user/stakeholder and the level of satisfaction with the methodology adopted in the development of the software is obtained by the project team, we considered that the participants of the evaluation could be the user/stakeholder and the project team. Bearing in mind that the evaluation method would be conducted by people, two methods were chosen: questionnaires and mixed interviews. A questionnaire is a data collection instrument, consisting of a sorted series of questions that should be answered in writing and without the presence of the interviewer. An interview combines pre-defined questions with the possibility of creating new questions based on the topics of the interviews [52
The choice of two different evaluation methods gives more variety to the approach, allowing it to be applied in any project context. The decision of which method to choose depended on several factors, such as the size of the team to be evaluated, the size of the evaluation team and the time available for data collection and analysis. Therefore, taking into account the advantages and disadvantages of the two methods, the use of questionnaire is recommended for the case when the team that is being evaluated is large and the evaluating team, data collection and analysis are small. The interview is recommended when the evaluated team is small and the quantity of members of the evaluation team, the data collection and analysis are considered adequate, for example. The questions were based on DT techniques and also based on the explanation of challenges observed in the elicitation of the identified requirements.
The best moment to do the evaluation was based on two points: to verify whether the application of DT technique had been perceived and whether its use contributed positively to the challenges of requirements elicitation. The first evaluation phase took place shortly after the collection of requirements and involved final users and other stakeholders. The second evaluation phase happened right after the delivery of the product and also involved the users/stakeholder with the objective of capturing the compliance of the defined requirements with the delivered product, the positive contribution of the use of DT regarding the challenges and the satisfaction with the product and the used methodology.
The questionnaire is composed of 11 questions which were answered by the participants of the requirements gathering. It has two parts: the first part was applied shortly after the workshop and the second right after the delivery of the product. Table 8
and Table 9
present the result of the workshop questionnaire and the results obtained from the participants’ responses. The user assigned a grade from 1 to 5 to indicate the level of agreement with the presented statements: the closer to one (1), the lower the degree of agreement/satisfaction, while, the closer to (5), the higher the degree of agreement/satisfaction.
The interviews were conducted with the members of the project development team who were grouped into four different profiles:
Project Manager/Requirements (PM): Responsible for the project planning, definition of goals and monitoring its implementation as well as for the collection and analysis of all the software requirements information.
Product Owner (PO): Responsible for the requirements, assisting in prioritizing them and validating the implementation in order to ensure the quality of the final product.
Developer (DV) Responsible for the development of the system.
User experience/interface UX/UI design: Responsible for planning the best practice and and interaction with the system.
The interview with the team was conducted by taking into account the main aspects obtained from the questionnaire with clients/users. The intention was to conflict the answers obtained in the questionnaires with the information that was given by the team. The questions used as the baseline of the interview were the same for all areas, but the answers took different directions according to the interviewed team. For instance, a conversation with a developer is more technical than an interview with a product owner, turning the interview to a more specific level in each area. Thus, after all interviews, it was possible to merge the information obtained and realize that they complemented each other and gave more support to the conclusions.
The interviews, in general, lasted a 30 min and, to synthesize the answers of the members, we considered the common views and divergences, by choosing to cite the individual points of view of each interviewee. Table 10
shows the results obtained and the questions can be found at: Questions
In this paper, we present an approach designed to evaluate the use of Design Thinking in the requirements elicitation in software development agile methodologies. The approach was elaborated during the case study, as observed elements served as input to identify the best evaluation method. From the nine techniques observed in the project, we concluded after the evaluation that eight of them were used and only one, Hypotheses and Tests, was partially considered. Additionally, out of the thirteen challenges perceived in the workshop, there was evidence of the use of Design Thinking in eight of them, evidence of a partial contribution in three of them and no contribution in two.
6.3. Discussion of the Results RQ.2. and RQ.3
Through the analysis of the answers obtained from questionnaires and interviews, we can see in the case study that there is evidence of an input to eight of the challenges identified through of the use of DT.
Participation of users/stakeholders: From the user’s point of view, they realized that there was a proper incentive to assist the project team in assessing the initial requirements, through the exposition of ideas and the underpinning during its construction, the discussing of needs and the reaching of consensus regarding the requirements. This incentive was caused by means of DT technics used in the workshop. From the point of view of the project team, the stakeholder participated actively in the course of the project, contributing to the prioritization and validation of requirements, testing and improvements. This contribution was due to the dynamics adopted in the project meetings, which used DT techniques and required the participation of the stakeholder.
Requirements Definition: From the users point of view, we may affirm that the use of DT techniques in the workshop helped both users and the team to define the requirements and deliver the product, and the defined requirements were reflected in the final product. From the point of view of the project team, the DT techniques used in the workshop gave users the opportunity to express their wishes and needs and indeed fulfilled their main purpose, which was to define the requirements of the system.
Requirement Validation: From the users point of view, it may be asserted from their responses that the use of DT techniques enabled an environment for discussing requirements, thus allowing to validate them. From the team’s point of view, the techniques used together with the diversity of the users present in the workshop with their different points of view, contexts of work and needs enriched the discussion and forced the project team to meet the most relevant requirements so that everyone could feel included and taken into account.
Project Schedule Estimation: According to the project team, the needs identified during the workshop allowed to estimate the complexity of the project in a way that the elaboration of an estimation of a more appropriate schedule could be enabled. As a result, a chain effect could be observed: the use of DT in the workshop provided a better identification of the project needs and allowed the preparation of a more reliable schedule.
Initial Activities Planning: According to the project team, there was a great effort in planning the initial activities, via the study of the DT and an analysis of its practical implementation, so that the main focus of the workshop was the stakeholder’s involvement in order to allow the extraction of system key features.
Details of the requirements: In accordance to the project team and considering the participation of the stakeholder, the system requirements were well detailed regarding the prototyping, the DT approach and the regular presence of the stakeholder in the course of the project, which was also fostered by DT.
Prioritization of requirements: In accordance with the project team, the dynamics applied in the meetings during the project, which were based on DT techniques, were important to prioritize the requirements.
Interdependence between requirements: In accordance with the project team, the dynamics applied in meetings in the course of the project, which were based on DT techniques, were important to identify the interdependence the requirements.
There is partial evidence of the contribution of this project to three specified challenges through the use of DT, following an analysis of the questionnaire and of the interviews.
Attention to non-functional requirements: In accordance to the users’ vision, questions related to non-functional requirements were addressed during the the workshop, but not so efficiently, so that the DT techniques used did not promote a discussion about this. Taking all the questions asked in the after-workshop questionnaire, those questions related to the non-functional requirements were the ones which obtained a lower average. Evaluating the view of the users after the delivery of the product, they observed the presence of some resources related to non-functional requirements, mainly concerning safety and the question of system usability. From the point of view of the team, they considered that the non-functional requirements were addressed superficially during the workshop, and that they were only considered more effectively during the development phase mainly related to the usability, which is due to the use of prototyping and performance which is due to the type of system they were developed. Due to the characteristics of the applied system, it is difficult to come to a conclusion on the influence of the DT on non-functional requirements. However, it can be asserted that DT fosters the discussion of some items related to non-functional requirements, such as usability.
A correct combination of artifacts and their proper use: In accordance with the vision of the team and because the workshop had well defined goals, they succeeded together with the users, identified in a well-defined way which were the key needs and characteristics of the system and, starting from this, elaborated the artifacts which would be necessary for each member of the team. However, it can be seen that, in the decision making of what artifacts should be elaborated in a project, the DT does not influence the choice. The methodology only provides the input needed to elaborate the artifacts, with the team being responsible for the definition of which artifacts will be used.
Change of requirements: According to the vision of the team, due to the regular participation of the stakeholder to discuss ideas and to create prototypes, items which are related to DT, the atmosphere seemed favorable for the dealing with changes of requirements in a way that it would not cause much impact. The only mistake perceived in the development of the system in the study was the lack of testing when discussing and prototyping these new ideas. This made it difficult to analyze the real need of change in the requirements. Hence, there is partial evidence for the contribution of a challenge related to the changes of the requirements, but this is more due to the negligence of the team than to the methodology itself.
Finally, it can be seen, through the analysis of the questionnaires and interviews, that there were no indications of a contribution in two specified challenges through the use of DT:
Cost estimate: According to the perspective of the team and because the project was carried out within a public institution by internal staff, the cost estimate was not dealt with in this project, only the timeline was relevant to the context. Therefore, the influence of DT on the resolution of the challenge of the cost estimate cannot be evaluated.
Time available to perform usability tests: In accordance with the team’s point of view, a usability testing was not realized. The only tests performed in the project were functional. Therefore, the influence of DT in the resolution of challenges related to the time to perform the usability tests cannot be evaluated.
In the questionnaires and interviews which were carried out, in addition to the collection of data to answer if DT techniques helped the team solve challenges arising from activities to elicit requirements in an agile software environment, the staff and users were asked about their level of satisfaction with the applied workshop/methodology and the final product, on a scale from 1 to 5.
The feedback obtained from the use of the workshop/methodology, for both the user’s development side and the team’s side, was generally satisfactory, since the average of responses was 4.46 from the perspective of users and 4.25 from the view of the team. It can be asserted that the users have been partially satisfied with the final product and the development team has been fully satisfied, since the average of the responses here are 3.71 and 5.0, respectively.
7. Limitations and Threats to Validity
The findings presented in this review study have the following limitations and threats to validity.
We considered all the studies which provided empirical, case study, experimental, practices, industrial and survey related information about challenges faced in the requirements elicitation of agile software development projects.
The findings of this systematic literature review cannot be generalized because the results are based on a specific set of keywords and the research repositories that were used for the data collection. Therefore, our results could be limited and cannot be applied to every projects and organizational setup.
We cannot guarantee that all relevant primary studies were selected in systematic literature review. It is possible that relevant papers were not chosen. To mitigate it, we performed the automatic search, and complemented it by performing manual search to try to collect all primary studies in this field. During the data extraction process, the primary studies were classified based on our judgment. To mitigate this threat, the classification process was performed using peer review.
Internal validity is mainly related to the capability of replicating similar findings. We addressed this aspect by defining and later following the systematic literature review procedure, described in Section 4
. Four researchers were involved in the review process, who, over a period of time, worked together to avoid duplications and achieved consensus in the acceptance of the identified primary studies. However, it could be possible that, if this study were replicated by other researchers, minor variations in the identified studies would be observed due to differences in personal aptitude and thinking. Regardless of this fact, the findings presented in this review will enable readers to obtain a clear picture of challenges for software requirements elicitation using Design Thinking.
Another threat is that perceptions of the interviewees could be biased towards their own beliefs. These beliefs could cause some distortions when interpreting the reality. To reduce this threat, the chosen practitioners were those who had more experience within organization.
The systematic review of the literature identified 12 challenges that organizations were confronted with in the elicitation of requirements in agile projects and 31 Design Thinking techniques, 13 of which are related to the immersion phase, 10 are related to the phase of ideation and 8 are related to the implementation phase (prototyping). These challenges and techniques served as an input to assess whether the use of DT helps the internal design team of the practical case study to meet the recurrent challenges of eliciting requirements in agile methodologies identified by systematic review in the project studied in this work.
To carry out this evaluation, a process of executing the case study was developed, in which the use of DT was observed during the development of a module belonging to the development of software by the TCU, e-staff (e-pessoal in Portuguese). During this observation, it was assessed which techniques of DT were used by stakeholders, both in the DT workshop and during the development of the software module. During the execution of the case study, a more generic approach was elaborated which can be used in other contexts in which DT is used with the same objective of assessing whether it helps staff face existing challenges in eliciting requirements via agile methodologies.
During the implementation of the process, stakeholders were asked via questionnaires and interviews whether they actually perceived the use of the observed techniques and whether they contributed positively to challenges identified in the systematic literature review. In addition to that, interested parties gave their opinion of the application of DT (workshop) and the final product, after the development of the elicited requirements.
Of the 31 techniques, the application of nine techniques was observed: Strategic challenge, Challenge selection, reframing, generative sections, brainstorming, cooperation workshop, list of ideas, prototype of paper and hypotheses and test. It can be concluded that, through the obtained results in the questionnaires and interviews, they were perceived by people interested in the use of these techniques, except for the technique of hypotheses and test which was partly used because, according to the interested parties, the hypotheses raised during the workshop were prototyped, but not tested.
With regards to the 13 challenges identified in the systematic review, evidence was found that DT strengthens stakeholders participation in the definition, detailing, validation, interdependence and prioritization of requirements, mainly in the prototyping; in the estimation of the schedule; and in the planning of the initial activities. What was not perceived in stakeholders/users view was an incentive to discuss non-functional requirements during the workshop for the assessment of requirements; only during the development of the project was a major concern with usability verified, due to the use of prototyping and performance, and also due to the characteristics of the software being developed.
In addition to that, as the system in the study is part of a larger system, several questions related to non-functional requirements were extended to the module, for instance, reliability, security and support. Due to this, the project team dealt less with non-functional requirements.
About the correct combination of artifacts and their use, the project team believes that DT provides sufficient input for the elaboration of coherent artifacts, however it cannot confirm that every project team using DT may choose these artifacts and use them correctly. This is in fact the task of a team rather than the use of DT. Finally, although DT provides support for teams to identify new requirements through prototyping, there was a negligence of the team which did not execute a DT technique correctly, which prevented this point from being evaluated. This is partially because, despite creating prototypes of these new requirements, they were never actually tested before their implementation.
Regarding the challenges of cost estimation and the time to perform usability testing, it was not possible to identify evidence of a contribution. It can be concluded through the application of the evaluation approach in the studied software project that there is evidence of a contribution, by the use of Design Thinking, for most of the challenges of the elicitation of requirements obtained from an agile methodology.
For future studies, we can be propose the realization of other practical studies for a better consolidation of the obtained results, with different types of systems and diverse teams, in a way that allows for an application of all the challenges found in the literature.