The Impacts of Design Heuristics on Concept Generation for a COVID-19 Brief

Abstract

Design heuristics (DHS) as a tool can help boost designers’ creativity in early conceptual design phases. We have developed a set of DHS for digital innovation (DHS10). There are numerous studies on DHS impacts/outcomes in the recent years. However, little research has been conducted to identify whether DHS has lasting benefits on designers’ ideation performance. This paper explores whether DHS10 can help designers achieve more creative ideas based on different design briefs, and if DHS has lasting impacts on or benefits for students. An empirical study was conducted with two groups (i.e., 32 students who learned DHS10 five weeks ago, and 24 students who studied DHS10 instantly). They were asked to address an open-ended design brief on COVID-19 and generate as many innovative ideas as possible. The results suggest DHS has impacts on students who learned DHS just now and five weeks ago. The effect is stronger on those who just learned DHS10. We suggest that DHS10 be provided for ideation as it provides texts and visual stimuli for designers. DHS10 also has the potential to help students understand digital innovation and generate ideas accordingly.

1. Introduction

Manufacturing firms must adapt to digital transformation to ensure long-term survival [1]. Digital innovation, the carrying out of new combinations of digital and physical components to produce novel products [2], is no longer just the business of software companies [3] but has the potential to revolutionize the manufacturing industry by changing existing product designs and business models [4,5]. Ideation originally determines the type of design, and plays an important role in the development of novel concepts and business success ultimately [6]. Design heuristics (DHS) are cognitive ‘shortcuts’ that point toward useful design patterns, emphasizing reducing search time [7]. DHS as a tool can help boost designers’ creativity in early conceptual design phases [8] with different approaches to DHS being developed [9], e.g., general ones such as SCAMPER [10], TRIZ [11], and 77 Design Heuristics [12]. Some scholars have developed DHS for specific contexts, such as DHSfX (design for one-handed use) [13] and DHS for additive manufacturing [14]. However, technical advances have triggered an opportunity for design innovation to generate a wealth of new products [15] but few DHS have been created for the digital era. Meanwhile, ICT and innovation have crucial impacts on economic growth [16]. To fill this gap, we extracted 10 design heuristics (DHS10) for digital innovation by systematically studying 1307 Red-Dot award-winning conceptual designs [17], and our new DHS10 for digital innovation were validated through a controlled experiment based on a design brief (see Appendix Figure A1) on the theme of Business communication [18] in our previous study. That study suggested that DHS10 can be employed as a new ideation tool to support design students and potentially design practitioners, in the generation of greater numbers of more innovative and diverse concepts for digital innovation, with reduced design fixation [19,20].

There have been numerous studies on design heuristics in recent years. Many researchers have reported the positive influences of DHS [8,14,21], especially on the novelty of concepts produced. Different design discipline students (i.e., mechanical engineering and industrial design) were observed for DHS use differences [22]. For example, more DHS use was related to higher creativity with the concepts, while a negative relationship was also found when more DHS use lead to less diversity [22]. Another study investigated DHS as a tool that can support student designers to iterate their initial concepts by providing specific suggestions [21]. DHS show advantages in the initial idea generation phase when compared with other design techniques (e.g., brainstorming and morphological analysis) [23]. However, quantitative analyses have not yet been conducted for exploring whether DHS has long-lasting impacts on designers’ ideation performance after they have studied DHS. As including a dimension of likelihood of success is highly valuable in using the design tools [7], different design briefs and contexts ought to be tested and evaluated when a new design stimuli tool is developed. Understanding the lasting and immediate impacts of DHS on design processes allow us to make an effective use of DHS to boost innovation and creativity in design practice and education.

In December 2019, reports emerged from Wuhan, China, about an outbreak of viral pneumonia caused by the novel Coronavirus SARS-CoV-2 [24]. The viral disease has become a global pandemic. By the 4 April 2020, the disease had infected nearly 1.2 million people and caused more than 60,000 deaths [25]. COVID-19 is not just a health crisis—it may trigger a devastating social, economic and political crisis. These issues harm sustainability dramatically. The fight against the COVID-19 pandemic appeals not only to medical and public safety experts, but also to engineers and designers who can contribute to creating solutions to slow or control the spread of the virus. In response to this emergent societal crisis, we conducted an empirical study centred around addressing a design brief about COVID-19 for investigating the potential of DHS, and we look forward to seeing how design students can generate novel solutions with our DHS during this study.

This study aimed to explore whether DHS10 can generate more novel ideas when facing emerging societal challenges such as COVID-19 and to evaluate lasting and short-term impacts of DHS.

This paper comprises four parts, background, methods, results, discussion and conclusions.

This study evaluates the DHS10’s effectiveness in addressing a new design brief (design for COVID-19 challenges) and analyses the DHS10’s lasting impact on students who studied it before. The findings have implications for design practice and education.

2. Background

2.1. Challenges for Ideation

Ideation originally determines the type of design, and plays an important role in the development of novel concepts and business success ultimately [6]. Designers naturally generate ideas, even without tools [26]. These natural approaches are developed based on experience and preferences for problem-solving [27]. A common technique for commercial ideation is team brainstorming [28] or its variants such as ‘brainwriting’ (e.g., developing a large quantity of ideas). However, this technique is not without its limitations [29]. For example, a dramatic decrease in idea quality was found when brainstorming was undertaken for longer than 20 min and the quantity of idea generation decreases after 30 min [30]. The ability to take a problem and generate multiple, varied solutions that can lead to new creative outcomes is often referred to as concept generation or ideation [31]. Despite the emphasis on creative exploration, industrial designers have been shown to experience limitations when attempting to generate diverse concepts [32]. Generating a diverse range of ideas becomes more challenging for designers when they become ‘fixated’, i.e., their attention is focused on a single, past example or on one new idea [19,20].

2.2. Cognitive Theories of Creativity

Cognitive theories have direct implications on utilising stimuli in creative ideation. Ideas are commonly considered as products of existing information in minds [33,34]. Ideation processes are often based on theories of memory processes. Search of Associative Memory (SAM) theory [35] and the Adaptive Control of Thought (ACT) theory [36,37] are two well-cited theories on memory processes. Both theories claim that long-term memory is an associative network of memory units. Short-term memory, or working memory, has limited capacity and contains elements that can be thought of as search cues. That is, these elements are sources of activation that probe long-term memory [38].

2.3. Design Tool Evaluation

According to a well-cited study [7], design principles should be readable by humans and understandable by designers, and require specific investigation beyond the scope of the literature in extraction of data for increasing the designer’s chances for success (i.e., including a metric or dimension of likelihood of success is highly valuable in using the principle). To explore the likelihood of success, different experiments should be conducted with different designer types, problems and contexts.

2.4. Design Heuristics

Because generating creative ideas is both valuable and difficult, various methods to support designers have been proposed [7]. Creative tools are required to aid the designer to produce more ‘creative’ ideas in short periods [6]. Design heuristics (DHS) are a context-dependent directive based on intuition, tacit knowledge or experiential understanding, which provide design process direction to increase the chances of reaching a satisfactory, but not necessarily optimal, solution [7]. DHS are evidenced to help generate ideas effectively in the conceptual design phase and play an important role in addressing issues of design fixation [8,39]. Howard [40] suggests that design tools can be categorized as creative analysis tools, thinking tools and stimuli tools. DHS can be categorized as a stimuli tool. Different DHS have been developed for different purposes, such as DHS for additive manufacturing [14] and DHS for assistive (one-handed) products [13].

2.5. Research Opportunities

Technical advances have triggered an opportunity for design innovation to generate a wealth of new products [15], but few DHS have been created for the digital era [9]. To fill this gap, we extracted 10 design heuristics (DHS10) (see Figure 1) for digital innovation by systematically studying 1307 Red-Dot award-winning conceptual designs by six steps: (1) narrowing the data sources’ dataset; (2) reading through descriptions; (3) defining a design’s critical innovative functions and features; (4) grouping the similar design features; (5) extracting heuristics; (6) communicating heuristics [17]. The success of this extraction method depends on the following factors, similar to Ref. [8]: (1) whether the proposed heuristic is observed by other evaluators when using the same dataset; (2) whether it is possible to identify the heuristic as described within other product designs; (3) whether the proposed heuristic provides a useful strategy in generating novel designs. Our study suggests that DHS10 can be employed as a new ideation tool to support design students and potentially practitioners in the generation of greater numbers of more innovative and diverse concepts for digital innovation, with reduced design fixation [19,20]. However, different design briefs ought to be tested for more comprehensive analysis and evaluation of this newly developed design stimuli tool [7]. Additionally, our previous study and other researchers have reported the positive influences of DHS [8,14,21] on novelty of concepts produced, while there is little research that explores whether DHS has long-lasting benefits on designers’ ideation performance. Thus, we conducted this study with a different design problem and context to evaluate the effectiveness of DHS10. Through this study, we also intend to investigate whether DHS has long-lasting impacts and benefits and further identify the DHS10’s dimensions of likelihood of success.

3. Methods

3.1. Research Questions

To identify the effectiveness of the DHS and long-lasting benefits, an empirical evaluation was undertaken. The following three research questions guided our study, and three research hypotheses were proposed:

RQ1: Do students show evidence of DHS use in the ideation process after instant DHS and 5-week-lapse DHS?

Hypothesis 1 (H1).

Students will use design heuristic in their concept generation whether instant DHS or 5-week-lapse DHS. This hypothesis is based on prior work that shows the evidence of DHS use in concept generation [22]. However, it was unknown if students will recall the DHS and employ it 5 weeks later.

RQ2: Do different periods of DHS affect the ideation outcomes?

Hypothesis 2 (H2).

DHS10 will have lasting impacts on students, but immediate effects are stronger than after a lapse of time. Prior studies have investigated instant DHS effects on ideation outcomes [13,14,23]. However, it was unknown whether DHS10 has long-lasting impacts on students.

RQ3: How does DHS10 assist idea generation when facing emerging societal challenges such as COVID-19?

Hypothesis 3 (H3).

DHS10 will help students to generate more novel, useful and creative ideas. This hypothesis is based on prior studies [13,14,23].

3.2. Ethics

The experiment protocol and consent form were approved by the institution’s ethics committee. In order to protect anonymity, we removed any potential identifiers of locations and individuals.

3.3. Participants

Eighty-four students studying industrial design at a UK university were recruited for this study. The students were first-year undergraduates enrolled in a design sketching module. The experiment protocol is illustrated in Figure 2. Participation in the research was voluntary with no remuneration. Participants were informed of the learning opportunity afforded by participating in the study. The students were randomly divided into two groups based on the first letter of their surnames. One group was called Group A and the other Group B.

In total, 43 students were introduced to the DHS10 on 19 March 2020 (first experiment). No individual approached explicitly refused to participate in the second experiment, 5 weeks later on 30 April 2020, but 11 out of the 43 did not reply to our invitation to the second experiment, leaving 32 participants; they were named Group A2 (to test the lasting impact of DHS10 after 5 weeks). Twenty-four students who never learnt any DHS methods formed Group B2 (to test the instant impact of DHS10).

3.4. Procedure

The empirical evaluation was conducted by comparing participants’ ideation outcomes through the use of two different DHS conditions (i.e., instant DHS and 5-week-lapse DHS). The experiment protocol is illustrated in Figure 2. In this paper, our study focuses on comparing the second experiment ideation outcomes from two different groups (i.e., A2 and B2). The objective of conducting the first experiment was to have the students of Group A study DHS 5 weeks earlier than Group B so that we can conduct a comparison of the ideation outcomes from different DHS period conditions.

During the experiment, Group B2 (instant DHS) students were asked to watch a pre-recorded video introduction to the DHS10, and Group A2 (5-week-lapse DHS) students were asked to watch a video of Individual Brainstorming (IB). The DHS video introduction was developed by the research team and IB video instruction came from a LinkedIn learning resource [41]. Both videos were approximately 10 minutes long. Additionally, the students were provided with experimental materials for use in the controlled experiment, i.e., a set of DHS10 cards (see Figure 1) or IB guidance (see Appendix Figure A2), the design brief, sketching sheets (see Appendix Figure A3 and Appendix Figure A4) and an information sheet plus consent form.

The students were asked to generate as many concepts as possible within 60 min using a pre-defined design brief, as follows: ‘The outbreak of COVID-19 has made a huge impact on our everyday life. As a designer, what can we do to help support people through this challenging time? Products/service systems for self-protection? Self-isolation? Better communication? Community support? Healthcare provision? Please provide your solution—be that small or big’. The brief was chosen for three reasons: (1) it was neither abstract nor narrow; (2) it removed the potential bias of experience; (3) it was beneficial to see whether students can generate novel and useful solutions for this pandemic crisis.

Group A2 students were asked to answer the design brief by brainstorming (DHS cards were no longer provided) and to communicate their design concepts using sketches and text descriptions. This group was asked to identify whether they recall any Design heuristics (DHS10) in generating concepts (see Appendix Figure A3). Group B2 students were asked to answer the same design brief using DHS10. This group was also asked to identify which specific DHS they used (see Appendix Figure A4). All students were permitted to use their individual working space for completing the task to prevent discussing with and influencing each other. When students completed the design task, they uploaded the sketching sheets and the informed consent form online for assessment and analysis.

3.5. Metrics for Evaluation

Metrics for evaluation can play an important role in the product/idea evaluation process [42]. The ideation outcomes for each experimental group were assessed using three evaluation criteria: novelty, usefulness and quantity. These criteria have been widely employed to evaluate design ideation outcomes in published studies [13,43,44,45]. To assess the creativity of newly designed products, one must be able to assess the ‘novelty’ and ‘usefulness’ of these products, where usefulness represents the value of the products [42]. To evaluate the design ideation outcomes more comprehensively, creativity (see Equation (1)) and fixation (see Equation (3)) were assessed based on the results of the three evaluation criteria of novelty, usefulness and quantity.

3.5.1. Novelty

Novelty is a measure of how unusual or unexpected an idea is as compared to others [45]. For novelty, the Sarkar and Chakrabarti [42] method was employed as it demonstrates consistency in results when compared with expert evaluation as the benchmark. The five steps of the proposed method was used for assessing novelty [42]. According to this approach, a concept can be classified as either having very high-novelty, high-novelty, medium-novelty, low-novelty or no-novelty. The evaluators were informed that aesthetic appeal is not included in novelty evaluation and limitations in sketching skills should not influence assessment. The 5-point rating scales for novelty is shown in Figure 3a.

3.5.2. Usefulness

Sarkar and Chakrabarti [42] suggest that usefulness includes three parameters as being equally important in assessing the overall usefulness of a product. These are importance of use, rate of popularity of usage (i.e., if more people use a product) and rate of use/duration of benefit. The detail explanations of three parameters mentioned above can be found in Ref. [42]. The 5-point rating scales for usefulness is shown in Figure 3b.

3.5.3. Creativity

A widely used definition of creativity of ideas or products requires both novelty and usefulness [46,47]. We average concept novelty and usefulness in order to get a creativity score with equal weights on each category. The creativity metric is provided in Equation (1).

$$\mathrm{Creativity}\left(\mathrm{C}\right)=\frac{\mathrm{Novelty}\left(\mathrm{N}\right)+\mathrm{Usefulness}\left(\mathrm{U}\right)}{2}$$

(1)

3.5.4. Quantity

Quantity is defined as the total of all generated ideas (Q_Total) [45]. The quantity of repeated ideas (Q_R) is based on when a repeated idea occurs (when a participant develops a slight variation of a previous idea) and quantity of non-repeated ideas (Q_NR) corresponds to the remaining number of Q_Total once repeated ideas have been removed [48]. The quantity metric is provided in Equation (2).

$${\mathrm{Q}}_{\mathrm{Total}}=\sum \mathrm{all} \mathrm{ideas} \mathrm{generated}={\mathrm{Q}}_{\mathrm{R}}+{\mathrm{Q}}_{\mathrm{NR}}$$

(2)

3.5.5. Evaluating Fixation

To assess whether design heuristics can help designers to reduce fixation and increase diversity of ideas, fixation was evaluated. The fixation metric is employed based on Moreno, Blessing, Yang, Hernandez and Wood [48]. The Fixation metric is shown in Equation (3). The fixation score was calculated for each group with repeated ideas being identified. Lower values indicate non-fixated designs.

$$\mathrm{Fixation}=\frac{\mathrm{Total} \# \mathrm{of} \mathrm{repeated} \mathrm{ideas} }{\mathrm{Total} \# \mathrm{of} \mathrm{generated} \mathrm{ideas}}=\frac{{\mathrm{Q}}_{\mathrm{R}}}{{\mathrm{Q}}_{\mathrm{Total}}}$$

(3)

3.6. Data Analysis

Concept sheets were collected by an online information system, while the time-out submissions were not collected for ensuring the participants using the same time. In total, 56 students generated 271 concepts in the 60-minute session, ranging from 1 to 13 concepts from each student. The 271 concepts were presented a different randomized order for each evaluator. Three experts were invited to assess the concepts, all with strong experiences in the industrial design domain (see

Table 1. The background information of the three evaluators.

Evaluator	Education Background	Working Experience	Occupation	Judge Experience	Professional Awards
E1	BE in Design and Technology	>10 years	Lecturer	IDEA	IDEA
	MA in Industrial Design
	PhD
E2	BE in Industrial Design	>10 years	Lecturer	SDG International	China Red Star
	MA Architectural Design and Theory
	PhD
E3	BA in Industrial Design	>3 years	Industrial designer	/	Red-Dot best of the best,
	MS in Industrial Design Engineering				IDEA, iF

). Two of the evaluators had served as judges in international design competitions, e.g., International Design Excellence Awards (IDEA) of the Industrial Designers Society of America (IDSA). The third evaluator was an industrial designer who has won more than 10 reputational industrial design awards. The Consensual Assessment Technique (CAT) was utilized for all measures besides quantity and fixation [49]. The evaluators were provided a document of prompts (e.g., the novelty and usefulness evaluation methods mentioned above) for reaching the similar mental model. They each subjectively and independently evaluated individual ideas by utilising the prompts for reference. Repeated ideas were noted in the analysis. A repeated idea means a participant develops a slight variation of a previous idea [48]. The fixation score was calculated for each participant with repeated ideas being identified.

Consistent with Consensual Assessment Technique (CAT), we asked the evaluators to use a 5-point Likert Scale to rate each idea, including novelty and usefulness (the number 1 denotes the lowest rating and number 5 indicates the highest rating) (see Figure 3). The 5-point scale used in this study aligns with the scales recommended for use in rating systems [50]. The condition of the experiments and hypotheses were ‘blind’ to the evaluators and the DHS identified on the sketching sheets for DHS were removed for assessment. The intraclass correlation coefficient (ICC) was used to test the interrater agreement [51,52]. The ICC was computed via SPSS V24. The results show that ICC is 0.63 and 0.617 in novelty and usefulness, respectively, indicating substantial agreement between the three evaluators [53].

Two trained coders were engaged in identifying whether design heuristics were employed based on concept sketches, written descriptions and the claims of DHS10 used (Group B2 was asked to identify which DHS they used; Group A2 was asked to identify whether they recall any DHS10 (i.e., YES, NO, and Can’t tell)). The two coders had bachelor’s and master’s degrees in industrial design. Before identifying the DHS used, the original claims of DHS used by participants were recorded in an Excel spreadsheet and then verified. Finally, discussion took place when required to reach a consensus.

Table 2. Different design directions inspired by DHS10.

Idea Sketch	Title and Design Description	DH(s) Used
	Portable Interactive Whiteboard This is an interactive whiteboard that parents can set up easily to help give structure to home-schooling their children and to keep organized.	#3 Utilizing Display Technology; #4 Utilizing Lights and Sounds; #7 Adding Smart Functions
	Social-distancing alert This is a pocket-sized scanning device that vibrates when it detects somebody breaking the 2-metre rule.	#7 Adding Smart Functions
	Light for the NHS Currently during the mornings, communities are clapping for the NHS. Thus, this light movement could allow for the display of ‘THANK YOU’ to be shown, to provide morale.	#4 Utilizing Lights and Sounds; #9 Sharking Product for Novel Effects
	Hospital Bed This is a bed that displays the patient’s heartbeat and any other key info about the person using the bed.	#3 Utilizing Display Technology

shows the different design directions inspired by DHS10.

By averaging the three experts’ rating scores for each metric, the average scores were used for further statistical analysis. We computed the independent sample T-test to compare the mean difference for each metric between instant DHS and 5-week-lapse DHS. We used the Chi-square test to compare the nonparametric statistics (e.g., the high creativity proportion comparison). One-way analysis of variance (ANOVA) was applied to explore the statistical difference in three different conditions (i.e., YES, NO, and Can’t tell). We also used the correlation coefficients for the Correlation Analysis.

4. Results

Before the analysis, the data were pre-processed to remove identification of the DHS used. The sketching sheets were checked to ensure that the task had been undertaken as instructed. A blank sketching sheet in Group A2 was excluded from the analysis. Finally, 56 sketch sheets were analysed, with 32 using DHS/IB and 24 using DHS (instantly). The analysis focused on five aspects: (1) long-lasting effects, (2) quantity and fixation rate, (3) novelty, usefulness, and creativity, (4) correlation and (5) DHS use.

4.1. Long-Lasting Effects

The results showed that 18 out of 32 (56%) students from Group A2 who studied DHS 5 weeks before could recall and employ DHS. Based on the students’ self-reports, 33 (21%) ideas utilized DHS definitely; 89 (57%) ideas did not utilize any DHS; and 33 (21%) ideas were noted ‘Can’t tell’. Although the mean novelty scores of ‘Yes’ (M = 2.8, SD = 0.56) were larger than ‘No’ (M = 2.6, SD = 0.69) and ‘Can’t tell’ (M = 2.6, SD = 0.85), the mean novelty scores of the three conditions were not significantly different according to a one-way ANOVA, F(2, 152) = 1.82, p = 0.17, η² = 0.023. Although the mean novelty scores of ‘Yes’ (M = 2.2, SD = 0.55) were larger than the ‘No’ (M = 2, SD = 0.54) and ‘Can’t tell’ (M = 2, SD = 0.67), the mean usefulness scores of the three conditions were not significantly different according to a one-way ANOVA, F(2, 152) = 1, p = 0.37, η² = 0.013. Similarly, the mean creativity scores of ‘Yes’ (M = 2.51, SD = 0.46) were larger than the ‘No’ (M = 2.31, SD = 0.54) and ‘Can’t tell’ (M = 2.28, SD = 0.71), but the mean creativity scores of the three conditions were not significantly different according to a one-way ANOVA, F(2, 152) = 1.82, p = 0.17, η² = 0.023. More details can be seen in Figure 4.

4.2. Quantity and Fixation Rate

Group A2 (5-week-lapse DHS) (n = 32) generated 155 ideas, and Group B2 (instant DHS) (n = 24) generated 116 ideas. The average number of ideas for every participant was compared across the groups by employing the independent-samples t-test. The mean difference was not significant: t(54) = 0.019, p = 0.985. By eliminating repeated ideas, Group IB generated 154 distinct ideas (repeated ideas n = 1). Group DHS generated 115 distinct ideas (repeated ideas n = 1). The t-test results show that the mean difference was not significant, either: t(54) = 0.037, p = 0.97. The fixation rate was calculated based on Equation (3). The fixation rate of Group A2 (5-week-lapse DHS) was 0.006, and Group B2 (instant DHS) was 0.009. The difference in fixation rate was not significant.

4.3. Novelty and Usefulness

The statistical results compare expert opinions of Group A2 and B2 for mean novelty and usefulness (see

Table 3. The results of the t-tests comparing the two different methods in the mean novelty, usefulness, and creativity.

Metrics	Time Difference		T	p-Value	Cohen’s d (95% CI)
	5 Weeks Later M (SD)	Instantly M (SD)
Novelty	2.62 (0.71)	2.91 (0.72)	−3.3	0.001 **	−0.4
Usefulness	2.07 (0.57)	2.28 (0.61)	−2.95	0.003 **	−0.36
Creativity	2.35 (0.57)	2.6 (0.59)	−3.6	0.000 ***	−0.43

Note: ** denotes p < 0.01; *** denotes p < 0.001 (2-tailed).

). Students who studied and employed DHS instantly had higher novelty scores (M = 2.91, SD = 0.72) than students who studied DHS 5 weeks before (M = 2.62, SD = 0.71). The mean difference was significant: t(269) = −3.3, p = 0.00, d = −0.4 (see Figure 5). The t-test for mean usefulness also reveals a statistically significant difference between Group A2 (M = 2.28, SD = 0.61) and Group B2 (M = 2.07, SD = 0.57), t(269) = −3.3, p = 0.001, d = −0.34 (see Figure 5).

4.4. Creativity

Creativity scores were calculated by averaging concept novelty and usefulness, which is provided in Equation (1). Results indicate that students who utilized DHS (instantly) achieved higher creativity scores (M = 2.6, SD = 0.59) than those studied DHS 5 weeks before (M = 2.35, SD = 0.57). The mean difference is significant, t(269) = −3.3, p = 0.001, d = −0.43, with full details in Figure 5 and Table 3. High creativity (HC) refers to scores greater than or equal to 3 for novelty and usefulness. The HC ideas were identified in Group B2 (instantly) (n = 22) and Group A2 (5 weeks) (n = 15). The corresponding HC proportion of Group A2 was 0.19 whereas Group B2 was only 0.097. The difference between the two was significant, as indicated by the Chi-square test: χ²(1, n = 37) = 4.855, p = 0.028, suggesting that Group B2 generated a greater number of highly creative ideas than Group A2 (Figure 6 and

Table 4. Summary of high creativity proportions.

Method	HC	LC	Total	HCP	χ2	p-Value
Studied DHS before	15	140	155	0.097	4.855	0.028 *
Instant DHS	22	94	116	0.19

Notes: HC, high creativity; HCP, high creativity proportion; LC, low creativity; * p < 0.05 (2-tailed).

).

Table 5. Concept examples with high creativity score.

Idea Sketch	Title and Design Description	DH(s) Used
	Test at Home Drone This is a drone that allows people to test themselves for COVID-19 at home. They book a timeslot online, and the drone comes down and displays instructions for taking a swab from yourself. The user removes the test tube marked for them with a light, takes a swab, then places the test tube back into the drone using the pop lock seal. The user receives their results by text.	#1 Adding Autonomous Rotorcraft; #3Utilizing Display Technology

provides an example of student work with a high creativity score.

4.5. Correlation Analysis

Table 6. Correlation matrix for number of DHS used, quantity, novelty, usefulness and creativity.

	Quantity	Novelty	Usefulness	Creativity
Number of DHS used	+0.44 *	+0.12	−0.041	+0.053

Notes: * denotes p < 0.05 (2-tailed).

presents the correlation coefficients for the variables in the study. The data indicate a moderate positive correlation between the numbers of DHS used and the numbers of distinct ideas (r = 0.44, p = 0.031). Figure 7 shows a moderate correlation trend in which the greater numbers of DHS used lead to a greater number of distinct ideas (r² = 0.19, p = 0.031). However, the data reveal that the numbers of DHS used are unrelated to the novelty (r = 0.121, p = 0.2), usefulness (r = −0.041, p = 0.66) and creativity scores (r = 0.036, p = 0.7).

4.6. DHS Use

Table 7. Summary of DHS used in the second experiment (design brief 2).

N	Minimum	Maximum	Mean (SD)	NDU	NDU Rate	WID	WID Rate
116	0	5	1.45 (0.963)	11	9.5%	0	0

Notes: WID, wrongly identified DHS; NDU, no DHS used.

shows the summary of DHS used in Group B2 (instantly) for addressing the COVID-19 design brief.

Table 8. Different design directions inspired by DHS10 from one student.

Idea Sketch	Title and Design Description	DH(s) Used
	Face Mask This is a self-cleaning mask that uses UV light. This will reduce waste in PPE equipment. The mask cleaning is tracked by an app on your mobile device to update you on when to clean mask and how long is left for the clean mask.	#8 Manage and Control Remotely
	Drone Communication This is a drone that allows you to contact friends, family and colleagues through a camera. The drone feature allows a hands-free experience and means you can move around the area. It takes a full body image if necessary.	#1 Adding Autonomous Rotorcraft
	Smart Alert System This is a small keychain device that will alert you with sound and light to complete a certain task that you want. For example, for the COVID-19 situation, this could be to apply sanitiser on your hands.	#4 Utilizing Lights and Sounds; #10 Smart Reminder

shows three design concepts of a student generated by utilising DHS10. In the concepts generated by Group B2, only 11 concepts (9.5%) from four students (i.e., P₁(5 out of 5), P₂₄(1out of 5), P₃₃(4 out of 4), P₅₃(1 out of 5)) did not utilize any DHS. In total, 40 concepts were identified using more than two items of the DHS, which represents 35% of all the concepts generated. Figure 8 shows the frequency distribution in the number of DHS used, which indicates that multiple DHS can be employed in a design and lead to innovation.

5. Discussion

This section discusses the key findings, contributions, implications and limitations.

5.1. Findings

5.1.1. Long-Lasting Effects

More than half (56%) of the students still could recall and employ DHS during ideation 5 weeks after they studied DHS, which indicates that DHS10 could have lasting effects through long-term memory.

5.1.2. Quantity and Fixation

Differences in the average number of ideas/distinct ideas per participant are not significant, whereas differences in the average number of distinct ideas per participant were significant in the previous study (DHS > IB on quantity). Additionally, this study did not show a significant difference on design fixation, whereas the difference was significant in the previous study (IB > DHS on fixation rate). Two main factors may influence the outcomes: (1) the design brief is an emerging social issue being experienced by the students and (2) the students may obtain benefits from generating diverse ideas from DHS learned before. Further investigation is needed to explore whether different types of design briefs influence the outcomes on quantity. Additionally, based on the Group B2 (DHS instantly) analysis, there was a moderate correlation trend towards using more DHS and a greater number of distinctive ideas (r = 0.44), indicating that students were inspired by DHS text or image stimuli to generate more diverse ideas, similar to our previous study.

5.1.3. Novelty, Usefulness, Creativity and DHS Use

Through analysing the Group A2 (5-week-lapse DHS) students’ outcomes, we found ideas utilising DHS had higher scores on novelty, usefulness and creativity than those not utilising DHS. However, no significant difference was found. The students who studied DHS instantly had significantly higher scores on novelty, usefulness and creativity than the students who studied DHS 5 weeks before. No correlation was found between the numbers of DHS used and the scores of novelty, usefulness and creativity. However, in the previous study, we found that there was a weak correlation, and a significant relationship with creativity was observed in Ref. [22]. Further investigation is needed to explore what triggered the differences. We found DHS easy to employ as only 2% and 9.5% of students did not utilize any DHS, and around 35% ideas utilized more than two DHS.

There are three kinds of design fixations (i.e., unconscious adherence, conscious blocking and intentional resistance) based on previous research [54]. In this study, the results support our DHS10 as a design stimuli tool that could help design students in reducing design fixation in multiple dimensions. DHS plays a cognitive prompts role for linking interconnected memory, which allows designers to link irrelevant or remotely related items together and represents a common source of creative and novel ideas [55]. Specifically, DHS can be thought of as search cues in short-term memory, which provides many potential cues for inspiring the brain of human being’s long-term memory, leading to the generation of more creative and diverse ideas [35,36,38]. The mental burden and processing time for the memory search would be reduced by providing a DHS tool [13]. On the contrary, these DHS can also be remembered, and contribute to generating more creative and diverse ideas even though cues were not provided.

In this study, the group sizes differed. Group A2 (5 week-lapse DHS) (n = 32) generated 155 ideas, and Group B2 (instant DHS) (n = 24) generated 116 ideas. Before conducting the independent samples t-test, we conducted the test for homogeneity of variance by Levene’s test, and the results indicated that the variance is equal across groups. Meanwhile, these samples were selected randomly from a relatively large population, and the rating results meet the normality.

Overall, the experiment results support our research hypotheses that students will use design heuristic in their concept generation. DHS10 strengthens the outcomes of concept generation and have lasting impacts on students, but immediate DHS10 effects are stronger than the 5-week-lapse group; DHS10 helps students to generate more novel, useful and creative ideas.

5.2. Contributions and Implications

5.2.1. Comprehensive Design Heuristics Evaluation

This study has filled the gaps in investigating if DHS has lasting impacts. Specifically, an empirical evaluation was conducted with student designers (n = 56). Comparisons were made between two groups. We found that DHS had positive lasting impacts on novelty, usefulness and creativity in the ideation generation phase when students had been studying them, either instantly or 5 weeks before. The impact was stronger when they learned the DHS instantly. A possible reason might be that DHS10 has provided the conscious prompts.

5.2.2. Implications for Design Practice and Education

Through the evaluation based on different design briefs, our study suggests that DHS10 can be employed as a new ideation tool to support design students and potentially practitioners in the generation of greater numbers of more innovative and diverse concepts for digital innovation, with reduced design fixation. This study also suggests that our new DHS10 has strong potential for helping designers to quickly generate creative ideas when facing social challenges. DHS10 may assist in generating digital-powered products for meeting digital transformation [3]. Students benefit from DHS10 whether they studied it previously or instantly, and the instant effect is greater. It was found that Group B2 (instant DHS) generated more good ideas (high usefulness and novelty) at a higher proportion than Group A2 (5-week DHS). In addition, the industrial design experts who rated the generated ideas commented that the DHS Group generated more specific and digital technology-driven ideas; the 5-week-lapse DHS Group generated more ‘traditional’ industrial design ideas. This finding indicates that if the external stimuli is not provided, the students will repeatedly search their stored long-term memory. We suggest that the DHS be provided for generating ideas during ideation as it provides texts and visual stimuli similar to the use of mood boards [56].

5.2.3. Contribution for Sustainability

Countries are racing the COVID-19 pandemic, trying to slow the spread of the virus as quickly as possible by different measures (e.g., tracing contacts, restrictions on travel, social distancing, cancellation of mass gatherings). COVID-19 is not just a health crisis, it may trigger devastating social, economic and political crises. These issues harm sustainability dramatically. During this pandemic, many papers have been published on the design of different products for facing COVID-19 [57,58,59]. However, few studies investigated the applications of design tools or methods during this pandemic crisis [58,60,61]. This study seeks to inspire people to focus on utilising different design tools for helping designers/engineers ideation to create innovative ideas to addressing emerging societal crisis in a time-efficient manner. Our DHS10 provides many heuristics that can help designers to generate innovative ideas for sustainable economic growth [16], not only for the COVID-19 design brief, but potentially for addressing the United Nations’ Sustainable Development Goals (SDGs) on good health and well-being [62,63].

6. Conclusions

The results of this study suggest that DHS10 can support the generation of concepts with greater novelty, usefulness and creativity in the ideation phase. DHS10 appeared to be easy to remember, understand and employ, and may contribute to design practice (as a stimuli tool that DHS can boost wider explorations of potential designs, and provide useful experience for considering more design candidates) and design education (to help students understand digital innovation and generate concepts accordingly). DHS10 can be remembered or learned as valuable design experience and knowledge for supporting the generation of creative ideas. The results also suggest DHS10 has lasting impacts on students, while the effect is stronger on those who learned DHS10 instantly than those who learned it 5 weeks before. DHS as a design tool has the potential to greatly strengthen design practitioners’ ideation outcomes in a low-cost way to face emerging societal challenges such as COVID-19, and may facilitate sustainable economic growth by inspiring digital innovation.

The originality of our study is twofold: (1) we comprehensively investigated DHS10’s effectiveness by designing solutions for emerging society challenges, namely COVID-19; and (2) we investigated the lasting impacts of DHS10 on student designers. The rigour of the study is demonstrated by the comprehensiveness of the evaluation, involving comparison between two groups and a longitudinal aspect of one group.