Insights from a Latent Semantic Analysis of Patterns in Design Expertise: Implications for Education

Abstract

Design and design thinking are increasingly being taught across several disciplines—ranging from arts, architecture, and technology and engineering to business schools—where expertise plays a central role. A substantial corpus of literature on research in regard to design expert and design expertise has accumulated in the last decades. However, in spite of its importance for design and design education, the topic has remained largely unframed. A major goal of this study was to carry out an assessment of literature through leading interdisciplinary design journals in order to identify main topics and categorize them into thematic patterns concerned with expertise in design. A structured assessment approach based on Latent Semantic Analysis (LSA) was used over 861 abstracts. The emerging thematic patterns associated with design expertise allowed a categorization at different levels of detail, which included 80, 50, 20, and 12 factors analyses, respectively. The major contribution of the study was to offer a structured assessment of key design literature that enabled to gain a clearer and more comprehensive understanding of the thematic patterns in the discussion on design expertise. Implications of the identified key factors for design education are discussed.

1. Introduction

Design plays a central role in a broad spectrum of disciplines, ranging from arts and architecture, to technology and engineering, and business studies. Accordingly, disciplines view design from different perspectives, what includes subjective and social aspects in the arts, to more objective and scientific issues in the engineering disciplines. In spite of such differences, a common interest among these disciplines has been the study of design expertise. What expertise is [1], and what the features of a design expert are [2], continue to be a major topic of research interest in design. Bearing in mind the importance that these notions have for the development of design, and for design education in particular, e.g., [3], a comprehensive understanding of the emergent patterns and trends in interdisciplinary design literature is necessary, mainly considering the increasing demand for interdisciplinary approaches in design. It is therefore expected that a literature assessment will help to unveil some of the key factors that characterize design expertise, and facilitate its development and acquisition. In spite of the relevance that expertise has for design education, the subject has remained largely unframed.

Expertise is a human attribute that is closely associated with performance, quality, and productivity. This notion can be defined as a superior performance in a certain domain [4], and is based on knowledge, skills, and experience [5,6]. Expertise has been studied for more than 50 years across varied domains, and from different perspectives, including design. Given the broad significance of expertise in the design domain, there is a large body of existing literature that makes its review an ambitious task, especially as a comprehensive manual process. Typically, manual literature review involves a pre-selection of key articles that is often based on search criteria such as the number of citations of an article. This type of review approach, while insightful and quite common, is prone to be subjective and limiting in terms of the number of articles that can be covered. The potential tendency to focus on the more cited and influential papers might develop into a bias towards representative literature. In this study, we distinguish between critical review and the review of thematic patterns in the distribution and occurrence of the dominant themes in literature. Thus, instead of conducting a subjective evaluation regarding which themes are more important than others, a major goal of the present study is to use a semantic text analysis approach to carry out a quantitative and objective assessment to identify how the dominant themes appear across the corpus of assessed design literature. We would like to understand the main trends and topics associated with design expertise, and whether and how these can be organized and categorized into main thematic patterns. Eventually, understanding what topics relate to what kind of patterns, and at what level of detail may enable to gain a deeper understanding of what constitutes design expertise. A semantic analysis of existing literature can also provide further hints about what are the important issues that design students, educators, and practitioners should be aware of in regard to the development of expertise.

The frequency and occurrence of themes will be assessed through semantic grouping of the keywords using techniques such as Latent Semantic Analysis (LSA), which is a text data mining and natural language processing (NLP) technique [7]. Such methods are well established as valid and useful for identifying and understanding thematic patterns in literature, e.g., [8,9]. LSA allows a systematic, computational, and comprehensive analysis of a large corpus of literature for matching between query and document at topic level. In previous studies it was also showed that performing LSA over abstracts is adequate and suitable to identify the thematic patterns, since abstracts tend to provide the key summary of a paper and key terms, e.g., [10,11,12].

Therefore, by systematically applying LSA techniques to 861 paper abstracts selected from leading interdisciplinary design journals, this research presents a structured representation of key thematic patterns that have emerged in design expertise research over the last 25 years. The identified patters are of importance for intervention programs aimed to promote the acquisition and development of design expertise, for example in the design studio.

2. Expertise in Design

Expertise has been investigated extensively in the design domain, and it continues to be a major research topic (for an overview see Cross [13], and for special issues see Design Studies [14], and the Journal of Design Research [15]). Most design literature centers on questions about what are the competencies and skills that are necessary to be considered a design expert, and how an expert behaves during the design process. Design is characterized by extremely demanding standards set by the experts themselves [13]. Whereas literature on design expertise is vast, and a complete analysis and discussion of the topic is beyond the scope of this study, in this section we attempted to organize recurrent themes identified in relevant related studies (see

Table 1. Categories and themes identified in the design expertise literature.

Domain Knowledge

Domain-specific knowledge; procedural knowledge; procedural skills; procedural actions; procedural know-how; declarative knowledge; representational knowledge; conceptual knowledge; body of knowledge; discipline knowledge; tacit knowledge; implicit knowledge; domain-specific knowledge; experiential knowledge; technical knowledge; multi-disciplinary knowledge; knowledge structures; knowledge integration; consistent knowledge

Knowledge Strategies

Strategic knowledge/strategy; heuristics; rules of thumb; trial and error; solution oriented strategies; problem oriented strategies; procedures; goals/sub-goals/limited strategy; trial and error; breadth first approach; depth-first approach; top-down/bottom-up; deductive reasoning/approach

Knowledge Representation

Problem representation; mental representation; external representation; chunks of information; knowledge structure; encoding; visual representation; drawing; sketching; graphic/spatial modeling; knowledge storage; level of abstraction; geometric descriptions; symbolic references

Problem Solving

Ill-defined problem solving; routine/non-routine problems; problem constraints; needs and requirements; design space/search space; problem-solution interaction; problem situation; task; productive thinking; adductive thinking; singular problems; repetitive problems; uncertainty; incomplete information

Design Activities and Design Process

Design actions/moves; design analysis; problem analysis; problem framing/structuring; information gathering; goal analysis; hypotheses; concept/idea/solution identification/recognition; concept/idea/solution generation/development; solution analysis; solution clarification; solution evaluation/assessment; solution selection; knowledge access and retrieval; solution adaptation/adjustment/transformation/development; decompose/recompose; solution exploration; decision making; cyclical process; linear process; transitions/phases

Design Performance and Approach

Efficiency; opportunism; problem treatment; problem interpretation; problem criticism; learning; reflection; improvement; mental simulation; criticism; superior performance; information prioritization; speed; prediction; elaboration of conjectures about problem

Conceptualization

Abstraction; schema/schemata; idea; concept; pattern; principles/first principles/solution principles; symbolic reference; level of detail

Tools and Resources

Existing solution examples; precedents; typologies; visual displays

Analogizing

Identification; access; retrieval; mapping; correspondences; transference; between-domain sources; within-domain sources; distance; deep relationships; surface relationships; analogical principles

Creativity

Creative discovery; imagination; exploration; uncertainty; challenge the familiar; novelty; originality; new perspectives; innovation; fluency; elaboration; functionality/usefulness; value; fixation

).

Expertise is concerned with the designer ability to put into practice some level of superior competency. The nature of design ability can be defined by certain modes of behaving and thinking that designers express while tackling design problems. Cross [3] suggested core features of expert design competencies that conforms main design abilities. He identified eight abilities that included: solve ill-design problems, adopt solution-oriented approaches, produce novel and unexpected solutions, tolerate uncertainty when working with incomplete information, apply imagination to deal with practical problems, employ abductive and productive thinking, use non-verbal, use graphic/spatial modeling as a media of representation, and employ drawings and sketches as a means of problem solving.

Expertise can be generally defined as the possession of a large body of domain knowledge and procedural skills [16,17]. An expert designer is an individual who has acquired considerable experience through time, master a vast body of well-integrated and consistent domain-specific knowledge [18,19,20], and has the capability to retrieve, analyze, and apply that knowledge in particular tasks [16,17,21]. Expert designers use a systemic approach to deal with the design task [22,23,24]. They do not follow a linear process, but iterate cyclically, with many transitions through various design stages [18].

While novice designers tend to rely heavily on declarative knowledge and simple or standard heuristics to understand and solve design problems, experts assume the possession of procedural knowledge to explore multiple concepts, and adapt them to novel situations [25,26]. Thus, design expertise can be understood as tacit knowledge that entails more procedural know-how than declarative representations (or know-that) [14]. Gaining expertise is therefore seen as transforming declarative knowledge into procedural knowledge [27], which is spontaneously in the form of heuristics, rules of the thumb, integrated procedures and strategies [28] (for an overview see Ball and Ormerod [29]).

Accordingly, design experts possess strategic knowledge, what means that they own knowledge of processes and strategies that they can use during the acquisition or employment of new knowledge [16]. They acquire such strategies by solving related problems time after time. Therefore, they tend to use such strategies quite often and more effectively than novices [30]. They are also very good at making spontaneous decisions regarding what are the best circumstances to use a certain strategy [31]. Their experiential and organized domain-specific knowledge allows them to create a large number of sub-goals by means of problem decomposing approaches that novices lack [13,16,20]. In fact, expert designers are able to carry out opportunistic decompositions for tackling ill-structured design problems [19,32,33,34].

Novice’s strategies are generally related to a depth-first approach to problem solving, meaning successively recognizing, exploring, and analyzing sub-solutions exhaustively and in depth before moving to the next one [13,32]. By using trial and error strategies, they develop only a part of the solution at gradual levels of detail. Experts, in contrast, usually adopt top-down and breadth-first approaches [35,36]. Based on years of experience solving an assortment of diverse types of problems in different situations, after exploring multiple concepts, expert designers make preliminary and rapid evaluations of a broad range of design options before opting for one of them [18,33]. Eventually they combine both approaches when the design situation demands it [37].

Differences in the quality of knowledge and problem representations were identified between experts and novices. Design experts possess better representations that capture the more relevant aspects of the design domain [19]. Rather than storing detailed or concrete entities, experts sort information by means of more abstract and meaningful features [38]. Indeed, experts process information at a higher level of abstraction, what enables them to simplify large amounts of information embedded in multiple sources [39]. As a result, they can easily recognize, remember, and interpret relevant knowledge and underlying principles from past solutions accumulated through professional experience, which they utilize extensively and effectively [40]. In many cases, rather than investing a large amount of time in redefining design problems, experts frequently use a solution-driven approach from the initial stages of the process. Employing familiar solutions enables them to act quickly and more efficiently than when relying on problem analysis. This is why some scholars believe that experts can be seen as more solution than problem oriented [40,41]. A consequence of this is that experts consider a larger number of alternatives during the design development before making final decisions [18].

Moreover, design experts extensively use knowledge from existing solution examples to create analogies with the problem at hand [42,43]. Whereas novices tend to organize problems that have superficial features in common, experts group problems that are related to solutions by similar principles [44]. Thus, they know how to identify the circumstances under which such solutions best apply to problems. Literature on analogical-reasoning showed that establishing a qualitative analogy is an outcome of the distance between a target (the problem) and source (the solution example). Accordingly, while experts generally establish between-domain analogies, which are remote to the problem at hand, novice designers prefer to use within-domain analogies, which are close to the problem. On the other hand, experts are good at recognizing structural similarities between a source and target. Experience in the design domain contributes to the creation of abstract problem representations, and augments the chances of structural mappings from source to target [45,46]. Thus, between-domain analogies are difficult for novices to access and transfer, as they have weak knowledge structures to establish deep analogies [47].

Gaining expertise was of not only observed to be related to the capability of making abstractions and creating analogies, but also to forming design concepts and knowledge schemas [48,49]. Schemas allow representing design problems according to key features, and establishing a mapping of relationships with similar problems [46,50]. Expert designers construct their own knowledge schemas, and provide them with personal and singular meanings [51]. These are characterized by intentions, ideals, and values that are used as affordances to direct their design processes [52,53]. As expertise develops, designers access and retrieve familiar design schemas from a collection of examples and precedents—either external or stored in their minds—and adapt them to the design needs [43,54]. The use of familiar and well-established precedent solutions allows them to act faster and more effectively than when employing pure problem analysis [40]. Sharedness of schemas and principles is critical to enhance communication and performance in design teams [55], as well as to effectively interact and negotiate with users, clients, and entrepreneurships [56].

Since experts approach design problems more critically, they better understand the needs and constraints of a task [57]. As such, they are skillful enough to perceive more information needs and more information sources, prioritize criteria from such sources, and as noted above, decompose the problem by defining a hierarchy of goals and sub-goals, before making more detailed refinements of the solution [32,58]. The opportunistic decomposition of the problem helps them successfully deal with the ill-structure of design problems when searching for solutions [29,59,60].

A reason the master designer can move efficiently in the search space is because he or she stores, accesses, and manipulates information in larger and meaningful cognitive chunks (or blocks of information) than the novice designer [34,61,62]. They can identify underlying principles, instead of paying attention to the irrelevant or surface features of the design problems [13,38]. Therefore, expert designers are very efficient at framing complex problems from initially incomplete requirements [63]. By recalling chunks of information, they identify from the outset many conflicts that assist them frame the problem, and thereafter search feasible solutions [57,64]. Due to their organized thinking, experts are more efficient than novices in controlling their cognitive activity and performance [13], what enable them to restructure problems without being stuck to the initial conditions [32]. In this regard, Cross [65] found that experts are good at contextualizing the problem in a personal manner, and take a broad systemic approach to the task, in contrast to just applying narrow problem criteria. These not only contribute to dealing with complex and dynamically interrelated design issues that arise when structuring the problem [66], but to speculating and making inferences and conjectures about potential solutions. Elaboration of conjectures has an important function to define and explore problems and solutions in a more integrated fashion [13,57].

Experts also showed evidence of accomplished spatial reasoning skills [56], and to possess knowledge structures that are particularly rich in terms of visual representations [67]. Whereas novice designers were seen to sketch to focus early on concrete forms and design solutions, experts tend to use this tool mainly with the aim of generating ideas and concepts [51,68]. In this regard, more experienced designers are prone to make use of abstract symbolic references to represent their designs, while novices employ formal geometric descriptions more frequently [69]. As the process develops, experts produce representations at various levels of detail, including both conceptual designs and solutions with technical elements [70].

Besides retrieving familiar knowledge from existing design sources, or the systematic generation of expected solutions, design expertise is also about creating something new [27,51]. Important creative outcomes in several domains are frequently made by individuals who possess domain-specific knowledge. Thus, the exploration of original ideas is particularly pertinent to the study of design expertise [71]. From the initial stages of the process, experts unceasingly explore ideas from new perspectives, what leads to unveiling a broader assortment of designs [30]. Casakin [50] observed that while novice designers apply known principles from visual displays that belong to the same domain to the problem at hand, experts use structural principles accessed from remote displays leading to creative solutions. Other researchers found that differences between the creativity of experts and novices can be attributed to their dissimilar cognitive processes [72]. Kokotovich [73] showed that novices, who lack well established skills and methodologies for properly representing their thoughts and ideas, tend to run into a solution too early in the design process. This, of course, constraints their creative abilities due to a limited and rather prosaic understanding of the critical aspects of the design problem.

Since design experts can effectively move through the search space, they easily restructure ill-defined problems without being stuck to initial problem conditions [27], which facilitates creative discovery. Thus, expertise in creativity is seen as the ability to organize information in a way that facilitates the intuitive identification of creative and unorthodox solutions, and the employment of heuristic procedures that serve to transform inert knowledge into active exploration [30,74]. Creativity in design expertise is also very powerful in the context of multidisciplinary design teams [38]. Interaction among team members intensifies creativity by meeting different perspectives, information sources, approaches, and technologies. These interplays can trigger unusual thinking, and stimulate experts to think out of the box, and reflect about their own views.

Although expertise has been largely associated to creative potential, some studies showed that in some cases it may also be associated with negative effects [74]. Although experts are more efficient than novices in solving problems, their structured knowledge can also prevent them from looking for creative solutions away from their familiar search space [75]. Thus, instead of taking risks in exploring vague and uncertain and perhaps unsuccessful solutions, their commitment to prior design concepts and known examples can lead to higher levels of fixation than novices [76,77]. Petre [38] summed up expert creative behavior as identification of creative solutions based on developed chunks, which allow moving efficiently through the search space, restructuring problems in support of creative discovery, and using heuristic procedures that transform inert knowledge into dynamic exploration.

3. Methodology and Research Data

The research data was collected from leading interdisciplinary design journals (see

Table 2. Selected interdisciplinary design journals and corresponding articles organized according to their relevance for the keywords ‘expertise’ and ‘expert’.

Journal Name	Selected Articles
Design Studies	455
Journal of Engineering Design	79
Research in Engineering Design	50
AIEDAM - Artificial Intelligence in Engineering Design Analysis and Manufacturing	78
CoDesign	147
International Journal of Design Creativity and Innovation	44
International Journal of Design	8
Total	861

), part of which were indexed in the Design Society internet homepage (Accessed: December 2018). The search was limited to the online databases of the seven journals listed in Table 2, even though a wider search was possible. This was done under the assumption that most of the ‘design expertise’ related articles belonging to these journals would cover the trending themes at any given time.

The collection and selection of the data involved multiple steps. Initially, articles were identified using keywords such as expertise, expert, and semantic variants of the term such as proficient and skillful. A preliminary reading of the abstracts of the resulting articles was conducted to make a qualitative filtering according to their relevance for the study. For instance, articles related to expert systems or empirical studies where expert feedback was obtained to validate findings, were considered to have no direct relevance to the present study, and therefore were filtered out. Finally, the filtering and grouping process led to the inclusion of 861 article abstracts into the corpus for text analysis. Table 2 shows the selected design journals and the number of articles included in the study after the filtering process.

Why LSA on abstracts and not the full-text? From LSA perspective, abstracts have less noise in the data since they mostly provide uninterrupted and self-containing text that summarizes the whole paper. In contrast, in the main text of the paper, authors often use figures and tables that are integral to communicating the key themes of the paper. Since figures and tables are not accessible to semantic text analysis, it is likely that LSA on the main body of the paper will miss some critical information. Another disadvantage of carrying out an LSA analysis through the main text is that authors may repeat the same themes to emphasize key messages, and this could affect the term frequency calculations [9]. Consequently, in LSA based studies, use of abstracts for identifying thematic patterns in the literature is an accepted approach, e.g., [10,11,12].

3.1. Data Analysis

The collected dataset (corpus) was subjected to LSA [8,78,79] to identify the latent concepts within the textual data at the semantic level [80]. LSA uses a set of algorithms to convert unstructured text into structured data objects, and analyze these data objects to identify patterns for the discovery of knowledge [81]. The main idea behind LSA was to collect all the contexts belonging to the words in the corpus, and derive associated factors that represent related concepts.

In this paper, a factor is a latent class representing multiple observed entities which have similar patterns that are associated with the latent class. LSA was used to cluster thematic patterns at the different levels of detail using 12-, 20-, 50-, and 80-factor analyses. For example, in a 12-factor analysis, all the entities that were found in the corpus to be relevant to the analysis were classified and represented through 12 latent classes. Initially, five and eight factors were also considered for analysis. However, given the large variety of terms associated with design expertise in literature, anything less than 10 factors was deemed too abstract to be labeled in a way that would provide useful insights. Accordingly, an analysis on five and eight factors was not continued further.

Thus, each factor represented a certain part of the overall observed entities, whereas the factors were organized in the order of the entities that they explain. A sample representation of factors and their associated entities is presented in

Table 3. Latent classes and corresponding entities identified in the 12-factor analysis.

	Latent Classes	Entities
F12.1	Training for improving quality of outcome	Maxim, situated, coach, quantify, idea_product, knowledg_structur, us framework, prospect, durat, fact, interact_experi, meaning, articul_design, unstructur. media_design, lean, ill, meet_requir, novic_design, partner. novic
F12.2	Scoping and framing of ill-defined problems	III, us_techniqu. articul_design. applic_condit, coach. hinder, fixat_effect, situated, durat, cross, interact expert, industri_engin, assumpt. problem_design, schon, major, develop creativ. service, examin_design, session, idea_product, static
F12.3	Interaction and experience-based	interact_experi, cross, quantify, ill, articul_design, develop_creativ, model_gener, durat, coach, applic_condit, hinder, situated, us_techniqu, result_detail, assumpt, us_framework, service, entir, findindic, lean, van, master
F12.4	Model-driven for idea-solutions	model_gener, idea_product, situated, articul_design, quantify, static, deliber, assumpt. interact_experi, ill, build_user, durat, sketch_design, coach, facility, nnodel_tool, partner, advantage, fixat_effect, model_valid, primari_school, group_perform
F12.5	Operations and tools	oper_process, musician, design_heurist, ill, coach, master, applic_condit, interact_experi, analys_us, us framework, fund, develop_user, divers, freshmen, hi, quantify, design_invoiv, paco, futur_develop, van, articul_design. emerg_design
F12.6	Mastery and assessment	Master, interact_experi. articul_design, sketch_design, maxim, fixat_effect, hi, review_literatur, quantify, complementari, us comput. situated, lean, method_assess, model_gener, failure, durat. ill, partner. prospect, fact. Descript
F12.7	Individual and teamwork experience	individu_team, emphasi_place, driven_process, situated, convent, inhabit, applic_condit, interact_experi, articul_design. ill, hi, exclude, knowledg_structur, complementari, equip, group_perform, review_literatur. innov_activ, idea_product, service, musician, advantage
F12.8	Training for blending and concept development	Static, articul_design. conceptu_blend, coach, fact, interact_experi, us techniqu, equip, paco, quantify, us_comput, examin_design, maxim, samara, constant, us framework, idea_product, design_base, assumpt, design_conceptu, model_gener, fr
F12.9	Quality and assessment	review_literatur, maxim, quantify. lean, durat, articul_design. coach, descript, interact_experi. complementari, attitud_design, designinnov, analys_us, secondli, knowledg_structur, design_qualiti, prospect, cross, design_particip, ill, us_techniqu, model_gener
F12.10	Creativity and added value	provid_insight, develop_creativ, complementari, assumpt, applic_condit, situated, equip, ill. conceptu_blend, interact_experi, advantage. idea_product, driven_process, emerg_design. develop tool, design_emploi, design_altem, fact, prospect, fr, underli_design, problem_us
F12.11	Reflective, participatory, and interactive approaches	us_techniqu, articul_design. idea_product, quantify, problem_us, assumpt, coach, examin_design, paco, situated, participatori_action, ill, interact_experi, primari_school. schan, nnitig, lean, deliber, us_framework, prospect, us_comput, applic_condt
F12.12	Design know-how and concept exploration	Hinder, paco, conceptu_blend, situated, analys_us, ill, us_framework. durat, quantify, investig_role, optimis. applic_condit, articul desn, static, fact, extend, interact_experi, equip, examin_design, prospect, maxim, develop_user

. In any analyzed text, multiple words may share the same meaning, and one word may have many synonyms in different contexts. LSA “loads” the words that share the same meaning to their associated concept, and also “loads” one word to various latent semantics other than its main associated concept.

LSA differs from traditional factor analysis in using Singular Value Decomposition (SVD) to reduce the dimension of the original data. This helped to present the collected 861 papers under the categories. By employing SVD, LSA generated two sets of loadings in a matrix format: one aimed for the terms and another for the documents. Each factor in the matrices is associated with a high-loading term and a high-loading document, where each factor represented a research theme in the corpus.

Higher term/document loading values for each factor indicates a greater possibility that the related term/document discloses a key theme. The level of detail considered for identifying research themes is modified by changing the number of factors in the analysis. Lower level factor aggregation represents common research areas, whereas higher level factor aggregation is an indicator of main research themes. Since the use of LSA for identifying thematic patterns in literature is well-established and the techniques are well documented, e.g., [9,10,11,12], this paper only briefly outlines the key steps involved in the process of analysis.

3.1.1. Step 1: Applying Text-Mining Process Including Tokenization, Stemming, Stop Word Filtering, N-Grams in Text-Mining Software

The following series of text-mining procedures was applied within Rapidminer 5.0 as follows: (i) abstracts were tokenized with non-letter separators such that each abstract was aggregated to singular words that make it an individual word-bag; (ii) all the letters in the word bags were transformed to lowercase; (iii) stop words of English language such as “and,” “the,” and “so” were predefined in Rapidminer 5.0 (in addition, keywords like “expert” and “expertise” that are used during document collection were custom defined and removed from the word-bags.); (iv) tokens of the words that had less than two letters were removed, since they do not add any meaning to the words; (v) term stemming techniques were applied to the word-bags in order to remove word variations; i.e., the words “experts” and “expertise” were transformed to a single token “exper-”; and, (vi) the n-gram model was applied, which creates word groups that were considered as an individual token in the word-bag. For example, the phrase “Design Expertise” or “Design Expert” was aggregated to tokens “Design-Exper” as a bigram. Similarly, the phrase “Expert design methodology” or “Expert design method” was aggregated to token “Exper-Design-Method” as a trigram.

3.1.2. Step 2: Output of Text Mining Processes: Term Occurrences (TO) and Term Reduction

Initially, the dataset was analyzed using both bigrams and trigrams. Term occurrences were sorted in descending order. In order to reduce the dimension, the entities that occurred once, and the entities that appeared in only one abstract, were removed from the matrix. However, during the dimensionality reduction, approximately 90% of the entities were removed from the matrix due to their existence in one document or only one time. This included most of the three-word entities, provided that these documents did contribute in any meaningful way to the entire corpus. Therefore, the trigram analysis was discarded, and instead the bigram analysis was used for further analysis. Additionally, following the steps in [82], the entities that appeared less than five times in the corpus were removed from the matrix.

3.1.3. Step 3: Calculating Term Frequency–Inverse Document Frequency (TF–IDF), SVD, and Factor Loadings

The raw t–d matrix was transformed through weighting and normalization methods. Term frequency (tf) and inverse document frequency (idf) values of each entity were calculated consecutively [79,83,84]. The tf–idf value of each entity represents how important that entity is to a corpus. The tf–idf values increase as the occurrence of the entity increase. However, in counterbalance to this, the frequency of words in the corpus help to account for the fact that some entities are more common than others [85]. The tf–idf, which was transformed from the raw t–d matrix, was subjected to SVD and decomposed to term-to-concept matrix, document-to-concept matrix, and square roots of eigenvalues (singular values), whereas strengths of concepts appear in descending order. Multiplication of term-to-concept and singular value matrixes produces the factor loadings of terms for each study. The multiplication of document-to-concept matrix and singular value matrix generates the factor loadings of documents for each research pattern. The maximum number of factors generated in this way is equal to the number of documents (i.e., 861). To assess the principal research areas and the research themes, the number of factors was adjusted by reducing the dimension of factor loading matrix.

3.1.4. Step 4: Rotation of Factor Loadings and Factor Threshold Values for Research Areas and Themes

In factor analysis, the rotation of the matrix displays the loadings of variables on factors, and simplifies the factor associations. Entities with a high loading value in factor are associated with a theme. Through rotation of factor axes, the loading of an entity on one factor is maximized while minimizing its loading on other factors. Associated high-loading terms and documents of each research pattern are clustered and matched. Varimax rotation [86] was applied to the term and document loadings. Aggregation level varies for each factor solution. In order to parse the significant and non-significant entities/documents, an appropriate threshold value was selected for each factor solution following the approach used in [9]. Documents with factor loading below the related threshold value were eliminated.

3.1.5. Step 5: Factor Evaluation and Research Theme Labeling

In each factor solution, high-loading entities and documents within the calculated threshold value were grouped with the absolute values of the loadings in descending order. High-loading entities and documents were reviewed and labeled by the authors (12 through 20, 50, and 80 factors).

The process of labeling is a subjective and iterative exercise, dependent on interpretation and on the labels selected by the researcher(s) naming a thematic cluster. Therefore, in the first round the clustered themes were labeled independently by the two authors. After the initial labeling, the labels of the two authors were compared. At this stage, nearly 25% of the labels differed between the two authors. Following a discussion, authors agreed on most of the labels, leaving only 5% for which agreement was not reached. After a third round of discussion that included three doctoral students, a final agreement was obtained for the remaining themes.

4. Results and Discussion

Table 4. Themes identified in the 12-, 20-, 50-, and 80-factor analyses.

12 Factors
Training for improving quality of outcome (F12.1)	Individual and team experience (F12.7)
Scoping and framing of ill-defined problems (F12.2)	Training and concept development (F12.8)
Interaction and experience-based (F12.3)	Quality and assessment (F12.9)
Model-driven for idea-solutions (F12.4)	Creativity and added value (F12.10)
Operations and tools (F12.5)	Reflective, participatory, and interactive approaches (F12.11)
Mastery and assessment (F12.6)	Design know-how and concept exploration (F12.12)
20 Factors
Context and fixation in ill-defined problems (F20.1)	Forecasting by making conjectures about ideas (F20.11)
Mastery and breadth of skills (F20.2)	Managing uncertainty and complexity (F20.12)
Collaboration experience (F20.3)	Training and experience (F20.13)
Interaction and solution focus (F20.4)	Design management (F20.14)
Operations and design process (F20.5)	Exploration and analysis (F20.15)
Efficiency in novel concepts and solutions (F20.6)	Idea and creativity development (F20.16)
Individual and team performance (F20.7)	Design techniques and methods (F20.17)
Co-creation of concepts (F20.8)	Concepts and knowledge-based design (F20.18)
Assessment: effectiveness and efficiency (F20.9)	Framing and assessing design (F20.19)
Idea generation process (F20.10)	Assumptions and design management (F20.20)
50 Factors
Ill-defined problem and reflective practice (F50.1)	Process and outcome: alternative solutions for ill-defined problems (F50.26)
Efficiency and mastery (F50.2)	Efficiency and learning (F50.27)
Interaction and representation (F50.3)	Decision criteria and value (F50.28);
Effective design (F50.4)	Search and reflection (F50.29)
Heuristics and techniques (F50.5)	Dealing with facts and fiction (F50.30)
Modeling tools (F50.6)	Knowledge structure and innovation (F50.31)
Team and problem context (F50.7)	Modelling process (F50.32)
Combine concepts and iterate (F50.8)	Deliberation and knowledge exchange (F50.33)
Efficiency and assessment (F50.9)	Control (F50.34)
Reflection and design strategies (F50.10)	Performance assessment strategy (F50.35)
Articulation and knowledge elicitation (F50.11)	Iterative decision making (F50.36)
Dealing with complexity (F50.12)	Team knowledge and performance (F50.37)
Sharing experiences and mentoring (F50.13)	Emergence and discovery (F50.38)
Pro-active and multi-disciplinary/interdisciplinarity (F50.14)	Contextual knowledge (F50.39)
Conjectures, investigations, and analysis (F50.15)	Analysis and review (F50.40)
Insights and assessment (F50.16)	Alternative solutions (F50.41)
Creative skills development (F50.17)	Participatory action and concept finding (F50.42)
Concept and context awareness (F50.18)	Multiple perspectives (F50.43)
Evaluating effectiveness of design process (F50.19)	Measuring user-product experience (F50.44)
Design assumptions and design strategy (F50.20)	Dealing with complexity (F50.45)
Reflection and representation (F50.21)	Systematic analysis and knowledge transfer (F50.46)
Participation and engagement (F50.22)	Representation, mastery and control (F50.47)
Methodical approach and structuring (F50.23)	Design experience and computational techniques (F50.48)
Interaction, participation and learning experience (F50.24)	Reasoning and expanding knowledge for novelty (F50.49)
Complementary perspectives (F50.25)	Suggestions for design development (F50.50)
80 Factors
Engagement (F80.1)	Develop design alternatives (F80.41)
Level of expertise (F80.2)	Conceptualization and ideation (F80.42)
Integration (F80.3)	Considering alternatives and being non-fixated (F80.43)
Knowledge structure (F80.4)	User-centered problem-solution co-evolution (F80.44)
Operation techniques and tools (F80.5)	Dealing with the unknown (F80.45)
Context and conditions (F80.6)	Systemic and concept driven (F80.46)
Team interaction (F80.7)	Communication and representation (F80.47)
Reflection and concept formation (F80.8)	Design procedures and rules (F80.48)
Efficient and effective (F80.9)	Broaden knowledge (F80.49)
Control process (F80.10)	User participation and experiential learning (F80.50)
Product modelling (F80.11)	Training and experience (F80.51)
User participation (F80.12)	Exchanging complementary knowledge (F80.52)
Training and learning (F80.13)	User awareness and interaction (F80.53)
Knowledge base (F80.14)	Teamwork (F80.54)
Review, analysis, and investigation (F80.15)	Methodology and process (F80.55)
Skills development (F80.16)	Situated and reflective practice (F80.56)
Assessment and judgement (F80.17)	Knowledge elicitation (F80.57)
Clarity on facts and fiction (F80.18)	Design reuse and strategies (F80.58)
Measurement (F80.19)	Knowledge typology and representation (F80.59)
Design conjectures (F80.20)	Dealing with complex products (F80.60)
Contextual knowledge (F80.21)	Suggestions for continuous improvement (F80.61)
Domain experience, participation, and involvement (F80.22)	Involvement and participation (F80.62)
Mental block (F80.23)	Design interaction (F80.63)
Novice and expert interaction (F80.24)	Design generation as synergistic activity (F80.64)
Efficiency and creativity (F80.25)	Capturing design intentions and requirements (F80.65)
Exhibition of expertise (F80.26)	Team collaboration and interaction (F80.66)
Predict and anticipate (F80.27);	Design attitudes and iteration (F80.67)
Inform decision making (F80.28)	Team performance (F80.68)
Discussing and share (F80.29)	Problem decomposition and process (F80.69)
Develop context awareness (F80.30)	Finding solutions for ill-defined problems (F80.70)
Structuring and formulating measures for innovation (F80.31)	Training for innovation (F80.71)
Modelling and assessment (F80.32)	Team management (F80.72)
Deliberate practice (F80.33)	Information and fact based (F80.73)
Control strategy (F80.34)	Value addition for creativity (F80.74)
Discerning and dealing with complexity (F80.35)	Experience and reflective decision process (F80.75)
Design evaluation (F80.36)	Creating new knowledge (F80.76)
Proactive process (F80.37)	Skills and knowledge for de-fixation (F80.77)
Emergent design and development process (F80.38)	Making sense of problem context (F80.78)
Goal oriented (F80.39)	Complementary perspectives (F80.79)
Exploration and conjecturing (F80.40)	Detailing and uncovering (F80.80)

lists the results of the 12-, 20-, 50-, and 80- factor analyses. The analysis at different factor levels helped to gain better insight into ‘expertise in design’ related themes at different levels of detail. As discussed previously, the labeling of the factors is a subjective process, based on an interpretation of the clustering of the terms contained within the factors. Accordingly, similarities were identified in some themes at different levels of detail.

4.1. Review of 12-Factor Analysis

This section briefly reviews each of the research areas that emerged through the 12-factor analysis. Moreover, several similarities were observed when comparing the themes identified in the 12-factor analysis and the manual literature review carried out at the beginning of the manuscript (See

Table 5. Themes identified using the Latent Semantic Analysis (LSA) technique for the 12-factor analysis and the manual literature review.

12-Factor Analysis	Manual Literature Review
F12.1 Training for improving quality of outcome
F12.2 Scoping and framing of ill-defined problems	Problem solving Design activities and design process
F12.3 Interaction and experience-based
F12.4 Model-driven for idea-solutions	Knowledge strategies
F12.5 Operations and tools	Knowledge strategies Tools and resources
F12.6 Mastery and assessment	Design performance and design approach Design activities
F12.7 Individual and teamwork experience
F12.8 Training for blending and concept development	Analogizing
F12.9 Quality and assessment	Design activities and design process
F12.10 Creativity and added value	Creativity
F12.11 Reflective, participatory, and interactive approaches	Design performance and design approach
F12.12 Design know-how and concept exploration	Domain knowledge Design activities and design process

). In some cases, the factors were found to superpose with more than one category.

4.1.1. Training for Improving Quality of the Outcome

This area combines two key aspects of expertise, ‘training’ and ‘quality’. Training not only refers to being taught or coached under supervision, but is also concerned with self-learning and practice. Whereas training sets a focus on the effort involved in the development of expertise [1], the second aspect denotes that training experts can set higher levels of benchmark in the expected quality of the outcome. Thus, training provides the means needed to improve and achieve higher quality outcomes.

4.1.2. Scoping and Framing of Ill-Defined Problems

This area is concerned with two frequently related aspects of expertise: ‘framing’ and dealing with ‘ill-defined problems’. Framing and scoping refer to the ability of experts to clearly formulate and identify what the crux of the problem is, and to focus on the most relevant issues demanding a solution, e.g., [63,64]. However, the ill-defined nature of design problems is a major concern reflecting that a potential solution space is determined by the way that a design problem is formulated. The ability of experts to define and frame ill-defined design problems contributes to arriving at better solutions. This factor is closely related to two categories identified in the traditional literature review: one is Design activities, which includes problem framing/structuring and the other is Problem solving, focusing on ill-defined problems.

4.1.3. Interaction and Experience-Based

In addition to knowledge and skills, experience is a basic condition of expertise [5] necessary for reaching successful solutions [87]. The present factor adds to these the view that the ability to build on interactions with others when solving problems, and the exposure to other disciplines, supported by the sharing and exchange of experiences are also critical aspects to enhance expertise.

4.1.4. Model-Driven for Idea-Solutions

According to literature, experts are known for their large, organized, and well-integrated structures of knowledge. These enable them to encode and represent significant domain-specific knowledge. Models, on the other hand, offer the means to articulate and represent the external world in abstract, conceptual, and often symbolic terms. Models allow to make assumptions, interpretations, and decisions at various levels of abstraction under different situations [2]. What the present factor adds to these, is that due to their well-developed knowledge structures of problem contexts, experts are driven by models that enable them to generate ideas and develop solutions. The present factor shares similarities with the category identified from the traditional literature review dealing with Knowledge strategies. In particular, the strategic knowledge used by experts when tackling a design problem, mainly the solution-driven approach [41].

4.1.5. Operationalizing and Tools

Using the right methods and tools is critical to deal with design problems. Experts are known to possess a wide repository of well-established tools, procedures, and methods [29,68]. This increases their chances of choosing adequate approaches to tackle design problems, when operationalizing ill-defined contexts. The present pattern reaffirms that tools, procedures and methods embrace techniques such as search strategies, best practices and processes—including those related to managing tasks and context, as well as external aids relevant to the problem. Similarities were found between the Operationalizing and tools factor and the categories of Knowledge strategies and Tools and resources from the traditional literature review. While the former focuses on the different strategies employed by designers for solving design problems, the latter is about the means that designers use in support of the design problem solving activity [71].

4.1.6. Mastery and Assessment

Design literature defines mastery as an outstanding level of competence, knowledge, and skills that expert designers possess [13,21]. The present pattern suggests that assessment demands expertise. Thus, master designers have high levels of breadth and depth that they employ in not only understanding a problem, but also assessing the design situation. By efficiently assessing design outcomes in the framework of major design goals, needs, and requirements, master designers are able to focus on the important aspects of the design situation. The factor was found to be related to Design performance and design approach, and to Design activities categories from the traditional literature review. Above all, Mastery has to do with superior performance [4], characterizing all the themes in the former category.

4.1.7. Individual and Teamwork Experience

Teamwork was confirmed to be a fundamental factor for acquiring design experience. Design experience entails the capability of complementing and integrating divergent ideas and perspectives, including multidisciplinary knowledge and skills. The design literature discusses experience both in terms of the individual and his or her ability to work as a team [42]. When interacting with other team members, sharing individual knowledge affects not only the design outcomes produced, but also the coordination and communication of tasks, processes, and activities [55].

4.1.8. Training for Concept Blending and Concept Development

The ability to develop and blend concepts, and build on new ideas and concepts, is known in design literature as a most critical stage of the design process [23]. Generally speaking, blending is defined as the process of mixing. In the design domain blending is concerned with the ability to connect between two domains that are remote to the design problem, which can contribute largely to idea generation and concept development. Since concept development is by no means an inborn feature of the designer, the current factor stresses that coaching designers to enhance their knowledge and skills can contribute largely to this aim. Often, training in concept blending and concept development requires experience over multiple problems and disciplinary domains, and abstraction skills. With experience, experts acquire the ability to reflect and self-train themselves in concept development. In this factor, concept blending was found to be in some way related to Analogizing, identified in the traditional literature review [43,44]. It is well-known that creating a qualitative analogy is a consequence of the distance between the design problem and the source example. Experts excel in establishing between-domain analogies, which are distant to the design problem. As such, these kinds of analogies support the development of outstanding concepts.

4.1.9. Creativity and Added Value

In domains such as design, the production of creative and worthwhile outcomes is particularly relevant. Existing literature accounts that the creativity of the designers and their ability to generate innovative outcomes were studied extensively, specifically in relation to expertise [71]. As noted before, rather than applying familiar knowledge from existing sources to deal with a problem, or the mechanical use of expected solutions, design expertise is also about creating something novel [51]. In addition to the ability to generate unfamiliar outcomes, the present pattern suggests that as expertise develops, creativity in design should be concerned with the production of solutions with added value (e.g., social, cultural, environmental values) [88]. This factor was found to be strongly related to the Creativity category observed in the traditional literature review. This suggests that the topic is at the core of design expertise.

4.1.10. Quality and Assessment

In spite of the significance of design creativity and innovation, many new products failed due to their lack of satisfactory quality. The expert ability to understand and assess attributes that can affect the quality of a design is another important theme identified in literature [89]. This includes the knowledge and procedural skills necessary to employ tools and methods [90] to enhance quality control, delivery, and assessment.

4.1.11. Reflective, Participatory, and Interactive Approaches

Expertise in design is a thoughtful activity that was seen to be associated with reflective practice, wherein designers self-assess their design process and own way of working [91,92]. Interestingly, literature mainly focuses on the reflection of the individual expert. However, reflection about what has been done and what needs to be achieved can not only be performed by an individual designer, but can also be the outcome of a participatory practice in a design team, as the present factor suggests. During participatory activities, reflective practice can be enhanced while maintaining interactions with other experts. This can contribute to a sharing of knowledge and gaining a deeper understanding of the design situation [93], and thus to the development of expertise. Consequently, this factor was found to be party related to the Design performance and approach category found in traditional literature, mainly reflection, but at the individual level.

4.1.12. Design Know-How and Concept Exploration

This theme emphasizes know-how as a critical element necessary for the development of expertise. As discussed before, experts are able to transform declarative representational knowledge (know about or know that) into procedural knowledge (know-how) [14]. Indeed, design expertise encompasses more procedural knowledge (know-how) than declarative representations (know-that) [26]. In addition to these, what the present factor indicates is that procedural knowledge is essential to not only structure the design problem, but also to guide the exploration of design concepts and ideas, while searching for alternative solutions. Accordingly, this factor showed to be strongly related to two categories from the traditional literature concerned with Domain knowledge and Design activities and design process.

The review of the themes identified in the 12-factor analysis shows congruence with the main topics from the literature review of leading interdisciplinary design journals. This is interesting considering that the analyses were carried out using two independent approaches. However, what the present study adds to the existing literature about design expertise, is that the mapping of the factors shows how themes are related across the different factors, and at various levels of detail (see

Table 6. Mapping themes from 12-factor analysis to themes in 20-, 50-, and 80-factor analyses.

F12	Themes	12+ - Mapping with 50-Factor and 80-Factor Terms
F12.1	Training for improving quality of outcome	[Training: 20.13, 50.27, 80.13, 80.16, 80.50, 80.51] [Quality of outcome: 20.6, 50.4, 80.9,80.36]
F12.2	Scoping and framing of ill-defined problems	[Scoping/framing: 20.19, 50.43, 80.31, 80.35, 80.79] [Ill-defined:20.1, 20.12, 50.1, 50.12, 50.45, 80.35, 80.60]
F12.3	Interaction and experience-based	[Interaction: 20.4, 20.8, 50.3, 50.13, 50.22, 50.33, 80.1, 80.7, 80.24, 80.29, 80.53, 80.54, 80.62, 80.63, 80.66] [Experience: 20.3, 20.13, 50.13, 50.24, 50.44, 50.48, 80.22, 80.49,80.51, 80.52, 80.75]
F12.4	Model-driven for idea-solutions	[Model-driven: 20.17, 50.6, 50.32, 80.11., 8032, 80.55]
F12.5	Operations and tools	[Operations: 20.5, 80.5] [Tools: 20.17, 50.6, 50.48, 80.5]
F12.6	Mastery and assessment	[Mastery: 20.2; 50.2; 50.47; 80.2; 80.22; 80.24; 80.26] [Assessment/evaluation /judgement: 20.9; 20.19; 50.9; 50.16; 50.19; 50.36; 80.17; 80.32; 80.36]
F12.7	Individual and team experience	[Team: 20.7; 50.7; 50.37; 80.7; 80.54; 80.66; 80.68; 80.72] [Experience: 20.3; 20.13; 80.22; 80.50; 80.51; 80.75]
F12.8	Training and concept development	[Training: 20.13; 50.13; 50.27; 80.13; 80.50; 80.51; 80.71] [Concept/idea: 20.6; 20.8; 20.10; 20.11; 20.16; 20.18; 50.8; 50.18; 80.42; 80.46] [Development: 20.16; 50.17; 50.50; 80.16; 80.30; 80.38; 80.41]
F12.9	Quality and assessment	[Assessment/evaluation/judgement: 20.9; 20.19; 50.9; 50.16; 50.19; 50.36; 80.17; 80.32; 80.36]
F12.10	Creativity and added value	[Creativity/innovation/novelty: 20.16; 50.17; 50.31; 50.49; 80.25; 80.74; 80.31; 80.71] [Value addition: 50.28; 80.74]
F12.11	Reflective, participatory, and interactive approaches	[Interaction/iterate: 20.4; 50.8; 50.3; 50.36; 50.33; 80.7; 80.24; 80.53; 80.63; 80.66; 80.67] [Reflective practice/conjectures/assumptions: 20.11; 20.20; 50.1; 50.10; 50.15; 50.21; 50.29; 80.8; 80.20; 80.56; 80.75] [Participation/involvement/engagement/collaboration: 20.3; 50.22; 80.1; 80.12; 80.50; 80.62; 80.66]
F12.12	Design know-how and concept exploration	[Knowledge/skills: 20.2; 20.18; 50.11; 50.17; 50.31; 50.33; 50.37; 50.39; 50.46; 80.4; 80.14; 80.16; 80.21; 80.45; 80.49; 80.52; 80.57; 80.59; 80.76; 80.77] [Concept/idea: 20.6; 20.8; 20.10; 20.11; 20.16; 20.18; 50.8; 50.18; 80.42; 80.46] [Search/exploration: 20.15; 50.29; 80.40]

and

Table 7. Mapping themes in 20-factor analysis to themes in 50- and 80-factor analyses.

F20	Themes	20+ - Mapping with 50-Factor and 80-Factor Terms
F20.1	Context and fixation in ill-defined problems	[Ill-defined problems/solutions: 50.1; 80.69; 80.70] [Fixation: 80.15; 80.43; 80.77] [Context: 50.7; 50.18; 80.6; 80.21; 80.30; 80.78]
F20.2	Mastery and breadth of skills	[Mastery: 50.2; 50.47; 80.2; 80.22; 80.24; 80.26] [Knowledge/skills: 50.17; 80.4; 80.14; 80.16; 80.21; 80.45; 80.49; 80.52; 80.57; 80.59; 80.76; 80.77]
F20.3	Collaboration experience	[Participation/involvement/engagement/collaboration: 50.22; 80.1; 80.12; 80.50; 80.62; 80.66] [Experience:80.22; 80.50; 80.51; 80.75; 80.66]
F20.4	Interaction and solution focus	[Interaction/Iterate: 50.8; 50.3; 50.33; 50.36; 80.7; 80.24; 80.53; 80.63; 80.66; 80.67] [Solution/alternative: 50.41; 80.41; 80.43; 80.44; 80.70]
F20.5	Operations and design process	[Operations, techniques, methods, and tools: 50.5; 50.6; 50.10; 50.23; 50.35; 50.20; 50.48; 80.5; 80.34; 80.58; 80.55] [Design process: 50.19; 50.26]
F20.6	Efficiency in novel concepts and solutions	[Concept/idea: 50.8; 50.18; 50.42; 80.8; 80.42; 80.46] [Solution/alternative: 50.41; 80.41; 80.43; 80.44;80.70] [Creativity/innovation/novelty: 50.49; 80.25; 80.74; 80.31; 80.71] [Effective/efficient: 50.2; 50.4; 50.9; 50.19; 50.27; 80.47; 80.59; 80.9; 80.25]
F20.7	Individual and team performance	[Performance: 50.19;50.35;50.37; 80.19;80.31;80.68] [Team: 50.7; 50.37; 80.7; 80.54; 80.66; 80.68; 80.72]
F20.8	Co-creation of concepts	[Co-creation: 50.13; 50.14; 50.22; 50.42] [Concept/idea: 50.8; 50.18; 80.8; 80.42; 80.46]
F20.9	Assessment: effectiveness and efficiency	[Assessment/evaluation/judgement: 50.9; 50.16; 50.19; 50.36; 80.17; 80.32; 80.36] [Effective/efficient: 50.2; 50.4; 50.9; 50.19; 50.27; 80.47; 80.59; 80.9; 80.25]
F20.10	Idea generation process	[Concept/idea: 50.8; 50.18; 50.42; 80.8; 80.42; 80.46] [Generation: 80.44; 80.64]
F20.11	Forecasting by making conjectures about ideas	[Concept/idea: 50.8; 50.18; 50.42; 80.8; 80.42; 80.46] [Reflective practice/conjectures: 50.1; 50.10; 50.15; 50.21; 50.29; 80.8; 80.20; 80.56; 80.75]
F20.12	Managing uncertainty and complexity	[Complexity: 50.12; 50.35; 80.35; 80.60]
F20.13	Training and experience	[Training: 50.13; 50.27; 80.50; 80.13; 80.51; 80.71] [Experience: 80.22; 80.50; 80.51; 80.75]
F20.14	Design management	[Management: 80.72]
F20.15	Exploration and analysis	[Search/exploration: 50.29; 80.40] [Analysis: 50.15; 50.40; 50.46; 80.15]
F20.16	Idea and creativity development	[Concept/idea: 50.8; 50.18; 50.42; 80.8; 80.42; 80.46] [Creativity/innovation/novelty: 50.49; 80.25; 80.74; 80.31; 80.71] [Development: 50.17; 50.50; 80.16; 80.30; 80.38; 80.41]
F20.17	Design techniques and methods	[Operations, techniques, methods, and tools: 50.5; 50.6; 50.10; 50.23; 50.35; 50.20; 50.48; 80.5; 80.34; 80.58; 80.55]
F20.18	Concepts and knowledge-based design	[Concept/idea: 50.8; 50.18; 50.42; 80.8; 80.42; 80.46] [Knowledge/skills: 50.11; 50.17; 50.31; 50.33; 50.37; 50.39; 50.46; 80.4; 80.14; 80.16; 80.21; 80.45; 80.49; 80.52; 80.57; 80.59; 80.76; 80.77]
F20.19	Framing and assessing design	[Framing/structure: 50.23; 50.31;80.40] [Assessment/evaluation/judgement: 50.9; 50.16; 50.19; 50.36; 80.17; 80.32; 80.36]
F20.20	Assumptions and design management	[Reflective practice/conjectures/assumptions: 50.1; 50.10; 50.15; 50.20; 50.21; 50.29; 80.8; 80.20; 80.56; 80.75] [Management: 80.72]

. For example, it can be observed that whereas some themes emerged at the 50- and 80-factor analyses respectively, they were absent in the 12- and 20-factor analyses. Thus, the factor analyses helped to understand the relative emergence of a theme in relation to the others. This should be considered an important contribution to the design expertise field, and to design education in particular, as is discussed in the following section.

In this regard, the mapping of a theme can be used to assess its breadth and dominance across the different factor analyses, as is covered in the design literature. Thus, themes identified in factors with low level of detail or that appeared in two or more consecutive factors, can be seen as having higher significance for design expertise than those that emerge once in factors with a high level of detail. For example, F12.12 ‘Design know-how and concept exploration’ is a comprehensive theme covering notions such as knowledge, skills, and concepts and ideas, which are frequently referred to in the design expertise literature. This was reflected in the mapping established between this theme at the 12-factor analysis, and other themes identified across the 20-, 50-, and 80-factor analyses. Another example is F12.11 dealing with ‘Reflective, participatory, and interactive approaches’. Since design is a practice-oriented discipline, several topics in the design expertise literature referred to the conduct or practices of the designers, as identified in theme F12.11. In contrast, other themes showed to be narrower and more specific. For example, F12.4 ‘Model based idea generation’, F12.5 ‘Operations and tools’, and F12.9 ‘Quality and assessment’ had fewer mapping associations with the list of themes identified across the 20-, 50-, and 80-factor analyses. It is also important to note that themes emerging at certain level of analysis were not mutually exclusive, nor was the mapping across the different levels of analysis. That certain themes can appear more than one time at a certain level, as well as across several levels of analysis, reflected the subjectivity of the terms and concepts in the research on expertise in design.

The mapping of themes in 20-factor analysis with those in 50- and 80-factor analyses also revealed differences in breadth for each theme, as reflected in the design literature. For instance, themes such as 20.16 ‘Idea and creativity development’ are rather broad and largely covered in design expertise literature. In contrast, themes such as F20.14 ‘Design Management’ and F20.12 ‘Managing uncertainty and complexity’ are more specific, and tend to be infrequent in the design literature. A reason that themes like these were not observed in factors with higher level abstractions (e.g., 12-factor analysis) is because they are not considered to be central to design expertise. Although the use of LSA in the present study provided preliminary insights into the main themes and patterns that emerged from existing literature, further qualitative research and expert feedback may reveal additional reasons for the observed themes and patterns.

5. Implications for Design Training and Design Education

With ‘design thinking’ gaining wider recognition as a mainstream ‘critical thinking’ approach across various industry sectors, there is an increasing demand for multi-disciplinary design knowledge, skills, and education. Thus, a better understanding of the patterns and trends associated with design expertise will be beneficial for students and teachers interested in design training and education. To this aim, findings from the present study revealed thematic patterns and factors associated with design expertise. Some of the identified themes and factors relate to what characterizes design expertise, whereas others are concerned with how design expertise is acquired. These factors are domain independent, and can be combined and applied to various disciplines where design education is encouraged.

A main contribution of this paper was to present a structured assessment that allowed identification of these factors at different levels of detail, ranging from 12 to 80 factors. The different levels of abstraction offer a convenient reference that can be used in support of pedagogical goals, and for planning training. For instance, general categories of 12–20 factors may be relevant for curriculum development tasks, and for intervention programs to define teaching objectives in design related courses. On the other hand, more detailed categories of 50–80 factors might be suitable for dealing with specific design problem-solving tasks.

Not only factors, but also the connections between factors and related themes can be used as guidelines and checklists for intervention programs in design training and education. It is likely that experienced design educators, expert practitioners and mentors, and well-planned design training programs already account for some or most of the identified factors. Even so, the present findings can be used as a general “route map” or “check-lists” to identify opportunities for improvement in educational environments such as the design studio.

Future Work

Further analysis of the main themes and contents in design expertise literature focusing on the chronology of studies, and the disciplinary background of key authors, will be carried out in a future study. It is possible that the journals considered in the literature review largely focus on design thinking. Additional studies should consider including other design sources that have an interest in the social context as well. An international survey on the perception of design expertise among students, academics, and professionals is currently underway, and the findings will be compared to a literature review. Such investigation will provide additional insights into the evolving pattern of major topics in design expertise through time, as well as into implications for design education.

Findings from this research, in particular the connections and links between factors and related themes, can be considered as a basis for developing an ontological framework aimed at representing ‘Design Expertise’ knowledge in information systems. Such a semantic representation—e.g., using the Resource Description Framework (RDF) schema [94]—can be useful for enhancing information management, as well as search and navigation of content related to design expertise. In order to implement the RDF representation, further iterations on the different labels and cross-factor linking will be required, including labeling of the relationships between the factors. A preliminary attempt was made to build an ontological framework of the ‘DesignExpertiseFactors’ using an RDF representation in Protégé, which is a work in progress. An XML output of the ‘DesignExpertiseFact’ can be accessed in [95].

6. Conclusions

With the increasing significance of design across different disciplines, there is a need to understand what the characteristics that define design expertise are, and the conditions that may foster the acquisition of design expertise. However, the broad corpus of existing literature makes its objective review an ambitious enterprise, particularly when it comes as an exhaustive manual process. It is often the case that the manual review approach is subjective and limiting in regard to the number of articles that can be covered, which are not always representative of the topic under consideration. Therefore, the main goals of this study were to understand what the major topics and trends related with design expertise in the current literature are, and to explore whether and how these can be organized and categorized into main thematic patterns.

This paper presented a systemic analysis and assessment of the thematic patterns in design expertise literature, primarily focusing on sources that publish articles from different design disciplines. This allowed gaining a comprehensive insight into design expertise with regard to main trends and topics. Congruence with the traditional literature review was found, suggesting that LSA can be considered as a promising complementary supporting tool. On the other hand, the analysis carried out by LSA also showed some differences with respect to the emerging categories. The patterns that turned up through the factor analysis indicated that the design expertise literature not only covers themes related to cognitive processes and cognitive studies on design expertise, but also managerial aspects of design expertise. That is, besides design problem solving skills and cognitive processes, other issues such as interaction, training, operationalization of tools and methods, quality, and assessment also showed to be concerned with design expertise. All these have important implications for design training and education.