Viability in Multiplex Lexical Networks and Machine Learning Characterizes Human Creativity

Abstract

Previous studies have shown how individual differences in creativity relate to differences in the structure of semantic memory. However, the latter is only one aspect of the whole mental lexicon, a repository of conceptual knowledge that is considered to simultaneously include multiple types of conceptual similarities. In the current study, we apply a multiplex network approach to compute a representation of the mental lexicon combining semantics and phonology and examine how it relates to individual differences in creativity. This multiplex combination of 150,000 phonological and semantic associations identifies a core of words in the mental lexicon known as viable cluster, a kernel containing simpler to parse, more general, concrete words acquired early during language learning. We focus on low (N = 47) and high (N = 47) creative individuals’ performance in generating animal names during a semantic fluency task. We model this performance as the outcome of a mental navigation on the multiplex lexical network, going within, outside, and in-between the viable cluster. We find that low and high creative individuals differ substantially in their access to the viable cluster during the semantic fluency task. Higher creative individuals tend to access the viable cluster less frequently, with a lower uncertainty/entropy, reaching out to more peripheral words and covering longer multiplex network distances between concepts in comparison to lower creative individuals. We use these differences for constructing a machine learning classifier of creativity levels, which leads to an accuracy of $65.0\pm 0.9\%$ and an area under the curve of $68.0\pm 0.8\%$, which are both higher than the random expectation of 50%. These results highlight the potential relevance of combining psycholinguistic measures with multiplex network models of the mental lexicon for modelling mental navigation and, consequently, classifying people automatically according to their creativity levels.

1. Introduction

The creative process—generating novel and useful ideas—has been shown to involve participant’s search processes to “move away” from prototypical ideas [1] and has been related to individual differences in semantic memory structure and executive processes that guide such search processes [2,3,4,5,6,7]. The associative theory of creativity [6] argues that creativity involves the connection of weakly related, remote concepts into novel and applicable concepts. Furthermore, this theory argues that low and high creative individuals differ in their structure of semantic memory, where high creative individuals have a structure that facilitates such a process [6]. However, this theory has been challenging to investigate due to the complexity of modelling and representing semantic memory and the search processes that operate over it, which would allow examination of this theory [8].

In the current study we apply a computational multiplex network analysis [9,10] to examine how low and high creative individuals search through their memory to retrieve animal names. We outline one such approach, based on the application of network science methodologies [11]. Network science is based on mathematical graph theory and allows quantification of complex systems as networks. The application of network science to study cognitive phenomena is steadily developing, providing quantitative means to study the structure and dynamics of cognitive systems [11] and the interplay between cognition and language in terms of modelling word learning [12], picture naming [13], semantic relatedness [14], stance detection [15], and personality traits [16].

1.1. Previous Research: Assessing Creativity with Cognitive Networks

1.1.1. Semantic Networks Capture Knowledge Structure and Search as Related to Creativity

Recently, computational methods to study knowledge and memory structure in creativity are paving the way to uniquely examine their role in the creative process [17,18,19,20] and examine the associative theory of creativity [5,6].

Several studies have applied network science methodologies to investigate how individual differences in creativity relate to differences in their semantic memory [21,22,23]. These investigations have consistently shown that the semantic memory structure of higher creative individuals is more connected and flexible than lower creative individuals [4].

Other studies explored how models of simple search processes implemented on semantic spaces. Such approaches examined how these search processes can capture memory retrieval [24,25,26] and performance in creative tasks which require cognitive search [27,28,29]. In regard to individual differences in creativity, Kenett and Austerweil [30] examined how the semantic memory structure of low and high creative individuals constrain a simulated model of a simple search process (random walk). The authors hypothesized that the structure of the semantic network of high creative individuals enables them to use simple search processes that reach further and weaker connected concepts, than low creative individuals. In line with the associative theory of creativity, the authors found that random walks over the semantic network of high creative individuals visits more unique and weaker nodes. Finally, a recent study proposed a method to quantify the semantic distance between associative responses generated by participants in a chained free association task [31]. The authors found that high creative individuals reach farther distances than low creative individuals, providing empirical support to the findings of [30].

Thus, applications of network science to study creativity have demonstrated their strength in quantitatively assessing the role of semantic memory and how people search through it in relation to creative thinking.

1.1.2. Studying Cognitive Search with the Semantic Fluency Task

The associative theory of creativity [6] highlights the role of cognitive search in the creative process. As described above, network science research has highlighted the role of memory structure in creativity and are slowly moving towards studying search processes operating over semantic networks. A parallel line of research applies computational methods to more generally examine how people search through their memory [24,32,33,34,35]. The general approach of these studies is representing semantic memory as a network or space, a representation that allows the simulation of search processes and predictions about search behavior.

These studies have mostly focused on modelling how people perform during a semantic fluency task. In a semantic fluency task, participants are required to retrieve from memory as many category members they can think of to a certain category (e.g., animals) in a fixed amount of time [36,37]. Previous studies have analyzed such semantic fluency data to infer the structure of semantic category networks [23,38,39,40]. For example, Kenett et al. [23] examined the relationships between semantic memory structure, creative ability, and intelligence in a large sample of individuals. All participants completed the animal category semantic fluency task. Next, participants were divided according to two dimensions—low/high creativity and low/high intelligence, finally, the animal category networks of these four groups were estimated and compared. The authors found that creative ability and intelligence were associated with different structural aspects of the animal category semantic network as estimated from the verbal fluency task [23].

The sequences that individuals produce in a semantic fluency task can also be used to study the search processes involved when individuals search through their semantic memory. For example, classic findings that retrieved category members with high semantic relatedness and shared features tend to cluster together and that participant’s “jump” between such categories [41] was proposed to be related to an active search process that dynamically switches between retrieval cues [33,35,42].

1.2. Beyond Semantics: A Multiplex Approach to Study the Mental Lexicon

However, all the above studies focus only on one type of semantic relations, either thematic or taxonomic [33]. Instead, increasingly more studies report empirical evidence that multiple types of relations between concepts, such as phonological similarities in how words are pronounced [43,44,45] or meaning in overlap through synonyms [46] or even hierarchical relationships of conceptual generalizations (hypernyms/hyponyms) [47,48], all play a role in how people search, successfully and unsuccessfully, through their memory. In particular, research on failed retrieval of words from memory such as the tip-of-the-tongue phenomenon [43] in healthy participants (a failed recall where only some features from the target word are identified, hence the name) and mistakes in picture naming in patients with aphasia [13,48], highlight an interplay between semantic and phonological similarities among concepts in influencing lexical access and recall that cannot be accounted for in single-layer network models of the mental lexicon structure. Hence, the need for exploring new modelling approaches that can analyze the mental lexicon as a multilayer network. More information about the lexical structure can indeed be obtained by accounting, simultaneously, for multiple types of word–word interactions. The above multidimensional nature of conceptual similarities in the mental lexicon can be computationally investigated through a multiplex network approach. Multiplex networks represent a natural and suitable framework for this purpose, as they simultaneously encode multiple types of interaction among units of a complex networked system [49]. Therefore, multiplex lexical networks can be used to extract information about linguistic structures beyond information available from single-layer network analysis [9,50,51,52,53].

Stella et al. [9,10] combined semantic and phonological relations between words, represented by free associations, co-occurrences, semantic feature sharing, and phonological similarities within a unique multiplex representation. The authors showed that the multiplex combination of different aspects of semantics and phonology boosted the lure of associates, a phenomenon where children tend to acquire new words that are similar to ones they already know, thus making the multiplex lexical representation of words in the mental lexicon a suitable tool for predicting early word acquisition [9,50,52]. The multiplex interplay between semantic, syntactic and phonological similarities of words also highlighted phonological priming effects [53] and critical differences in picture naming performance in patients with aphasia [51], a clinical condition affecting language understanding and production. Furthermore, examining the structure of this multiplex lexicon revealed a core of words that are highly connected to each other [10].

This core, identified as the largest viable cluster (LVC), has been found to be made up of words that are more frequent, learned earlier, more concrete, shorter in length, and are more easily identified in a lexical decision task, compared to words that are “outside” of the LVC [10]. On a multiplex network there can be several ways of connecting two nodes, either by using links only from one layer or by mixing links from different layers. Viability relies on paths of the first type, i.e., paths made by links all located in one multiplex layer. By definition, a viable cluster is a group of nodes interconnected with each other at the same time across every individual layer in a multiplex network [49]. For instance, as reported in Figure 1, in the highlighted viable cluster all nodes are connected on the red layer and, at the same time, also on the blue layer. This makes a viable cluster equal to or even smaller than the intersection of groups of connected nodes in each network layer. Furthermore, viability translates into the existence of several paths, each one living on a single multiplex layer but all connecting the same couple of nodes in the viable cluster.

In our multiplex lexical network, all words in the LVC are connected with each other by paths using either only free associations, synonyms, generalizations, or phonological similarities. Such simultaneous connectedness across phonology and semantic aspects of the mental lexicon would intuitively make these words of relevance for cognitive processes influenced by the multiplex structure of the mental lexicon. For instance, Stella and colleagues [10] showed that words in the LVC are quicker to be identified in a lexical decision task compared to words outside of the LVC. We will test the role played by words in the LVC for assessing creativity levels in the current investigation.

1.3. Current Research: Outlook and Aims

In the current study we re-analyze data collected by Kenett et al. [23] to examine whether a multiplex representation of the mental lexicon, and the notion of the core LVC, sheds light on how lower and higher creative individuals perform in a semantic fluency task—i.e., the generation of a sequence of category members of a certain category (e.g., animals). The main contribution of this work is investigating how a multiplex lexical network can unveil additional features of cognitive search that dissociate low and high creative individuals. These features are then used and tested to automatically assess, through machine learning, an individual’s creativity level.

In line with previous studies [5], we predict to find differences in how these two groups rely on the LVC in retrieving animal category members. Furthermore, to the best of our knowledge, we examine for the first time how a machine learning classifier performs in predicting an individual’s creativity level. This prediction is based on their performance and on how participants access and exploit connections in the core LVC of the mental lexicon during their fluency tasks.

2. Materials and Methods

2.1. Participants

In this study, we re-analyzed data collected by Kenett et al. [23] in that study, the authors analyzed data from 182 participants (153 female, 29 male; mean age = 19, SD = 2.65). All participants provided written informed consent. The study was performed in accordance with the guidelines and regulations of the University of North Carolina at Greensboro’s Institutional Review Board, who approved the study (see Kenett et al. [23] for more detail).

2.2. Behavioral Tasks

All participants were assessed for their intelligence and creativity levels, and then divided into low/high intelligence/creativity groups. Participant intelligence levels were assessed via a battery of fluid intelligence tasks that included (1) an abbreviated version of the Ravens Advanced Progressive Matrices [54]; (2) a paper folding task, which asks people to determine the final state of a piece of paper that has been folded, punched with holes, and unfolded [55]; (3) a letter sets task, which presents a series of four-letter combinations and requires people to determine which set does not follow a rule governing the other four [55]; and (4) a number series task, in which participants complete a sequence of numbers by discovering a guiding rule [56]. Principal component analysis was applied to compute for each participant a compiled Gf score [23]. Participant creativity levels were assessed via a subjective creativity questionnaire, the Creative Achievement Questionnaire [57], that measures real-world creative accomplishments in 10 domains: visual arts, music, dance, architectural design, creative writing, humor, inventions, scientific discovery, theater/film, and culinary arts. Most participants receive low CAQ scores, resulting in a skewed distribution of sample results [58]. To compute a general CAQ score, we used a log-transform on the sum of all CAQ sub-domain scores.

Participants were than sorted into low/high intelligence/creativity groups based on median splits across these two dimensions. While the groups did not significantly differ in age and vocabulary knowledge [55], they significantly differed with respect to the low/high classification of the two independent variables (Gf/CAQ). In the current study we only analyze the two extreme groups—low Gf/low creativity and high Gf/high creativity, to maximize differences across groups.

Finally, all participants completed the animal category semantic verbal fluency task. According to standard procedure [36], participants had 60 s to generate as many animal category members they could think of. For each participant, repetitions and non-category members were excluded from final analysis.

Participants produced in total 252 unique responses. $80.5\%$ of these responses were contained in the multiplex lexical network. Words that were not present in the multiplex lexical network represent either highly specific types of animal such as “tyrannosaurus”, “meerkat”, etc. or composite names such as “sugar glider”, “mountain lion”, etc. When counting the number of responses in participant’s lists, we included these specific animal responses. In other network measures normalized by response length, we excluded specific animal responses and counted only words present in the multiplex lexical network, in order not to alter the nature and specificity of the fluency data originally provided by participants. Of the 203 animal responses present in the multiplex lexical network, only $11.3\%$ were in the language kernel represented by the LVC of the multiplex network.

2.3. Construction of the Multiplex Lexical Network

We represent word–word semantic and phonological similarities as a multiplex lexical network [9,10,51,53], where concepts are represented by nodes and are linked according to:

• Free associations [59,60,61], indicating empirical conceptual associations elicited by participants to a cognitive task (e.g., reading “bed” elicited the concept “sleep” x times). The data for this layer was gathered from the Small World of Words project by De Deyne and colleagues [59]. Only links elicited more than $x\ge 11$ times were considered, in order for the association layer to feature the same link density of other multiplex layers. This layer is treated as undirected, as in previous approaches in cognitive network science [14,51].
• Synonyms [62], indicating overlap in meaning between concepts (e.g., “character” can mean also “font”). This layer is undirected by definition.
• Generalizations [62], representing which words are a special/more general type of concepts (e.g., “hawk” is a type of “bird”). These relationships are treated as undirected, as in previous approaches in cognitive network science.
• Phonological similarities [63,64], connecting words whose phonological transcription differed for the substitution/deletion/addition of one phoneme (e.g., “cat” sounds similar to “can”).

The synonyms, generalizations and phonological similarities were all obtained from the WordData[] repository curated by WolframResearch, Champaign, IL, US, available through Mathematica 11.3. Notice that the WordData[] dataset is based on WordNet 3.0 [65], a dictionary including synsets and additional information about word–word similarities as computed from English dictionaries (for additional references we refer the reader to [10]).

The resulting multiplex network included aspects of the mental lexicon such as phoneme overlap, meaning sharing, hierarchical generalizations, and semantic memory patterns, which were all reported to influence language processing and acquisition in both healthy [11,14,44,61,62] and clinical [46,48,51] populations. Based on recent results demonstrating how semantic memory structure relates to individual differences in creativity [20], we assume that the multilayer, networked representation of the mental lexicon serves as a valid approach to assess differences across our two levels of creativity groups. In comparison to previous works [10,51,53], the multiplex lexical network used here for the first time is $200\%$ larger, as it includes 15,886 English concepts and almost 150,000 conceptual associations.

2.4. Multiplex Lexical Networks and Viable Clusters

The combination of semantic and phonological similarities from the mental lexicon in the same multiplex network gives rise to a largest viable cluster that acts also as a multiplex core, cfr. [10]. Viability is a generalization of the definition of connected component from single- to multiplex networks [49]. In a single-layer network, a set of nodes is a connected component if it is possible to connect through a sequence of links any two nodes in the set. In a multiplex network, a set of nodes is a viable cluster if it is possible to connect any two nodes in the cluster by using only links of a given layer and this holds for all layers.

Despite the multiplex network used here including twice as many words (16,000 ca.) compared to the ones in previous studies [10,51,53], the resulting network structure features a LVC of 1030 nodes, which is analogous in size to the 1173-words LVC reported previously. Interestingly, doubling the size of the words included in the multiplex network does not alter the core of this networked representation of the mental lexicon, indicating that LVC size is robust even to drastic changes in the increase of the considered multiplex lexical network.

Notice that the LVC identifies a multiplex core, i.e., a cluster of words that tend to preferentially connect with each other than with the rest of the network (that is usually called network periphery). Labelling with $p_1,p_2,p_3$ the link densities within the core, between the core and the periphery and within the periphery, even on the current larger dataset we find that $p_1>p_2>p_3$ consistently across all multiplex network layers, analogously to [10]. Such inequality of link densities characterizes a marked core-periphery structure [66] across all semantic and phonological network layers. It is important to underline that in this case the network core is defined by the whole multiplex structure: The LVC is determined by the combination of semantics and phonology together. If only individual layers were to be considered, viability would reduce to connected components and lose the characterization of core/periphery structure. For instance, in the layer of free associations, the connected component would include almost all words and there would be no observable differences in terms of psycholinguistic features between viable/connected and non-viable/disconnected words. These differences in psycholinguistics features are, instead, quite well marked when words are partitioned into being within or outside of the LVC, cfr. [10].

Location equivalence tests indicated that words within the LVC had a higher median frequency, were learned earlier, were identified in a shorter median reaction time in a lexical identification task, were more concrete and had more meanings than words outside (cfr.

Table 1. Median psycholinguistic features of words within and outside the LVC and location equivalence test results (KW stands for Kruskal-Wallis). The LVC is richer in easier to parse/learn, concrete, shorter words than the rest of the multiplex network.

Psycholinguistic Feature	Median in LVC	Median Outside the LVC	Test Statistics
Word Length	4	7	KW, 1546, $p<{10}^{-10}$
Log Frequency	7.71	5.66	KW, 957, $p<{10}^{-10}$
Age of Acquisition	6.22 yrs	8.95 yrs	KW, 720, $p<{10}^{-10}$
Concreteness	3.97	3.26	KW, 293, $p<{10}^{-8}$
Reaction Time	552 s	605 s	KW, 560, $p<{10}^{-10}$
Number of Meanings	7	2	KW, 1560, $p<{10}^{-10}$

for the statistics and median values). These results indicate that the LVC is made of “special” words, easier to identify, more common in language, and less abstract. Notice that the results reported here agree with previous findings on a different multiplex lexical network [10].

2.5. Mental Navigation Modelling of Fluency Data over Multiplex Lexical Networks

Let us consider a fluency list $l=l_1,l_2,\cdots ,l_N$. Let us indicate cardinality with $|.|$, so that for instance the number of responses can be indicated with $\left|l\right|=N$. We can also count the fraction of nodes in the LVC, namely:

$$\frac{|l\cap LVC|}{\left|l\right|}.$$

(1)

We denote with $d_{ij}$ the network distance between nodes i and j over the whole multiplex lexical network, i.e., the smallest amount of links connecting any pair of nodes i and j. Also, let $p_{ij}=(i,k),(k,m),\cdots ,(l,j)=(i,m,\cdots ,l,j)$ be the path between nodes i and j going in order through nodes $(i,m,\cdots ,l,j)$. Let us denote path length with $|p_{i,j}|$. For simple graphs, shortest path length is equivalent to the number of visited nodes minus one, as there are no repetitions in terms of edges or vertices. The number of visited nodes during a given path is also called coverage. Building upon previous results [23,38,39,40], we consider the assembly of a fluency list a recall process navigating the structure of the mental lexicon by means of shortest network paths, so that the recall itself can be influenced by network distance.

In a given list, a couple of consecutive responses $l_i,l_{i+1}$ identifies a path of length $d_{i,i+1}$ (i.e., made of $d_{i,i+1}$ links) and coverage $|d_{i,i+1}|+1$ (i.e., visiting $c_i=\left|d_{i,i+1}\right|+1$ nodes). The collection of all these shortest paths represents a walk or sequence of associated concepts, which are explored sequentially in the mental lexicon and might be inside or outside of the largest viable cluster. We call this collection of paths collective walk. We measure the collective walk of a fluency list because we assume lexical recall is influenced mainly by shortest paths in the networked structure of the mental lexicon, as supported by previous studies in single-layer [13,23,38,39,40] and multiplex [13] lexical networks. Furthermore, considering the sequences of consecutive response $l_i,l_{i+1}$ for $i=1,\cdots ,N-1$ we can consider the resulting sequence $c_i$ of coverage induced by the shortest network paths on the multiplex lexical network. We can describe this sequence by considering its average, i.e., the average coverage per response:

$$\sum _{i=1}^{N-1}\frac{c_i}{N}=\sum _{i=1}^{N-1}\frac{d_{i,i+1}+1}{N}.$$

(2)

Rather than counting all nodes in the multiplex network being visited by each shortest path, we can restrict the count to only those nodes/words being part of the LVC, thus obtaining the average LVC coverage per response.

By always considering the sequence of responses as an ordered list, we can define an entropy measure for the appearance of lexical items within the LVC or outside. If we denote with $h(.)$ the base e information entropy:

$$h\left(X\right)=-\sum _s{p}_{s}log\left(p_s\right)$$

(3)

where s are the different outcomes of the stochastic variable X, then in our case $h\left(l\right)$ would be relative to identifying how close to uniform randomness (with entropy $log\left(2\right)$) is a given sequence of lexical items being within the LVC (state $s=0$) or outside the LVC (state $s=1$). In this way, the entropy h, also called Shannon entropy, can be considered to be a measure of uncertainty in the way the cognitive navigation of the mental lexicon produces responses. Notice that in case all responses were inside (outside) of the LVC, then the entropy would be 0, as it can be easily verified from the definition. In this study, we measured both the entropy of responses in the LVC (considering l as a list of states) and also the entropy of conceptual items in the LVC visited during the sequences of shortest paths (considering the list obtained by joining consecutively $p_{1,2},p_{2,3}\cdots ,p_{N-1,N}$ as a list of states, considering the visited nodes rather than the visited links and avoiding repetitions between the ending node of a path and the starting node of the next path).

Beyond the quantification of how the LVC is accessed by a given fluency list, we can also identify for how long the mental search building up a given fluency list remains within the LVC. We define “permanence” as the number of nodes in the LVC visited consecutively during a collective walk. The maximum (i.e., longest) permanence and its median are both features we test in our analysis.

We consider network distance between responses and the concept “animal” in the multiplex network, which defines the fluency task (for the animal category). For any given fluency list, we measure the shortest paths connecting each response to the node “animal” in the multiplex lexical network. On the ensemble of these paths we measure: (i) the coverage, as in the number of visited nodes, per response, (ii) how many times these paths involve nodes from the LVC or the accesses to the LVC, and (iii) the graph distance entropy of the related network distances between the node “animal” and every response in the multiplex network. These measurements quantify the average distance covered by the mental search process, and the access to the LVC and its variance, respectively.

We test additional features of fluency lists such as the location of the first response (inside or outside of the LVC) and the presence of misspelled words. Misspellings are based on the SpellingCorrectionList function implemented in Mathematica 11.3.

All the above measurements are reported in

Table 2. Table of tested numerosity and network features of the fluency lists. Example values are reported for the mock fluency list $l=l_1,l_2,\cdots ,l_N=${“dog”, “cat”, “bird”, “giraffe”, “monkey”, “zebra”, “hedgehog”,“donkey”, “elephant”, “tiger”, “bear”, “snake”, “lizard”, “hippo”}.

Name	Definition	Example Value
Number of responses	Number of responses in the list.	14
Number of repeated words	Number of words repeated at least once.	0
Number of all repetitions	Total number of repetitions of all repeated words.	0
Coverage per response	Average number of visited nodes in the multiplex shortest paths from one response to the next one.	16/7
Fraction of responses in LVC	Fraction of words in the list being part of the LVC.	2/7
LVC Coverage per response	Average number of visited nodes being part of the LVC in the multiplex shortest paths from one response to the next one (collective walk).	9/32
Entropy of LVC Coverage	Entropy of the collective walk $w_{iN}$, including nodes not in l but in the multiplex lexical network and being inside or outside the LVC	0.621
Entropy of LVC Responses	Entropy of nodes inside/outside the LVC as contained in the list l	0.598
Maximum Permanence in LVC	Maximum number of visited nodes in the collective walk $w_{iN}$ being consecutively in the LVC	15
Median Permanence in LVC	Median number of nodes in all the visits to the LVC during the collective walk $w_{iN}$	3
Coverage from “animal” per response	Average number of visited nodes in the multiplex shortest path between a response and “animal”	37/14
Accesses to LVC from “animal”	Average number of visited nodes in the LVC in the multiplex shortest path between a response and “animal”	6/37
Graph distance entropy from “animal”	Graph distance entropy of all multiplex shortest paths between responses and “animal”	1.0582
Start in the LVC?	Flag for the first response being in the LVC	True
Contains typos?	Flag for any response containing mistakes or typos	False

. All these measurements were tested across the two groups of highly and lowly creative individuals. No statistically significant differences were found between the two groups ($\alpha =0.05$) except for number of responses, coverage per response, fraction of responses in LVC, accesses to LVC from “animal” and entropy of LVC responses, which are discussed in the main text.

2.6. Machine Learning Classification of Low and High Creativity Individuals

We implemented a machine learning classifier using the features that were significantly different between the two groups of low and high creativity individuals.

Each group has 47 fluency lists, which we label as “low” and “high”. We tackled this balanced classification problem by separating the data into a training and a validation set with a 50/50 split. A Monte Carlo cross-validation was performed assigning each list to training or validation uniformly at random. In this way, 1000 random splits were obtained. For each partition, a logistic regression classifier was trained on the data. Logistic regression was chosen over other methods such as random forests and support vector machines because it provided slightly higher classification accuracy. The trained classifier was then used to predict “low” and “high” labels in the test set. The accuracy and area under the receiver operating characteristics curve (AUC) [67] were registered for every partition trained and tested in this way.

A flowchart summarizing the adopted methodology and the training of the machine learning classifier is reported in Figure 2.

3. Results

We examine how the presence of a viable cluster in the mental lexicon differentiates lower and higher creative individuals and how a machine learning classifier based on network features and behavioral performance predicts classification of participants into the low and high creative groups. Low and high creative groups (both groups N = 47) were based on participants performance on a battery of tasks measuring fluid intelligence and a subjective questionnaire measuring creative achievements (see Methods; [23]). First, we computed the LVC (see Methods). Next, we classified words in the multiplex as words that are inside the LVC (LVC-in) and words that are outside of the LVC (LVC-out). Finally, we examined how both groups differ in how they rely on the LVC and how a machine learning classifier can be used to classify participants into their creativity groups.

3.1. Distinct Features of Multiplex Core Relate to Low/High Creativity Levels

Provided that the LVC is richer in easier to learn, concrete, shorter words than the rest of the multiplex network (see Methods), we examine whether lower and higher creative individuals differ in the way of generating fluency lists with concepts inside or outside the LVC. In this semantic fluency task, participants are required to name as many animals as they can in one minute (see Methods). In a given list of fluency responses we measure both performance related measures (e.g., number of responses) and network features (e.g., fraction of responses in the LVC) and compare these features across the two groups.

First, we examined the medians m of the tested fluency measures for the two groups, namely higher creative individuals ($m_H$) and lower creative individuals ($m_L$) via non-parametric Mann-Whitney U-tests (Figure 3). In agreement with previous results, higher creative individuals generated on average longer fluency lists ($m_H=20.0\pm 0.8$ responses) than lower creative individuals ($m_L=16.0\pm 0.8$ responses), $U=779$, $p=0.01$. Next, we examined how the two groups rely on LVC words when generating fluency lists (Figure 3 top, middle panel). This analysis revealed that lower creative individuals generated fluency lists that contained, on average, a higher fraction words from the LVC ($m_L=0.29\pm 0.02$) compared to higher creative individuals ($m_H=0.24\pm 0.02$), $U=1383$, $p=0.03$. Notice that the reported fraction is normalized by the length of responses in the list that were part of the multiplex lexical network for both groups. This result indicates that lower creative individuals produced many simpler, easier to parse words than higher creative individuals (cfr. Table 1).

Furthermore, higher creative individuals covered slightly longer network distances connecting together consecutive animal category members in their fluency lists through shortest paths on the whole multiplex lexical network ($m_H=2.59\pm 0.02$ path length) in comparison with lower creative individuals ($m_L=2.50\pm 0.02$ path length), $U=822$, $p=0.02$ (Figure 3 top, right panel). Notice that the multiplex network distance used here has been shown to be relevant in capturing a variety of language-processes concerning word acquisition, production and mental search [9,50,51,52].

Finally, consider that the analyzed fluency task is relative to the animal category. “Animal” is also a node in the multiplex lexical network. Hence, the multiplex network enables us to measure the distance and other aspects of conceptual relatedness between fluency responses and the categorical node “animal”. We find no difference in terms of network distance between the two groups of higher and lower creative individuals. Instead, we report that shortest paths from lower creative individuals contained a higher median of words in the LVC ($m_L=0.17\pm 0.01$) compared to higher creative individuals ($m_H=0.15\pm 0.01$), $U=1936$, $p=0.02$, see also Figure 3 (bottom, left panel). In other words, assuming that mental navigation in the human mental lexicon follows shortest paths, as supported by previous studies [14,52,68], these findings indicate that lower and higher creative individuals remain at the same network distance from “animal” when exploring related concepts, nonetheless, lower creative individuals may exploit more conceptual associations involving simpler words from the LVC in comparison to higher creative individuals.

The above results indicate that lower creative individuals rely more on LVC words when generating responses in a semantic fluency task. A quantification of the entropy of accesses to the LVC (see Figure 3, bottom, right panel) indicates that lower creative individuals tend to produce LVC-in words during the fluency task in a way more similar to uniform randomness ($m_L$ entropy = $0.60\pm 0.01$) than higher creative individuals ($m_H$ entropy = $0.55\pm 0.01$), $U=1402$, $p=0.030$. This difference indicates that lower creative individuals tend to access words in the LVC in a way closer to uniform randomness compared to higher creative individuals.

3.2. Multiplex-Based Machine Learning Classification of Low/High Creativity Levels

Considering the above differences found across the two groups in generating animals in a fluency task, we examined whether these metrics can be used to classify participants into low and high creativity groups, using a machine learning classifier. Logistic regression was chosen given its superior performance in comparison to other classifiers such as random forest, support vector machine and decision tree.

The logistic regression classifier that we use combines the above features of fluency lists (i.e., number of responses, coverage per response, access to LVC from “animal”, entropy of LVC responses and fraction of responses in the LVC) together with demographic information of participants (gender, age, vocabulary size). Cross-validated analysis (see Methods) indicated that gender and age inserted to the classifier only add noise, lowering the prediction accuracy, and were therefore excluded. The best classifier, including the metrics in Figure 3 and participant’s vocabulary size, achieved a classification accuracy of $65.0\pm 0.9\%$ in classifying participants to their correct creativity group, solely based on their fluency lists. The achieved prediction is significantly higher than random expectation ($50\%$ for our balanced case), thus indicating the possibility of classifying participants as low or high creative individuals in small populations by combining psycholinguistic and multiplex network features.

Beyond accuracy, which focuses only on correctly identified classification, we also measured the area under the Receiver Operating Characteristics curve (AUC) (see Figure 4), which measures how many true and false positives a given classifier captures. The baseline expectation for the AUC is 0.5, which is relative to classifying groups uniformly at random. Our classifier achieved a much higher AUC, of $68.0\pm 0.8$. In the clinical sciences, the rule-of-thumb threshold for the AUC is 0.75 [67], so that automatic classifiers above that value are considered acceptable for clinical diagnosis. AUC lower than 0.75 are considered to indicate reasonable classification performance, despite some deficiency in the diagnostic accuracy that might depend either on the quality and quantity of the data or on the nature of the classifier itself. Even though our classifier achieves an AUC lower than 0.75, its classification being considerably higher than random indicates the possibility of using such an approach in synergy with other features in order to achieve even better prediction performance.

Importantly, not including any network-based features in the classifier led to a drop of accuracy to $62.1\pm 0.6\%$. Furthermore, including network-based features while excluding the measure of vocabulary size led to a drop of accuracy to $63.0\pm 0.5\%$. These findings indicate that the features based on the multiplex lexical network and measure of vocabulary size are both beneficial in improving the classification of lower and higher creativity groups.

4. Discussion

In the current study we investigate how a multiplex representation of the mental lexicon relates to the way low and high creative individuals retrieve animal category members in a semantic fluency task. We examine, for the first time, how a multiplex representation of the mental lexicon relates to differences between low and high creative individuals. Furthermore, we examine how well a machine learning approach, based on semantic fluency and multiplex lexical networks, can classify participants into groups that relate to their creativity level. In line with recent literature considering proficiency in linguistic tasks [11,12,69] and fluency in particular [5,20,39,40] as the outcome of a network-based mental exploration of the mental lexicon, we quantify performance in generating fluency lists in terms of both performance-based (i.e., length of semantic fluency list) and network-based features (i.e., reliance on the LVC).

Previous research highlighted the presence of a core in the mental lexicon, a language kernel made of several highly general, frequent and conceptually concrete words which facilitates language comprehension and processing [10,70]. In this regard, the LVC represents highly general, “common” words in English and is fully identified by the connectedness of lexical items across semantic and phonological network layers. In other words, the LVC emerges from the multiplexity of the mental lexicon and cannot be identified in single-layer modelling approaches. In our study, we show how features of the multiplex mental lexicon, such as the fraction of responses originating from the LVC, the entropy of lexical access to items in the LVC, the number of visited concepts during the mental search and the accesses to the LVC from the “animal” concept all highlight differences between lower and higher creative individuals.

Our results indicate that higher creative individuals generate fluency lists differently from lower creative individuals, not only in terms of mere numerosity (e.g., higher creative individuals generate more responses than lower creative individuals in the same amount of time), but also in terms of their qualitative structure in relation to a multiplex representation of the mental lexicon. First, we found that lower creative individuals tend to provide more responses that are within the LVC. Since words in the LVC are frequent, concrete words that are learned early on and possess more different semantic meanings, we interpret this distinction as higher creative individuals providing less “common” responses in their fluency tasks. Such a finding is in line with the ability of higher creative individuals to generate more uncommon responses [71].

Second, we found that higher creative individuals can cover longer distances over the multiplex network when jumping from one response to the next (i.e., higher coverage per response). This finding is analogous to previous findings [1,72] about higher creative individuals being able to cover longer network distances when searching through their mental lexicon, findings that currently have only been shown at the semantic level [1,30,31].

Third, we found that when generating responses to the stimuli word “animal”, lower creative individuals tend to rely more on words in the LVC, compared to higher creative individuals. Hence, in addition to lower creative individuals providing more responsive in the LVC, this finding indicates that lower creative individuals tend to exploit more words in the LVC as “bridges” for their mental exploration in comparison to higher creative individuals, who exploit more concepts outside of the language kernel represented by the LVC. This finding may be related to previous studies demonstrating how higher creative individuals’ search processes being broader and allowing them to reach more weakly connected nodes, potentially words that are outside the LVC [6,30,31].

Finally, we found that in fluency lists generated by lower creative individuals, the entropy of LVC and non-LVC responses is higher than in fluency lists generated by higher creative individuals. This difference indicates that lower creative individuals may access the LVC in a way that is closer to uniform randomness than higher creative individuals do. Such a finding may be related to lower creative individuals more relying on prototypical responses [6].

Thus, our results suggest that higher creative individuals have a less random and more strategic way of searching for concepts during the fluency task, minimizing or reducing random accesses to the LVC. Given that such a potential strategy leads to higher creative individuals generating more complex animal responses, the strategic access detected here might be used by higher creative individuals for connecting more remote concepts, in agreement with previous findings [5,14,16,20,73]. However, it is important to note that this potential strategic access of words outside the LVC can be due to various search strategies or differences in the structure of the mental lexicon in higher creative individuals. Indeed, previous studies have found effects of both lexicon structure and executive processes in relation to individual differences in creative ability [2,7,21]. Disentangling retrieval strategies from lexicon structure is a main open issue in the application of network science to cognitive science [11].

Importantly, the differences we found across the performance-based and network-based features of the two groups can be used for building a machine learning classifier automatically partitioning participants into low and high creative groups. Such an approach has relevant potential future applications in educational and employment settings because of its simplicity and data parsimony (i.e., fluency data can be produced in a few minutes and easily processed). Taking as reference the threshold of 0.75 for the AUC, as indicated by best practices in machine learning for clinical applications [67], the performance of our classifier (0.68 AUC) is not valid enough for clinical applications. However, it could be used in synergy with other mainstream psychology measures to provide novel automatic tools for the assessment of creativity scales beyond task-specific and time-consuming assessments. Nonetheless, future studies that replicate our classifier in a larger sample are needed to examine the validity of this approach.

Limitations and Future Work

Our study includes a few limitations. First, our groups are assessed for creativity by the CAQ, a questionnaire that assesses subjective creative achievements across different domains [57]. Future studies should replicate our findings with other measures of individual differences in creativity, such as divergent thinking tasks [71].

Furthermore, while creativity is an individual difference measure, we aggregate over individuals into two groups, of lower and higher creative ability. Thus, our aggregated groups may obscure important relations between creativity and mental navigation at the individual level. Approaches to represent individual semantic networks are currently being developed [21,39,40]. Future work should replicate our findings at the individual level, based on a multiplex representation of participant’s mental lexicon.

From a methodological perspective, beyond the standard machine learning algorithms tested here, deep learning is quickly becoming a promising approach for pattern detection even in noisy and small-sized cognitive datasets [74]. Although the logistic regressor used here is already capable of capturing important differences between high and low creativity groups, future research with neural networks and deep learning might achieve higher accuracy and AUC than the ones achieved in this first-of-its-own approach.

From a network science perspective, the aspects of the mental lexicon represented here as multiplex layers constitute only a limited representation of the multi-relational nature of word–word similarities in the human mind [11]. The inclusion of additional types of word–word similarities such as co-occurrences or feature sharing, cfr. [9,50,52], as multiplex layers, might improve the accuracy of the automatic creativity assessment reported here, at the cost of building large enough datasets.

Beyond psycholinguistic features, additional data could enrich automatic models of creativity assessment. For instance, increasingly more data has been gathered and analyzed in relation to how individuals perform collaborative knowledge building with online encyclopedias [69,75]. Tracking and measuring the cognitive performance of knowledge creators might shed important additional information for achieving more accurate classifiers of creativity levels and better understand which role of people with different creativity levels for the construction of online knowledge resources [75].

Finally, we use only the extreme groups from Kenett et al. [23], in order to reduce the number of classified classes and thus enable a better training in presence of scarce data ($N=47$ for both groups). Further research is needed to replicate our findings in larger samples across more graded levels of creative ability.

5. Conclusions

In conclusion, we examine how the performance of low and high creative individuals in a semantic fluency task can be quantified by a multiplex network account of the mental lexicon. Thus, our findings support and extend previous studies on individual differences in creativity and mental lexicon structure [5].

Furthermore, we apply for the first time a machine learning classifier to classify participants into their respected low and high creative groups. To the best of our knowledge, this is the first study that demonstrates how a classifier based on linguistic and behavioral variables can be used to automatically classify participants into accurate groups of various creative levels starting from relatively scarce data (i.e., a list of a dozen word responses). This approach demonstrates the feasibility of achieving accurate automatic predictions of creativity levels through cognitive data and machine learning.