Cognitive Network Science: A review of research on cognition through the lens of network representations, processes, and dynamics

Network science provides a set of quantitative methods to investigate complex systems, including human cognition. Although cognitive theories in different domains are strongly based on a network perspective, the application of network science methodologies to quantitatively study cognition has so far been limited in scope. This review demonstrates how network science approaches have been applied in the study of human cognition, and how it can uniquely address and provide novel insight on important questions related to the complexity of cognitive systems and the processes that occur within those systems. Drawing on the literature in cognitive network science with a focus on semantic and language networks, we argue three key points. (i) Network science provides a powerful quantitative approach to represent cognitive systems. (ii) The network science framework enables cognitive scientists to achieve a deeper understanding of human cognition by capturing how the structure, i.e., the underlying network, and processes operating on a network structure, interact to produce behavioral phenomena. (iii) Network science provides a quantitative framework to model the dynamics of cognitive systems, operationalized as structural changes in cognitive systems that can occur on multiple timescales and resolutions. Finally, we highlight key milestones that the field of cognitive network science needs to achieve in the future as it matures in order to provide continued insights into the nature of cognitive structures and processes. 2 COGNITIVE NETWORK SCIENCE Introduction Networks are everywhere. The friends you interact with in real life and on social media form your social network. Webpages form a network that you navigate through when you browse the World Wide Web. The same holds for roads, train tracks, or waterways for navigation in the real world. Over the past two decades, an increasing number of studies have been applying network science methodologies across diverse scientific fields to study complex systems (Barabási, 2012, 2016; Sporns, 2011). Complex systems involve multiple components that interact with each other to give rise to complex behavior. Examples of complex systems include traffic infrastructure, social interactions, economic markets, the human brain, as well as human cognition, most prominently in the domains of memory and language (e.g., Baronchelli, Ferrer-iCancho, Pastor-Satorras, Chater, & Christiansen, 2013; Beckage & Colunga, 2015; De Deyne, Kenett, Anaki, Faust, & Navarro, 2016). Network science is based on a domain of mathematics known as graph theory and provides a set of powerful quantitative methods to investigate complex systems, such as human cognition (e.g., Boccaletti, Latora, Moreno, Chavez, & Hwang, 2006). In recent years, network science has become a popular tool in the study of structures and dynamics at the neural level of the brain (Bassett & Sporns, 2017; Sporns, 2011). Despite its rich potential, this has been the case to a lesser extent in the study of cognitive phenomena. This review aims to discuss how network science approaches have been applied in the study of human cognition, and how it can uniquely address and shed novel light on important questions related to cognitive systems and the processes that occur within those systems. In particular, we aim to establish the following three points by an in-depth discussion of the extant literature on cognitive 3 COGNITIVE NETWORK SCIENCE network science. This review is organized into three sections that will attempt to address each of these issues in turn. 1. Network science provides an attractive quantitative approach to representing cognitive systems. One important goal of cognitive science is to model cognitive structures, such as those facilitating semantic memory—our memory for facts or events (Beckage & Colunga, 2015; Borge-Holthoefer & Arenas, 2010; Jones, Willits, & Dennis, 2015). This goal of formalizing cognitive representations is reflected in the diversity of approaches that have been employed including symbolic approaches (e.g., Anderson, 1996), connectionist or neural network approaches (e.g., Dell, Chang, & Griffin, 1999; McClelland, McNaughton, & O’Reilly, 1995; Seidenberg & McClelland, 1989), and combinations of the two (e.g., Chater & Manning, 2006; Griffiths, Chater, Kemp, Perfors, & Tenenbaum, 2010), among others. We argue that a network science approach can provide a powerful alternative framework for modeling and quantifying cognitive representations. Network science provides a suite of computational tools, allowing the cognitive scientist to explicitly examine the structural properties of cognitive systems—something that can be difficult to achieve with, for instance, connectionist approaches where the structure of a cognitive system is obfuscated within a black box of “hidden” neural layers (Lake, Ullman, Tenenbaum, & Gershman, 2017). Section 1 focuses on cognitive representations and architectures and demonstrates how network science approaches can be used to represent and describe the structural properties of cognitive systems. This section also introduces the reader to the terminology of networks. 2. A deeper understanding of human cognition can be achieved when structures, i.e., the underlying network, and processes operating on a network structure, are considered in tandem within a network science framework. Another strength of the network science 4 COGNITIVE NETWORK SCIENCE approach is the ability to not only quantify aspects related to the structure of cognitive systems, but also to model the processes that operate within these systems. For instance, a model of memory retrieval typically requires two core components, a representation of memories and a process to retrieve them. Within a network approach, these might be modeled, for instance, as a network representation that depicts semantic memory as a network of concepts that are connected based on their semantic similarity and a so-called random walk process that walks randomly across the semantic network to emit the nodes it traverses. Generally, the joint consideration of structure and process emerges naturally from a network approach and could provide a parsimonious account of human behavior and cognition in domains such as semantic memory and lexical retrieval. Section 2 focuses on cognitive and language-related processes that occur in cognitive networks and highlights how a thorough understanding of cognitive processes requires close consideration of how the structure of the cognitive system interacts with processes to give rise to complex human behavior. 3. Network science provides a framework to model structural changes in cognitive systems on multiple scales. Another area of research that cognitive scientists are deeply involved concerns the development and decline of cognitive systems. Research in areas such as language acquisition and cognitive aging have revealed that cognitive systems are dynamic and are acutely sensitive to changes in the linguistic environment (Hart & Risley, 1995), accumulation of experience over time (Ramscar, Hendrix, Shaoul, Milin, & Baayen, 2014), and deficits or other age-related decline in sensory processing (Baltes & Lindenberger, 1997). Another area of research demonstrating dynamic changes at shorter timescales is research on creativity (Kenett & Thompson-Schill, 2017). We aim to show how network science methods can provide new ways of quantifying and modeling the dynamics in these areas. Section 3 focuses on 5 COGNITIVE NETWORK SCIENCE the dynamics of cognitive networks, and specifically discusses research focusing on the factors that lead to structural changes in the network on multiple timescales. Aside from applications to social relationships (e.g., Lazer et al., 2009), which are not the primary focus of this review, cognitive science has employed network science methodologies largely to study the relationships between words and concepts (e.g., Beckage & Colunga, 2015; Borge-Holthoefer & Arenas, 2010; Collins & Loftus, 1975; Steyvers & Tenenbaum, 2005). While a wide range of cognitive constructs can be represented as a network, the review will primarily focus on research studying memory and language-related phenomena using networks, as these domains constitutes the majority of the literature on cognitive network science to date. Nevertheless, we note that network science can be a valuable tool far beyond the study of words and concepts (see Table 1), and encourage researchers to consider how network science methods can be used to address a broad spectrum of research questions in the cognitive sciences. Section 1: Cognitive Constructs as Networks Networks are composed of two elements: nodes that represent the conceptual entities of interests, e.g., persons, websites, or words, and edges that represent the relationship those units, e.g., friendship, hyperlinks, or semantic similarity. While more fine-grained distinctions are sometimes considered such as bipartite and multiplex networks (defined below), identifying these two basic elements in the system of study is sufficient to formalize the system as a network and to employ the powerful tools provided by network science. It is worth noting that a network science approach assumes that the relationship between nodes (i.e., edges) is as important as the nodes themselves, if not more important. A first challenge in studying cognitive systems and 6 COGNITIVE NETWORK SCIENCE processes as networks is to represent these systems or processes in a meaningful way in terms of nodes and edges. 1.1 Network representations of cognitive systems Cognitive science traditionally has a strong interest in words and concepts as the basis for thought, reasoning, and communication (e.g., Pinker & Jackendoff, 2005). Much research has been dedicated to studying the properties of words, such as their frequency in natural language, their valence, or their concreteness (e.g., Warriner, Ku


Introduction
Networks are everywhere. The friends you interact with in real life and on social media form your social network. Webpages form a network that you navigate through when you browse the World Wide Web. The same holds for roads, train tracks, or waterways for navigation in the real world. Over the past two decades, an increasing number of studies have been applying network science methodologies across diverse scientific fields to study complex systems (Barabási, 2012(Barabási, , 2016Sporns, 2011). Complex systems involve multiple components that interact with each other to give rise to complex behavior. Examples of complex systems include traffic infrastructure, social interactions, economic markets, the human brain, as well as human cognition, most prominently in the domains of memory and language (e.g., Baronchelli, Ferrer-i-Cancho, Pastor-Satorras, Chater, & Christiansen, 2013;Beckage & Colunga, 2015;De Deyne, Kenett, Anaki, Faust, & Navarro, 2016). Network science is based on a domain of mathematics known as graph theory and provides a set of powerful quantitative methods to investigate complex systems, such as human cognition (e.g., Boccaletti, Latora, Moreno, Chavez, & Hwang, 2006).
In recent years, network science has become a popular tool in the study of structures and dynamics at the neural level of the brain (Bassett & Sporns, 2017;Sporns, 2011). Despite its rich potential, this has been the case to a lesser extent in the study of cognitive phenomena. This review aims to discuss how network science approaches have been applied in the study of human cognition, and how it can uniquely address and shed novel light on important questions related to cognitive systems and the processes that occur within those systems. In particular, we aim to establish the following three points by an in-depth discussion of the extant literature on cognitive network science. This review is organized into three sections that will attempt to address each of these issues in turn.
1. Network science provides an attractive quantitative approach to representing cognitive systems. One important goal of cognitive science is to model cognitive structures, such as those facilitating semantic memory-our memory for facts or events (Beckage & Colunga, 2015;Borge-Holthoefer & Arenas, 2010;Jones, Willits, & Dennis, 2015). This goal of formalizing cognitive representations is reflected in the diversity of approaches that have been employed including symbolic approaches (e.g., Anderson, 1996), connectionist or neural network approaches (e.g., Dell, Chang, & Griffin, 1999;McClelland, McNaughton, & O'Reilly, 1995;Seidenberg & McClelland, 1989), and combinations of the two (e.g., Chater & Manning, 2006;Griffiths, Chater, Kemp, Perfors, & Tenenbaum, 2010), among others. We argue that a network science approach can provide a powerful alternative framework for modeling and quantifying cognitive representations. Network science provides a suite of computational tools, allowing the cognitive scientist to explicitly examine the structural properties of cognitive systems-something that can be difficult to achieve with, for instance, connectionist approaches where the structure of a cognitive system is obfuscated within a black box of "hidden" neural layers (Lake, Ullman, Tenenbaum, & Gershman, 2017). Section 1 focuses on cognitive representations and architectures and demonstrates how network science approaches can be used to represent and describe the structural properties of cognitive systems. This section also introduces the reader to the terminology of networks.

2.
A deeper understanding of human cognition can be achieved when structures, i.e., the underlying network, and processes operating on a network structure, are considered in tandem within a network science framework. Another strength of the network science approach is the ability to not only quantify aspects related to the structure of cognitive systems, but also to model the processes that operate within these systems. For instance, a model of memory retrieval typically requires two core components, a representation of memories and a process to retrieve them. Within a network approach, these might be modeled, for instance, as a network representation that depicts semantic memory as a network of concepts that are connected based on their semantic similarity and a so-called random walk process that walks randomly across the semantic network to emit the nodes it traverses. Generally, the joint consideration of structure and process emerges naturally from a network approach and could provide a parsimonious account of human behavior and cognition in domains such as semantic memory and lexical retrieval. Section 2 focuses on cognitive and language-related processes that occur in cognitive networks and highlights how a thorough understanding of cognitive processes requires close consideration of how the structure of the cognitive system interacts with processes to give rise to complex human behavior.
3. Network science provides a framework to model structural changes in cognitive systems on multiple scales. Another area of research that cognitive scientists are deeply involved concerns the development and decline of cognitive systems. Research in areas such as language acquisition and cognitive aging have revealed that cognitive systems are dynamic and are acutely sensitive to changes in the linguistic environment (Hart & Risley, 1995), accumulation of experience over time (Ramscar, Hendrix, Shaoul, Milin, & Baayen, 2014), and deficits or other age-related decline in sensory processing (Baltes & Lindenberger, 1997).
Another area of research demonstrating dynamic changes at shorter timescales is research on creativity (Kenett & Thompson-Schill, 2017). We aim to show how network science methods can provide new ways of quantifying and modeling the dynamics in these areas. Section 3 focuses on the dynamics of cognitive networks, and specifically discusses research focusing on the factors that lead to structural changes in the network on multiple timescales.
While a wide range of cognitive constructs can be represented as a network, the review will primarily focus on research studying memory and language-related phenomena using networks, as these domains constitutes the majority of the literature on cognitive network science to date.
Nevertheless, we note that network science can be a valuable tool far beyond the study of words and concepts (see Table 1), and encourage researchers to consider how network science methods can be used to address a broad spectrum of research questions in the cognitive sciences.
Section 1: Cognitive Constructs as Networks Networks are composed of two elements: nodes that represent the conceptual entities of interests, e.g., persons, websites, or words, and edges that represent the relationship those units, e.g., friendship, hyperlinks, or semantic similarity. While more fine-grained distinctions are sometimes considered such as bipartite and multiplex networks (defined below), identifying these two basic elements in the system of study is sufficient to formalize the system as a network and to employ the powerful tools provided by network science. It is worth noting that a network science approach assumes that the relationship between nodes (i.e., edges) is as important as the nodes themselves, if not more important. A first challenge in studying cognitive systems and processes as networks is to represent these systems or processes in a meaningful way in terms of nodes and edges.

Network representations of cognitive systems
Cognitive science traditionally has a strong interest in words and concepts as the basis for thought, reasoning, and communication (e.g., Pinker & Jackendoff, 2005). Much research has been dedicated to studying the properties of words, such as their frequency in natural language, their valence, or their concreteness (e.g., Warriner, Kuperman, & Brysbaert, 2013). These efforts have been instrumental for predicting the behavior of human memory and lexical retrieval; however, researchers have also found that additional insights can be gained by considering the relationships between words. Instrumental to such insights was the representation of words and their relationships as networks. For instance, by mapping words onto nodes and associative strength between words, i.e., the likelihood that one word is named as an association to another word (see also De Deyne & Storms, 2008;Kenett, Kenett, Ben-Jacob, & Faust, 2011;Nelson, McEvoy, & Schreiber, 2004), onto edges, research has found that a word's degree, a popular node metric derived from a network (defined below), predicts how well words can be learned (Griffiths, Steyvers, & Firl 2007;Steyvers & Tenenbaum 2005).
Likewise, research has studied different linguistic units other than words by mapping letters (Choudhury et al., 2010;Mukherjee, Choudhury, Basu, & Ganguly, 2007), syllables or segments (Majerus, Van der Linden, Mulder, Meulemans, & Peters, 2004), or entire documents (Small, 1999) onto nodes. The minimum requirement for representing a cognitive system as a network is to identify nodes and edges. However, there is much more information that can be represented. For instance, networks can be specified with multiple types of nodes to distinguish between cues and responses in an associative network, creating a bipartite network (Dubossarsky, De Deyne, & Hills, 2017). Similarly, networks can be specified with multiple types of edges, for instance to represent both phonological and semantic similarity between words, creating a multiplex network (Stella, Beckage, & Brede, 2017, Stella, Beckage, Brede, & De Dominico, 2018. Finally, edges can be weighted and directed in order to reflect weight and direction of a relationship, respectively, to account for the fact that a cue triggers a response at a higher rate than the other way around such as the case with "dog" cuing the response "bone" but "bone" only infrequently cuing "dog", with higher associates to words like "skeleton" or "body" (Dubossarsky et al., 2017;Nelson et al., 2004).
The construction of networks leaves vast freedom to the researcher and renders it possible to represent a wide variety of cognitive systems as networks. For instance, the emerging area of network psychometrics represents statistical relationships between personality traits or items in a symptom checklist as a network, seeking to better understand causal structure of human personality and psychological disorders (see Borsboom & Cramer, 2013;Fried et al., 2017 for reviews). This approach is rapidly being established in personality and clinical research as a fruitful alternative to traditional approaches that use latent variable modeling approaches, which assumes the presence of a latent variable that accounts for psychological and personality disorders, whereas the network approach emphasizes the relationships (i.e., edges) between symptoms and the importance of considering the causal pathways that can lead to the emergence of a disorder (e.g., Forbush, Siew, & Vitevitch, 2016;McNally et al., 2015;Siew, Pelczarski, Yaruss, & Vitevitch, 2017). Moreover, networks have been used to represent the external environment that people are embedded in, such as their social network or the informational space that learners are exposed to. Emerging work is showing that quantifying such external structures could lead to new insights into a number of topics of deep interest to cognitive scientists, including the influence of a speaker's social network size on language processing (Lev-Ari, 2018b), language evolution (Lev-Ari, 2018a, Steels, 2011, problem solving and decision making in groups (Kearns, Suri, & Montfort, 2006;Mason & Watts, 2012), and how learners are able to extract the external structure of the world via statistical learning (Karuza, Kahn, Thompson-Schill, & Bassett, 2017;Karuza, Thompson-Schill, & Bassett, 2016). See Table 1 for a summary of different types of cognitive networks and relevant cognitive science topics. In summary, representing cognitive structures in terms of a network offers high degrees of flexibility to researchers investigating various cognitive phenomena (see Table 1). The nodes and edges in any cognitive network should represent theoretically motivated constructs, with nodes depicting an appropriate and relevant scale of representation and edges defining a meaningful relationship between nodes (Butts, 2009). Choosing a network representation can be likened to choosing a measurement instrument. Different network representations will reveal different aspects of the underlying cognitive system, and it is ultimately up to the researcher to decide which aspects are of focal interest.

How can network structures be characterized?
A key strength of studying cognitive systems as networks is the accessibility of reliable, well-established quantitative measures and tools, reflecting the long history of graph theory and its mathematical foundations (Euler, 1736), as well as its continual refinement and development (for instance, in the area of multiplex networks, Boccaletti et al., 2014). In this section, we examine common measures used to quantify aspects of networks on three scales of structure in network representations, beginning with (i) the microscopic structure, i.e., a "node's" eye view of structural properties of individual nodes and edges, (ii) the meso-scale, involving a subset of nodes and the substructures that they form, and (iii) the macro-scale, i.e., a "network's" eye view summarizing entire network structure. To highlight how these three different scales provide novel opportunities for the study of cognition, we review measures on each of these scales and how they are employed to foster our understanding of cognition.
shortest path between node i and node j d(n i , n j ) where n i , n j ∈ N average shortest path length, L average length of shortest path between pairs of nodes largest shortest path between nodes D= max n i ∈ N , n j ∈ N d ( n i , n j ) closeness centrality inverse of the sum of the length of the shortest paths between node i and all other nodes in the graph largest connected component largest group of nodes in the network that are connected to each other in a single component degree distribution, P(k) probability distribution of node degrees in the network γ power-law exponent for the degree distribution small-world structure network with short average path lengths and relatively high clustering coefficient (relative to a random graph with similar density) scale-free network network with a degree distribution that is power-law distributed

1 Microscopic Network Measures
At the microscopic level, network analysis examines different properties of nodes and edges, most commonly focusing on quantifying the "importance" of a node in the graph representation via measures of centrality (Brandes, Borgatti, & Freeman, 2016;Koschützki et al., 2005;Papo, Buldú, Boccaletti, & Bullmore, 2014).
One popular measure of node centrality is the node's degree, k i , i.e., the number of edges connected to a node. Nodes of higher degrees are connected to a higher number of nodes in the network and can have an important role in, for instance, exchanging information across the network. A node's degree also defines a node's "neighborhood size", a property that has often been used in the context of phonological and orthographic networks of word forms, where edges commonly represent an edit distance of 1 phoneme (Luce & Pisoni, 1998) or 1 letter (Coltheart, Davelaar, Jonasson, & Besner, 1977). Here, the degree of a node defines the level of similarity of a word form to other word forms.
Another property of nodes derived from its neighborhood is the local clustering coefficient, c i , which characterizes the extent to which the neighbors of a node are interconnected. Specifically, clustering coefficient measures the extent to which a node's neighbors are also neighbors of each other (similar to the notion of transitivity in social networks, i.e., are your friends also friends of each other; Watts & Strogatz, 1998). As shown in Figure 2, it is possible for a word with the same degree to have different values of c i , reflecting differences in the internal structure of their neighborhoods. Similar to a node's degree, a node's clustering coefficient will control the flow of information through and around a node.
Along these lines, recent psycholinguistic research has shown that people are sensitive to nuances in the similarity structure of words as operationalized by c i . The clustering coefficient of words has small but measurable influences on how people recognize spoken (Chan & Vitevitch, 2009) and written words (Siew, 2018;Yates, 2013), produce speech (Chan & Vitevitch, 2010), learn new words , and recall words in memory tasks (Vitevitch, Chan, & Roodenrys, 2012). Both node degree and local clustering coefficient consider only the immediate neighborhood of the node of interest. To characterize the importance of a node beyond its immediate neighborhood measures such as closeness centrality or PageRank centrality are available. Closeness centrality is computed as the inverse of the average shortest path length to all other nodes in the network to identify whether a node is more central to the network than others (Beauchamp, 1965). Closeness centrality of words computed from phonological and orthographic similarity networks have been shown to influence spoken (Goldstein & Vitevitch, 2017) and visual word recognition (Siew, 2018), picture naming performance among people with aphasia (Castro & Stella, 2018), and performance in a mental navigation task (Iyengar, Madhavan, Zweig, & Natarajan, 2012). PageRank centrality became widely known as an algorithm used to rank websites in Google search results (Brin & Page, 1998). PageRank centrality can be thought of in terms of a "fluid" that flows throughout the network and pools at the most important nodes. The general idea is that more important nodes (websites) receive more "fluid" (endorsement) from nodes (websites) that are themselves important in a recursive fashion. Although PageRank centrality was developed for the purpose of optimizing web search, Griffiths, Steyvers, and Firl (2007) showed that an implementation of the PageRank algorithm on language networks was better able to account for people's responses in a fluency task as compared to traditional predictors such as word frequency or associate frequency--suggesting strong parallels between the mechanisms underlying successful information retrieval in search engines and in human memory.
Finally, the shortest path, or shortest distance between two nodes, d(n i , n j ), can be used to reveal something about the relationship of (non-neighboring) nodes rather than the nodes themselves. For instance, path length between two nodes in cognitive and language networks influence the misperception of spoken words (Vitevitch, Goldstein, & Johnson, 2016), judgments of semantic relatedness (Collins & Loftus 1975;Kenett, Levi, Anaki, & Faust, 2017), and picture naming performance in people with aphasia (Castro & Stella, 2018).
Many other network measures that were not discussed here in detail have been shown to have measurable effects on behavioral tasks. These include assortative mixing by degree (Vitevitch, Chan, & Goldstein, 2014), key players (Borgatti, 2006;, whether a node resides in the largest connected component of the network or in smaller connected components Vitevitch & Castro, 2015), network connectivity of a node's broader neighborhood that included its immediate neighbors and nonneighboring words (Siew, 2017), along with many others that have yet to be thoroughly explored in the cognitive sciences, such as betweenness and eigenvector centrality (see Vitevitch, Goldstein, Siew, & Castro, 2014 for a review detailing the influence of various network metrics on language processing, and Borgatti, 2005, for a review of various centrality measures).

Mesoscopic Network Measures
At the mesoscopic level, research has focussed on network community structure.
Communities refer to the grouping of nodes into sub-networks, usually based on their interconnectedness. To identify communities in networks, several existing algorithms aim to maximize connectivity within clusters while minimizing connectivity between clusters (Newman, 2006). The identification of communities and the nodes they include can in of itself produce interesting insights regarding a cognitive system. In the domain of language, for instance, it can be used to identify semantic fields or categories (Borge-Holthoefer & Arenas, 2010). This approach can also be used to characterize the overall tendency of a network to produce communities, which is known as modularity (Fortunato, 2010;Newman, 2006).
Modularity statistics measure the extent to which a network can be easily partitioned into subcommunities. The larger the modularity measure, the greater extent to which the network is comprised of sub-networks (Newman, 2006). The notion of modularity is extensively investigated at the neural level in the brain (Bullmore & Sporns, 2012;Hilgetag & Hütt, 2014;Meunier, Lambiotte, & Bullmore, 2010). Such research has consistently shown how modularity of neural networks changes with the progression of several different psychopathologies (Stam, 2014;van Straaten & Stam, 2013).
Recent studies have also highlighted the significance of modularity in cognitive networks in both healthy and clinical populations (Kenett, Gold, & Faust, 2016;Kenett et al., 2018;Shai et al., 2015;Siew, 2013). For example, the semantic network of individuals with high functioning autism (Asperger's syndrome) has been found to exhibit higher modularity than matched controls, which offers one possible account of their rigidity in processing language . Moreover, higher modularity observed in the phonological network was suggested to constrain the spreading of activation in lexical retrieval (Siew, 2013) and higher modularity observed in the semantic network was negatively related to individual differences in creative ability (Kenett, Anaki, & Faust, 2014;Kenett et al., 2018).

Macroscopic Network Measures
Measures of the macroscopic structure of networks speak to the overall organization of networks. These measures can reveal emergent properties of a system visible only when considering the network as whole; these properties may play an important role in the system's behavior. Below we describe macroscopic network measures that have been used to study cognitive systems.
One pattern that frequently emerges in a variety of systems is known as a small world structure, characterized by high local clustering and moderate average shortest path lengths, relative to a similarly-sized, density matched, randomly drawn networks (Watts & Strogatz, 1998). This property may be important in the domain of cognition for two reasons: First, small world structures have been found to be an almost universal property of real-world networks across diverse domains including biological (e.g., Sole & Montoya, 2001;van den Heuvel & Sporns, 2013), social (e.g., Lewis, Kaufman, Gonzalez, Wimmer, & Christakis, 2008), and information (e.g., Albert, Jeong, & Barabási, 1999) networks. Second, small world structures may emerge from systematic growth processes that may adapt to environmental constraints to give rise to a beneficial structure. For instance, the small world structure of brain networks has been said to reflect the trade-off between short neuronal distances between brain regions and the costs associated with creating these connections, with the surprising result that the small world structure may provide a unique means to optimize organizational structure of neurons (Bullmore & Sporns, 2012). This interpretation is related to the idea of network efficiency, E G , referring to a network's ability to quickly exchange information (Table 2; Latora & Marchiori, 2001).
"Small worldness 1 " is also a ubiquitous feature of many types of language networks, including semantic networks (Steyvers & Tenenbaum, 2005), phonological networks of various languages (Arbesman et al., 2010;Vitevitch, 2008), the orthographic network of English (Siew, 2018), and syntactic networks (Ferrer i Cancho et al., 2004). Similar to the argument concerning brain networks, small-world properties in language networks might arise due to two competing aspects of language learning and use: distinctiveness (e.g., each object having a unique word mapping) and memory constraints (e.g., the easiest language to learn is one where a single word refers to everything). These two competing features of language may result in the emergence of local clusters of similar meaning and form but a low average path length due to the influence of memory constraints resulting in the reuse of words and sublexical segments (Ferrer i Cancho & Solé, 2001;Zipf, 1949). Finally, the small world structure in semantic and language networks could provide important clues into how the structure of such cognitive systems might be 1 We note that some people debate the usefulness of measuring small world structure as Watts and Strogatz (1998) showed that even an extremely structured lattice will exhibit small world structure when a small amount of random rewiring of edges is introduced. exploited in order to maximize the efficiency of search processes within semantic memory or prevent catastrophic failures in a language system (Borge-Holthoefer & Arenas, 2010; Goñi et al., 2011).

Degree distribution
The degree distribution, P(k), of a network indicates how many nodes have a given number of connections in the network (i.e., its degree). In most naturally occurring networks many nodes have low degree (few connections) and a few nodes (so called hubs) have very high degree (many connections). The degree distribution of some networks is often best approximated by a power law 2 such that P ( k ) ≈ k −γ , and is typically referred to as scale-free networks when the exponent, γ, is between 2 and 3 (Clauset, Shalizi, & Newman, 2009;Newman, 2005). The term scale-free refers to the fact that the second and higher order moments tend to go to infinity, implying an infinitely sized variance of node degrees. The scale-free property of networks has been linked to a network's resilience to random node failure. That is, studies of percolation processes have found that the connectivity in scale-free withstand a continued deletion of random edges longer than networks with other degree distributions (Cohen, Erez, Ben-Avraham, & Havlin, 2000).
The degree distributions of semantic networks studied by Steyvers and Tenenbaum (2005) also approximate a power law distribution, with the exponents of the best fitting power laws converging at ~3. This is consistent across semantic networks constructed from free associations (Nelson et al., 2004) and from more complex semantic relationships (e.g., WordNet; Miller, 1995), suggesting commonalities in the semantic organization of word meanings (Kello et al., 2010; but see Morais, Olsson, & Schooler, 2013 of phonological networks of various languages appeared to be best fit by a truncated power law (Arbesman et al., 2010), suggesting that there are some words that are hubs but that the evolution of form similarity networks may have involved a process different from preferential attachment, a prominent model of network growth known to produce a scale-free structure. To account for the ubiquity of scale-free degree distributions observed in naturally occurring networks, Barabási and Albert (1999) proposed an influential model of network growth known as the preferential attachment where, as new nodes are added to the network, they are more likely to be connected to nodes with higher degree (i.e., more connections). Therefore, highly connected words are more likely to acquire new connections, resulting in a "rich-getricher" effect. Steyvers and Tenenbaum (2005) suggested that the growth of semantic networks could have occurred in a similar manner, by conceptualizing preferential attachment as a process of semantic differentiation in which words are likely to be learned if they connect to other words with many varied meanings, helping increase the child's vocabulary and helping the child learn the various meanings of words.

Summary
This section provided an overview of various network measures at the micro-, meso-, and macro-levels and highlighted cases where these network measures were predictive of human behavior, predominantly drawn from the domain of language processing and semantic memory.

Section 2: Processes in Cognitive Networks
Now that we have discussed how cognitive systems can be represented as networks, we turn to models that may account for processes that occur in these cognitive networks. In this section we begin with an overview of classic theory of spreading activation in cognitive psychology, and discuss an extension to this classic model as inspired by random walk models in network science research. This sets the stage for an in-depth discussion of how spreading activation and random walks, when implemented in a network representation, can account for behavioral findings from various domains in human cognition, including lexical retrieval, creativity, and cognitive search and navigation.

Cognitive processes in networks
One of the earliest attempts to conceptualize a cognitive system as a network of nodes and edges was made in research on human semantic memory (Quillian, 1967; in order to explain an intricate taxonomy of human reasoning. When asked to verify statements such as "a robin is an animal" or "a robin is a bird", human participants typically take longer to verify the former as compared to the latter (Collins & Quillian, 1969; c.f., Conrad, 1972;Nelson & Kosslyn, 1975). To account for this finding, human semantic memory was assumed to exhibit a tree-like organization, in which nodes represented concepts and edges represented whether words were elements of a higher order concept. In such a network, the triad, "robin", "bird", and "animal", forms a line with "robin" being connected to "bird" and "bird" being connected "animal". Any reasoning process seeking to identify whether two concepts are, at least indirectly, connected via "is-contained-in" relationships would consequently have to traverse two edges to verify the first sentence ("a robin is a bird, and birds are animals") but only one to verify the second ("a robin is a bird"). Today, this represents one of the classic examples explaining behavioral phenomena as a combination of an underlying mental network organization and a process operating on it.
Later, Collins and Loftus (1975) generalized this approach and developed the theory of spreading activation in semantic memory to account for various behavioral findings (see also Anderson, 1983). The key contribution of this model was to hypothesize a process operating on a semantic network when executing a cognitive task. Specially, it was assumed that, for instance, reading or thinking about a concept would activate the concept and that this activation would spread to neighboring concepts in the network, priming related concepts and making them easier to retrieve. The theoretical process of spreading activation proposed by Collins and Loftus (1975) implicitly assumes the presence of some cognitive resource (i.e., activation) that can be assigned to specific nodes, spread among connected nodes in a pre-defined network, and decay over time (as formally implemented in Anderson (1983); Dell (1986)). This spread of activation quickly decays over time and distance in the semantic network (Balota & Lorch, 1986). Overall, the success of the spreading activation account demonstrates the significance of formalizing a process that captures search within a cognitive system.
Network science provides new ways of expanding the original conceptualization of spreading activation by Collins and Loftus (1975) by formalizing diffusion models over network representations. These diffusion processes have been used to extensively study and predict epidemics of disease spread and of contagious ideas in a population (e.g., Christakis & Fowler, 2007;Valente, Palinkas, Czaja, Chu, & Brown, 2015). The models developed in these domains can be similarly used and adapted to better suit the purposes of a cognitive scientist to study how information "spreads" in a cognitive network. Although different implementations of network diffusion models exist, one core idea that illustrates the power of such network process models can be captured by the notion of a random walk on a network. A random walk model is a naïve search process that moves from node to node as function of a set of transition probabilities specifying the probability that the process transitions from one node to any of its directly connected neighboring nodes. Note that when combined with a decay parameter, a random walk model essentially becomes a model of spreading activation (De Deyne et al., 2016). In recent years, various empirical studies have demonstrated how memory search can be modeled as a random walk process over semantic memory (Abbott, Austerweil, & Griffiths, 2015;Bourgin, Abbott, Griffiths, Smith, & Vul, 2014;Fathan, Renfro, Austerweil, & Beckage, 2018;Smith, Huber, & Vul, 2013) and have shown predictive power in accounting for human behavior.
It is important to note that there are differences in the implementations and goals of spreading activation in the tradition of Collin and Loftus (1975) and random walk models by network scientists. That is, random walk models produce individual paths taken by the walk COGNITIVE NETWORK SCIENCE (e.g., an ordered list of words), whereas a process of spreading activation produces a pattern of activation levels among nodes in the network and how they change over time. In contrast to random walks, spreading activation, thus, represents the aggregate, long-run behavior arising from some underlying basic process, which could be a random walk. In the rest of this section, we review more recent empirical work showing how behavioral data from experiments can be used to provide a deeper understanding of the interaction of structure and processes that occur in cognitive and linguistic networks. Each subsequent section focuses on a different cognitive domain-specifically, lexical retrieval, creative processes, and search and navigation in cognitive networks-that draw on either the process of random walks or the theoretical construct of spreading activation to account for relevant behavioral findings. The final section focuses on a key debate in the cognitive science regarding the complexities of disentangling structure and process in the domain of cognitive search and retrieval and attempts to show how network science methods can contribute to this important debate.

Lexical retrieval
The notion of spreading activation as proposed by Collins and Loftus (1975) (Chan & Vitevitch, 2009) and visual word recognition tasks (Siew, 2018;Yates, 2013), and more slowly produced (Chan & Vitevitch, 2010). To account for clustering coefficient effects in word recognition, Chan and Vitevitch (2009) provide a theoretical account that assumes a spreading activation process operating on a phonological network, where a word's structural characteristics affect how activation spread through the network. Words with low clustering coefficients are hypothesized to receive more activation (and hence a processing advantage) from its neighbors as compared to those with high clustering coefficients because activation in the latter case is more likely to be shared among neighbors resulting in similar levels of activation between the target word and its neighbors. This account of a possible mechanism for word confusability within a theoretical spreading activation framework was further formalized and validated in computer simulations conducted by Vitevitch, Ercal and Adagarla (2011).
Although spreading activation process as discussed above is mainly related to the local structural properties of words, the spreading activation process can also be used to account for findings showing that global structural aspects of words in the network influence lexical retrieval. Such findings have found evidence that closeness centrality of words influence spoken and visual word recognition (Goldstein & Vitevitch, 2017;Siew, 2018), that the network components that words reside in (i.e., whether it is in the largest connected component or an isolate) influence spoken word recognition, serial recall, and picture naming Vitevitch & Castro, 2015), and that assortative mixing by degree affects failures in lexical retrieval . Among these results, the finding of a processing advantage for high closeness centrality words in spoken word recognition (Goldstein & Vitevitch, 2017) is especially interesting as it goes against the general finding that greater similarity does not tend to help recognition (e.g., degree and local clustering effects result in poorer recognition; Luce & Pisoni, 1998;Chan & Vitevitch, 2009). Nevertheless, the same spreading activation framework can be used to account for these counterintuitive findings as well. To account for the processing advantage for high closeness centrality words, Goldstein and Vitevitch (2017) suggest that as words are retrieved over time, lexical representations are activated and activation can spread to connected words. Over time, lexical representation accrue small quantities of activation, with high closeness centrality words acquiring more activation due to their topologically advantageous location in the network. Such an account is analogous to Morton's (1969) classic logogen model, where the advantage for higher frequency words is attributed to higher resting levels of activation due from repeated retrieval-although in this case, words that reside in structurally important locations in the network are suggested to also have higher resting activation levels due to receiving spreading activation from other words in the network.

Creative processes
Another cognitive domain where spreading activation has proven to be a powerful model is creativity. Theories of the creative process attribute a key role of spreading activation over memory to account for individual differences in creativity (Mednick, 1962;Volle, 2018). These theories propose that creativity is related to the ability to combine together weakly related (i.e., distant) concepts into new concepts that are both novel and appropriate. The farther the original concepts are in a semantic network, the more novel the new concept will be (Mednick, 1962).

Mental navigation and cognitive search
A large body of research has shown that people search their internal cognitive spaces in similar ways as they would in an external, physical space (e.g., Hills, Todd, & Goldstone, 2008;Hills, Jones, & Todd, 2012;Wulff, Hills, & Hertwig, 2013), with the implication that spatial and cognitive search processes have similar evolutionary roots (Hills, 2006). Representing semantic memory as networks essentially produces a map of semantic space that allows one to mathematically trace search processes as paths in a network and make predictions about how structural properties of such network maps influence cognitive search behavior.
In one study, Iyengar et al. (2012) had participants play a word-morph game that required them to transform one word into another by only changing one letter at the time (e.g., ball--tall-tale--take). This is analogous to finding a path in a lexical network of word forms. The results showed that over time participants began to actively utilize network "landmarks," hub words, and words of high closeness centrality, to drastically improve their performance both in terms of time and minimal word forms used. In another study, individuals were able to successfully identify the shortest path between a start word to a target word within a pre-defined semantic network based on free association norms (Beckage, Steyvers, & Butts, 2012, Fathan et al., 2018. This suggests that participants are able to actively exploit both global and local structure of semantic memory in order to estimate the distance between two words and to further use that information to guide their search (see also, Kleinberg, 2000;West & Leskovec, 2012;West, Pineau, & Precup, 2009).

Similar conclusions were drawn from more open-ended search tasks requiring individuals
to actively explore their mental representations, such as category and letter fluency tasks. In fluency tasks, individuals are required to retrieve from memory as many elements belonging to a semantic category (e.g., animals) or words beginning with a specific letter (e.g., the letter S) as they can (Bousfield & Sedgewick, 1944;Romney, Brewer, & Batchelder, 1993). Cognitive network research based on behavioral data from the semantic fluency task has led to important insights into the structure of semantic networks across the lifespan (Goñi et al., 2011;Wulff, Hills, Lachman, & Mata, 2016), between low creative and high creative individuals , between the structure of the first and second languages of bilinguals (Borodkin, Kenett, Faust, & Mashal, 2016), and between people with low or high openness to experience (Christensen, Kenett, Cotter, Beaty, & Silvia, 2018).
The sequences that individuals produce in verbal fluency tasks (in particular when listing items from a semantic category) typically exhibits a high degree of clustering with regard to semantic relatedness and shared features (e.g., Troyer, Moscovitch, & Winocur, 1997). This type of clustered recall was proposed to be related to an active search process that dynamically switches between retrieval cues Hills, Mata, Wilke, Samanez-Larkin, 2011;Hills & Pachur, 2012;Wulff, Hills, & Hertwig, 2013). Other work on this type of verbal fluency has shown that micro (node-level) information such as PageRank centrality was most predictive of word recall (Griffiths et al., 2007) or random walks (Abbott et al., 2015, Zemla & Austerweil, 2018. Collectively, this work shows a promising application of studying network navigation and search processes as a means to formalize human processes operating on physical as well as mental spaces.
The studies discussed above provide important insights into how people navigate and retrieve information from their semantic and linguistic networks. It appears that people are able to successfully search their network in flexible ways to accomplish some kind of cognitive task, such as converting a word to another word, searching the semantic space as quickly and efficiently as possible, or generating creative ideas.

Disentangling structure and process
A critical and open debate in cognitive network science is whether investigating network structure and representations can be achieved independently from the retrieval processes operating upon it. This debate arises from the fact that usually both the underlying structure and the process operating on it are flexible enough to produce a wide array of behavior. This indeterminacy is illustrated in a recent debate on models of retrieval from semantic memory.
Modeling the semantic fluency task using a semantic space extracted from a text corpus, Hills et al. found evidence for an active search process that dynamically switches between sub-categories of the semantic space . Shortly after, Abbott, Austerweil, and Griffiths (2015) argued that a simple random walk model operating on a semantic network constructed from free associations-a model that does not require a switching process-is equally plausible as a mechanism of search in verbal fluency tasks, suggesting that a simpler model could reproduce the original results found by Hills, Jones, and Todd (2012).
However, note that Abbott et al. evaluated the verbal fluency search on a network estimated from free association data, which -as subsequently argued by Jones, Hills, and Todd (2015) -may already contain traces of the underlying search processes in semantic memory, rendering it unnecessary to account for such traces using an elaborate search model.
There have been a few further attempts to disentangle process and structure (e.g., Nematzadeh, Misceivie, & Stevenson, 2016;Avery & Jones, 2018); however, we will focus on the analysis conducted by  who examined the relation of semantic memory structure, creative ability, and intelligence, to provide an example of how network science methods can help unravel the contributions of structure and process on behavior. In creativity research, there are currently two main opposing theories on the creative process: The bottom-up associative theory of creativity emphasizes the importance of a flexible memory structure in facilitating the generation of novel ideas (Mednick, 1962;Volle, 2018), whereas the top-down cognitive control theory of creativity emphasizes the importance of cognitive control processes operating upon semantic memory in guiding the generation of novel ideas (Volle, 2018).  collected semantic fluency responses from a large sample of participants. This sample was then divided into four sub-groups according to the combination of two dimensions -low/high creative ability and low/high intelligence. Finally, animal category networks were estimated and compared for all four groups. The authors hypothesized that if creativity was more related to top-down processes, the semantic networks will differ along the low/high intelligence dimension (thus, related to network processes). Alternatively, if creativity was more related to bottom-up processes, the semantic networks will differ along the low/high creativity dimension (thus, related to network structure). The authors found that intelligence was related to measures of average shortest path length and modularity, creativity was related to the "small world-ness" measure, and the semantic network of the high intelligence/high creative group demonstrated a balance between these two opposing "forces" . Such an approach can potentially sidestep the challenge of disentangling structure from process, by examining different potential effects of processes on the same representation. Importantly, this study further illustrates the need for developing suitable empirical experimental designs in the field of cognitive network science.
The bottom line of this open debate is that unless one of the two, structure or process, is clearly identified, it is difficult, if not impossible, to make strong inferences from data about the other. Currently, a growing body of work focuses on examining the reliability and reproducibility of estimating networks in both cognitive and psychological networks Zemla & Austerweil, 2018;Wulff et al., 2016) and it remains one of the major challenges for network approaches to retrieval from semantic memory.

Summary
In this section, we discussed recent empirical work that provided a deeper understanding of the interaction of structure and processes that occur in cognitive and linguistic networks. We focused on the cognitive domains of lexical retrieval, creativity, and cognitive search to illustrate how the process of random walks or spreading activation can be implemented in a cognitive network representation to account for a variety of behavioral phenomena. These studies not only emphasize the importance of considering how the structure of the underlying network interacts with processes operating in it, but also the complexities of disentangling such structure and processes, particularly in the domain of cognitive search. We conclude that the consideration of structure and process emerges naturally from a network science approach by compelling researchers to explicitly define and model the relationship between structure and process in order to account for human behavior and cognition, enriching our theoretical understanding of the interplay between cognitive processes and cognitive structures in various domains.

Section 3: Network Dynamics across Multiple Timescales
Conceptualizing cognitive systems in terms of a network representation not only motivates cognitive scientists to think more explicitly about the structure of cognitive systems (Section 1: Network Representations of Cognitive Systems) and how cognitive processes might be captured by processes operating on a network (Section 2: Processes in Cognitive Networks), but also stimulates the question of how a particular structure came to be in the first place and how it develops across time.
In this section, we posit that semantic and language networks are inherently dynamicthe structure of such cognitive systems changes over multiple timescales in response to internal factors, such as the structural properties of the network itself or cognitive or memory constraints, and external factors, such as exposure to linguistic input and accumulation of semantic knowledge. The first part focuses on developing semantic networks to capture the process of language acquisition and the second part focuses on semantic networks of older adults. Finally, the third section will focus on more transient, dynamic changes that occur over semantic memory during creative insight problem solving.

Developing networks
Rather to empirically study normative language development using networks. Using this data, they constructed semantic networks of an average child at various time points and performed a statistical comparison of three different models of network growth (see Figure 6): preferential attachment, preferential acquisition, and lure of the associates. In the spirit of Barabási & Albert's model (1999), the preferential attachment model predicts that words that connect to well-connected words already known by the (normative) child would be learned earlier. The preferential acquisition model predicted that words were learned earlier if they were highly connected in the learning environment as approximated by the full (adult) semantic network.
Finally, the lure of associates model predicted that new words are more likely to be learned if they form more connections with the network of words already known by the child. Comparing these models, Hills et al. found that the preferential attachment model was in fact not a good fit to the CDI data and that the preferential acquisition model was able to best account for vocabulary growth in early semantic networks (Hills, Maouene, Riordan, & Smith, 2010;Hills et al., 2009aHills et al., , 2009b. This highlights that the learning environment plays an important role in language acquisition--words that occur in many different contexts (and hence are well-connected) in the learning environment are more likely to be acquired before less connected words in the learning environment. More importantly, these results suggest that in addition to preferential attachment (and other related variations), other growth models such as preferential acquisition could also lead to scale-free degree distributions in semantic networks.
The growth mechanisms underlying semantic and language networks remain, however, imperfectly understood. For instance, Hills et al. (2009b) found that the preferential acquisition model accounted for growth of semantic networks with edges constructed from free associations, but not for semantic networks with edges constructed from shared features, suggesting that different growth processes may govern different aspects of language. In phonological networks, evidence seems to support the lure of associates model (Carlson, Sonderegger, & Bane, 2014;Storkel, 2009), although it remains to be empirically tested against other growth models such as preferential attachment and preferential acquisition.
Studies have recently pursued new approaches by examining language acquisition in terms of feature networks (Engelthaler & Hills, 2017;Sizemore, Karuza, Giusti, & Bassett, 2017), or multiplex networks representing both semantic and phonological information (Stella et al., 2017;Stella et al., 2018). One particularly informative approach has been to study atypical acquisition processes. Examining the semantic networks of children who were classified as late talkers, Beckage, Smith, and Hills (2011) found that the network structure of late talkers had higher average path length and lower clustering as compared to the semantic networks of typically developing children, even when controlling for differences in network size or age of the child. This suggested a maladaptive tendency in late talkers to acquire "odd" words that were less connected in the semantic network (see also Ke & Yao, 2008). Studying the structure of semantic networks of children with cochlear implants, Kenett et al. (2013) also found differences compared to typically developing children. Specifically, they observed shorter average path lengths for children with cochlear implants relative to normative acquisition, suggesting an under-development of the semantic network due to impoverished input.

Aging networks
The development of semantic networks does not halt with onset of adulthood. Language learning and change continues throughout the lifespan. Wulff et al. (2016) examined the semantic networks of younger and older adults (as inferred from semantic fluency data) and found that the small world index, a measure of small-worldness that controls for network size (Humphries & Gurney, 2008), was smaller for older adults as compared to younger adults. Using free associations obtained from lifespan sample, Dubossarsky, De Deyne, and Hills (2017) similarly found that structure accumulated in early life (as evidenced by lower entropy, shorter path length, smaller clustering coefficient, and smaller small world index) reverses, in parts, in later life (see Figure 7). These findings fuel a captivating hypothesis on age-related decline in cognitive function. Compared to younger adults, older adults usually take more time and perform worse on variety of cognitive tasks involving memory, concentration, and reasoning which is commonly attributed to cognitive slowing (Healey & Kahana, 2016;Salthouse, 2010).
An interesting alternative to this account based on a network approach, however, is that changes in the underlying representation of knowledge might be held partly accountable for these behavioral changes (Ramscar et al., 2014;2017). It is well documented that older adults have access to a larger vocabulary, implying a larger network representation exacting higher search costs in accessing the representation. Assuming some process akin to spreading activation, it is further conceivable that other changes in structure affecting, for instance, the average path length or clustering coefficient, of older adults' networks may play an additional role, offering a different perspective on what is normally attributed to cognitive slowing due to aging. Thus, in the case of age-related cognitive decline, the application of a network approach has led to an interesting rival explanation to established theories paving the way for an entirely new perspective on an important issue in our aging society. densely connected in adulthood, and becomes less connected in old age. Adapted from Dubossarsky, De Deyne, and Hills (2017).

Young adult
In light of the growing importance of age-related degenerative diseases, such as dementia and Alzheimer's disease, it is especially worthwhile to study models that may capture possible mechanisms underlying these changes in memory representations. Towards this goal, studies have focused on comparing healthy individuals to clinical patients. In one study, Borge-Holthoefer, Moreno, and Arenas (2011) conducted simulations based on a network model of language degradation in order to account for hyperpriming among people with Alzheimer's disease. Hyperpriming refers to the finding that the priming effect for related words (e.g., 'lion' and 'tiger') in verbal fluency tasks was counterintuitively larger in patients with Alzheimer's than in those of healthy controls (Laisney et al., 2011). Their simulations showed that a process of edge degradation was able to account for the effects among people with Alzheimer's disease.
Specifically, the effect appeared to be driven by the fact that weak (distinctive) associations were lost earlier than stronger (common) associations, leading to a loss of distinctiveness between primes and targets and an increase in priming effects for related words. This result highlights that changes in aging semantic networks are not simply the "inverse" of early development.

Creative insights as network dynamics
Here we will look at network dynamics that occur at more immediate timescales, by considering examples from research on higher-order cognitive processes related to conceptual representations and creativity. Specifically, solving problems by insight-when a solution suddenly "pops" into one's head (Shen et al., 2018)-could be a research question amenable to being quantitatively investigated via a cognitive network perspective. The study of creative insight is not straightforward, and many psychologists struggle to provide a mechanistic explanation for the moment of insight (e.g., Shen et al., 2018). By reframing the search for a solution to a problem as the discovery of a "shortcut", or path, in the semantic network, Schilling (2005) suggests that when a non-obvious solution to a problem is found, changes in the semantic network structure may occur as a creative solution typically involves the addition of nodes or edges that lead to a drastic reduction in the average path length in the network (e.g., Durso, Rea, can examine Schilling's hypotheses empirically, by comparing, for example, the structure of a semantic network before and after insight problem solving. The provocative ideas proposed by Schilling (2005) are largely in line with current theories of semantic memory that posit a dynamic representation of conceptual structures that can be flexibly recombined in different contexts and in response to various task demands (Yee & Thompson-Schill, 2016, and in the case of semantic search, Fathan et al. 2018). While we note the importance of carefully designing empirical studies that can disentangle structure from process (see Section 2.5), we argue that the network science approach can help make hypotheses about creative processes explicit and mathematically tractable. For instance, preliminary work by Kenett and Thompson-Schill (2017) measured semantic networks before and after participants completed a conceptual combination task. This conceptual combination task primed participants to focus either on the attributes of concepts or on the relations between concepts. The authors found that the semantic network post-manipulation showed more connectivity if participants combined concepts relationally (e.g., tennis ball is a ball used to play tennis) whereas the network showed less connectivity if participants combined concepts based on their attributes (e.g., zebra clam is a clam with stripes like a zebra). Furthermore, both post-manipulation semantic networks exhibited lower average shortest path length and lower modularity, indicating that participants can adapt their semantic representations to the task at hand (Kenett & Thompson-Schill, 2017).
Relevant to this discussion is recent work by Christensen, Kenett, Cotter, Beaty, and Silvia (2018) who found that the semantic networks of people with higher openness to experience are more interconnected and better organized as compared to the semantic networks of people with lower openness to experience. As being partial to new and diverse experiences is frequently related to enhanced cognitive flexibility and creative achievement (Kaufman et al., 2016), the results reported by  and Kenett & Thompson-Schill (2017) suggest a plausible mechanistic account of the differences observed in the semantic networks of highly creative and less creative people . Openness to experiences encourages the accumulation of diverse types of associations and information, whereas combining concepts relationally stimulates the association of concepts that are not immediately obvious. Hence, the less modular structure of semantic network of highly creative people  may be due to the perpetual construction of links between disparate concepts (that less-creative people are less likely to do), reducing path length and modularity in the network (Schilling, 2005). The convergence of these separate lines of research (personality, semantic memory) show how one can quantitatively investigate cognitive and psychological constructs as dynamics that occur on a network structure. Thus, cognitive network science offers an exciting avenue to investigate and identify more complex interactions across such psychological and cognitive domains.
Re-conceptualizing higher order cognitive processes (such as creativity and insight problem solving) as dynamic processes implemented on a cognitive network provides a powerful quantitative framework for investigating both the cognitive mechanisms that lead to creative insights, and how these processes (when they occur consistently over time) can lead to structural differences observed in the semantic networks of creative and less creative people.

Summary
The tools of network science have revealed interesting topological differences in the network structure of various populations (e.g., young vs. older adults, healthy vs. clinical populations, low creative vs. high creative persons). Such findings build on the idea that models using network representations and processes are inherently dynamic, compelling cognitive scientists to formalize theoretical and algorithmic explanations for how these differences emerge in the first place. By combining the process or growth models provided by network science with empirical data from the cognitive sciences, such models can formalize relationships between (network) representation and processes that operate within the representation, and also external factors such as the linguistic environment of young language learners, transient changes that are due to creative insights and memory restructuring, or the accumulative effects of semantic knowledge acquired in a person's lifetime. This formalism results in a framework that can offer explanations and predictions of changes in the structure of cognitive representation, which in turn affects language and cognitive processes in a continual, interacting cycle.

Section 4: Summary and Conclusions
In this review, we have attempted to demonstrate the usefulness of the network science approach to the study of cognition in at least three ways: 1. Network science provides an attractive quantitative approach to representing cognitive systems.
To demonstrate this point, Section 1 discussed how networks can represent a variety of cognitive systems, including language, semantic memory, personality traits, and the information environment of individuals (see Table 1). Furthermore, we highlighted a host of network measures that are available to the researcher when he or she commits to the theoretical decision of representing the cognitive system of interest as a network, and reviewed previous research using these tools to characterize the structure and behavior of networks on the micro-, meso-, and macroscopic level in order to derive novel insights.

2.
A deeper understanding of human cognition can be achieved when structure, i.e., the underlying network, and processes operating on a network structure are considered in tandem within a network science framework.
Section 2 focused on processes operating on networks. Network representations of cognitive systems, particularly in the area of language and semantic memory, are often used to represent a latent mental structure that require the assumption of some process to link the network to observable behavior. Adopting a network science approach naturally compels researchers to consider the interaction between structure and process, which has been especially useful in the three research domains discussed in Section 2 (lexical retrieval, creativity, and cognitive search). Finally, we briefly discuss the difficulties in dissociating structure and process, particularly as it relates to the modeling of behavioral outputs in retrieval tasks from semantic memory, and suggest ways in which network science methods can enrich the investigation of such cognitive phenomena.

Network science provides a framework to model structural changes in cognitive
systems at multiple timescales.
In Section 3, we discussed how network science can be used to study the development of cognitive systems, enabling a better understanding of cognition at both the early and late stages of human life, as well as structural changes that occur at more immediate timescales, as related to higher order cognitive processes such as creative insight and problem solving. Such cognitive systems can be modeled as a dynamic network representation that changes in response to the external environment (e.g., linguistic input) and the internal processes (e.g., problem solving and search) that operate within the system. The research discussed in this section demonstrates how network science approaches can be used to quantify structural changes and the dynamics of cognitive systems at multiple timescales.

Future directions
It is clear from this review that network science approaches have contributed much to the study of human cognition. But it is important to emphasize that cognitive network science is a relatively young field and many methodological and theoretical challenges remain to be addressed. For example, as previously discussed in Section 2.5, to what extent can specific aspects of the network structure and the processes operating in the network be disentangled?
Below we briefly highlight three milestones that cognitive network science needs to achieve in the near future in order to become a mature research paradigm in the cognitive sciences.
One critical milestone needed in cognitive network science is the development of inferential methodologies to analyze empirical networks, that is, networks that are inferred from empirical or behavioral data. Currently, statistical models that allow for hypothesis testing when comparing empirical networks remain a major challenge. This is mainly due to difficulties in estimating or collecting a large sample of empirical networks and the existence of few statistical methods to compare networks (Moreno & Neville, 2013). In these cases, bootstrapping methods over comparable networks might be a solution (Baxter, Dorogovtsev, Goltsev, & Mendes, 2010).
Similarly, issues in how to minimize spurious connections in psychological and cognitive networks are currently debated and require further methodological development .
Another crucial milestone is the development of methods to represent psychological and cognitive networks at the individual level. Psychological and cognitive constructs vary across individuals and aggregating across participants in group-based cognitive network analysis may conceal nuanced differences across individuals. To date, only a few attempts have been made at representing individual semantic networks (Benedek et al., 2016;Morais et al., 2013;Zemla & Austerweil, 2018;Zemla, Kenett, Jun, & Austerweil, 2016). Developing a reliable and easy to apply methodology to represent an individual's semantic network will allow researchers to design studies that relate such networks to other cognitive and neural measures.
Finally, a third milestone is the development of new network science methodologies to quantitatively study specific theoretical issues across different cognitive domains. Two such examples were briefly discussed in the review: the application of multiplex network analysis to examine how different cognitive domains interact (e.g., Stella et al., 2017, and the application of percolation theory to study cognitive phenomena such as memory decline or flexibility of thought (e.g., Borge-Holthoefer & Arenas, 2011; Kenett et al., 2018). Cognitive network science could also benefit from adopting state-of-the-art network methodologies used to study neural systems and brain dynamics, such as network dynamic analysis (Garcia, Ashourvan, Muldoon, Vettel, & Bassett, 2018) and network control theory (Medaglia, 2018;Tang & Bassett, 2018). Network dynamic analysis examines time varying community assignments over brain functional connectivity networks and has attributed state flexibility-variation of assignment of a brain region to a specific community across time-to capacities such as motor-skill learning and language comprehension (Garcia et al., 2018). Network control theory quantifies the extent that different nodes in a network drive dynamics over the network. Recent studies have applied network control theory to the analysis of white-matter connectivity networks to examine the 2018). Importing such state-of-the-art methods to the cognitive domain could greatly advance the study of dynamics in cognitive networks.

Cognitive network science: A new frontier
The aim of this review was to highlight and emphasize the feasibility and significance of applying network science methodologies to study cognition. Such applications allow the quantification of theoretical cognitive constructs and direct examination of cognitive theories in domains such as language and memory. The network framework also provides a means to model and formalize theoretical and mathematical descriptions of dynamics operating in cognitive systems. The work that has been conducted in this new field of cognitive network science has already provided novel insights on cognitive issues such as lexical retrieval, language acquisition, memory search and retrieval, bilingualism, learning, creativity, personality traits, and clinical populations.
In this review, we have placed a particular focus on language and semantic networks that capture various types of relationships between words. However, as alluded to where relevant research was available, the usefulness of network science in cognitive science is by no means limited to the linguistic domain. Network science is a general-purpose toolbox for many types of cognitive systems provided they can be represented as a network. This also implies that the usefulness of network science depends on both the problem at hand and, of course, the researcher, who will make theoretically motivated decisions about what aspects of the problem can and should be represented as a network, what tools and measures should be applied to analyze the network representation, and how to link network structure and process to offer meaningful insights into empirical and behavioral data. Network science is, thus, no panacea for the challenges faced by cognitive research, but when used appropriately it can produce important and novel insights for cognitive research, as adeptly demonstrated by the vast array of research on cognitive network science covered in this review.
Our understanding of any cognitive or language-related process is necessarily incomplete if we do not consider the structural properties of the cognitive system that the process is occurring in. While network science provides cognitive scientists with a language to quantify and study the structure of these cognitive systems, the cognitive and language sciences has developed a suite of experimental tasks that can provide crucial behavioral evidence that constrains and informs network models of cognition. The judicious combination of these two approaches will lead to continued insights into a variety of behavioral and cognitive phenomena and strengthen psychological theories of lexical access, memory retrieval, cognitive search, language acquisition, cognitive decline, and creativity, as well as many other related domains of cognitive science that will certainly benefit from the application of network science methods.