This paper covers similarity analyses, a subset of multivariate pattern analysis techniques that are based on similarity spaces defined by multivariate patterns. These techniques offer several advantages and complement other methods for brain data analyses, as they allow for comparison of representational structure across individuals, brain regions, and data acquisition methods. Particular attention is paid to multidimensional scaling and related approaches that yield spatial representations or provide methods for characterizing individual differences. We highlight unique contributions of these methods by reviewing recent applications to functional magnetic resonance imaging data and emphasize areas of caution in applying and interpreting similarity analysis methods.

Researchers who engage in neuroimaging methods face many daunting challenges associated with the vastness and complexity of the data gathered in even a modest experiment with few participants. These data can be analyzed at several different levels, each of which may serve a different theoretical purpose. Recent methodological advances in multivariate pattern analyses (MVPA) have shifted the focus from examining the responses of individual voxels to examining patterns of neural activity associated with different cognitive processes and mental representations (for reviews of MVPA approaches, see [

Similarity based methods are very flexible. A variety of methods can be used to construct a pairwise similarity matrix to represent the proximity relationships among the entities of interest. In fMRI research these entities are often voxel activation patterns associated with the corresponding states or cognitive representations elicited by presentation of different stimuli, tasks, or conditions. Additionally they may correspond to different brain regions or even individuals themselves. While many MVPA methods use patterns of activity to classify different states or representations, similarity based methods examine the relationships among those patterns to make inferences about relationships in the data at the neural, cognitive, or behavioral levels of analysis. These methods provide valuable insights into processes and representations that may be inferred from the data. They have received a considerable interest in recent neuroimaging literature, as seen from numerous applications, as well as methodological advances [

Similarity analyses have a long history of wide-ranging applications in the sciences. For example, multidimensional scaling (MDS) has been used to visualize data in such diverse fields as psychology, biology, geography, marketing, sociology, physics, and political science. Many applications in psychology have been directed toward understanding perceptual and conceptual representations and processes associated with interobject similarity (e.g., [

When used in conjunction with MVPA methods, the examination of similarity relationships offers several advantages over simply focusing on activation patterns of conditions directly. Analyzing the similarity structure of activation patterns allows one to evaluate hypotheses without specifying brain regions or locations [

The flexibility of similarity based methods allows for comparison of internal representations derived from fMRI data to those based on behavioral responses, computation, or physical characteristics of stimuli. These comparisons can ground hypotheses about neural representations [

The construction of similarity measures of activation patterns between conditions, instead of distributed patterns themselves, has been successfully used for object decoding across individuals [

We will refer to the entities under investigation as objects, for consistency with the multidimensional scaling literature. For example, objects can refer to stimuli, brain regions, or individuals. Similarity analyses focus on the object-by-object matrix of proximities, a generic term that refers to either similarity or dissimilarity. The

Each object can be represented by a multivoxel pattern of brain activity values. These patterns of activity can be viewed as points in a multidimensional space with dimensionality equaling the number of voxels. Multivoxel patterns of activity can be either estimated or extracted from neuroimaging data. The reader may benefit from general discussions of data used for MVPA [

Once the multivoxel activity pattern has been abstracted from the data, one has to decide which voxels to include in the analysis. The total number of voxels is typically large, and inclusion of voxels that are not relevant introduces noise that will obscure the systematic relationships in the data. There are several possibilities, and a particular choice largely depends on the application area. Analyses often focus on theoretically motivated regions of interest (ROI) and may additionally be constrained by further criteria. For example, Kriegeskorte and colleagues [

There are several ways to measure proximities between pairs of objects. Generally, measures of dissimilarity (e.g., distances) are used to compare items, and measures of similarity (e.g., correlation) are used to compare variables. Proximities for each pair of objects are organized into a square matrix of proximities. A proximity matrix is assumed to be symmetric with minimum distances (or maximum similarities) on the diagonal. When proximities are calculated from patterns of activities, these assumptions generally hold.

In summary, proximities are easily computed with neuroimaging data based on correspondence between pairwise activation patterns. One difficulty lies in the selection of relevant input variables (i.e., voxels), as the inclusion of large numbers of irrelevant variables will typically mean that relevant proximities are obscured by noise. Extreme caution should be exercised in choosing unbiased variable selection criteria.

In the neuroimaging literature, there are examples of proximity matrices created from activation patterns or confusability patterns. Proximity for a pair of objects has been defined as pairwise Euclidean distances [

Objects × objects proximity matrices can be derived from many different sources. For example, they may correspond to different brain regions, individuals, or data collection methods. The relatedness of two proximity matrices can be evaluated with a correlation coefficient and tested by randomization [

We discuss two sets of exploratory multivariate techniques that are commonly used in neuroimaging applications for visualization of similarity structure, multidimensional scaling, and cluster analysis.

Multidimensional scaling is a set of techniques for analysis of proximities (similarities or dissimilarities) that reveals structure and facilitates visualization of high dimensional data. MDS has a long history in psychology and neuroscience and has been used extensively for analyzing behaviorally derived data (e.g., [

Assume that a measure of proximity (

MDS attempts to find a configuration

Determining the MDS solution is typically an iterative process in which the badness-of-fit measure for the MDS representation, called stress, is minimized. The objective function that is minimized is a normed sum-of-squares of representation errors,

There is no statistical test for selecting the correct number of dimensions. Typically researchers conduct the MDS analysis for several successive values of the number of dimensions and select the solution that seems most appropriate. A plot of dimensionality versus fit, called a scree plot, is useful in selecting the appropriate number of dimensions when there is a clear elbow. Increasing the number of dimensions reduces stress values. Choosing too many dimensions results in over fitting the data, so that the configuration reflects unstable influences of noise. For

There are a number of factors to consider in deciding on the number of stimuli to use in a prospective study or the appropriateness of using MDS on a given data set. The number of fitted dimensions depends on the number of stimuli, as a perfect solution may be achieved with

The previous basic algorithm was presented for a single matrix of proximities (two-way data: objects × objects). However, most neuroimaging data is collected for a group of individuals, and so a methodological question arises concerning how to aggregate individual proximity matrices into a single analysis. If little commonality exists between individual proximity matrices, aggregating the data is not meaningful, as the average will not represent any of the constituents. On the other extreme, if the differences between individual proximity matrices are not systematic, interpreting the differences is not meaningful. Most data sets, however, lie between these two extremes. Each proximity matrix can be analyzed separately, although it is difficult to summarize the results for a group of individuals or compare the results across groups. Moreover, additional data may be needed to obtain stable results for an individual [

Several algorithms have been proposed to simultaneously analyze multiple proximity matrices (three-way data: objects × objects × individuals). These approaches offer two key advantages over analyzing each proximity matrix individually or averaging the matrices together. First, in cases when individual proximity matrices are noisy, these methods take advantage of commonalities among individuals. Second, group space provides a useful basis for comparison of individuals [

The most popular algorithm for individual differences scaling is INDSCAL [

STATIS, which stands for

Cluster analysis seeks to discover natural nonoverlapping groupings of objects. Hierarchical clustering techniques are perhaps most popular in neuroimaging applications for similarity structure visualization. Hierarchical clustering techniques produce a nested sequence of partitions and can be either agglomerative or divisive. In agglomerative hierarchical clustering each object starts out in its own group. In a series of successive mergers similar objects get grouped together until finally all objects are grouped together. Divisive hierarchical methods operate in an opposite direction. Types of hierarchical clustering vary on how the similarity is defined for groups of objects. For instance, average linkage computes an average distance, complete linkage computes maximum distance, and single linkage computes minimum distance between clusters. Once objects have been grouped together in hierarchical clustering, they cannot be regrouped. Hierarchical clustering results are visualized with a dendrogram, a tree diagram showing successive groupings of the objects. Selecting a partition is thus equivalent to cutting the dendrogram at a given height. The clustering results depend both on choice of proximity and linkage methods. A challenging decision in cluster analysis is to select a number of clusters and to check the validity of the solution. For a detailed treatment of the cluster analysis the reader is referred to Johnson and Wichern [

We will next review the applications of similarity analyses in fMRI literature. Both MDS and cluster analysis have been used as exploratory tools for visualization of similarity structure derived from fMRI data. Additionally, RSA has been used to test hypothesized relationships between similarity matrices. We will group the studies based on entities under investigation: stimuli (focusing on internal representations), individuals, and brain regions.

Object representation in the brain has been extensively studied with the aid of similarity analyses of single-cell recordings data (e.g., [

Object representation across different tasks has been examined by Tzagarakis et al. [

Representation of objects that come from different categories has also been examined. O’Toole et al. [

In an investigation of category structure of objects, Op de Beeck et al. [

Some of the aforementioned studies have used representational similarity analysis and compared the internal representation derived from fMRI data to another representation. The internal representation of objects derived from fMRI data has been compared to internal representation derived from perceptual space [

In addition to object representation studies, MDS has been used to examine the internal representation of affective states [

Similarity analysis techniques can also be applied to an individuals × individuals proximity matrix. For example, MDS has been used as part of the algorithm for assessing group homogeneity [

Another application of MDS focuses on the representational space of cortical areas. In one of the earlier applications of MDS to neuroimaging data, Friston et al. [

MDS and cluster analysis are implemented in most statistical packages. MATLAB (statistics toolbox), R, SAS, SPSS, and SYSTAT provide functions for classical and nonmetric multidimensional scaling and cluster analysis. Individual differences scaling is implemented in R with indscal() function in the SensoMineR package and the smacof package [

We have reviewed the advantages and applications of different methods for examining similarity structures of activation patterns, along with potential cautions for interpreting these analyses. Using similarity as a level of analysis allows for comparison of representational structures across individuals, brain regions, and data collection methods. These analytic methods provide useful exploratory visualization tools. More importantly, used in conjunction with other methods of fMRI data analysis, similarity analysis methods provide a means for testing correspondence between similarity structure derived from imaging data and that derived from other sources, such as physical similarity or perceptual similarity. Similarity based methods, representational similarity analysis in particular, have been instrumental in examining hypotheses of neural representation of objects through comparison of internal representations derived from fMRI data to those derived from behavioral data and those derived from physical stimulus attributes.

The authors have no conflict of interests to declare.