Large-scale models of neuronal structures are needed to explore emergent properties of mammalian brains. Because these models have trillions of synapses, a major problem in their creation is synapse placement. Here we present a novel method for exploiting consistent fiber orientation in a neural tissue to perform a highly efficient modified plane-sweep algorithm, which identifies all regions of 3D overlaps between dendritic and axonal projection fields. The first step in placing synapses in physiological models is neurite-overlap detection, at large scales a computationally intensive task. We have developed an efficient “Staggered Walk” algorithm that can find all 3D overlaps of neurites where trillions of synapses connect billions of neurons.
Simulating brain structures with large-scale neuronal models lets researchers precisely manipulate features of simulated neural tissues and observe both local and global properties of neural systems. During the last decade, large-scale brain modeling has risen in prominence, with a wide range of publications on brain-scale models[
Most large-scale modeling research groups focus either on networks that are highly realistic down to the individual axon collaterals and dendrite branches of each neuron [
Large-scale neuronal models range in accuracy from simple, randomly probabilistic networks [
A critical and complex part of large-scale neuronal modeling is the creation, or initialization, of the myriad of specific details within the model. Even though many modeling studies focus on analyzing and improving simulation run times, the setup and initialization of models with trillions of details can be time intensive and represent significant computational challenges when connectivity patterns are complex [
Synapse placement is of great concern in large-scale models, since synaptic connection patterns control the flow of excitation and inhibition through a neuronal network. A typical approach to initialize synapse placement is to prespecify a few pools of interconnected neurons and define the likelihood that any two neurons in a given pool are connected by a synapse[
Rather than specifying neuronal pools, our approachdraws from bothstatistical connectivity and spatial location mappings. Instead of defining neuronal pools without any sense of spatial geometry, we placeneurons in a three-dimensional (3D) Cartesian (
The most critical step for our approach is determining axonal-dendritic intersections once all volumes of potential connectivity have been specified by placement of all neuron instances. The task requireswalkingthroughthree-dimensional spaceand determining where synaptic volumes overlap. For neuronal simulations involving many millions of neurons, finding overlapping volumes poses a significant computational challenge.
To perform neurite-overlap detection efficiently, we have developed a practical algorithm that performs a walk through axonal and dendritic volumes. There are two variants of the algorithm: one “Staggered Walk” (SW) that performs the walk blindly in one predetermined dimension (e.g.,
The algorithm is particularly useful in the rapid determination of connectivity between millions of neurons in moderate-scale to large-scale models. Staggered walks provide an automated method to create verifiable simulation models by reproducibly specifying details of neuron placement and synaptic connectivity. Our algorithm scales well and within hours permits the rapid creation, or replication, of models containing trillions of synapses.
Our method uses C++ programs crafted to give the same results on many different computers. Models are specified by textual descriptor files. Results of model creation are checked by visualization programs that show neuronal soma locations and synaptic connections.
Our staggered walk algorithm is implemented in C++ and uses the C++ Standard Template Library (STL) for data structures [
Our approach to building brain models starts with the creation of detailed parameter specifications. The program reads in a parameter text file that statistically describes the neuronal and synaptic configuration of the brain tissues to be modeled. Our program can be modified to accept input files in other formats, such as neuronal description parameters in NeuroML [
We have tested our algorithm with cerebellar models based on statistical and morphological data collected from the literature [
Human Cerebellar Parameters $PARAMS NCELLT = 12, MSIMYMX = 6000, MSIMXMX = 2800, $END P N G B A S T I C M R D P IS FOR PURKINJE CELLS $CELL MCY = 50, MCX = 75, MAYY = 2, MAXX = 2, MAZZ = 2, MAZO = −10000 DN = 2, MDY = 10,2, MDX = 240,2, MDZ = 320,2, MDZO = 160, −5, ... SYNDA = 0,0,0,0,0,0,0,0,0,0,0,1000, $END N IS FOR GRANULE CELLS $CELL MCY = 5, MCX = 5, MCZLVL = −20, MCZ = −20, CZLN = 16, TCZP = 750,750,750,750,750,750,750,750,750,750,750,750,750,750,750,750, MAYY = 6000, MAXX = 2, MAZZ = 2, MAZO = +0330, MDY = 10, MDX = 10, MDZ = 20, ... SYNDA = 1000,0,250,750,750,750,750,750,0,0,0,0, $END
The input parameter file contains a list of cell types, what pairs of cell types can form synapses, the density of each type of soma in the model space, the axonal and dendritic process shapes and positions relative to the soma of each instance of a cell type, plus statistical properties about synapse concentrations and whether they are inhibitory or excitatory. See the Appendix section for more details. The distances of each synapse from its axonal and dendritic somas combined with parameters specifying propagation speeds in axons and speeds at different distances within dendritic trees determine propagation delays (and attenuations) for action potentials passing though each synapse. To lessen computer memory needs and execution times, our present modeling system does not consider branching details within dendritic trees.
After parsing the input file, the program places all cell instances in the 3D space. The axonal and dendritic regions, where all synapses are located, are approximated as axis-aligned bounding boxes (AABBs), as seen in Figure
Illustration of four cerebellar neurons and their synaptic regions.
Figure
Axonal and dendritic AABBs can be broken into several smaller regions to model cell synaptic positions more accurately. In a test of a cerebellar model that contained 65 million synapses, we halved each dendritic region along
We use our visualization code to inspect and analyze models created with the SWD algorithm. Figure
Computer-generated image of Purkinje, Golgi, and granule cell axonal (blue) and dendritic (green) synaptic regions.
In this section we define the problem of neurite overlap detection, discuss the limitations of previously published search methods, and explain our staggered walk methods in detail.
The task of finding volume overlaps in a three-dimensional space of billions of axonal and dendritic volumes can be expressed by the following generalized problem definition: There are billions of densely packed rectilinear boxes. Sides form axis-aligned bounding boxes (AABBs) Box sides are axis-aligned in three dimensions: Boxes vary drastically in sizes and shapes. There are two classes of boxes: axonal (blue) and dendritic (green) Boxes can be colinearly located; faces may end at the same Boxes can be enclosed within other boxes.
The goal is to identify all axonal and dendritic intersections and compute overlapping pairwise (axonal-dendritic) regions in 3D space.
The most straightforward way to determine all pairs is to compare all axons to all dendrites. The obvious first optimization is to look for overlaps between neuronal synaptic regions only for cell type pairs that can form synapses. We refer to this pairwise constrained full comparison as the Naïve Algorithm. For each type-pair allowed to form synapses, the locations of all axonal boxes versus all dendritic boxes need to be compared to find all
From computer science, we have many traditional one-dimensional sweep line algorithms, such as Shamos and Hoey [
Kozloski et al. [
Kozloski’s touch-detection algorithm has a similar goal to ours of locating synapses, but it targets models with much greater morphological detail and spatial precision than ours [
An important part of the approach of Kozloski et al. is the decomposition of each model into many small parts, distributing the work evenly onto thousands of Blue Gene processors. Touch-detection efficiency is improved by slicing the model into small volumes in which only segments close enough to touch are analyzed to compute precisely whether a touch occurs. To find touches rapidly in parallel, they distribute the segment data across 4096 supercomputer nodes, with four processing cores per node.
Apart from general differences in the applicability of Kozloski’s and our synapse placement algorithms, the methods differ markedly in their steps to find actual touches where a synapse should be placed. Within each volume of Kozloski’s volume-based data decomposition, the initial determination of a possible touch is calculated by finding intersections between segment bounding spheres. The complexity of their algorithm is
Touch detection algorithms differ by the bounding objects being compared, for example, spheres or boxes. Many factors should be considered when choosing bounding objects that both tightly fit the model and can be compared efficiently on the available computing hardware. AABBs can be described easily by a minimum and maximum 3D point pair. Determining whether two AABBs have a 3D overlap is computationally easy because it requires only simple subtractive comparisons and allows fast rejection as soon as a comparison fails in any dimension. Spheres have an advantage that they require slightly less storage space, just a single 3D point and a 1D radius length, but their comparisons involve four multiplicative operations and have fewer fast rejection points. For instance, when comparing two AABBs, touch detection fails if the minimum
The cerebellar cortex has a highly regular structure with specific dendritic and axonal ramification patterns and locations [
Our solution to finding axonal-dendritic 3D overlaps in which to place synapses uses a sweeping plane algorithm, similar to existing sweeping line predecessors [
Consider only pairs of cell types that are allowed to make connections. Count the overlaps per sweep in each dimension, Set the order for axes to be searched in future comparisons of region face coordinates to match the axes in the ascending order of overlap counts from Step 2; the axis with the fewest 1D-overlaps along its dimension is walked first.
Primary => Secondary => Tertiary (e.g., Find all overlaps along the primary axis using a sweeping plane search, which we call a staggered walk*.
*The staggered walk step analysis assumes that the counts of axonal and of dendritic boxes have roughly the same value,
Sort the list of green boxes and the list of blue boxes by primary dimension (ordered by begin-face coordinates). Traverse both lists with a plane sweep in the primary dimension, queuing “open” items (ordered by end-face coordinates). Any new green or blue faces are compared against their complementary (blue versus green) “open” box items. The “open” items are “closed” when they are no longer able to intersect any item newly reached on the opposite side. For each overlap in the primary dimension (1D-overlap), see if there are coordinate overlaps in both the secondary and tertiary dimensions (3D overlap). Calculate the volume of each overlap which occurs in all three dimensions and place the expected number of synapses randomly within the overlap volume.
Total cost for SW is
Total cost for SWD is also
Other interesting data structures that can be implemented for storage of “open” items during a staggered walk may change the costs of Steps (b), (c), and (d). At present, our C++ code uses a variable length vector for simplicity and optimal data locality. The number of vector element reorderings is small when compared to the number of traversals of the open list for comparisons (
Determining the best dimension (
To minimize the danger of
Without SWD, there is no quick way to know the best possible dimension to perform the walk. Traversing all three dimensions or guessing the wrong primary dimension can require a large number of comparisons. As seen in Figure
Number of primary comparisons to detect all one dimensional overlaps of granule axons with Purkinje dendrites.
In the SWD version, additional Step 2 determines the best dimension along which to perform the primary plane sweep for the selected cell type pair. It requires sorting both classes of neuritic boxes three times, once for each dimension; complexity is
Table
Difference in performance between the naïve full comparison and our staggered walk dynamic (SWD) algorithm.
56,462,784 Overlaps |
222,400,035 Overlaps |
496,905,207 Overlaps |
14,163,610,444 Overlaps |
|
---|---|---|---|---|
Naïve algorithm | 51 minutes | 4 hours | 10 hours | 28–34 days* |
Staggered walk dynamic | 1 minute | 3 minutes | 6 minutes | 150 minutes |
*The 28–34 days value is extrapolated based on two factors: the required number of calculations of overlaps and the calculations per time demonstrated by the Naïve and SWD data runs for smaller patch sizes on the same computer.
Computing hours for Staggered Walk (SWD) to find the billions of overlaps in twelve sizes of cerebellar models.
The data for Table
In this paper, we have targeted efficient model initialization as a key aspect of large-scale morphological neuronal modeling. We describe a staggered walk algorithm that successfully solves the problem of neurite overlap detection and synapse placement between axonal and dendritic regions in three-dimensional space. On a supercomputer, creation of models with trillions of synapses can be achieved within hours, rather than days or months.
Despite the demonstrated utility of our SWD approach, several limitations should be noted before attempting to implement this algorithm. Rather than just knowing roughly which areas project to which other areas, as is the case in simpler, faster brain modeling methods, our algorithm requires information about the geometric spans of axons and dendrites. This information may be prohibitively difficult to gather in some regions of the brain. Cerebellar cell and synapse structures are very consistent and regular; our algorithm makes use of the regularity when determining the critical primary dimension for the walk to locate synapses. Irregular axonal and dendritic regions may require many smaller bounding boxes to accurately represent their complex morphology.
The staggered walk dynamic algorithm allows us to generate dense neuronal models with trillions of overlaps and synapses in only a few hours of processing, hundreds of times more efficiently than naive
After one section ($PARAMS … $END) with overall model parameters, each input file has a section ($CELL … $END) dedicated to each cell type. The section for a cell type specifies the density and placement of its somas in the model space, the shapes and soma-relative positions of the axonal and dendritic synaptic regions for each soma, the density of synapses from its axons to each other cell type wherever their axonal and dendritic regions overlap, and electrical characteristics of its dendrites, somas, and axons.
The section for Purkinje (P) cells places the centers of all P-cell somas into a single plane at
Parameters in the section for granule cells (N) specify that granule centers are packed
This sample parameter file shows an excerpt from a text file for a model of a 6,000
In Figure
Axonal (blue) and dendritic (green) regions were generated from a tiny cerebellar model. Individual synapses are identified by red dots and can occur only where blue and green boxes overlap. The two horizontal lines, at the top of the image, are the horizontal segments of the parallel fibers from the granule cells. See Figure
Three nondynamic staggered walks produced drastically different comparison counts depending on which was the fixed starting dimension (
Data for these models were generated on a single processor of a desktop computer using our full human cerebellar cortex parameters for different sizes of cortical patches. Data were collected for twelve models ranging from 2.8 mm
Only models of modest sizes can be created with the