^{1, 2}

^{2}

^{1}

^{2}

With the advent of novel biomedical 3D image acquisition techniques, the efficient and reliable analysis of volumetric images has become more and more important. The amount of data is enormous and demands an automated processing. The applications are manifold, ranging from image enhancement, image reconstruction, and image description to object/feature detection and high-level contextual feature extraction. In most scenarios, it is expected that geometric transformations alter the output in a mathematically well-defined manner. In this paper we emphasis on 3D translations and rotations. Many algorithms rely on intensity or low-order tensorial-like descriptions to fulfill this demand. This paper proposes a general mathematical framework based on mathematical concepts and theories transferred from mathematical physics and harmonic analysis into the domain of image analysis and pattern recognition. Based on two basic operations, spherical tensor differentiation and spherical tensor multiplication, we show how to design a variety of 3D image processing methods in an efficient way. The framework has already been applied to several biomedical applications ranging from feature and object detection tasks to image enhancement and image restoration techniques. In this paper, the proposed methods are applied on a variety of different 3D data modalities stemming from medical and biological sciences.

The analysis of three-dimensional images has gained more and more importance in recent years. Particular in the medical and biological sciences, new acquisition techniques lead to an enormous amount of 3D data calling for automated analysis. In this paper, we show how the harmonic analysis of the 3D rotation group offers a convenient and computationally efficient framework for rotation covariant image processing and analysis. Most of the state-of-the-art techniques rely on “low”-order features such as intensities, gradients of intensities, or second order tensors like the Hessian matrix or the structure tensor [

From such a distribution, we can estimate the probabilities for the absence or presence of lesions in a voxel-by-voxel manner. Instead of solely using 0-order features, such as the Laplacian-pyramid, higher order tensor fields can be used to derive further scalar valued quantities. Such features can be the smoothed intensity gradient magnitudes (1-order features), or the eigenstructures of a Hessian matrix field or a structure tensor field (2-order features). However, due to their mathematical and computational complexity, features of order three or even higher order are rarely used. This paper proposes a unified framework that can cope with high-order features in a systematic way. The proposed framework is based on the harmonic, irreducible representations of the 3D rotation group. This guarantees the most sparse tensor representation. Consequently, in comparison with ordinary Cartesian tensor analysis, the algorithms and the handling are operationally clearer and more efficient.

Given a Cartesian tensor

In this paper, we want to review the basics of spherical tensor analysis and how it can be applied to image processing problems. In Section

Let

Note that we treat the space

We denote the standard basis of

A function

Now, we define a family of bilinear forms that connect tensors of different ranks.

For every

The characterizing property of these products is that they respect the rotations of the arguments.

Let

The components of the left-hand side look as

If

The proposition is proved by the symmetry properties of the Clebsch-Gordan coefficients

We will later see that the symmetric product plays an important role, in particular, because we can normalize it in an special way such that it shows a more gentle behavior with respect to the spherical harmonics.

For every

For the special case

The introduced product can also be used to combine tensor fields of different rank by point-wise multiplication.

Let

In fact, there is another way to combine two tensor fields: by convolution. The advantage of the convolution is that the evolving product also is covariant with respect to translation; that is, the product is covariant to 3D Euclidean motion.

Let

Given a translation

The product

Due to their special properties, the spherical harmonics (see, Appendix

A band-limited spherical harmonic representation of two images is illustrated in Figure

A spherical harmonic decomposition of images can be seen as some kind of frequency decomposition. A band limited expansion of a volumetric images is illustrated. We see that lower frequency components (right-hand side) are roughly representing the important characteristics of the objects. However, higher frequency components are necessary to represent the details. For the expansion here, we use a Fourier-like basis for representing the images in radial direction. Here,

The expansion coefficients of the rotated function

Besides the spherical harmonics, the so-called solid harmonics, often appear in the context of harmonic analysis of the 3D rotation group. They are the homogeneous solutions of the Laplace-equation and are just related by

The correspondence of spherical and Cartesian tensors of rank

This section proposes the concepts of differentiation in the context of spherical tensor analysis. First, we will introduce the spherical derivative operator which connects spherical tensor fields of different ranks by differentiation. The basic idea is simple; formally replace the coordinates

Let

In fact there are much more rotation covariant differential operators than the two defined previously. Given a tensor field

Let

See [

Both statements are direct consequences of the Fourier correspondences for the ordinary partial derivatives. For scalar fields, we can generalize this statement also for higher orders.

For

See [

We want to emphasize that both statements only hold for scalar-valued fields, and generalizations to tensor-valued do not hold in general due to the nontrivial associativity rules.

Let

It is well known that convolutions commute with differentiation, and actually there are generalized commutation rules for spherical tensor fields.

Let

Both assertions are founded by the associativity of the spherical product. Consider the first statement in the Fourier domain by using (

This proposition shows again the importance of the up- and down-derivatives. For general derivative operators

To get a better understanding of what happens during the differentiation via spherical derivatives, we consider their properties in polar representations.

Given a spherical tensor field

See [

Previously, we already stated that

The Gaussian windowed polynomials

See [

Gabor functions, that is, Gaussian-windowed plane waves, play an important role in image processing due to the fact that the different frequency components of signals can be studied locally. This information is, for example, used for tracking [

In the following, we show that there exists a very similar way to represent the Gaussian windowed wave in terms of the derivatives of the Gaussian windowed Bessel functions. Let

The spherical derivatives

Consider that

See [

In most image processing applications, the data to be processed is of scalar nature; that is, for each voxel, we observe one single intensity value. But there are actually acquisition techniques, where the measurement itself is already a tensorial quantity. For example, in diffusion weighted magnet resonance imaging (DW-MRI), rank 2 tensors are common. Or, in phase contrast MRI velocity, vectors are measured. Thus, there is a great interest to represent these measurement in an appropriate way. In [

The functions

In this section, we discuss the properties of expansion coefficients of specific tensor fields, expanded in terms of tensorial harmonics. We show that symmetries in a tensor field are simplifying the tensorial harmonic expansion coefficients. This is similar to the ordinary spherical harmonic expansion. For example, the point symmetry

The rotation symmetry of a spherical tensor field

We call such a rotation symmetric field torsion-free if

Finally, consider the reflection symmetry with respect to the

In the context of rotation covariant image processing, the applications of the proposed framework are manifold. The mathematical representation might appear unfamiliar, but the provided tools can be used quite easily. Basically, there are two types of operations: differentiation by spherical tensor derivatives and multiplication by spherical tensor products. The spherical derivatives can be used in two ways. On the one hand, the up-derivatives can be used to “create” new tensor fields out of existing fields by incorporating neighborhood relations. This can be regarded as a simple and efficient way to compute local meaningful image descriptors in a covariant way. On the other hand, the down-derivatives can be used to gather information from a local point neighborhood and form a lower ranked tensor field via superposition. Due to the tensorial nature, the information is able to interfere in a destructive or constructive way. The spherical products are the basic nonlinear ingredient in the framework. They can be used to combine tensor fields in a nonlinear, covariant manner.

Several principles in the image processing and pattern recognition [

In the following, we give examples of the proposed framework in several application domains.

For implementing the discrete spherical derivatives, we propose to utilize central differences of 4th order accuracy for computing the partial derivatives (see Figure

The discrete differential operators we use for realizing the discrete spherical derivative operators. On the left-hand side, the corresponding global weights are depicted. The red dot denotes the current image position.

The theory in practice: Laguerre expansion of a volumetric image with

Ground truth: an image is convolved with each basis function

Differential approach using the discrete operators shown in Figure

Differential approach using the standard Laplace operator considering only six neighbors results in strong artifacts and leads to unusable results (lower images)

The Tensor Voting framework was originally proposed by [

In the following, we briefly show how to design trainable rotation covariant image filters which can be used for rotation invariant object or landmark detection. The idea is that expansion coefficients of a spherically expanded voting function are learned in a data driven way. The filter is mainly based on two steps. Rotation covariant image descriptors are densely computed in a voxel-by-voxel manner. Then, a weighted superposition of these image descriptors is used to form expansion coefficients of a spherical voting function. The expansion coefficients are formed such that each voting function votes for the presence or absence of landmarks or objects. The weights are found by a least square fit to a given training data set. For a fast implementation, we propose to use voting functions based on an expansion of spherical functions having a differential relationship in terms of spherical derivatives. In [

Filter response.

Training set: A 3D image of airborne-pollen recorded by a confocal microscope

Centered slices of some datasets of the test-dataset together with the maximum intensity projection of the filter responses

Especially in the field of biomedical imaging, the third dimension becomes more and more important due to the fact that organism can be studied in their natural constellation. Objects and organism can be located in any number at any position and, much more challenging, in any orientation. The third dimension does not only lead to larger datasets, but also the interrelation of neighboring intensity values becomes more complex. With a fast voxel-wise transformation of volumetric images into the harmonic domain, we are capable to compute rotation invariant image descriptors in an analytical way. In [

Voxel-wise classification of cells. For a voxel-wise classification, we first use a manually labeled image (a) for training a support vector machine (SVM) based on local rotation invariant image descriptors. Then, the SVM classifier is used to detect and classify cells in unclassified images (b). In (c) we depict an isosurface rendering of the classified root. Further details concerning the experiment can be found in [

Diffusion weighted magnetic resonance imaging (DWI) plays a substantial role in neuroscience and clinical applications. One field of interest is the investigation of the neuronal fiber architecture located in the brain white matter connecting different regions in the brain. The fibers themselves cannot be recorded directly. However, the data is usually recorded using the high angular resolution diffusion imaging (HARDI) technique [

For the analysis of the fiber structure, a preprocessing step that identifies the brain white matter within the image is required. For group studies, the parcellation of the human brain into anatomical regions is of great interest. Preliminary results have been published in conference papers [

We utilize the fact that the given recordings are tensor valued. We first transform the local measurements into the spherical harmonic domain (e.g., see [

This is done by first comprising the voxels surrounding using the spherical down derivative operators. This can be seen as some kind of Taylor expansion of the given data. Then, we compute rotation invariant image features by computing the power spectrum of the resulting expansion coefficients. We finally use a random forest classifier [

In Figures

The ground truth regions that we used to train and evaluate our algorithm shown together with our algorithm’s regions prediction. We can clearly see that our predictions are much more consistent with the data.

Isosurface showing the predictions for dataset 3 using GND and a random forest (RF) classifier. The classifier can distinguish between background, brain white matter (green), and gray matter (red).

The confidence of the classifier represents the probability that a certain voxel belongs either to the background class, gray matter class, or the white matter class. The probability is represented by the intensity. A final decision is made by decision by majority (as shown in Figure

Heat maps representing the probability for all regions used in an experiment (continued in Figure

Heat maps representing the probability for all regions used an experiment (starting in Figure

Group studies often require the coregistration of images or partial image structures of different individuals. In such applications, the detection of characteristic landmarks is often an indispensable prerequisite.

Similar to [

Second-order features which are sufficient for most applications are not providing enough information to solve the detection task in a human brain; they are invariant against reflection about an axis. Hence, they cannot distinguish the left and the right hemisphere. It is known that the spherical triple-correlation [

The resulting filter has shown very promising results on a training set of 7 and a test set of 14 images. For the experiment, we placed about 20000 landmarks within the brain gray and white matter in an equidistant manner. For each dataset, the computation of the features and the detection of of all landmarks took about 5 minutes. We show some detection results in Figures

Differently weighted linear combinations of the feature images lead to different detection results.

Differently weighted linear combinations of the feature images lead to different detection results.

Differently weighted linear combinations of the feature images lead to different detection results.

Differently weighted linear combinations of the feature images lead to different detection results.

The Schmidt seminormalized spherical harmonics

Orthogonality

The components of

The spherical Bessel functions

Using the addition theorem of the spherical harmonics, we can express the spherical expansion of the plane wave (e.g., see [

The associated Laguerre polynomials [

M. Reisert and H. Skibbe are indebted to the Baden-Württemberg Stiftung for the financial support of this research project by the Eliteprogramme for Postdocs. This work was partly supported by Bioinformatics for Brain Sciences under the Strategic Research Program for Brain Sciences, by the Ministry of Education, Culture, Sports, Science, and Technology of Japan (MEXT).