The blood vessels and nerve trees consist of tubular objects interconnected into a complex tree- or web-like structure that has a range of structural scale 5
Humans (indeed all biological multicellular organisms) are made of multiscale hierarchy of structures ranging from subcellular structures (10−7 m) to cells (10−5 m), basic functional units (the smallest aggregation of diverse cells that behaves like the parent organ 10-4 m), organs (10−2–10−1 m), and bodies (100 m). In addition to the local scale variation, biological structures are also characterized by shape. For example, the blood vessels are tubular objects interconnected into a complex network and have a range of structural scale (5
Without prior information for a scale description of the image content, an image has to be studied at all scales. The basis for design of an automatic tool for such description could be derived from human perception model [
To build the multiscale representation of the image, the proper aperture (windowing) function as an operator should be chosen. Formalism for the scale-space representation was introduced by Witkin [
Smoothing the finer spatial image structures out with increase of Gaussian kernel size: original image with structure sizes sigma = 2; 10; 40 (a). Convolution with Gaussian using various kernel size sigma = 2 (b), 10 (c), 40 (d). With Gaussian kernel size increasing, finer image structures are disappearing.
The scale spaces could be generated by using various kernel functions. Recently, wavelets and their applications to signal and image processing have attracted attention of the scientists in many fields. A very good collection of papers on the wavelet theory and its applications can be found in the book by Heil and Walnut [
Even though different kernel functions have been proposed to generate the scale-space representations, the Gaussian kernels remain the best candidates so far [
Another motivation to use the Gaussian scale spaces has some support from neurophysiological and psychological experiments which has shown that receptive field profiles in the mammalian retina and visual cortex can be well modeled by sums of Gaussian [
All these properties make the linear Gaussian scale spaces the best choice for development of the automatic unsupervised systems for multiscale signal and image analysis, when there is no in advanced information available concerning preferable scales. The recipe which allows adaptively choosing the proper local scale parameter at every geometrical location was suggested by Lindeberg [
The strongest response of such function (with respect to
Original image convolved with first-order Gaussian derivatives. (a) Original image having an object with Gaussian intensity profile of kernel size
Original image convolved with second-order Gaussian derivatives. (a) Original image has an object with Gaussian intensity profile of kernel size
Responses to convolution of the original image with the first- (a) and second- (b) order Gaussian derivatives. Original image has an object with Gaussian intensity profile of kernel size
Unfortunately, the amplitude of Gaussian derivative operators tends to decrease with increasing scale due to the fact that with increasing scale, the response is increasingly smoothed. This gives more preference to smaller scales. To compensate such increase and, thus, improve accuracy of the automatic scale selection, Lindeberg [
In the case of tubular-like structures in an image, ridge detection with automatic scale selection can be done using a second derivative of Gaussian kernel function [
Original image convolved with second-order Gaussian derivatives. (a) Original image having an object with bar-like intensity profile of size
Intensity profiles of responses to Gaussian (a) and bar-like (b) ridge models.
Early vision perception occurs at all scale simultaneously and can be modeled by generating the image scale-space representation that introduces an additional variable, spatial scale size [
The Hessian matrix
Following the scale-space ideas described earlier, the partial second derivatives of the image
Various algorithms for multiscale tubular object tracking and enhancement were developed depending on the way Hessian eigenvalues are combined in the objectness measure function. For instance, Sato et al. [
To take the full advantage of the power of the multiscale shape detector filter in object tracking algorithms applied for a large variety of medical applications, in this work we focus on the process of automation of this filter.
The filter itself has many control parameters which can be separated into several groups: Brightness measure (objects are bright relative to background); objectness measure (shape description), scale description (range plus scale step function), and background noise suppression parameter (Frobenius norm scale factor) [
We present our development of the multiscale Hessian-based tubular object-tracking filter with automatic selection of the parameter used for suppression of background noise. That finalizes the automation of the filter. In our approach, the information required for the parameter calculation is acquired from the image being processed thus it automatically takes into account all the individual properties of the particular image such as voxel size and noise level. This allows for increased automation as well as parallel processing—thereby greatly decreasing processing time.
For our studies, we used both gray-scale images numerically derived and acquired by scanners. The modeled images were programmed so to model environment with certain features. Tubular objects with various widths were placed amid different background: Gaussian random noise, nontubular objects, background with noise, and polynomial varying intensity. For simulations of images degraded by noise, we used the C++ classes contributed to the ITK Insight-Journal by Lehmann [
The custom-made micro-CT scanners generate images up to
To be able to process large images using the developed algorithms, we built a specialized server with four 64 bit AMD Opteron 8350 Quad Core 2.0 GHz CPUs and 128 GB memory. The server is located in a server room and it is accessible in multiuser mode through the local network using remote clients.
For our software development, we used the library of C++ classes from the National Library of Medicine Insight Segmentation and Registration Toolkit (ITK) [
The developed multiscale shape detector filter is based on the objectness measure function suggested by Frangi et al. [
Let
Frangi et al. [
The parameter
The method for automating the selection of the parameter for suppression of background noise uses a scalar function (nondirectional) of the image voxels the Laplacian of the image. The Laplacian is a well known operator in image processing which is easy to calculate [ For each voxel in the image, calculate the Laplacian. In the calculated Laplacian array, find the maximum value of Laplacian, that is, Take one tenth of that maximum value of Laplacian, that is, Assign the calculated value to parameter
In this approach, the information required for parameter calculation is acquired solely from the image being processed; thus, it automatically takes into account all the individual properties of the particular image such as a voxel size and noise level.
There are two measures commonly used for objective evaluation of the perceptual quality of images: Mean Square Error (MSE) and Peak Signal to Noise Ratio (PSNR) [
The MSE and PSNR measures were developed using C++ classes from the ITK library [
In manual mode, the control parameters have to be provided by the operator before running the code. If the result is not acceptable, the operator has to change parameters and rerun the code again. The MSE and PSNR measures were used for objective quality evaluation of the processed image. As an “ideal image”, there was used a modeled image comprised three tube-like objects with Gaussian intensity profiles of different width. Then, the ideal image was degraded by the random Gaussian noise with Standard Deviation SD = 100.0 and Mean M = 0.0 and processed in manual mode by the algorithm. The background suppression parameter value was sampled over a wide range 10–500 to surely cover prospective optimal control parameter. The results are depicted in Figure
Mean Square Error (MSE) and Peak Signal-to-Noise Ratio (PSNR) measurements for manual image processing.
To test efficiency of the algorithm in fully automatic mode, we used the modeled images described above. The control parameters were chosen and fixed: brightness “on”, objectness measure “tubular”, scale range “1–30”, scale steps “20”, step function “logarithmic”, noise suppression mode “automatic”. Scale range was chosen wide enough to cover all possible diameters. The step function was made “logarithmic” so as to emphasize finer scales.
First, we processed the images with curved tubular objects with various widths which were placed amid nontubular objects. The images were degraded by Gaussian random noise with SD = 25.0, 50.0, and M = 0.0. As can be seen from Figure
Automatic processing. Input images with noise mean = 0 and standard deviation (a) SD = 25 and (b) SD = 50; processed images (c) and (d).
We also tested performance of the algorithm for neurological confocal microscopy image processing affected by tiling, shading, Gaussian noise, nonlinear background, and so forth (see, e.g., Figure
Neurological confocal microscopy image of Hippocampal CA3 Interneuron with polynomial noisy background: (Image data courtesy from Professor German Barrionuevo).
Program-simulated images with polynomial noisy background and tubular objects with Gaussian intensity profiles.
Intensity profiles across the tubular objects with Gaussian intensity profiles in the processed images with various polynomial noisy backgrounds added. (a) Noise SD = 0; (b) SD = 25; (c) SD = 50.
To evaluate time efficiency of our automatic method, we processed the
The result of applying our automatic algorithm to the micro-CT image of the coronary arteries in a heart in concert with the nonprocessed image is shown in Figure
Maximum Intensity Projection (MIP) of micro-CT image of rat heart coronary arteries. (a) Input image; (b) Processed image.
In Figure
Histological image of cerebellar climbing fibers. (a) Input image; (b) Processed image (Image data courtesy from Professor Giorgio Ascoli).
We also explored the efficiency of the algorithm as an initial filter in the central line extraction pipe line. The original image Sample H61 was processed with the developed filter and then segmented using the region growing connected threshold algorithm [
Specimen H61 (coronary artery branch within a human heart wall). (a) Not processed image with extracted center line; (b) Segmented processed image with extracted center line.
We have presented a method for automation of adaptive nonsupervised system for tracking tubular objects that is based on analysis of local structures performed in multiscale framework. The designed filter has demonstrated a great potential for complete automation and showed very good performance in both background noise suppression and tubular object tracking.
The developed approach can be used in the reconstruction pipeline right after image deconvolution operation. Even though the convolution operator will reconstruct the object features at finer scales, those features will appear in increased noise environment which in return might require additional postprocessing for noise suppression yet to preserve extracted features.
Another application is the object feature extraction pipeline. This filter can be used as a preprocessing filter for vessel enhancement and background noise suppression right before segmentation or immediately in the segmentation algorithms itself, for instance, in the family of segmentation algorithms which require distributed seeds [
Since the response function is built using exponents, with proper normalization this function can be considered as a probability function with values distributed over the interval “0.0-1.0”. In this case, after processing, the output image holds voxels with values of probability of the event that “a voxel belongs to the object with tubular shape”. These probabilities can be used in many ways. The most traditional way is to rescale it back to a gray-scale image. Although such images do not keep a proper intensity calibration, they still can be used for morphometric analysis. If calibration is of concern, the probabilities could be converted to a mask for sampling the original micro-CT image from which the calibration could be recovered.
Since the filter generates the response function with only one maximum across scale space at a scale that is proportional to the diameter of the tubular object and that maximum is located at the center of the object, the probability image is more suitable to construct various cost functions. The images with cost functions can further be used as the “feature image” in various image processing pipelines, for instance, such as in flux-driven centerline extraction algorithms [
This work was supported in part by NIH Grant no. EB 000305. The authors wish to thank Ms. Diane R. Eaker for her help with the micro-CT reconstructions and Point Spread Function deconvolution and Ms. Delories C. Darling for formatting this manuscript. The authors are exceedingly grateful to Professor Giorgio Ascoli and Professor German Barrionuevo for the permission to use their images in our work.