Pathological classification through transmission electron microscopy (TEM) is essential for the diagnosis of certain nephropathy, and the changes of thickness in glomerular basement membrane (GBM) and presence of immune complex deposits in GBM are often used as diagnostic criteria. The automatic segmentation of the GBM on TEM images by computerized technology can provide clinicians with clear information about glomerular ultrastructural lesions. The GBM region on the TEM image is not only complicated and changeable in shape but also has a low contrast and wide distribution of grayscale. Consequently, extracting image features and obtaining excellent segmentation results are difficult. To address this problem, we introduce a random forest- (RF-) based machine learning method, namely, RF stacks (RFS), to realize automatic segmentation. Specifically, this work proposes a two-level integrated RFS that is more complicated than a one-level integrated RF to improve accuracy and generalization performance. The integrated strategies include training integration and testing integration. Training integration can derive a full-view
Primary glomerular disease is the most common renal disease in China [
The diagnosis of many renal diseases is closely related to the glomerular basement membrane (GBM) [
Interimage shape variations and intraimage grayscale inconsistency of GBM (GV: gray value).
Early in 1993, Ong et al. [
Two common difficulties associated with GBM segmentation are interimage shape variations and intraimage grayscale inconsistency. Figure
To address the first challenge of autoextracting features, we employ a pixel-wise classifier, namely, random forest (RF) [
An enhanced generalization effect based on a single RF classifier is hardly obtained because of the grayscale inconsistency intraimage. To address this second challenge, we propose an RF stack (RFS) model based on a wider grayscale range of images. After assigning TEM images to different grayscale groups, we sample from all these groups and train a full-view RF classifier as
The remaining sections of this paper are organized as follows. In Section
Renal biopsy specimens were immediately fixed with 2.5% cold glutaraldehyde with 0.1 M phosphate buffer at pH 7.3 for 4 h, washed with phosphate buffer, postfixed with 1% osmium tetroxide in the same buffer, dehydrated with a graded series of ethanol, and embedded in Spurr resin. Ultrathin sections (70 nm) were contrast enhanced with uranyl acetate and lead citrate and examined using a Hitachi H-7500 electron microscope (Tokyo, Japan) at 60 kV. All of the sections were imaged with MORADA G3 (EMSIS Corporation of Japan) at 5000x magnification. In the field of vision, a whole glomerulus, including the glomerular capillaries and the basement membrane, was selected. Continuous filming was conducted using the attached digital imaging system and controlled by the Pathological Image Workstation of the NanFang Hospital in Guangzhou, China.
The pathologists collected 351 images from the obtained glomerular TEM images to build a GBM image database. Of these 351 images, 330 were used as a training set and divided into different groups (
TEM image (a) and the corresponding binary mask image of GBM (b).
RF is an ensemble machine learning method, which can be applied to image segmentation by classifying pixels into target or background. The proposed RFS is a RF-based multilevel integrated structure that mainly involves two phases: hierarchical training and refinement testing. The implementation process of RFS is shown in Figure
Flowchart of the proposed RFS method.
Train phase: from the prebuilt GBM database, an image is randomly selected from each image group with different GBM grayscale ranges. Hence,
Test phase: each pixel of the test image is classified by
The image-processing and analysis software FIJI (ImageJ) is developed by the US National Health Administration, and FIJI-based secondary development is well known. In this paper, we selected a FIJI plug-in named Trainable Weka Segmentation (TWS) (
Software tools.
RF [
Considering that the intensities of GBM in various TEM images are significantly different, a single RF classifier cannot extract different grayscale features of all TEM images and the segmentation performance is unstable. For example, given an RF classifier sampled and trained from Figure
Segmentation results of TEM images with different grayscale ranges by using the same RF classifier.
To address this problem, we introduced level 2 integration. We first assigned all training images to
The zoom-view method separately takes samples from
In the prebuilt GBM database, the TEM images in the training set were divided into
TWS has 15 applicable features, and 14 of them were selected as the inputs of the decision tree in this paper: common grayscale features (mean, minimum, maximum, median, and variance), boundary features (Sobel filter, Hessian, and difference of Gaussians), texture features (Gaussian blur, entropy, and Kuwahara filter), and other features (membrane projections, Lipschitz filter, and neighbors).
Other RF training parameters include the number of decision trees (
Given an image to be segmented or tested, two candidate segmentation results can be separately obtained by
For each image pixel to be segmented, equation (
In the probability map, a large gray value of a pixel corresponds to a high probability to become GBM. Therefore, by maximizing the similarity belonging to the same category or avoiding it to reach the minimum, the fuzzy C-means (FCM) [
Not every coarse segmentation results
Coarse segmentation results with an obvious error GBM area are eliminated on the basis of the preset threshold The remaining The Jaccard similarity The Jaccard similarity After the loop ends, the candidate result
Figure
Segmentation process of
For a test image, the candidate segmentation results Whether the foot process of the epithelial cell or the cytoplasm of the endothelial cell is inappropriately contained in the region of the basement membrane because its electron density is similar to that of the basement membrane Whether the subepithelial immune deposit is erroneously excluded from the basement membrane because its electron density is higher than that of basement membrane The continuity of the basement membrane should be cautiously analyzed because pathological fracture defects of the basement membrane are few
The accuracy of the proposed method is evaluated by Jaccard coefficient, which is widely utilized to evaluate the performance of segmentation methods [
In this study, 21 TEM images with different grayscale ranges, sizes, and basement membrane morphologies are used for evaluation. These images are manually segmented by pathologists as the gold standard.
The RFS method provides robust segmentation results of GBMs with different morphologies and grayscale ranges.
Segmentation results of RFS with different morphologies of GBM. (a) Strip type: size, 217∗307; Jaccard, 0.75. (b) Close type: size, 150∗206; Jaccard, 0.84. (c) Compound type: size, 282∗367; Jaccard, 0.76.
Figure
Segmentation results of RFS with various grayscale ranges of GBM (RoGV: range of gray value). (a) RoGV: 54–255; size: 282∗274; Jaccard: 0.70. (b) RoGV: 43–187; size: 168∗308; Jaccard: 0.71. (c) RoGV: 12–167; size: 297∗408; Jaccard: 0.65.
As shown in Figures
A multilevel integrated RFS classifier is constructed to address the generalization problem of GBM segmentation. This is based on the hypothesis that, for an RF classifier, the closer the grayscale range of the image to be segmented is to the training image, the better the segmentation effect will be. The experimental results from the heat map in Figure
Heat map of segmentation results of different grayscale ranges of RF classifiers.
In this experiment, each test image is separately segmented by these 37 RF classifiers of different grayscale ranges, and the corresponding Jaccard value is shown in different colors. An accurate segmentation result corresponds to a high Jaccard value, and its color turns to bright yellow. It can be seen from the color distribution in Figure
The multilevel RFS is constructed in full view and zoom view as shown in Figure
Jaccard values of zoom-view
Figure
The stability of
The following methods are adopted to validate the effect of postprocessing and iterative refinement on
Effects of postprocessing and refinement methods.
Ensemble methods construct a set of classifiers and then classify new data points by taking a weighted vote of their predictions. Dietterich [
The number of decision trees is among the most important parameter in the application of RF algorithm in medical image segmentation [
The number of sampling points is another critical parameter in the RFS method. In our experiment, as the number of sampling points increases from 200 to 2000 per training image, the accuracy rate of the RFS classifier increases by approximately 10%, whereas the accuracy is not improved greatly if the number of sampling points continuously increases. Thus, the total sampling point is set to 74,000, where
In TWS, 15 available image feature attributes are provided in the decision tree construction. In our experiment, the application of most features can improve the accuracy of segmentation, but entropy (
Effects of different image features with or without entropy and anisotropic diffusion.
Our experiment results reveal that the proposed RFS method obtains poor performance for some cases. For example, for a low-contrast image, the accuracy rate of voting is almost 0. Only the accuracy rate of
The segmentation of the whole GBM region in TEM pathological images can provide more rapid and intuitionistic observation for the morphological change and can reduce the tedious and expensive manual workload of the pathologist. This work proposed a two-level integrated RFS method involving training integration and testing integration to autosegment a GBM image. A total of 351 clinical images were included in the experiment. The accuracy and generalization ability of the RFS method were validated. Experimental results illustrated that the proposed method could be used for the automatic segmentation of GBM with different morphological characteristics and grayscale ranges. Further study is underway to improve segmentation accuracy of the automated CAD system and to implement GBM thickness measurement and deposit autorecognition for auxiliary pathological diagnosis.
The image data used to support the findings of this study are available from the corresponding author upon request.
The authors declare that there are no conflicts of interest regarding the publication of this article.
This work was supported in part by the grants from the National Natural Science Foundation of China (no. 81771916) and the Guangdong Provincial Key Laboratory of Medical Image Processing (no. 2014B030301042). Many thanks are due to Xi Yu, Yuanyuan Liao, Can Xu, Zhenxing Li, and Zitao Zhang, who participated in this study during their graduation project in Southern Medical University and made some contributions.