Asphalt Mixture Segregation Detection : Digital Image Processing Approach

Segregation determination in the asphalt pavement is an issue causing many disputes between agencies and contractors.The visual inspection method has commonly been used to determine pavement texture and in-place core density test used for verification. Furthermore, laser-based devices, such as the Florida Texture Meter (FTM) and the Circular Track Meter (CTM), have recently been developed to evaluate the asphalt mixture texture. In this study, an innovative digital image processing approach is used to determine pavement segregation. In this procedure, the standard deviation of the grayscale image frequency histogram is used to determine segregated regions. Linear Discriminate Analysis (LDA) is then implemented on the obtained standard deviations from image processing to classify pavements into the segregated and nonsegregated areas. The visual inspection method is utilized to verify this method. The results have demonstrated that this new method is a robust tool to determine segregated areas in newly paved FC9.5 pavement types.


Introduction
Characteristic of asphalt material impacts on asphalt quality and driving safety [1].Segregation of aggregates alters material properties and accelerates deterioration's rate [2,3].Pavement segregation is one of the main concerns affecting the performance of asphalt pavements.Segregation is defined as separation of aggregates gradation, so that coarse and fine aggregates are separated in the asphalt mixture [4].Also, existence of enough bituminous and stickiness between asphalt materials may reduce deterioration of asphalt concrete [5].Detection of segregated areas in pavements has always been a disputable issue between agencies and contractors.Segregated areas are primarily detected by the visual inspection and verified by core density [4].Segregation can also be determined by measuring the surface texture of the pavement.Florida Texture Meter (FTM) and Circular Track Meter (CTM) are laser-based devices that can be used for this purpose [6,7].
Image processing is another tool that is used to qualify pavement texture [8].Scanners and cameras are two types of digitalizing devices to produce a 3D image model [9].Digitalizing can be used to measure different size and scale objects [10].Blais (2004) developed a scanning machine, which captures 3D data from the surface, and then categorized the road texture using two algorithms [10].The first algorithm calculates the estimated texture depth (ETD), and the second calculates the texture profile level (TPL).The measurements from nine pavements were collected to validate these algorithms.The results show that there is a good agreement between the traditional Sand Patch method results and the 3D scanning prototype equipment [11].
The feasibility of using image processing to determine the gradation without separation of bitumen and aggregates was studied by Bruno et al. [12].The image, at first, subdivided into different distinct areas where each distinct area has homogeneity with the area around it.This procedure helps to separate voids, aggregates, and bitumen areas.Frequency histogram of each distinct area has been used to detect the gradation of the asphalt mixture.Dividing the image into distinct regions can also be used to detect pavement distresses [13].In this case, the transform method to designate lines and pixels is used to define the distresses in each distinct area.The Discriminant Analysis, -Nearest Neighbor, and Discrete Choice Method have also been used to classify each area.
More recently, the concrete pavement texture has been evaluated through digital image processing [14].Visual inspection has been used to verify this method.This validation approved that the flatbed scanner can be used to measure the pavement texture accurately.The scanner was used to take digital images of the surface texture.Simplicity to control the light source is the advantage of scanner application because the source of light, during the photography, has an effect on the color frequency.The colored digital image that has been produced by scanning the pavement surface was later converted to a grayscale image to generate a grayscale image frequency histogram.The pavement texture, then, has been classified based on the standard deviation of the frequency histogram.Bug-hole was argued as the major parameters that affect the frequency histogram standard deviation.In other words, pavement surface with more bug-holes would show higher standard deviation in comparison with surface textures with fewer bug-holes.
Furthermore, Chen et al. performed an investigation on the determination of pavement texture and pavement Mean Profile Depth (MPD) using image processing [15].Pavement MPD was determined using the scanned pavement cross-sections.Stationary Laser Profilometer (SLP) was then used to verify asphalt texture detected with discrete Fourier transform method.This technique was shown to be useful to design pavement when texture noise and friction are important.
Although image processing techniques have frequently been used to measure pavement texture in previous studies, little or no work has used image processing to determine segregation.The method introduced in this research is a nondestructive tool and potentially can be used as a smartphone application to detect asphalt pavement segregation in place.
1.1.Digitalization.Pictures consist of divided elements named pixel.Each pixel has some information which shows color at that particular point.In other words, the color in each part of the image is represented by a pixel.Pixels are stored in bits which can be 0 or 1.The intensity of pixels in an image shows the quality of that image.The more pixels in the image, the higher quality and resolution it has [16].
Photos can be either colored or black and white.While colored images have a color level of pixels, black and white pictures have a gray level.The gray level can vary from 0 to 255 resulting in 256 color spectra.The basic colors to produce a color image are red, green, and blue (RGB) which form 16.7 million spectra [17].It is possible to add some other colors to the basic colors of a colored image.For example, CMYK color image which contains cyan, magenta, yellow, and black is another kind of color image which has 4.3 billion spectra colors.Extracting this numerical information from an image is named image digitization [18].
Using a black and white image to extract numerical information from the image is easier than the colored image because colored images have three layers (one layer for each basic color) and millions and billions of spectra; therefore, categorizing information extraction from a color image is difficult.On the other hand, using black and white color image is preferable because it has 256 spectra colors.The image frequency histogram can be applied to extract numerical information from any picture.

Segregation Detection Methodology
In this study, the frequency histogram was used to conduct digital image processing.Frequency histogram of an image is a graph which uses vertical columns to show the number of repetitions of each spectrum in an image [18].When the asphalt mixture frequency histogram is used, it is expected that nonsegregated area's pictures have uniform colors.In other words, it is supposed that most of the colors in nonsegregated asphalt image have gray colors.In contrast, in segregated pavement surface, the separation of bitumen and aggregates or fine and coarse aggregates may occur.This separation can cause darker or lighter color intensity in segregated areas of the image which can lead to a higher standard deviation of segregated pavement images in comparison with nonsegregated pavements images.In the following, this hypothesis will be investigated by using a MATLAB program which defines the frequency histogram standard deviation.Then, these results will be classified through Linear Discriminate Analysis (LDA).
LDA is a useful tool to classify and predict the group's membership based on the linear distinction between measured factors.This theory first was developed by R. A. Fisher in 1936.The robustness of this theory is defining the probability of an expected portion of data belonging to each group.However, this method demonstrates the threshold and membership of each group almost the same as the Analysis of Variance (ANOVA) [19].
In ANOVA means of groups are verified to be the same; so ANOVA performs an eigendecomposition and finds its eigenvalues.In contrast, LDA performs eigendecomposition and looks at the eigenvectors.These eigenvectors define directions in the variable space and are called discriminant axes.Therefore, the advantage of implementing LDA is in predicting the membership of each group in generating a big gap between groups [20].
The concept of LDA is searching for a linear combination of variables that best separates two classes.In order to define each group member, it first defines a linear equation between variables, which determines a Discriminate Factor (DF).Then, the data is classified based on the calculated DF and the cutoff score.The cutoff score defines the threshold between two groups of data.
The LDA classification method has been used to predict each group's membership and classify the segregated and nonsegregated pavements.In this case, the standard deviation of each picture was determined and classified.Two scenarios were considered for the LDA: (1) each site location standard deviation result was classified and the threshold line was defined separately and (2) the standard deviation results from both locations were analyzed together and the threshold line was designated.These two scenarios were considered to identify if there is the same threshold for separating the segregated asphalt and nonsegregated asphalt for all the test sites.To perform LDA analysis according to the considered scenarios, the SPSS software was used.

Experiments
Two sites in Florida that were recently paved with FC9.5 asphalt mixture were chosen for this study because FC9.5 is a commonly paved asphalt mixture type in Florida [21,22].FC9.5 is a Superpave mixture with the nominal maximum aggregate size of 9.5 millimeters [23].In this research, newly paved pavements were selected to eliminate the effect of pavement aging and contamination on results.Experts from the Florida Department of Transportation (FDOT) designated the segregated and nonsegregated areas on the pavement by the visual inspection method.Also, the pictures were evaluated by the other three FDOT experts.The average of these experts inspection was considered for the spot locations where the inspectors' ideas did not match each other.Fifteen nonsegregated and five segregated points were selected at each test site.Since the type and resolution of the camera may affect the results, all the pictures were taken by the same camera and the same operator (same quality of the pictures) from the same elevation.The setup shown in Figure 1 was used to capture the pictures.In the center of the table, there is a hole where the camera was placed to take the picture.
To take pictures, the pavement surface was inspected, and clean and dry surfaces were selected.Field surfaces were free of loose particles and the surfaces were cleaned of the residue by brush.Then, the segregated and nonsegregated areas were chosen and marked within the same asphalt lane.The table was placed exactly on the marked surface areas.The pavement surface was flat and the size of the table was approximately 45 * 45 centimeter, which was small enough that the pavement slope had no effect on the pictures.Then, the camera was placed on the table and the picture was taken in such a way that it was oriented with the direction of the pavement.As mentioned before, the light source is an important factor and may affect frequency histogram.In this study, all the pictures were taken at the same time, so that the light was same for all photos and also the pictures were free of shadows.To control this problem, the standard deviation of frequency histogram is used to detect segregation.In fact, the ambient light may affect the whole picture.The ambient light source can impact on the mean of frequency histogram, but it cannot affect the standard deviation of frequency histogram [24].Thus, the standard deviation of frequency histogram is used to detect segregation areas in asphalt pavement.

Results and Discussion
A MATLAB code was used to find image frequency histogram.The program imports the colored picture, converts it to a grayscale image, and obtains the frequency histogram.Afterward, the frequency histogram standard deviation is calculated as a symbol representing the pavement surface texture to be used to detect pavement segregation.Figure 2 represents the frequency histogram of the asphalt grayscale image which is the conversion of color asphalt image.All the pictures follow this trend.Figure 3 shows the standard deviation of segregated and nonsegregated frequency histogram in both site locations, respectively.
As shown in Figure 3, nonsegregated areas have a lower standard deviation in comparison with segregated areas.This trend makes it possible to define a threshold between pictures indicating segregated and nonsegregated areas.This difference between segregated and nonsegregated frequency histogram standard deviation can be caused by the closer texture of nonsegregated pavements in comparison with the segregated pavement.In other words, as shown in Figure 4, when segregation occurs, on one hand, coarse aggregates may separate from fine aggregates [25]; therefore, aggregates in segregated locations may not have enough interlock to each other which can lead to an increase of voids between the aggregates in pavement texture [26].This rise in voids in the pavement surface can cause brighter or darker colors, which can, in turn, result in higher standard deviation of segregated areas.
The standard deviation of both site locations is represented together in Figure 5.The threshold line between segregated and nonsegregated areas is not clearly obvious as shown in Figure 3. Due to this fact, the LDA classification method has been used to classify the segregated and nonsegregated pavements.
The LDA analysis was conducted using SPSS software.The results and prediction of each group membership are represented in Tables 1 and 2, respectively.Table 3 presents the membership prediction of both projects based on standard deviation of two projects using LDA method.
As shown in Tables 1 and 2, the threshold line which separates segregated and nonsegregated pavements is different.But, the overall estimation of segregated and nonsegregated group's membership in each site location is high enough.In other words, by comparison of Tables 1 and 2 with Table 3, the overall estimation decreased in Table 3, because each group has a specific threshold line to separate segregated and nonsegregated pavements which are different from the other.Therefore, the threshold which can separate segregated and  nonsegregated pavements is site-dependent.For each site, first the threshold line and membership prediction analysis should be conducted, and the results then can be used to determine segregated and nonsegregated pavement areas.In this research, for these two projects, it is obvious that the overall accuracy to determine the pavement segregation in each site location is beyond 80%.

Conclusion
A new image processing-based method to detect segregation in asphalt pavements was introduced.In the proposed method, the frequency histogram standard deviation of the grayscale image was used to identify segregated asphalt mixture pavement areas.The LDA was then performed to classify segregated and nonsegregated image processing results.The visual inspection has been applied by FDOT experts to verify this new method's result.This new method, which is nondestructive, can provide an alternative and decent solution to overcome disputes between contractors and agencies on the determination of segregated areas.Results from this work showed that images from segregated pavement areas have a higher standard deviation in comparison with those from nonsegregated areas.This outcome is due to the voids increase in segregated asphalt surface which can lead to brighter colors in pavement images.It is indicated that, in nonsegregated area's image, the color distribution is uniform and spectra include gray level colors while segregated area's image has brighter or darker colors.To apply these results to categorize segregated areas, first, the threshold line between segregated and nonsegregated pavements should be identified through LDA.The results then can be used to define segregated areas.For two studied projects in this research, LDA results show over 80% accuracy for detecting segregated areas.

Disclosure
The work represented herein was the result of a team effort.An earlier version of this work was presented as a poster at the 96th Annual Meeting of the Transportation Research Board 2017.

Table 1 :
Classification results a,b for site location 1.
a 90.0% of original grouped cases correctly classified.b Threshold line is in standard deviation of 40.

Table 2 :
Classification results a,b for site location 2.
a 80.0% of original grouped cases correctly classified.b Threshold line is in standard deviation of 46.

Table 3 :
Classification results a,b for both sites together.
b Threshold line is in standard deviation of 41.