Intelligent Identification of Childhood Musical Murmurs

Heart murmurs are often the first signs of heart valvular disorders. However, most heart murmurs detected in children are innocent musical murmurs (also called Still’s murmurs), which should be distinguished from other murmur types that are mostly pathological, such as regurgitant, obstructive, and flow murmurs. In order to reduce both unnecessary healthcare expenditures and parental anxiety, this study aims to develop algorithms for intelligently identifying musical murmurs in children. Discrete wavelet transform was applied to phonocardiographic signals to extract features. Singular value decomposition was applied on the matrix derived from continuous wavelet transform to extract extra features. The sequential forward feature selection algorithm was then utilized to select significant features. Musical murmurs were subsequently differentiated via a classification procedure consisting of three classification techniques: discriminant analysis, support vector machine, and artificial neural network. The results of 89.02% sensitivity, 84.76% specificity and 87.36% classification accuracy were achieved.


INTRODUCTION
Heart disease is a major cause of death in the United States, with 631,636 deaths due to heart disease in 2006, accounting for 26% of total deaths in that year [1].Heart disease was expected to cost the US $316.4 billion in 2010, which included the cost of health care services, medications, and lost productivity [2].Prevention and control of heart disease are therefore of primary importance.Heart murmurs always deserve great attention because they are often the first signs of heart valvular disease [3,4].Murmurs are usually caused by turbulence in blood flow or vibration of heart tissues.They can occur in the systolic period, the diastolic period, or both.Murmurs frequently appear in children [5], but fortunately, most heart murmurs detected in children are innocent [6].
To relieve parental and patient anxiety, it is important to recognize an innocent murmur as soon as it is detected [7,8].However, without diagnosis by a doctor with experienced auscultation skills or a cardiologist, innocent and pathological heart murmurs cannot be readily distinguished [9][10][11].As many as 87% of the patients with heart murmurs referred to cardiologists for evaluation turn out to have benign murmurs [12].The standard clinical method for diagnosing cardiac disease is echocardiography.However, ordering an echocardiogram is costly, usually between US$750 and 1500 [13].Therefore, there is a need to develop a reliable and affordable method that is capable of distinguishing innocent from pathological murmurs, as an assistive confirming tool of diagnosis.
A considerable amount of research has been conducted to address this challenging task [14][15][16][17][18][19][20][21][22][23][24][25][26].Noponen et al. [14] conducted a phono-spectrographic analysis to differentiate innocent from other murmurs caused by ventricular septal defect (VSD), aortic stenosis (AS), pulmonic stenosis (PS), atrial septal defect (ASD), and patent ductus arteriosus (PDA).Thompson et al. [15] designed a system to distinguish innocent from other murmurs with different conditions (e.g., AS, ASD, PS, VSD) and examined the correlation between relative murmur intensity and likelihood of pathology.Andrisevic et al. [16] proposed an algorithm that utilized wavelet analysis and artificial neural network for the detection of heart murmurs, distinguishing normal heart murmurs from those associated with AS, PS, and mitral stenosis (MS).Ahlstrom et al. [18] presented multiple techniques to extract various features, from which a suitable feature subset was selected for the task of classifying murmurs into normal murmurs, murmurs with AS, and murmurs with mitral insufficiency (MI).Reed et al. [24] developed a system for heart sound analysis and evaluated the system using heart sounds with five different conditions: normal, mitral valve prolapse (MVP), coarctation of the aorta (CA), VSD, and PS.Ölmez et al. [25] used wavelet transform and dynamic programming to extract and analyze features, and applied artificial neural network for the classification of seven different heart sounds: AS, MI, MS, PS, aortic regurgitation (AR), summation gallop, and normal heart sound.
As depicted above, most of the existing works focused on the differentiation of innocent murmurs from pathological murmurs.This study is to address the problem from a different perspective by considering different murmur types.Since heart murmurs occurring in a diastole period are mostly pathological [5], similar to Ahlstrom et al. [18], this paper only considers systolic murmurs which include four types: musical murmurs, flow murmurs, obstructive murmurs, and regurgitant murmurs.Musical murmurs, also called Still's murmurs, are caused by vibration of tissues.They are innocent in children and probably organic in adults.Both flow murmurs and obstructive murmurs are caused by turbulent blood flow.Flow murmurs are innocent if the splitting of a heart sound component S2 is normal, and are organic otherwise.Obstructive murmurs are pathological, resulting from heart conditions such as aortic stenosis.Regurgitant murmurs are also pathological.In this study, we considered child patients only.Since musical murmurs are innocent in children, differentiating musical murmurs from the other three murmur types is useful and helpful in relieving parents' concern and avoiding expensive echocardiogram.This study aims to contribute to the development of a reliable intelligent diagnostic device by developing algorithms that help distinguish musical murmurs from other types of murmurs in children.
In this work, phonocardiographic signals of child patients were analyzed.Murmur segmentation is a very difficult task which usually needs the assistance of electrocardiogram [18,19].In the current study, this task was manually accomplished by experienced personnel.Features were then extracted from the murmur segments.Feature extraction methods include time-frequency analysis [20], instantaneous energy and frequency estimation [21], wavelet transform [22], recurrence quantification analysis [23], and others [18].Wavelet transform was employed in the present work since it is suitable for analyzing non-stationary signals like heart sounds.Discrete wavelet transform was used to extract the first set of features.Additional features were extracted by applying singular value decomposition to the time-scale matrix derived by continuous wavelet transform.In order to reduce the computational complexity associated with a large number of features extracted, the most significant features were selected by the sequential forward feature selection algorithm.Different from previous works, most of which utilized artificial neural network to perform classification [16][17][18]25], we proposed an ensemble classification method based on three classification techniques including discriminant analysis, support vector machine and artificial neural network.The computational results demonstrate high sensitivity and specificity, indicating good differentiation of musical murmurs from other murmur types.
This paper is organized as follows: Section 2 describes the data source and the adopted methodology, with each inline method introduced in detail in the subsequent subsections, Section 3 presents the results, Section 4 provides discussions, and Section 5 summarizes the work.

DATA AND METHODOLOGY 2.1. Data Description and Preprocessing
The heart sound data used in this study were extracted from an auscultation skill training CD [27], which has pre-classified murmur types.Thirty heart sound recordings were selected, all from children aged 16 years and under.This research does not need approval by the IRB concerning protection of human subjects in research.A typical heart sound recording contains a number of cardiac cycles (see Figure 1).A cardiac cycle is defined as a heart sound segment between two consecutive S1's (or S2's).The segment between S1 and S2 is the systole period, while the segment between S2 and the next S1 is the diastole period.Heart murmurs can occur in either period or both.Murmurs occurring solely in a diastole period are pathological [5]; therefore, in this study, only systolic murmurs were taken into account.For each of the 30 selected heart sound recordings, we manually extracted several systole segments that contain murmurs.In total, 269 murmur segments were acquired, including 105 musical murmur segments and 164 other murmur segments.The subsequent analysis was conducted on these 269 murmur segments.

Figure 1.
A segment of heart sound recording.
In order to eliminate heart sound intensity variations resulting from different auscultation locations and conditions, the acquired heart sound data were normalized according to the following formula: (1) where x(•) is heart sound data, ͉x(•)͉ is the absolute value of x(•), representing the magnitude, y(•) is the normalized heart sound data, and N is the number of data points.

Methodology Framework
Figure 2 delineates the methodology framework for developing an intelligent system that is able to specify musical murmurs.Firstly, appropriate data representations of murmurs (i.e., features) were extracted based on the rule that the favorable features should be capable of keeping similarities within classes while exposing differences between classes.Next, a small number of the most significant features were selected, as a large number of features generally cause computational difficulties in the later classification steps.Heart murmurs are subsequently differentiated into different types via an appropriate classification method.The specific method for each stage is described in detail below.

Figure 2.
The methodology framework.

Feature Extraction
Heart sound is a non-stationary signal.Wavelet transform was applied to the extracted heart sound segments, since it is a time-scale representation method very suitable for analyzing non-stationary signals.Wavelet transform is superior to other time-frequency or time-scale analysis methods, such as short-time Fourier transform, in that not only it is capable of providing time and scale information simultaneously, but also it has varying time and scale resolution.Both discrete wavelet transform (DWT) and continuous wavelet transform (CWT) were employed in this study.DWT has been widely used in published works for murmur classification, while CWT has not.New features based on DWT were first extracted.In order to extract more features, CWT, which provides more information than DWT, was further employed.The feature derivation methods based on DWT and CWT were different as demonstrated below.The adopted wavelet was Daubechies-2, which had excellent performance for heart sound classification [25,26,28].

Features Derived from DWT
The first set of features was extracted via DWT.A six-level decomposition using Daubechies-2 wavelet was implemented.Only six levels were considered because of the constraint of the number of data points in a murmur segment.Each level of decomposition reduces the number of points to half of the original.The first-level approximation coefficients and detail coefficients were calculated according to the following formulas: (2) where a 1 (•) and d 1 (•) stand for approximation and detail coefficients, respectively, and g(•) and h(•) represent low-pass and high-pass filters associated with Daubechies-2 wavelet, respectively.The second-level decomposition was conducted by repeating execution of the above two equations but using a 1 (n) instead of y(n).The newly determined approximation coefficients were used in the next level decomposition, and the decomposition continued until the six-level approximation and detail coefficients were obtained.As depicted in Figure 3, the normalized heart sound recording y(n) was first processed by the low-pass and high-pass filters, respectively, and was then subsampled by 2. The first-level approximation and detail coefficients were thus determined.The same procedure was then applied to the newly determined approximation coefficients to calculate the approximation and detail coefficients at the next level.The procedure was repeated up to level six.In each level, the sequence of detail coefficients was segmented.Since the sequence length on each level was different, the number of segments on each level was set to be different as well.They were 15, 10, 6, 3, 2, and 1, respectively.Such segmentation made the number of detail coefficients in each segment (regardless of which level) differ slightly.For each segment, absolute values of the detail coefficients were taken and averaged.The averaged values were taken as features.

Features Derived from CWT
For CWT, the normalized heart sound signal y(•) was regarded continuous by using a piecewise constant interpolation, and was then transformed according to the following formula [29]: (4) where a and b denote scale and position parameters, respectively, Y(•) denotes a mother wavelet (here it is Daubechies-2 wavelet), and '*' is the operator of complex conjugate.By appropriately setting the ranges of a and b, a time-scale representation matrix composed of CWT coefficients was formulated for each of the 269 murmur segments.Subsequently, the singular value decomposition (SVD) technique was applied on these matrices to extract features.
SVD is a powerful tool that was successfully applied in the field of signal processing and for statistical data analysis [28,30].The SVD [31] of an m × n matrix X is given by (5) where U is an m × n orthonormal matrix (i.e., U T U = I) V is an m × n orthonormal matrix (i.e., V T V = I) and S is an m × n diagonal matrix with nonnegative diagonal elements in decreasing order.The columns of the matrices U and V are called left and right singular , ( vectors, respectively.The diagonal elements in the matrix S are called singular values, which represent the importance of the corresponding left and right singular vectors in the composition of the matrix X.In this study, the first 10 largest singular values were chosen as features. Since singular vectors are orthonormal, their squared elements can be regarded as probability density function.We adopted the method of [30] to extract additional features, where both left and right singular vectors need to be considered simultaneously for extracting features.The first left and right singular vectors correspond to the largest singular value and they were selected for analysis.Take the first left singular vector as an example, denoting it as U 1 = (u 11 , u 21 ,•••,u m1 ) T .By squaring each of its elements, a probability density function was achieved as follows: (6) where . Its cumulative distribution function was then: where A 10-bin histogram computation was implemented on F U 1 and the calculated 10 values were chosen as features.The same procedure was applied on the first right singular vector and another 10 features were obtained.

Feature Selection
In total, 67 features were extracted based on the procedures described above.If all these features were adopted, it would require an enormous computational effort to implement the subsequent classification task.In order to decrease the number of features, the sequential forward feature selection (SFFS) algorithm was applied.SFFS is a feature selection algorithm that sequentially adds a new feature into the currently selected feature subset, such that this addition is the best among all possible additions, based on some designated criteria [32].Specifically, at the first step, a single feature that achieves the best criteria value is selected.The second feature is selected such that its inclusion in the current feature subset results in the best criteria value compared to other possibilities.Such a procedure is continued until the required number of features is obtained or other terminating conditions are reached.In this study, the selection criterion was the minimization of the average classification error rate via 10-fold cross validation.Linear discriminant analysis (LDA) was the classification method.The terminating condition was that the addition of a new feature does not further decrease the criteria value.The results of feature selection are presented in Section 3.

Classification
After appropriate features were selected, murmur signals were classified.Since the aim of this study was to distinguish musical murmurs from other types of murmurs, only two classes were considered: the class of musical murmurs (class 1) and the class of other types of murmurs (class 0).In most of the published works, artificial neural networks (ANNs) were applied for classification.An artificial neural network is a nonlinear statistical data modeling tool.It is an adaptive system that adjusts its own structure based on external or internal information that passes through it during the learning phase.It includes an input layer, one or more hidden layers, and an output layer, with a number of neurons on each layer.ANNs can model complex relationships between inputs and outputs, and therefore can be used for classification.
In this study, we also considered other classification techniques such as discriminant analysis (DA) and support vector machine (SVM), which have never been used in the literature of murmur differentiation.DA is a statistical technique for classifying new observations into predefined classes [33].For a k-class problem, k discriminant functions are constructed.Given a new observation, all the k discriminant functions are evaluated and the observation is assigned to class i if the i-th discriminant function has the highest value.Discriminant functions can be of several types, including linear and quadratic.SVM is a classification method that performs classification tasks by constructing hyperplanes in a multi-dimensional space to optimally separate classes [33,34].For the linearly separable case, SVM is to find a hyperplane or a set of hyperplanes with the maximum margin between two classes.For the linearly nonseparable case, SVM produces nonlinear boundaries by constructing linear boundaries in a large, transformed version of the feature space.In other words, SVM first maps the original input space into a higher dimensional, linearly separable feature space, and then constructs linear boundaries in the feature space.The mapping is a mathematical function called a kernel.There are many types of kernel functions, including linear kernel, polynomial kernel, radial basis function, etc.
For each classification method, the best classifier that achieves the highest classification accuracy was determined through searching different parameters, kernels, or numbers of layers, etc.In order to take advantage of these three classification techniques, an ensemble classification method was proposed, which applied a majority rule.A new objective was assigned to the class if it was the result of at least two of the above three classifiers.The concept of ensemble classification is very popular in the field of data mining and machine learning.This method can include different classifiers and combines the results from these classifiers together.It thus takes advantage of all the classifiers included.In implementing ANN, DA, and SVM, 10-fold cross-validation was applied.The detailed classification results are presented in the following section.

RESULTS
The extracted features are summarized in Table 1.The first column lists feature numbers, the second column shows the source based on which the corresponding features were derived, and the third column explains the features.Using the criterion of minimizing the average classification error rate from 10-fold cross validation, the following subset of 8 features was selected by the SFFS algorithm: {24, 29, 37, 39, 51, 52, 59, 66}.Among The criterion value in each SFFS step.
The three classification techniques were applied to the selected features.For the DA method, different types of discriminant functions were examined.For the SVM method, different kernel functions with different parameter values were considered.For the ANN method, two networks were investigated: one with two hidden layers, and the other with three hidden layers.For each network, different combinations of the number of neurons on each layer were investigated.The best classifiers obtained from the three methods are an LDA classifier, an SVM classifier with a linear kernel function, and an ANN with three hidden layers of 2, 8, and 15 neurons, respectively.Applying the ensemble method, the final classification result is shown in Table 3 which demonstrates that, of the 105 musical murmur segments, 16 were misclassified.Of the 164 other types of murmur segments, 18 were misclassified.

Table 3. Result of classification.
As a common practice for binary classification methods, sensitivity and specificity were assessed to evaluate the performance of our ensemble classification method.Since musical murmurs are innocent in children while other types of murmurs are mostly pathological, sensitivity was calculated as the proportion of other types of murmurs that were correctly classified as such, and specificity was calculated as the proportion of musical murmurs that were correctly classified as such.After calculation, the current classification method provided a sensitivity of 146/164 = 89.02%and a specificity of 89/105 = 84.76%.The classification accuracy was (89+146)/269 = 87.36%.Furthermore, Figure 5 displays the Receiver Operating Characteristic (ROC) curve (blue) for the current ensemble classification method.The diagonal line (red) represents the ROC curve based on random guessing.The area under the curve (AUC) reflects the performance of a classifier.The larger the AUC, the better performance of the classifier.The ideal or perfect classifier has the AUC of 1. Figure 5 shows that the performance of the proposed ensemble classification is very good, with an AUC value of 0.841.

Figure 5.
The ROC curve for the ensemble classification.

DISCUSSION
Previous works mainly focused on differentiating innocent from pathological murmurs such as aortic atenosis and mitral insufficiency.In this study, we considered child patients only and aimed to distinguish murmurs in terms of types.To the best of our knowledge, this study is the first to address identifying musical murmurs in children.
Wavelet transform is a useful method for analyzing non-stationery signals such as heart sound.Previous studies have widely applied discrete wavelet transform on murmur differentiation.Few studies have adopted continuous wavelet transform.In this study, we employed discrete wavelet transform, but different features were extracted than those in previous works.In addition, we implemented continuous wavelet transform and performed singular value decomposition on the derived coefficient matrix to extract useful features.In view of the size of the extracted features, the SFFS algorithm was executed to select features.Based on the reduced subset of features, musical murmurs were distinguished from other types of murmurs by a combination of three classification techniques: DA, SVM, and ANN.ANNs were often applied for classification in previous studies [16][17][18]25], while DA and SVM have never been used before.The combination of three classification methods demonstrated good performance, with a sensitivity of 89.02%, a specificity of 84.76%, and an accuracy of 87.36% achieved.
The combination of these three classifiers is superior to the best individual classifier.Tables 4 to 6 exhibit the results of classification by the three classification methods, respectively, while Table 7 compares the sensitivity, specificity and accuracy of three classification methods and the proposed combination method.LDA correctly classifies 95 musical murmurs (90.48% specificity) and is better than linear SVM, ANN and even the ensemble method.However, this benefit is at the cost of high misclassification rate of other types of murmurs (80.49% sensitivity).Linear SVM has the same misclassification rate of other types of murmurs with the ensemble method (89.02% sensitivity), while the number of musical murmurs that it is able to correctly classify is the lowest (80% specificity) among all methods.As for ANN, its overall performance is inferior to the ensemble method in terms of sensitivity, specificity, and accuracy.The above comparison suggests the advantage of the current ensemble classification method.The accuracies achieved by the current methods are in general significantly better than the average auscultation accuracy of cardiologists, which is approximately 80% [13,16].The results are also comparable to previous studies on the differentiation between innocent murmurs and pathological murmurs.Andrisevic et al. [16] applied wavelet analysis and ANNs to classify normal and abnormal heart sounds, with a specificity of 70.5% and a sensitivity of 64.7%.Strunic et al. [17] developed a system that classified normal heart sounds, aortic stenosis, and aortic regurgitation with up to 85±7.4% accuracy and 95±6.8%sensitivity.Ahlstrom et al. [18] took into account a large number of features extracted from different areas and the selected feature set achieved 86% classification accuracy.Noponen et al. [14] used phono-spectrograms to analyze heart murmurs with 90% specificity and 91% sensitivity.Ahlstrom et al. [23] applied recurrence quantification analysis to distinguish innocent murmurs from murmurs caused by aortic stenosis, and demonstrated a sensitivity of 90% and a specificity of 88%.Due to the variety of classification targets and the differences among heart sound data sources in earlier works, a strict comparison between our study and previous studies is impossible.
It was noted that 16 misclassified musical murmur segments came from 6 child patients, with the following numbers of segments misclassified for each patient: {2, 3, 3, 4, 3, 1}.Because we extracted about 8-10 murmur segments from each patient, this reflects that the developed technique only misclassifies some musical segments from a patient rather than all of them.On the other hand, 18 misclassified other types of murmur segments came from another 6 patients, with the number of misclassified segments for each patient being {6, 2, 5, 2, 2, 1}.It implies that the current method only missed some instead of all of other types of murmur segments from a patient.This result suggests an advantage of the proposed method: it does not completely misdiagnose a patient.Some aspects of this study could be improved.Firstly, wavelet transform was the only perspective from which the features were extracted.Other feature extraction methods such as Shannon energy and S-transform [18] could be employed to enlarge our feature base in the future work.Secondly, the adopted feature selection algorithm has a limitation: once a feature is selected, it will always stay in the selected feature pool.It disregards the possibility that other feature combination could result in better classification performance.Other feature selection algorithms, such as principle components analysis (PCA) or logistic regression model [35], may improve this classification method.Thirdly, in this work, we manually executed murmur segmentation from heart sound recordings.For a more intelligent murmur differentiation system, an automatic murmur segmentation algorithm needs to be developed.

CONCLUSION
In this paper, a methodology framework is proposed for identifying musical murmurs in children.Features were extracted from phonocardiographic signals, then selected to make computation possible, and finally employed for murmur classification.Novel features were extracted by discrete wavelet transform.To extract additional features, singular value decomposition was applied to the coefficient matrix derived from continuous wavelet transform of heart sound data.Features were selected by the sequential forward feature selection algorithm to reduce feature size.Three classifiers including discriminant analysis, support vector machine, and artificial neural network were combined to perform an ensemble classification.Favorable differentiation results with 89.02% sensitivity, 84.76% specificity and 87.36% classification accuracy were demonstrated.These results are superior to those from the three individual classifiers.
It is also noted that the proposed method does not completely misdiagnose a patient if it misses some murmur segments.