Pneumonia is a very common and fatal disease, which needs to be identified at the initial stages in order to prevent a patient having this disease from more damage and help him/her in saving his/her life. Various techniques are used for the diagnosis of pneumonia including chest X-ray, CT scan, blood culture, sputum culture, fluid sample, bronchoscopy, and pulse oximetry. Medical image analysis plays a vital role in the diagnosis of various diseases like MERS, COVID-19, pneumonia, etc. and is considered to be one of the auspicious research areas. To analyze chest X-ray images accurately, there is a need for an expert radiologist who possesses expertise and experience in the desired domain. According to the World Health Organization (WHO) report, about 2/3 people in the world still do not have access to the radiologist, in order to diagnose their disease. This study proposes a DL framework to diagnose pneumonia disease in an efficient and effective manner. Various Deep Convolutional Neural Network (DCNN) transfer learning techniques such as AlexNet, SqueezeNet, VGG16, VGG19, and Inception-V3 are utilized for extracting useful features from the chest X-ray images. In this study, several machine learning (ML) classifiers are utilized. The proposed system has been trained and tested on chest X-ray and CT images dataset. In order to examine the stability and effectiveness of the proposed system, different performance measures have been utilized. The proposed system is intended to be beneficial and supportive for medical doctors to accurately and efficiently diagnose pneumonia disease.
Chronic diseases and epidemics have taken the lives of a large number of people and created numerous crises in the countries, which takes a long time for a country to recover from the loss caused by both of these major outbreaks. Some diseases that ascend in a specific time period within a population are termed outbreaks and epidemics [
Pneumonia is an infective disease that inflames the air sacs in a single or both lungs caused by fungi, bacteria, and viruses [
Pneumonia infected and normal lung.
In 2017, more than 850,000 people died from pneumonia. The death ratio due to pneumonia is very high in South Asia and Sub-Saharan Africa. According to a report published in 2017, the death ratio in five countries, i.e., Pakistan, India, Ethiopia, Nigeria, and the Republic of Congo, was more than half of the deaths from childhood pneumonia and was called the ultimate disease of poverty [
Death rate from pneumonia across the world by age from 1990 to 2017 [
At the beginning of the 21st century, several coronaviruses have passed through the species fence to produce lethal pneumonia in human beings. In order to know the origin and development of these fatal pandemics, the experts need to inspect the structure of the virus and the method of how this virus causes infection. Furthermore, doing so will help the specialists in finding the right solution and providing proper treatment and possibly developing vaccines [
Past epidemics occurred over time.
Epidemic name | Duration | Approximate deaths |
---|---|---|
Antoine plague | 170–180 | 5.0 million |
Prague of Justinian | 541–543 | 35–50 million |
Japanese smallpox outbreak | 734–736 | 1.2 million |
Black death | 1349–1353 | 200 million |
Smallpox outbreak | 1520+ | 56–60 million |
Italian plague outbreak | 1628–1630 | 1.0 million |
Yellow fever (US) | 1886–1891 | 1.5 million |
Spanish flu | 1918–1920 | 45–50 million |
Third plague (China and India) | 1985+ | 12 million |
Asian flu | 1959–1960 | 1.2 million |
HIV/AIDS | 1980–to present | 30–35 million |
SARS | 2002-03 | 700–800 |
Swine flu | 2009-10 | 0.2 million |
Ebola | 2014–2017 | 10,000–11,000 |
MERS | 2015–to present | 860 |
COVID-19 | 2019–(12/04/2021) | 2.94 million |
Severe Acute Respiratory Syndrome Coronavirus (SARS-CoV) [
The Middle East Respiratory Syndrome Coronavirus (MERS-Cov) is a viral respirational infection produced by a virus [
Nowadays, the world is facing a hazardous pandemic which occurred due to a virus and is named COVID-19, acknowledged in December 2019, for the first time in China, Wuhan Province, and lead to the death of many people [
Medical image analysis plays a vital character in the diagnosis of various ailments like MERS, Covid-19, pneumonia, etc., and is considered to be one of the auspicious approaches [
The remaining paper is organized as follows. Section
Pneumonia is one of the fatal diseases, which is more dangerous for children and old age people. Toğaçar et al. [
Behzadi-Khormouji et al. [
Medical imaging plays a significant part in the identification of numerous diseases [
The following subsection describes the resources used and the approaches followed in carrying out this research study.
The development of an automated and intelligent system extensively depends on the problem-related dataset. It means that a problem-specific dataset has a very high influence on the efficiency of an intelligent model. Considering the significance of the dataset, a chest X-ray and CT image dataset was used which is available online in the UCI Kaggle databases. The dataset consists of a total of 5856 images of two categories/classes, i.e., pneumonia and normal images. The dataset contains 1583 normal and 4273 pneumonia images. The dataset is distributed in two parts (training and testing), where 70% of the data is used to train the models while 30% of the data is used to test and validate the model. Figure
An example of chest X-ray images (a) Normal, (b) Pneumonia.
The main goal and objectives of the proposed system are to diagnose whether a person has pneumonia or not at the early stages through their chest X-ray images in order to prevent them from more damage. In this study, the recent DCNN architectures based on the fine-tuned versions of (CNN, AlexNet, SqueezeNet, VGG16, VGG19, and Inception V3) are used to extract useful features from the images. Several preprocessing techniques are used, in order to present the data in a normalized form to the classification models. Various ML classification models such as KNN, SVM, LR, NB, AB, and ANN are used in this study. Different performance assessment metrics are computed to measure and track the performance of each utilized ML model. Keras deep learning framework is deployed which uses TensorFlow at the backend for building and training our proposed system. The libraries and packages used in the implementation of the work include TensorFlow, Keras, Sklearn, Matplotlib, Seaborn, and NumPy. All the experiments were performed using the Jupyter NoteBook of the Anaconda integrated development environment (IDE). Figure
Proposed framework for pneumonia identification.
Data preprocessing is a vital technique used to provide data to the classification models in a well-organized manner, which are then trained and tested while using the normalized data. For the improvement of visual information quality (removal of noise, increasing contrast, deletion of high or low frequencies, etc.) of each input image, these images are preprocessed with the help of numerous techniques before being used in the classifiers. Preprocessing techniques such as intensity normalization, Contrast Limited Adaptive Histogram Equalization (CLAHE), and Min-Max normalization have been investigated in this study. Intensity normalization, CLAHE, and Min-Max normal distribution are interesting and important preprocessing techniques in image processing applications. Figure
Images before and after applying preprocessing techniques: (a) original, (b) normalize, and (c) CLAHE.
Looking at the dataset which represents two classes, i.e., pneumonia and normal images, almost 75% of the images represent pneumonia and the remaining 25% describe normal images which means that the dataset is imbalanced. To resolve the issue of unbalanced dataset and overfitting and to increase the accuracy of the models, various augmentation techniques have been used. The data augmentation techniques used include geometric transformations like rotations, zooms, rescale, shift, flips, and shears.
CNN is a popular deep learning model particularly used for image classification problems. It normally consists of five layers which include the input layer, convolution layer, pooling layer, fully connected layer, and output layer. The practical assistance of CNN is having fewer parameters which significantly decrease the time it takes to learn and reduce the amount of data needed for training the model. In addition, CNN can be trained end-to-end for the extraction and selection of features from an image and, at last, can be used to predict or classify the images. It seems a bit difficult to know how a network understands or processes an image, but features conquered at various layers of a network perform better as compared to human-built features [
Basic architecture of CNN model.
The CNN architecture used for the experimental work in this study has the following properties: Input layer: X-ray images are used as input and are provided at the input layer. The image dimensions are kept to Convolutional layer: we have used 3 convolution layers having Pooling layer: we have used max pooling for calculating the maximum value at every patch for each feature map. The max-pooling size is set to 2 × 2 while the stride value used is 2.0. Fully connected layer: this layer used in the proposed architecture utilized the sigmoid activation function at the outer layer. Output layer: the output layer gives us the predicted result that whether the person has pneumonia or not.
DL architectures are extensively used in image processing specifically in healthcare for diagnosing various diseases. These DL techniques extract useful features from the images and present them to the models for further investigation. Here, in our study, we have used five important DL architectures such as AlexNet, VGG16, VGG19, Inception-V3, and SqueezeNet. A brief description of the investigated DL architectures is given hereinafter.
AlexNet is a type of CNN, which comprises various layers such as input, convolution, max pooling, dense, and output layers that are its basic building blocks. In 2012, it won the ILSVRC competition. It solves the problem of image classification where the input image is one of 1000 different classes and the output is a vector of those classes. The
VGG (Visual Geometry Group) is a type of CNN architecture proposed for the first time by two researchers Simonyan and Zisserman in 2014 [
Inception models are a type of deep neural network (DNN) architecture developed by a researcher named Szegedy et al. for the first time in 2014, and the model was named as inception model [
SqueezeNet is a type of deep neural network developed by the researchers of Stanford University and was released on 22nd February 2016 for the first time. It is a type of CNN architecture consisting of 18 layers, particularly used in computer vision and image processing. The main objectives and goal of the authors from developing SqueezeNet were to create a smaller neural network, which consists of fewer parameters, can fit into computer memory easily (requires less memory), and can be easier to transmit over a computer network (requires less of bandwidth). Firstly, the original version of this architecture was implemented on top of a DL framework named Caffe. After a short period of time, the researchers started the use of this architecture in a number of open-source DL frameworks. SqueezeNet was firstly labeled in a paper in which it was compared with the AlexNet and was mentioned that it achieves AlexNet level accuracy with “50X” fewer parameters. AlexNet contains 240 MB parameters while SqueezeNet consists of only 5 MB parameters. Both the SqueezeNet and AlexNet are two different DNN architectures, and they have just one thing in common, i.e., their accuracy when evaluated on the ImageNet image dataset.
Various ML classification algorithms have been investigated for the diagnosis of whether a person has pneumonia disease or not. Each classification algorithm has its own importance, and its significance varies from application to application. In this paper, 6 distant natures of classification algorithms, namely, KNN, SVM, LR, NB, AB, and ANN, are applied in order to select the best and generalized prediction model.
In order to track the performance of each classifier used in this study, several performance measures have been utilized such as accuracy, specificity, sensitivity,
Confusion matrix.
Predicted (−) | Predicted (+) | |
---|---|---|
Actual (−) | TN | FP |
Actual (+) | FN | TP |
All of the abovementioned formulas are carried out from the confusion matrix which consists of the following basic components: True positive (TP): it means that the model prediction is positive and in actual fact the person has pneumonia. So, a pneumonia subject is diagnosed correctly by the model. True negative (TN): it means that the model prediction is negative and in actual fact the person does not have the pneumonia disease. Hence, a healthy person is diagnosed correctly by the classification model. False positive (FP): it means that the model did a wrong prediction by classifying a healthy person as a pneumonia patient. This is also known as type-1 error. False negative (FN): it means that the model did a wrong prediction by classifying a pneumonia patient as healthy. This is also known as type-2 error.
The simulation results of various ML classification algorithms by using different DL architectures such as AlexNet, SqueezeNet, VGG-16, VGG-19, and Inception-V3 are discussed in this section. These DL architectures, also called transfer learning techniques, extract useful features from the images which are very useful in classifying the normal and pneumonia patients in an efficient way. The performance of all utilized ML classifiers, i.e., KNN, SVM, LR, NB, AB, and ANN, was checked on the pneumonia chest X-ray dataset on full feature space generated by the transfer learning techniques. For measuring the performance of ML classifiers, different performance measures are used. In addition, preprocessing techniques are also applied to all features before being used by the classification algorithms.
This subsection represents the experimental results attained by the CNN classification algorithm. We performed multiple experiments on the basic CNN model by using various epoch numbers. First, we used 100 epochs and then 150 epochs, and at last, we used 200 epochs and got that the accuracy was increasing from epoch 0 to epoch 10, and after, that it becomes stable and remained 92.30%. Figure
ROC curve of the Convolutional Neural Network (CNN) classification algorithm.
This section represents the simulation results carried out through all the utilized ML classification algorithms using the AlexNet transfer learning technique. Table
Performance of all classifiers using the AlexNet transfer learning architecture.
Classification model | Accuracy | Sensitivity | Specificity | AUC | MCC | |
---|---|---|---|---|---|---|
KNN ( | 94.10 | 95.40 | 90.08 | 96.76 | 0.94 | 0.73 |
SVM (rbf) | 51.13 | 54.52 | 50.98 | 53.45 | 0.54 | 0.51 |
SVM (linear) | 88.65 | 92.98 | 72.53 | 88.94 | 0.85 | 0.70 |
AB | 89.72 | 92.63 | 80.74 | 86.62 | 0.89 | 0.73 |
NB | 87.89 | 87.56 | 88.62 | 92.68 | 0.88 | 0.72 |
LR | 95.94 | 96.98 | 91.40 | 98.42 | 0.95 | 0.89 |
ANN | 96.44 | 96.82 | 92.62 | 98.84 | 0.96 | 0.91 |
Table
Figure
Performance of all classifiers using the AlexNet transfer learning technique.
Figure
Figure
ROC curves of all 6 ML classifiers using the AlexNet transfer learning technique.
From Figures
The experimental results and performances of all the utilized 6 ML classifiers, using the SqueezeNet transfer learning technique, are discussed here in this subsection. The transfer learning techniques are used to extract valuable features from the images and then present them to the classifiers to classify it. Table
Performance of all classifiers using the SqueezeNet transfer learning architecture.
Classification model | Accuracy | Sensitivity | Specificity | AUC | MCC | |
---|---|---|---|---|---|---|
KNN ( | 95.16 | 96.44 | 91.05 | 97.80 | 0.95 | 0.73 |
SVM (rbf) | 52.03 | 55.43 | 51.88 | 54.33 | 0.55 | 0.52 |
SVM (linear) | 88.71 | 93.96 | 73.37 | 89.85 | 0.86 | 0.70 |
AB | 90.12 | 93.13 | 81.43 | 87.34 | 0.90 | 0.74 |
NB | 88.51 | 88.23 | 89.26 | 93.20 | 0.89 | 0.73 |
LR | 96.24 | 97.62 | 91.94 | 99.20 | 0.96 | 0.90 |
ANN | 96.97 | 97.52 | 92.99 | 99.40 | 0.97 | 0.92 |
Table
Figure
Performance of all classifiers using the SqueezeNet transfer learning technique.
Figure
Figure
ROC curves of all 6 ML classifiers using the SqueezeNet transfer learning technique.
The experimental results and performances of all 6 ML classification algorithms using VGG-16 transfer learning techniques are described in this subsection. Table
Performance of all classifiers using the VGG16 transfer learning techniques.
Classification model | Accuracy | Sensitivity | Specificity | AUC | MCC | |
---|---|---|---|---|---|---|
KNN ( | 95.41 | 96.00 | 93.66 | 98.32 | 0.95 | 0.88 |
SVM (linear) | 81.72 | 81.23 | 82.99 | 87.33 | 0.83 | 0.59 |
SVM (rbf) | 50.30 | 86.24 | 47.42 | 69.30 | 0.70 | 0.35 |
AB | 90.12 | 93.57 | 80.01 | 86.82 | 0.90 | 0.74 |
NB | 86.50 | 84.41 | 92.69 | 94.10 | 0.87 | 0.70 |
LR | 96.82 | 97.80 | 94.03 | 99.51 | 0.97 | 0.92 |
ANN | 96.56 | 97.52 | 93.28 | 99.22 | 0.97 | 0.91 |
From Table
The performances of all 6 ML classification models, using VGG-16 transfer learning techniques, are shown in Figure
Performances of all classifiers using the VGG16 transfer learning technique.
Figure
ROC curves of all 6 ML classifiers using the VGG16 transfer learning technique.
This subsection demonstrates the performance and experimental results obtained through all 6 ML classification models using the VGG-19 transfer learning technique. The transfer learning techniques extract useful features from the images and then present them to the classifiers for further processing. The performance of all 6 ML classification models is presented in Table
Performance of all classifiers using the VGG19 transfer learning techniques.
Classification model | Accuracy | Sensitivity | Specificity | AUC | MCC | |
---|---|---|---|---|---|---|
KNN ( | 96.10 | 96.09 | 94.11 | 97.89 | 0.96 | 0.89 |
SVM (linear) | 84.70 | 82.20 | 84.89 | 87.55 | 0.85 | 0.61 |
SVM (rbf) | 50.32 | 84.30 | 48.80 | 69.60 | 0.69 | 0.36 |
AB | 92.42 | 90.27 | 82.10 | 86.92 | 0.91 | 0.75 |
NB | 88.40 | 86.33 | 93.09 | 94.60 | 0.89 | 0.72 |
LR | 96.92 | 97.60 | 94.79 | 98.91 | 0.97 | 0.91 |
ANN | 97.01 | 97.62 | 93.80 | 99.12 | 0.97 | 0.92 |
Table
Figure
Performances of all classifiers using the VGG19 transfer learning technique.
Figure
ROC curves of all 6 ML classifiers using the VGG19 transfer learning technique.
The performance and experimental results attained through all 6 ML classifiers using Inception-V3 DL architecture are discussed here in this subsection. The performance of all 6 ML classification models using Inception-V3 architecture is illustrated in Table
Performance of all classifiers using the Inception-V3 transfer learning techniques.
Classification model | Accuracy | Sensitivity | Specificity | AUC | MCC | |
---|---|---|---|---|---|---|
KNN ( | 94.20 | 94.24 | 94.03 | 97.80 | 0.94 | 0.85 |
SVM (linear) | 85.02 | 92.03 | 86.41 | 87.33 | 0.85 | 0.78 |
SVM (rbf) | 50.30 | 84.23 | 48.39 | 50.20 | 0.74 | 0.33 |
AB | 87.40 | 91.33 | 78.91 | 83.62 | 0.87 | 0.67 |
NB | 91.50 | 90.89 | 93.13 | 95.90 | 0.92 | 0.80 |
LR | 97.08 | 97.90 | 94.33 | 99.52 | 0.97 | 0.92 |
ANN | 97.19 | 97.88 | 94.92 | 99.53 | 0.97 | 0.92 |
Table
The performance of all 6 ML classification models using the Inception-V3 transfer learning technique is demonstrated in Figure
Performances of all classifiers using the Inception-V3 transfer learning technique.
The MCC and
ROC curves of all 6 ML classifiers using the Inception-V3 transfer learning technique.
The performance of all five utilized DCNN transfer learning techniques and 6 ML classification algorithms have been evaluated by using various performance evaluation metrics as discussed above. From the abovementioned results, it is obvious that Inception-V3 and ANN performed brilliantly by attaining the classification accuracy of 97.19%, sensitivity of 97.92%, specificity of 94.92%, AUC of 99.53%,
In addition, a comparative study of the proposed system is conducted with the previous ML and DL methods used in the past [
A comparative study of the proposed system and previous approaches.
Publications | Approach | Accuracy (%) |
---|---|---|
Kermany et al. [ | Convolutional neural network (CNN) | 92.81 |
Stephen et al. [ | DL model with 4 conv-layers and 2 dense layers | 93.71 |
Saraiva et al. [ | DL model with 6 conv-layers and 3 dense layers | 95.29 |
Liang and Zheng [ | DL model with 48 conv-layers and 2 dense layers | 96.01 |
Wu et al. [ | CNN + random forest | 96.70 |
Proposed method | Intelligent framework (Inception-V3 + ANN) | 97.19 |
Table
Pneumonia is an infective disease and is very hazardous for all ages and is more dangerous specifically for smokers, alcoholics, recent surgical patients, asthma patients, people with weakened immune systems, and children having an age of less than 5 years. The death ratio caused by pneumonia can be condensed if the patients are diagnosed at the initial stages and on-time medication and treatment is provided to them. This study proposes an ML- and DL-based intelligent predictive system for the diagnosis of pneumonia. Chest X-ray and CT image dataset was utilized for both training and testing of the proposed system. In order to improve the quality of visual information of each input image, various preprocessing methods such as intensity normalization, CLAHE, and Min-Max normalization have been utilized in this study. Five fine-tuned versions of DL transfer learning techniques such as AlexNet, SqueezeNet, VGG-16, VGG-19, and Inception-V3 were utilized to extract useful features from the X-ray images and then present them to the classifiers for further processing. Six imperative ML classification algorithms such as KNN, NB, ANN, SVM, LR, and AB were used to examine the efficiency of the proposed system. Numerous performance evaluation measures including classification accuracy, sensitivity, specificity,
All data is available in this paper.
The authors declare that they have no conflicts of interest.
This paper was supported by the Taif University Researchers Supporting Project Number TURSP-2020/126, Taif University, Taif, Saudi Arabia.