An Enhanced Grey Wolf Optimization Based Feature Selection Wrapped Kernel Extreme Learning Machine for Medical Diagnosis

Li, Qiang; Chen, Huiling; Huang, Hui; Zhao, Xuehua; Cai, ZhenNao; Tong, Changfei; Liu, Wenbin; Tian, Xin

doi:https://doi.org/10.1155/2017/9512741

Computational and Mathematical Methods in Medicine

On this page

Abstract Introduction Background Experimental Results Conclusions Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2017 | Article ID 9512741 | https://doi.org/10.1155/2017/9512741

An Enhanced Grey Wolf Optimization Based Feature Selection Wrapped Kernel Extreme Learning Machine for Medical Diagnosis

Qiang Li,¹Huiling Chen,¹Hui Huang,¹Xuehua Zhao,²ZhenNao Cai,¹Changfei Tong,¹Wenbin Liu,¹and Xin Tian³

Academic Editor: Ezequiel López-Rubio

Received08 Oct 2016

Revised03 Dec 2016

Accepted21 Dec 2016

Published26 Jan 2017

Abstract

In this study, a new predictive framework is proposed by integrating an improved grey wolf optimization (IGWO) and kernel extreme learning machine (KELM), termed as IGWO-KELM, for medical diagnosis. The proposed IGWO feature selection approach is used for the purpose of finding the optimal feature subset for medical data. In the proposed approach, genetic algorithm (GA) was firstly adopted to generate the diversified initial positions, and then grey wolf optimization (GWO) was used to update the current positions of population in the discrete searching space, thus getting the optimal feature subset for the better classification purpose based on KELM. The proposed approach is compared against the original GA and GWO on the two common disease diagnosis problems in terms of a set of performance metrics, including classification accuracy, sensitivity, specificity, precision, -mean, -measure, and the size of selected features. The simulation results have proven the superiority of the proposed method over the other two competitive counterparts.

1. Introduction

In order to make the best medical decisions, medical diagnosis plays a very important role for medical institutions. As everyone knows, false medical diagnoses will lead to incorrect medical decisions, which are likely to cause delays in medical treatment or even loss of patients’ life. Recently, a number of computer aided models have been proposed for diagnosing different kinds of diseases, such as diagnostic models for Parkinson’s disease [1, 2], breast cancer [3, 4], heart disease [5, 6], and Alzheimer’s disease [7, 8]. As a matter of fact, medical diagnosis could be treated as a problem of classification. In the medical diagnosis field, datasets usually contain a large number of features. For example, colorectal microarray dataset [9] contains two thousand features with highest minimal intensity across sixty-two samples. However, there are irrelevant/redundant features in dataset which may reduce the classification accuracy. Feature selection is proposed to solve this problem. The process of a typical feature selection method consists of four basic steps [10]: (1) generation: generate the candidate subset; (2) evaluation: evaluate the subset; (3) stopping criterion: decide when to stop; (4) validation: check whether the subset is valid. Based on whether the evaluation step includes a learning algorithm or not, feature selection methods can be classified into two categories: filter approaches and wrapper approaches. Filter approaches are independent of any learning algorithm and often computationally less expensive and more general than wrapper approaches, while wrapper approaches evaluate the feature subsets with a learning algorithm and usually produce better results than filter approaches for particular problems.

In medical diagnosis scenario, high diagnostic performance is always preferred, even a slight lift in accuracy can make significant difference. Therefore, the wrapper approach is adopted to obtain the better classification performance in this study. Generally, metaheuristics are commonly used for finding the optimal feature subset in wrapper approaches. As a vital member of metaheuristics family, evolutionary computation (EC) has attracted great attention. Many EC based methods in the literature have been proposed to perform feature selection. Raymer et al. [11] suggested using genetic algorithms (GA) to select features. Authors in [12–14] proposed to use binary particle swarm optimization (PSO) for feature selection. Zhang and Sun applied tabu search in feature selection [15]. Compared with above-mentioned EC techniques, grey wolf optimization (GWO) is a new EC technique proposed recently [16]. GWO mimics the social hierarchy and hunting behavior of grey wolves in nature. Due to its excellent search capacity, it has been successfully applied to many real-world problems since its introduction, like optimal reactive power dispatch problem [17], parameter estimation in surface waves [18], static VAR compensator controller design [19], blackout risk prevention in a smart grid [20], capacitated vehicle routing problem [21], nonconvex economic load dispatch problem [22], and so on. However, it should be noted that the initial population of original GWO is generated in a random way. It may result in the lack of diversity for the wolf swarms during the search space. Many studies [23–26] have shown that the quality of the initial population may have a big impact on the global convergence speed and the quality of final solution for the swarm intelligence optimization algorithms, and initial population with good diversity is very helpful to improve the performance of optimization algorithms. Motivated by this core idea, we made the first attempt to use GA to generate a much more appropriate initial population, and then a binary version of GWO was constructed to perform the feature selection task based on the diversified population. On the other hand, to find the most discriminative features in terms of classification accuracies, the choice of an effective and efficient classifier is also of significant importance. In this study, the kernel extreme learning machine (KELM) classifier is adopted to evaluate the fitness value. The KELM is selected due to the fact that it can achieve comparative or better performance with much easier implementation and faster training speed in many classification tasks [27–29].

The main contributions of this paper are summarized as follows:(a)A novel predictive framework based on an improved grey wolf optimization (IGWO) and KELM method is presented.(b)GA is introduced into the IGWO to generate the more suitable initial positions for GWO.(c)The developed framework, IGWO-KELM, is successfully applied to medical diagnosis problems and has achieved superior classification performance to the other competitive counterparts.

The remainder of this paper is organized as follows. Section 2 gives some brief background knowledge of KELM, GWO, and GA. The detailed implementation of the IGWO-KELM method will be explained in Section 3. Section 4 describes the experimental design in detail. The experimental results and discussions of the proposed approach are presented in Section 5. Finally, the conclusions are summarized in Section 6.

2. Background

2.1. Kernel Extreme Learning Machine (KELM)

The traditional back propagation (BP) learning algorithm is a stochastic gradient least mean square algorithm. The gradient of each iteration is greatly affected by the noise interference in the sample. Therefore, it is necessary to use the batch method to average the gradient of multiple samples to get the valuation of the gradient. However, in the case of a large number of training samples, this method is bound to increase the computational complexity of each iteration, and this average effect will ignore the difference between individual training samples, thereby reducing the sensitivity of learning [30].

KELM is an improved algorithm proposed by Guang-Bin Huang, which combines the kernel function into the original extreme learning machine (ELM) [31]. ELM guarantees the network has good generalization performance, greatly improves the learning speed of the forward neural networks, and avoids many of the problems of gradient descent training methods represented by BP neural networks, like ease of being trapped into local optimum, large iterations, and so on. KELM not only has multidominance of the ELM algorithm, but also combines the kernel function, which nonlinearly maps the linear nonseparable mode to the high-dimensional feature space in order to achieve linear separability and further improve the accuracy rate.

ELM is a training algorithm of single hidden layer feed-forward neural networks (SLFNs). The SLFNs model can be presented as follows [32]: where is sample; is the output of neural networks, a class vector in classification; or is hidden layer feature mapping matrix; is hidden layer output layer link weight. In the ELM algorithm,where is a matrix consisting of class flag vectors of the training sample, is unit matrix, and is regularization parameter.

In the case where the hidden layer feature map is unknown, the KELM kernel matrix can be defined as follows [33]:

According to (2) and (3), (1) can be transformed as follows:

If the radial basis function (RBF) is used as kernel function, also known as Gaussian kernel function [34], which can be defined as follows:

therefore, the regularization parameter and the kernel function parameter are parameters that need to be tuned carefully. The configuration of and is an important factor affecting the performance of KELM classifier.

2.2. Grey Wolf Optimization (GWO)

The GWO is a new metaheuristic algorithm proposed by Mirjalili et al. [16], which mimics the social hierarchy and hunting mechanism of grey wolves in nature and is based on three main steps: encircling prey, hunting, and attacking prey. In order to mathematically model the leadership hierarchy of wolves, assume the best solution as alpha, and the second and third best solutions are named as beta and delta, respectively. The rest of the candidate solutions are assumed to be omega. The strict social dominant hierarchy of grey wolves is shown in Figure 1.

Grey wolves encircle prey during the hunt. In order to mathematically simulate the encircling behavior of grey wolves, the following equations are proposed:where indicates the current iteration, and are coefficient vectors, is the position vector of the prey, and is the position vector of a grey wolf. The vectors and are calculated as follows:where is linearly decreased from 2 to 0 over the course of iterations and and are random vectors in the interval of .

The hunt is usually guided by alpha. Beta and delta might also participate in hunting occasionally. In order to mathematically mimic the hunting behavior of grey wolves, the first three best solutions (alpha, beta, and delta) obtained so far are saved and the other search agents (omega) are obliged to update their positions according to (8)–(14). The update of positions for grey wolves is illustrated in Figure 2.

The pseudocode of the GWO algorithm is presented as shown in Pseudocode 1.

Begin
Initialize the parameters popsize, maxiter, ub and lb where
popsize: size of population,
maxiter: maximum number of iterations,
ub: upper bound(s) of the variables,
lb: lower bound(s) of the variables;
Generate the initial positions of grey wolves with ub and lb;
Initialize , , and ;
Calculate the fitness of each grey wolf;
alpha = the grey wolf with the first maximum fitness;
beta = the grey wolf with the second maximum fitness;
delta = the grey wolf with the third maximum fitness;
While
for : popsize
Update the position of the current grey wolf by Eq. (14);
end for
Update , , and ;
Calculate the fitness of all grey wolves;
Update alpha, beta, and delta;
;
end while
Return alpha;
End

2.3. Genetic Algorithm (GA)

The GA was firstly proposed by Holland [35], which is an adaptive optimization search methodology based on analogy to Darwinian natural selection and genetic in biology systems. In GA, a population is composed of a set of candidate solutions called chromosomes. Each chromosome includes several genes with binary values 0 and 1. In this study, GA was used to generate the initial positions for GWO. The steps of generating initial positions of population by GA are described below.(i)Initialization. Chromosomes are randomly generated.(ii)Selection. A roulette choosing method is used to select parent chromosomes.(iii)Crossover. A single point crossover method is used to create offspring chromosomes.(iv)Mutation. Uniform mutation is adopted.(v)Decode. Decode the mutated chromosomes as the initial positions of population.

3. The Proposed IGWO-KELM Framework

This study proposed a new computational framework, IGWO-KELM, for medical diagnosis purpose. IGWO-KELM is comprised of two main phases. In the first stage, IGWO is used to filter out the redundant and irrelevant information by adaptively searching for the best feature combination in the medical data. In the proposed IGWO, GA is firstly used to generate the initial positions of population, and then GWO is utilized to update the current positions of population in the discrete searching space. In the second stage, the effective and efficient KELM classifier is conducted based on the optimal feature subset obtained in the first stage. Figure 3 presents a detailed flowchart of the proposed IGWO-KELM framework.

The IGWO is mainly used to adaptively search the feature space for best feature combination. The best feature combination is the one with maximum classification accuracy and minimum number of selected features. The fitness function used in IGWO to evaluate the selected features is shown as the following equation:where is the accuracy of the classification model, is the length of selected feature subset, is the total number of features in the dataset, and and are two parameters corresponding to the weight of classification accuracy and feature selection quality, and .

A flag vector for feature selection is shown in Figure 4. The vector consisting of a series of binary values of 0 and 1 represents a subset of features, that is, an actual feature vector, which has been normalized [36]. For a problem with dimensions, there are bits in the vector. The th feature is selected if the value of the th bit equals one; otherwise, this feature will not be selected (). The size of a feature subset is the number of bits, whose values are one in the vector. The pseudocode of the IGWO algorithm is presented as shown in pseudocode 2.

Begin
Initialize the parameters popsize, maxiter, dim, pos, and flag where
popsize: size of population,
maxiter: maximum number of iterations,
dim: total number of features,
pos: position of grey wolf,
flag: mark vector of features;
Generate the initial positions of grey wolves using GA;
Initialize , , and ;
for : popsize
for : dim
if
;
else
;
end if
end for
end for
Calculate the fitness of grey wolves with selected features by Eq. (15);
alpha = the grey wolf with the first maximum fitness;
beta = the grey wolf with the second maximum fitness;
delta = the grey wolf with the third maximum fitness;
while
for : popsize
Update the position of the current grey wolf by Eq. (14);
end for
for : popsize
for : dim
if
;
else
;
end if
end for
end for
Update , , and ;
Calculate the fitness of grey wolves with selected features by Eq. (15);
Update alpha, beta, and delta;
;
end while
Return the selected features of alpha as the optimal feature subset;
End

4. Experimental Design

4.1. Data Description

In order to evaluate the proposed IGWO-KELM methodology, two common medical diagnosis problems were investigated, including the Parkinson’s disease diagnosis and breast cancer diagnosis. The datasets of Parkinson and Wisconsin diagnostic breast cancer (WDBC) publicly available from the UCI machine learning data repository were used.

The Parkinson dataset is composed of a range of biomedical voice measurements from 31 people, 23 with Parkinson’s disease (PD). Each column in the table is a particular voice measure, and each row corresponds to one of 195 voice recordings from these individuals. The main aim of the dataset is to discriminate healthy people from those with PD, given the results of various medical tests carried out on a patient. The time since diagnoses ranged from 0 to 28 years, and the ages of the subjects ranged from 46 to 85 years, with a mean age of 65.8. Each subject provides an average of six phonations of the vowel (yielding 195 samples in total), each 36 seconds in length [37]. The description of Parkinson dataset is presented in Table 1. The Parkinson dataset contains 195 cases, including 147 Parkinson’s cases and 48 healthy cases. The distribution of the Parkinson dataset is shown in Figure 5.

The WDBC dataset was created from the University of Wisconsin, Madison, by Dr. Wolberg et al. [38]. The dataset contains 32 attributes (ID, diagnosis, and 30 real-valued input features). Features are computed from a digitized image of a fine needle aspirate (FNA) of a breast mass. They describe the characteristics of the cell nuclei presenting in the image. Interactive image processing techniques and linear-programming-based inductive classifier have been used to build a highly accurate system for diagnosing breast cancer. With an interactive interface, the user initializes active contour models, known as snakes, near the boundaries of a set of cell nuclei. The customized snakes are deformed to the exact shape of the nuclei. This allows for precise automated analysis of nuclear size shape and texture. Ten such features are computed for each nucleus and the mean value largest (or “worst”) value and standard error of each feature are found over the range of isolated cells [39], and they are described as follows.

Descriptions of Features of the WDBC Dataset(a)Radius. The mean of distances from center to points on the perimeter(b)Texture. The standard deviation of grey-scale values(c)Perimeter. The total distance between consecutive snake points(d)Area. The number of pixels on the interior of the snake adds one-half of the pixels on the perimeter(e)Smoothness. The local variation in radius lengths(f)Compactness. Perimeter²/area - 1.0(g)Concavity. The severity of concave portions of the contour(h)Concave Points. The number of concave portions of the contour(i)Symmetry. The length difference between lines perpendicular to the major axis to the nuclear boundary in both directions(j)Fractal Dimension. “Coastline approximation” - 1

The mean value, worst (mean of the three largest values), and standard error of each feature were computed for each image, resulting in a total of thirty features for each case in the dataset. There are 569 samples’ data out of which 357 samples are labeled as benign breast cancer and the remaining as malignant breast cancer patients. The distribution of the WDBC dataset is shown in Figure 6.

4.2. Experimental Setup

The experiments were conducted in the MATLAB platform, which ran on Windows 7 ultimate operating system with Intel® Core™ i3-3217U CPU (1.80 GHz) and 8 GB of RAM. The implementation of KELM by Huang is available at http://www3.ntu.edu.sg/home/egbhuang. The IGWO, GWO, and GA were implemented from scratch.

In this study, the data were scaled into by normalization for the facility of computation. In order to acquire unbiased classification results, the k-fold cross validation (CV) was used [40]. This study took 10-fold CV to test the performance of the proposed algorithm. However, only one time of running the 10-fold CV will result in the inaccurate evaluation. So the 10-fold CV will run ten times.

Regarding the parameter choice of KELM, different penalty parameters and different kernel parameters were taken to find the best classification results. In other words, 11 × 11 = 121 combinations were tried for each method. The final experimental results demonstrate that when is equal to 2⁵ (32) and is equal to (0.5), KELM achieves the best performance. Therefore, and for KELM are set to 32 and 0.5 in this study, respectively. The global and algorithm-specific parameter setting is outlined in Table 2.

4.3. Performance Evaluation

Considering a two-class classifier, formally, each instance is mapped to one element of the set of positive and negative class labels. A classifier is a mapping from instances to predicted classes and produces a discrete class label indicating only the predicted class of the instance. A confusion matrix contains information about actual and predicted classifications done by a classification system. Performance of such systems is commonly evaluated using the data in the matrix as shown in Table 3.

Once the model has been built, it can be applied to a test set to predict the class labels of previously unseen data. It is often useful to measure the performance of the model with test data, because such a measure provides an unbiased estimate of generation errors. In this study, we evaluate the prediction models, utilizing the KELM classifier, based on different evaluation criteria described below.

Accuracy is the proportion of the total number of predictions that were correct. It is determined using

Sensitivity is the proportion of positive instances that were correctly classified, as calculated using

Specificity is the proportion of negative instances that were correctly classified, as calculated using

Precision is the proportion of the predicted positive instances that were correct, as calculated using

The accuracy determined using (16) may not be an adequate performance measure when the number of negative instances is much greater than the number of positive instances. Other performance measures account for this by including sensitivity in literature. For example, Kubat and Matwin [41] proposed the geometric mean (-mean) metric in 1998, as defined using Lewis and Gale [42] proposed the F-measure metric in 1994, as defined using

In (21), has a value from 0 to infinity and is used to control the weight assigned to precision and sensitivity. Any classifier evaluated using (21) will have a measure value of 0, if all positive instances are classified incorrectly. The value of is set to 1 in this study.

5. Experimental Results and Discussions

Comparative experiments were performed between IGWO-KELM and the other two competitive methods, including GWO-KELM and GA-KELM, in order to evaluate the effectiveness of the proposed method for the two disease prediction problems. 10-fold CV was used to estimate the classification results of each approach; the mean values over ten times of 10-fold CV were taken as the final experiment results.

5.1. Parkinson’s Disease Prediction

Table 4 illustrates the detailed classification results of the three methods in terms of the number of selected features, accuracy, sensitivity, specificity, precision, G-mean, and F-measure on the Parkinson dataset. It can be seen in Table 4 that, among the three methods, the IGWO-KELM method performs the best with the least number of selected features, with the highest values of 97.45% accuracy, 98.08% sensitivity, 96.67% specificity, 99.29% precision, 97.37% G-mean, and 98.68% F-measure and with the smallest standard deviation as well. The box plots in Figure 7 graphically depict comparisons among IGWO-KELM versus the other two methods in terms of accuracy, sensitivity, specificity, precision, G-mean, and F-measure. IGWO-KELM displays the greatest performance among the three methods. Specially, for the measurement specificity, as shown in Figure 7(c), the median value obtained from IGWO-SVM is 93.83%, much higher than GA-KELM and GWO-KELM by 88.74% and 91.99%, respectively.

(a)

(b)

(c)

(d)

(e)

(f)

To observe the optimization procedure of the algorithms including GA, GWO, and IGWO, the iteration process was recorded in Figure 8. It can be seen from Figure 8 that the fitness curve of IGWO completely converges after the 17th iteration, while the fitness curves of GWO and GA completely converge after the 30th iteration and the 45th iteration, respectively. It indicates that the proposed IGWO is much more effective than the other two methods and can quickly find the best solution in the search space. Moreover, we can also observe that the fitness value of IGWO is always bigger than that of GWO and GA in the whole iteration course.

The population size and the iteration number are two key factors in swarm intelligence algorithms; thus their suitable values were investigated on the Parkinson dataset. Firstly, to find the best value of the population size, different sizes of population from 4 to 20 with the step of 4 were taken when the number of iterations was fixed to 100. It can be observed from Table 5 that the performance of IGWO-KELM is shown to be the best when the iteration number is equal to 8. Secondly, to find the best value of the iteration number, the size of population was fixed to 8 and different numbers of iterations from 50 to 250 with step of 50 were tried. As shown in Table 6, IGWO-KELM achieves the best performance when the iteration number is equal to 100. Therefore, to obtain the best performance of the proposed method for the Parkinson dataset, the size of population and the number of iterations were set to 8 and 100, respectively, in this study.

Figure 9 shows the selected frequency of each feature of the Parkinson dataset in the process of the feature selection by three methods, including GA-KELM, GWO-KELM, and IGWO-KELM. It can be found from Figure 9 that the frequencies of the 8th feature, the 9th feature, the 11th feature, the 13th feature, the 14th feature, and the 20th feature selected by IGWO-KELM are higher than the counterparts selected by GA-KELM and GWO-KELM, and the frequency of these features selected by IGWO-KELM is more than five. It indicates that the 8th feature, the 9th feature, the 11th feature, the 13th feature, the 14th feature, and the 20th feature are much more important features than others in the Parkinson dataset.

Table 7 presents the average selected times of features of the Parkinson dataset, ranging from 1 to 10. Firstly, for IGWO-KELM, the average selected times of the 1st feature, the 4th feature, the 5th feature, the 8th feature, the 9th feature, the 10th feature, the 11th feature, the 13th feature, the 14th feature, the 16th feature, the 17th feature, the 18th feature, the 19th feature, the 20th feature, and the 22nd feature are more than five times. Secondly, for GWO-KELM, the average selected times of the 1st feature, the 4th feature, the 5th feature, the 6th feature, the 7th feature, the 10th feature, the 15th feature, the 16th feature, the 17th feature, the 18th feature, the 19th feature, the 20th feature, and the 22nd feature are more than five times. Thirdly, for GA-KELM, the average selected times of the 1st feature, the 17th feature, the 18th feature, the 19th feature, and the 22nd feature are more than five times. It is interesting to find that the average selected times of five features including the 1st feature (MDVP: Fo), the 17th feature (RPDE), the 18th feature (D2), the 19th feature (DFA), and the 22nd feature (PPE) are all more than five times for IGWO-KELM, GWO-KELM, and GA-KELM. It indicates that the three methods are highly consistent to pick out the most important features for the Parkinson dataset. It also suggests that these features should be paid more attention to in the decision-making process.

5.2. Brest Cancer Prediction

Table 8 presents the detailed classification results of the three methods in terms of the number of selected features, accuracy, sensitivity, specificity, precision, G-mean, and F-measure on the WDBC dataset. From the table, it can be seen that the IGWO-KELM method achieves the highest performance among the three methods with results of 95.78% accuracy, 94.88% Sensitivity, 96.75% Specificity, 95.24% Precision, 95.81% G-mean, and 95.06% F-measure. The boxplots are drawn to exhibit the general values of the accuracy, sensitivity, specificity, precision, G-mean, and F-measure and they are shown in Figure 10. As expected, compared with the other two methods, IGWO-KELM yields consistent increase of all performance measurements. For example, for the measurement sensitivity, as can be observed in Figure 10(b), the median value obtained from IGWO-SVM is 94.62%, higher than GA-KELM and GWO-KELM by 92.44% and 93.52%, respectively.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 11 shows the optimization procedure of the algorithms including GA, GWO, and IGWO. It can be observed from Figure 11 that the fitness curve of IGWO completely converges after the 24th iteration, while the fitness curve of GWO and GA just started to converge from the 28th iteration and the 43rd iteration, respectively. Moreover, it can also be observed that the fitness value of IGWO is always bigger than that of GWO or GA in the whole iteration course. It indicates that IGWO not only converges more quickly, but also obtains better solution quality than GA and GWO. The main reason may lie in that the GA initialization helps GWO to search more effectively in search space; thus outperforming both GA and GWO in converging to a better result.

As done for the Parkinson dataset, the population size and the iteration number were also investigated on the WDBC dataset. Firstly, to find the best value of the population size, different sizes of population from 4 to 20 with the step of 4 were taken when the number of iterations was fixed to 100. It can be observed from Table 9 that the performance of IGWO-KELM is shown to be the best when the iteration number is equal to 8. Secondly, to find the best value of the iteration number, the size of population was fixed to 8 and different numbers of iterations from 50 to 250 with step of 50 were tried. As shown in Table 10, IGWO-KELM achieves the best performance when the iteration number is equal to 100. Therefore, to obtain the best performance of the proposed method for the WDBC dataset, the size of population, and the number of iterations were set to 8 and 100, respectively, in this study.

Figure 12 shows the selected frequency of each feature of the WDBC dataset in the course of the feature selection by GA-KELM, GWO-KELM, and IGWO-KELM. It can be observed from Figure 12 that the frequencies of the 1st feature, the 2nd feature, the 3rd feature, the 5th feature, the 6th feature, the 8th feature, the 9th feature, the 12th feature, the 14th feature, the 16th feature, the 18th feature, the 19th feature, the 20th feature, and the 26th feature selected by IGWO-KELM are higher than the counterparts selected by GA-KELM and GWO-KELM. It indicates that these chosen features are important features in the WDBC dataset; they should be paid more attention to when the doctors make a decision.

Table 11 presents the average selected times of features of the WDBC dataset, ranging between 1 and 10. On the one hand, to IGWO-KELM and GA-KELM, the average selected times of the 21st feature, the 22nd feature, and the 25th feature are more than five times. On the other hand, to GWO-KELM, the average selected times of the 21st feature, the 22nd feature, the 23rd feature, the 24th feature, and the 25th feature are more than five times. Therefore, it can be deduced that the 21st feature, the 22nd feature, and the 25th feature are important features in the WDBC dataset, since they are selected consistently by the three methods in a high frequency.

6. Conclusions

In this paper, an IGWO-KELM methodology is described in detail. The proposed framework consists of two main stages which are feature selection and classification, respectively. Firstly, an improved grey wolf optimization, IGWO, was proposed for selecting the most informative features in the specific medical data. Secondly, the effective KELM classifier was used to perform the prediction based on the representative feature subset obtained in the first stage. The proposed method is compared against well-known feature selection methods including GA and GWO on the two disease diagnosis problems using a set of criteria to assess different aspects of the proposed framework. The simulation results have demonstrated that the proposed IGWO method not only adaptively converges more quickly, producing much better solution quality, but also gains less number of selected features, achieving high classification performance. In future works, we will apply the proposed methodology to more practical problems and plan to implement our method in a parallel way with the aid of high performance tools.

Competing Interests

The authors declare that there is no conflict of interests regarding the publication of this article.

Acknowledgments

This research is supported by the National Natural Science Foundation of China (NSFC) under Grant nos. 61303113, 61572367, and 61571444. This research is also funded by the Science and Technology Plan Project of Wenzhou of China under Grant no. G20140048, Zhejiang Provincial Natural Science Foundation of China under Grant nos. LY17F020012, LY14F020035, and LQ13G010007, and Guangdong Natural Science Foundation under Grant no. 2016A030310072.

References

R. Armañanzas, C. Bielza, K. R. Chaudhuri, P. Martinez-Martin, and P. Larrañaga, “Unveiling relevant non-motor Parkinson's disease severity symptoms using a machine learning approach,” Artificial Intelligence in Medicine, vol. 58, no. 3, pp. 195–202, 2013.
View at: Publisher Site | Google Scholar
H.-L. Chen, G. Wang, C. Ma, Z.-N. Cai, W.-B. Liu, and S.-J. Wang, “An efficient hybrid kernel extreme learning machine approach for early diagnosis of Parkinson's disease,” Neurocomputing, vol. 184, no. 4745, pp. 131–144, 2016.
View at: Publisher Site | Google Scholar
N. P. Pérez, M. A. Guevara López, A. Silva, and I. Ramos, “Improving the Mann-Whitney statistical test for feature selection: an approach in breast cancer diagnosis on mammography,” Artificial Intelligence in Medicine, vol. 63, no. 1, pp. 19–31, 2015.
View at: Publisher Site | Google Scholar
H.-L. Chen, B. Yang, J. Liu, and D.-Y. Liu, “A support vector machine classifier with rough set-based feature selection for breast cancer diagnosis,” Expert Systems with Applications, vol. 38, no. 7, pp. 9014–9022, 2011.
View at: Publisher Site | Google Scholar
N. C. Long, P. Meesad, and H. Unger, “A highly accurate firefly based algorithm for heart disease prediction,” Expert Systems with Applications, vol. 42, no. 21, Article ID 10101, pp. 8221–8231, 2015.
View at: Publisher Site | Google Scholar
J. Nahar, T. Imam, K. S. Tickle, and Y.-P. P. Chen, “Computational intelligence for heart disease diagnosis: a medical knowledge driven approach,” Expert Systems with Applications, vol. 40, no. 1, pp. 96–104, 2013.
View at: Publisher Site | Google Scholar
J. Escudero, E. Ifeachor, J. P. Zajicek, C. Green, J. Shearer, and S. Pearson, “Machine learning-based method for personalized and cost-effective detection of Alzheimer's disease,” IEEE Transactions on Biomedical Engineering, vol. 60, no. 1, pp. 164–168, 2013.
View at: Publisher Site | Google Scholar
E. Challis, P. Hurley, L. Serra, M. Bozzali, S. Oliver, and M. Cercignani, “Gaussian process classification of Alzheimer's disease and mild cognitive impairment from resting-state fMRI,” NeuroImage, vol. 112, pp. 232–243, 2015.
View at: Publisher Site | Google Scholar
U. Alon, N. Barka, D. A. Notterman et al., “Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays,” Proceedings of the National Academy of Sciences of the United States of America, vol. 96, no. 12, pp. 6745–6750, 1999.
View at: Publisher Site | Google Scholar
M. Dash and H. Liu, “Feature selection for classification,” Intelligent Data Analysis, vol. 1, no. 3, pp. 131–156, 1997.
View at: Publisher Site | Google Scholar
M. L. Raymer, W. F. Punch, E. D. Goodman, L. A. Kuhn, and A. K. Jain, “Dimensionality reduction using genetic algorithms,” IEEE Transactions on Evolutionary Computation, vol. 4, no. 2, pp. 164–171, 2000.
View at: Publisher Site | Google Scholar
K. Tanaka, T. Kurita, and T. Kawabe, “Selection of import vectors via binary particle swarm optimization and cross-validation for kernel logistic regression,” in Proceedings of the International Joint Conference on Neural Networks (IJCNN '07), pp. 1037–1042, IEEE, Orlando, Fla, USA, August 2007.
View at: Publisher Site | Google Scholar
X. Wang, J. Yang, X. Teng, W. Xia, and R. Jensen, “Feature selection based on rough sets and particle swarm optimization,” Pattern Recognition Letters, vol. 28, no. 4, pp. 459–471, 2007.
View at: Publisher Site | Google Scholar
H. L. Chen, B. Yang, S. J. Wang et al., “Towards an optimal support vector machine classifier using a parallel particle swarm optimization strategy,” Applied Mathematics and Computation, vol. 239, pp. 180–197, 2014.
View at: Publisher Site | Google Scholar | MathSciNet
H. Zhang and G. Sun, “Feature selection using tabu search method,” Pattern Recognition, vol. 35, no. 3, pp. 701–711, 2002.
View at: Publisher Site | Google Scholar | Zentralblatt MATH
S. Mirjalili, S. M. Mirjalili, and A. Lewis, “Grey Wolf Optimizer,” Advances in Engineering Software, vol. 69, pp. 46–61, 2014.
View at: Publisher Site | Google Scholar
M. H. Sulaiman, Z. Mustaffa, M. R. Mohamed, and O. Aliman, “Using the gray wolf optimizer for solving optimal reactive power dispatch problem,” Applied Soft Computing Journal, vol. 32, pp. 286–292, 2015.
View at: Publisher Site | Google Scholar
X. Song, L. Tang, S. Zhao et al., “Grey Wolf Optimizer for parameter estimation in surface waves,” Soil Dynamics and Earthquake Engineering, vol. 75, pp. 147–157, 2015.
View at: Publisher Site | Google Scholar
A.-A. A. Mohamed, A. A. M. El-Gaafary, Y. S. Mohamed, and A. M. Hemeida, “Design static VAR compensator controller using artificial neural network optimized by modify Grey Wolf Optimization,” in Proceedings of the International Joint Conference on Neural Networks (IJCNN '15), Anchorage, Ala, USA, July 2015.
View at: Publisher Site | Google Scholar
B. Mahdad and K. Srairi, “Blackout risk prevention in a smart grid based flexible optimal strategy using Grey Wolf-pattern search algorithms,” Energy Conversion and Management, vol. 98, pp. 411–429, 2015.
View at: Publisher Site | Google Scholar
L. Korayem, M. Khorsid, and S. S. Kassem, “Using grey Wolf algorithm to solve the capacitated vehicle routing problem,” in Proceedings of the 3rd International Conference on Manufacturing, Optimization, Industrial and Material Engineering (MOIME '15), Institute of Physics Publishing, Bali, Indonesia, March 2015.
View at: Google Scholar
V. K. Kamboj, S. K. Bath, and J. S. Dhillon, “Solution of non-convex economic load dispatch problem using Grey Wolf Optimizer,” Neural Computing and Applications, vol. 27, no. 5, pp. 1301–1316, 2016.
View at: Publisher Site | Google Scholar
R. L. Haupt and S. E. Haupt, 2. The Binary Genetic Algorithm, John Wiley & Sons, New York, NY, USA, 2004.
R. S. Rahnamayan, H. R. Tizhoosh, and M. M. A. Salama, “Opposition-based differential evolution,” IEEE Transactions on Evolutionary Computation, vol. 12, no. 1, pp. 64–79, 2008.
View at: Publisher Site | Google Scholar
S. Rahnamayan, H. R. Tizhoosh, and M. M. A. Salama, “A novel population initialization method for accelerating evolutionary algorithms,” Computers & Mathematics with Applications, vol. 53, no. 10, pp. 1605–1614, 2007.
View at: Publisher Site | Google Scholar
W.-F. Gao, L.-L. Huang, J. Wang, S.-Y. Liu, and C.-D. Qin, “Enhanced artificial bee colony algorithm through differential evolution,” Applied Soft Computing Journal, vol. 48, pp. 137–150, 2016.
View at: Publisher Site | Google Scholar
M. Pal, A. E. Maxwell, and T. A. Warner, “Kernel-based extreme learning machine for remote-sensing image classification,” Remote Sensing Letters, vol. 4, no. 9, pp. 853–862, 2013.
View at: Publisher Site | Google Scholar
T. Liu, L. Hu, C. Ma, Z.-Y. Wang, and H.-L. Chen, “A fast approach for detection of erythemato-squamous diseases based on extreme learning machine with maximum relevance minimum redundancy feature selection,” International Journal of Systems Science, vol. 46, no. 5, pp. 919–931, 2015.
View at: Publisher Site | Google Scholar
D. Zhao, C. Huang, Y. Wei, F. Yu, M. Wang, and H. Chen, “An effective computational model for bankruptcy prediction using kernel extreme learning machine approach,” Computational Economics, pp. 1–17, 2016.
View at: Publisher Site | Google Scholar
D. S. Huang, Systematic Theory of Neural Networks for Pattern Recognition, Publishing House of Electronic Industry of China, 1996 (Chinese).
G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, “Extreme learning machine: theory and applications,” Neurocomputing, vol. 70, no. 1–3, pp. 489–501, 2006.
View at: Publisher Site | Google Scholar
G.-B. Huang, H. Zhou, X. Ding, and R. Zhang, “Extreme learning machine for regression and multiclass classification,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 42, no. 2, pp. 513–529, 2012.
View at: Publisher Site | Google Scholar
G.-B. Huang, D. H. Wang, and Y. Lan, “Extreme learning machines: a survey,” International Journal of Machine Learning & Cybernetics, vol. 2, no. 2, pp. 107–122, 2011.
View at: Publisher Site | Google Scholar
L. Duan, S. Dong, S. Cui et al., “Extreme learning machine with gaussian kernel based relevance feedback scheme for image retrieval,” in Proceedings of ELM-2015 Volume 1: Theory, Algorithms and Applications (I), vol. 6 of Proceedings in Adaptation, Learning and Optimization, pp. 397–408, Springer, Berlin, Germany, 2016.
View at: Publisher Site | Google Scholar
J. H. Holland, “Genetic algorithms,” Scientific American, vol. 267, no. 1, pp. 66–72, 1992.
View at: Publisher Site | Google Scholar
D.-S. Huang and H.-J. Yu, “Normalized feature vectors: a novel alignment-free sequence comparison method based on the numbers of adjacent amino acids,” IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 10, no. 2, pp. 457–467, 2013.
View at: Publisher Site | Google Scholar
M. A. Little, P. E. McSharry, E. J. Hunter, J. Spielman, and L. O. Ramig, “Suitability of dysphonia measurements for telemonitoring of Parkinson's disease,” IEEE Transactions on Biomedical Engineering, vol. 56, no. 4, pp. 1015–1022, 2009.
View at: Publisher Site | Google Scholar
W. H. Wolberg, W. N. Street, and O. L. Mangasarian, “Machine learning techniques to diagnose breast cancer from image-processed nuclear features of fine needle aspirates,” Cancer Letters, vol. 77, no. 2-3, pp. 163–171, 1994.
View at: Publisher Site | Google Scholar
W. N. Street, W. H. Wolberg, and O. L. Mangasarian, “Nuclear feature extraction for breast tumor diagnosis,” in Proceedings of the IS&T/SPIE's Symposium on Electronic Imaging: Science and Technology, International Society for Optics and Photonics, San Jose, Calif, USA, January 1993.
View at: Google Scholar
R. Kohavi, “A study of cross-validation and bootstrap for accuracy estimation and model selection,” in Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI '95), C. S. Mellish, Ed., pp. 1137–1143, Morgan Kaufmann, Montreal, Canada, August 1995.
View at: Google Scholar
M. Kubat and S. Matwin, “Addressing the curse of imbalanced training sets: one-sided selection,” in Proceedings of the 14th International Conference on Machine Learning, 2000.
View at: Google Scholar
D. D. Lewis and W. A. Gale, A Sequential Algorithm for Training Text Classifiers, Springer, London, UK, 1994.

Copyright

Copyright © 2017 Qiang Li et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

6804

Downloads

3545

Citations