Intelligent Database Systems Lab
國立雲林科技大學
National Yunlin University of Science and Technology
1
GMDH-based feature ranking andselection for improvedclassification of medical data
Advisor : Dr. Hsu
Presenter : Yu-San Hsieh
Author : R.E. Abdel-Aal
2005. BI.456-468
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
2
Motivation
Objective
Method
Material
Results
Conclusions
Outline
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
3
Motivation
Accuracy is very important in classifiers usedfor medical application.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
4
Objective
Improved classification performance ofmedical data.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
5
Method
First stage – ranked feature
─GMDH algorithm
z1
Zm(m-1)/2
1. representation
2.Selection and stopping
x1
x2
x3
x4
y
An increasing rmin：model becoming complex,
1.Overfitting the estimation data
2.Performing poorly on the new selection data.
Iteration
Square error
r12
rm(m-1)2
rmin
r22
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
6
Method
First stage – ranked feature
─AIM abductive network
2.Selection and stopping
1.repesentation
1.repesentation
First stage – ranked feature
─AIM abductive network
2.Selection and stopping
Avoid overfitting
Using CPM control
1.CPM>1,simpler model that are less accurate but generalize.
2.CPM<1,complex model, overfit training data and decrease actual prediction performance.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
7
Method
Second stage – selected feature
─Selected k, performance on an evaluation dataset wouldfirst improve and starts to deteriorate due to the modeloverfitting the training data.
─A compact m-feature subset can be obtained by takingthe first m features starting from top of the ranking list.Ex: ranking list{2,6,7,8,1,5,3,4,9}, selected 6-features is{2,6,7,8,1,5}.
─The optimum subset of features is determined byrepeatedly forming subset of k features, starting from thetop of the ranking list.Ex: ranking list{2,6,7,8,1,5,3,4,9},{2,6,7,8,1,5},{6,7,8,1,5,3}…中選出最佳的subset
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
8
Material
Two standard medical diagnosis datasets fromthe UCI Machine Learning Repository wereused for this study.
─Wisconsin breast cancer dataset
─Cleveland heart disease dataset
70%
30%
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
9
Results
The breast cancer data
─Ranking for the feature set{2,6,7,8,1,5,3,4,9}
7
5
9
Feature selected
Feature ranked
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
10
Results
Rough set data analysis of dataset
Overfitting
Overfitting
3%
3%
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
11
Results
Standard error↓
Standard error↓
AUC↑
3%
3%
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
12
Results
The heart disease data
─Ranking for the feature set{13,12,9,3,2,10,8,4,5,11,1,7,6}
Feature selected
Feature ranked
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
13
Results
3%
6%
Overfitting
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
14
Results
AUC↑
AUC↑
Requires less than half the number of input features
Models using the reduced feature set will be more efficient.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
15
Conclusions
Improved implementation and performanceof classifiers for medical screening anddiagnosis.
Feature reduction is particularly useful withhigh-dimensional data characterized by alarge number of feature and a relatively fewtraining example.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
16
My opinion
Advantage: Preprocess
Disadvantage:
Apply：Clustering, Association Rule……