Classification: Cushing’s syndrome data
Data on 27 patients with Cushing’s Syndrome – a hypertensive disorder associated with over-secretion of cortisol by the adrenal gland. The observations are urinary excretion rates of two steroid metabolites.
Tetrahydrocortisone – urinary excretion rate (mg/24hr) of Tetrahydrocortisone
Pregnanetriol – urinary excretion rate (mg/24hr) of Pregnanetriol
Type – underlying type of syndrome, coded a (adenoma) , b (bilateral hyperplasia), c (carcinoma) or u for unknown (6 patients)
Linear Discriminant Analysis (LDA)
## Call:
## lda(tp ~ ., data = Cflog)
##
## Prior probabilities of groups:
## a b c
## 0.2857143 0.4761905 0.2380952
##
## Group means:
## lTetrahydrocortisone lPregnanetriol
## a 1.043285 -0.6034147
## b 2.007256 -0.2060408
## c 2.709728 1.5998004
##
## Coefficients of linear discriminants:
## LD1 LD2
## lTetrahydrocortisone 1.7511836 -0.9907670
## lPregnanetriol 0.2341177 0.7874075
##
## Proportion of trace:
## LD1 LD2
## 0.9299 0.0701##
##
## training set##
## tp a b c
## a 4 2 0
## b 2 7 1
## c 0 1 4## Misclassification error rate: 0.2857##
##
## leave-one-out CV## Misclassification error rate: 0.381
Quadratic Discriminant Analysis (QDA)
## Call:
## qda(tp ~ ., data = Cflog)
##
## Prior probabilities of groups:
## a b c
## 0.2857143 0.4761905 0.2380952
##
## Group means:
## lTetrahydrocortisone lPregnanetriol
## a 1.043285 -0.6034147
## b 2.007256 -0.2060408
## c 2.709728 1.5998004##
##
## training set##
## tp a b c
## a 6 0 0
## b 0 9 1
## c 0 1 4## Misclassification error rate: 0.0952##
##
## leave-one-out CV##
## tp a b c
## a 5 0 1
## b 1 8 1
## c 0 2 3## Misclassification error rate: 0.2381
Logistic Regression
## # weights: 12 (6 variable)
## initial value 23.070858
## iter 10 value 6.623970
## iter 20 value 6.214841
## iter 30 value 6.182968
## iter 40 value 6.172650
## iter 50 value 6.167699
## iter 60 value 6.162723
## iter 70 value 6.156685
## iter 80 value 6.155298
## iter 90 value 6.153807
## iter 100 value 6.152597
## final value 6.152597
## stopped after 100 iterations## Call:
## multinom(formula = tp ~ lTetrahydrocortisone + lPregnanetriol,
## data = Cflog)
##
## Coefficients:
## (Intercept) lTetrahydrocortisone lPregnanetriol
## b -19.09536 13.70353 -0.2491835
## c -27.95158 15.58629 3.3210951
##
## Residual Deviance: 12.30519
## AIC: 24.30519##
##
## training set##
## tp a b c
## a 4 2 0
## b 2 7 1
## c 0 1 4## Misclassification error rate: 0.2857
Support Vector Machine (SVM)
Support Vector Machine (SVM), linear basis
##
## Parameter tuning of 'svm':
##
## - sampling method: 10-fold cross validation
##
## - best parameters:
## cost
## 0.6
##
## - best performance: 0.2833333
##
## - Detailed performance results:
## cost error dispersion
## 1 0.1 0.5833333 0.3263150
## 2 0.6 0.2833333 0.2490724
## 3 1.1 0.2833333 0.2490724
## 4 1.6 0.3333333 0.2357023##
## Call:
## best.svm(x = tp ~ ., data = Cflog, cost = seq(0.1, 2, 0.5), kernel = "linear")
##
##
## Parameters:
## SVM-Type: C-classification
## SVM-Kernel: linear
## cost: 0.6
##
## Number of Support Vectors: 17
##
## ( 5 8 4 )
##
##
## Number of Classes: 3
##
## Levels:
## a b c##
## tp a b c
## a 5 1 0
## b 1 8 1
## c 0 0 5## Misclassification error rate: 0.1429
Support Vector Machine (SVM), quadratic basis
##
## Parameter tuning of 'svm':
##
## - sampling method: 10-fold cross validation
##
## - best parameters:
## degree cost
## 2 1.01
##
## - best performance: 0.4166667
##
## - Detailed performance results:
## degree cost error dispersion
## 1 2 0.01 0.5166667 0.2415229
## 2 2 0.51 0.4666667 0.2918650
## 3 2 1.01 0.4166667 0.3263150
## 4 2 1.51 0.4166667 0.3263150
## 5 2 2.01 0.4166667 0.3263150
## 6 2 2.51 0.4666667 0.2918650##
## Call:
## best.svm(x = tp ~ ., data = Cflog, degree = 2, cost = seq(0.01, 3,
## 0.5), kernel = "polynomial")
##
##
## Parameters:
## SVM-Type: C-classification
## SVM-Kernel: polynomial
## cost: 1.01
## degree: 2
## coef.0: 0
##
## Number of Support Vectors: 18
##
## ( 6 7 5 )
##
##
## Number of Classes: 3
##
## Levels:
## a b c##
## tp a b c
## a 2 2 2
## b 1 9 0
## c 2 2 1## Misclassification error rate: 0.4286
Support Vector Machine (SVM), radial basis
##
## Parameter tuning of 'svm':
##
## - sampling method: 10-fold cross validation
##
## - best parameters:
## cost
## 2
##
## - best performance: 0.2333333
##
## - Detailed performance results:
## cost error dispersion
## 1 1 0.3333333 0.2357023
## 2 2 0.2333333 0.2509242
## 3 3 0.2833333 0.2490724##
## Call:
## best.svm(x = tp ~ ., data = Cflog, cost = seq(1, 3, 1), kernel = "radial")
##
##
## Parameters:
## SVM-Type: C-classification
## SVM-Kernel: radial
## cost: 2
##
## Number of Support Vectors: 16
##
## ( 6 5 5 )
##
##
## Number of Classes: 3
##
## Levels:
## a b c##
## tp a b c
## a 6 0 0
## b 0 9 1
## c 0 1 4## Misclassification error rate: 0.0952
k-NN
##
##
## k= 1
## training set
## tp a b c
## a 6 0 0
## b 0 10 0
## c 0 0 5
## Misclassification error rate: 0
##
##
## leave-one-out CV
## tp a b c
## a 4 1 1
## b 1 8 1
## c 0 2 3
## Misclassification error rate: 0.2857##
##
## k= 3
## training set
## tp a b c
## a 6 0 0
## b 0 10 0
## c 0 0 5
## Misclassification error rate: 0
##
##
## leave-one-out CV
## tp a b c
## a 2 3 1
## b 1 8 1
## c 0 2 3
## Misclassification error rate: 0.381##
##
## k= 5
## training set
## tp a b c
## a 4 2 0
## b 0 9 1
## c 0 1 4
## Misclassification error rate: 0.1905
##
##
## leave-one-out CV
## tp a b c
## a 1 4 1
## b 0 9 1
## c 0 1 4
## Misclassification error rate: 0.3333##
##
## k= 7
## training set
## tp a b c
## a 2 4 0
## b 0 9 1
## c 0 1 4
## Misclassification error rate: 0.2857
##
##
## leave-one-out CV
## tp a b c
## a 0 5 1
## b 1 8 1
## c 0 4 1
## Misclassification error rate: 0.5714
Neural Network (1 hidden layer with 3 hidden
units)
##
## tp 1 2 3
## a 6 0 0
## b 0 9 1
## c 0 0 5## Misclassification error rate: 0.0476
Neural Network (2 hidden layers with respectively 3 and
2 hidden units)
##
## tp 1 2 3
## a 6 0 0
## b 0 10 0
## c 0 0 5## Misclassification error rate: 0
Classification Trees
Impurity measures for binary classification
misclassification \(\gamma=1-\max(p,1-p)=\min(p,1-p)\)
entropy/deviance \(\gamma=-p\ln p -(1-p)\ln(1-p)\)
Gini \(\gamma=1-(p^2+(1-p)^2)=2p(1-p)\)
Classification Tree
## node), split, n, deviance, yval, (yprob)
## * denotes terminal node
##
## 1) root 21 44.220 b ( 0.2857 0.4762 0.2381 )
## 2) Tetrahydrocortisone < 4.9 8 8.997 a ( 0.7500 0.2500 0.0000 ) *
## 3) Tetrahydrocortisone > 4.9 13 17.320 b ( 0.0000 0.6154 0.3846 )
## 6) Pregnanetriol < 2.05 7 0.000 b ( 0.0000 1.0000 0.0000 ) *
## 7) Pregnanetriol > 2.05 6 5.407 c ( 0.0000 0.1667 0.8333 ) *##
## Classification tree:
## tree(formula = tp ~ Tetrahydrocortisone + Pregnanetriol, data = Cf)
## Number of terminal nodes: 3
## Residual mean deviance: 0.8002 = 14.4 / 18
## Misclassification error rate: 0.1429 = 3 / 21##
## tp a b c
## a 6 0 0
## b 2 7 1
## c 0 0 5## Misclassification error rate: 0.1429
