Feature Scaling for Kernel Fisher Discriminant Analysis

Using Leave-One-Out Cross Validation

Liefeng Bo, Ling Wang, and Licheng Jiao

Xidian University

Abstract: Kernel fisher discriminant analysis (KFD) [2] is a successful approach to classification. It is well known that the key challenge in KFD lies in the selection of free parameters such as kernel parameters and regularization parameters. Here we focus on the feature-scaling kernel where each feature individually associates with a scaling factor. A novel algorithm, named FS-KFD [1], is developed to tune the scaling factors and regularization parameters for the feature-scaling kernel. The proposed algorithm is based on optimizing the smooth leave-one-out error via a gradient-descent method and has been demonstrated to be computationally feasible. FS-KFD is motivated by the following two fundamental facts: the leave-one-out error of KFD can be expressed in closed form and the step function can be approximated by a sigmoid function. Empirical comparisons on artificial and benchmark data sets suggest that FS-KFD improves KFD in terms of classification accuracy.


  1. Liefeng Bo, Ling Wang, and Licheng Jiao, Feature Scaling for Kernel Fisher Discriminant Analysis Using Leave-one-out Cross Validation, Neural Computation (NECO), vol. 18(4), pp. 961-978, 2006. [PDF] [BIB]

  2. S. Mika, G. Ratsch, and J. Weston, Fisher Discriminant Analysis with Kernels. In Proceedings of the IEEE Workshop on Neural Networks for Signal Processing, pp. 41-48, 1999.

Matlab Source Code

Description: FS-KFDA is a package for implementing feature scaling for kernel fisher discriminant analysis.

Requirement: Matlab 7.01.

Download: [code]. This package is free for academic usage. You can run it at your own risk.