FANDOM


Fisher's linear discriminant [1]is a quantity which is derived to solve binary supervised classification problems (see Classifiers_and_Discriminants).

It is assumed that every object or event  i

belongs to one of two classes  y_i \in \{-1,1\} 

, and may be represented by a vector of measurements referred to as a feature vector  x_i . Since the measurements have been chosen to distinguish between the classes, the distribution of each class in this feature space should be different. For simplicity the distributions of each class  k

have been summarised by their means  \mu_k 
and covariances  \Sigma_k 

.

Fisher's linear discriminant is a linear combination of the feature vectors  x_i

which maximises the separation of the two classes in one quantity. In general, two classes will be further apart when their means are far apart, and there is little variability within each class. Fisher summarised this with the measure of separation given by

Separation =  \frac{Between \; class \; variance}{Within \; class \; variance}


For this example, consider the linear combination of measurements

 \sum_i n_i x_i


It can be shown that the mean of each class in this new variable will be  n^T \mu_1

and  n^T \mu_2 

, and the variances will be  n^T \Sigma_1 n

and  n^T \Sigma_2 n 

. The Fisher separation will then be

Separation =  \frac{n^T (\mu_2-\mu_1)}{n^T(\Sigma_1 + \Sigma_2)n}


which should be maximised with respect to  n . It can be shown that the optimal  n

is given by

 n = (\Sigma_1+\Sigma_2)^{-1}(\mu_2-\mu_1)


and so Fisher's linear discriminant is given by  n^T x . A set of measurements can then be classified by a simple comparison of the Fisher discriminant with a threshold.

It is frequently stated that Fisher's discriminant assumes each class is normally distributed. This is probably due to confusion with the related Linear Discriminant Analysis (LDA), which is in fact identical to Fisher's discriminant when the covariance of both classes are equal.


[1] ``The use of multiple measurements in taxonomic problems," R.A.Fisher, Annals of Eugenics, Vol.7, pp.179--188, 1936.

Ad blocker interference detected!


Wikia is a free-to-use site that makes money from advertising. We have a modified experience for viewers using ad blockers

Wikia is not accessible if you’ve made further modifications. Remove the custom ad blocker rule(s) and the page will load as expected.