[pymvpa] extracting individual sample predictions from CV

[Francisco Pereira]

Hi Brian,

I don't know about the PyMVPA particulars ...

On Thu, Jun 2, 2011 at 11:23 AM, Brian Murphy <brian.murphy at unitn.it> wrote:
>
> And is there a standard way of getting the measure of confidence of the
> classifier in each prediction (binary, in my current case). This must be
> classifier specific,

... but it is indeed classifier specific (though there are some
nonparametric possibilities).

>  but I guess that GNB should give a probability for each  of its classifications.

In a 2 class (A/B) prob, you can indeed get P(A|x) or P(B|x). However,
GNB estimates of those posterior class probabilities are really bad,
in that they tend to be very extreme in one direction or another (see
Domingos 1997).

You can get these from logistic regression as well, using the weights
over features and passing them through a sigmoid function (see
http://www.cs.cmu.edu/~tom/mlbook/NBayesLogReg.pdf ). You can also get
them from an SVM, libsvm has an option that turns the outputs into
probability estimates (not sure how well it works in the nonlinear
case).

Francisco