[pymvpa] extracting individual sample predictions from CV
Francisco Pereira
francisco.pereira at gmail.com
Thu Jun 2 19:45:16 UTC 2011
Hi Brian,
I don't know about the PyMVPA particulars ...
On Thu, Jun 2, 2011 at 11:23 AM, Brian Murphy <brian.murphy at unitn.it> wrote:
>
> And is there a standard way of getting the measure of confidence of the
> classifier in each prediction (binary, in my current case). This must be
> classifier specific,
... but it is indeed classifier specific (though there are some
nonparametric possibilities).
> but I guess that GNB should give a probability for each of its classifications.
In a 2 class (A/B) prob, you can indeed get P(A|x) or P(B|x). However,
GNB estimates of those posterior class probabilities are really bad,
in that they tend to be very extreme in one direction or another (see
Domingos 1997).
You can get these from logistic regression as well, using the weights
over features and passing them through a sigmoid function (see
http://www.cs.cmu.edu/~tom/mlbook/NBayesLogReg.pdf ). You can also get
them from an SVM, libsvm has an option that turns the outputs into
probability estimates (not sure how well it works in the nonlinear
case).
Francisco
More information about the Pkg-ExpPsy-PyMVPA
mailing list