[pymvpa] How to evaluate the goodness of classification for an unlabelled example?
debian at onerussian.com
Fri Jan 18 14:23:40 UTC 2013
On Fri, 18 Jan 2013, Roberto Guidotti wrote:
> Dear all,
> I have a question that do not strictly concern to PyMVPA strictly.
> I trained a classifier to discriminate two classes (e.g. bananas and
> apples), using SVM, cross-validation etc. then I would like to try it with
> some "unlabelled" fruits, could be, bananas and apples but also melon,
> lemon, strawberries. If I try to classify a melon, the label assigned by
> the classifier could be banana. How can I establish a probability level
> for this fruit? I mean, if I use SVM distance from the hyperplane, the
> melon could be distant from bananas and further from apples (hyperspaces)
> and thus in my opinion this is not a good index for that. I would like to
> have an index that tries to tell me that is a banana only with higher
> probability than apples: p(bananas) = 0.3 p(apple) = 0.1 for example.
What about using SMLR -- as a logistic regression its decision is based
on the max of probabilities per each possible (trained) label. So just
enable_ca=['estimates'] and there (in .ca.estimates) you would get your
probabilities per each target label for the last .predict call
if for SVM - enable estimation of probabilities (I believe a sigmoid is
fit by libsvm in the decision boundary neighborhood) " probability=1"
and then get them from .ca.probabilities
or some other classifier? GDA/LDA/GNB...
would that help?
> Hope it is an xhaustive and an answerable question!�
> Thank you
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
Yaroslav O. Halchenko
Postdoctoral Fellow, Department of Psychological and Brain Sciences
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
More information about the Pkg-ExpPsy-PyMVPA