[pymvpa] General Question about Evaluating Results form a Multiclass Problem

[J.A. Etzel]

Another strategy is to look at subsets of the problem. For example, if 
you drop class d, are a, b, and c still distinguished? What if you just 
give the classifier classes a and b?

Jo


On 8/13/2014 5:45 PM, Hanson, Gavin Keith wrote:
> Hi all,
> I’m just wondering if anyone has any advice on some ways to deal with
> evaluating classifier performance on a 4-way problem.
> I’ve been using the BayesConfusionHypothesis tool which works quite
> well, but I just wondered what else was out there by way of quantitative
> evidence to insure that classifier accuracy isn’t being driven by
> perfect classification between 2 labels, and confusion between the other
> 2, or whatever. Just glancing at the confusion matrices can give us a
> good idea about what ROIs are confusing certain conditions, but a more
> objective solution would be nice.
> The problem seems to be kinda sidestepped in some of the literature on
> pattern classification in MRI.
>
> - Gavin
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Gavin Hanson, B.S.
> Research Assistant
> Department of Psychology
> University of Kansas
> 1415 Jayhawk Blvd., 534 Fraser Hall
> Lawrence, KS 66045
>
>
>
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
>

-- 
Joset A. Etzel, Ph.D.
Research Analyst
Cognitive Control & Psychopathology Lab
Washington University in St. Louis
http://mvpa.blogspot.com/