[pymvpa] Accuracy for distinguishing each class against all others

[Brian Murphy]

Hi Jian,

I don't have experience with the PyMVPA monte carlo routines, so I'm not
sure if/how you can get p-values for individual classes. If you don't
see what you need there you could look at Emanuele Olivetti's work,
e.g.:
http://www.sciencedirect.com/science/article/pii/S0031320311001841
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=7270850&tag=1

Brian

On Mon, 2016-04-11 at 09:56 +0800, jian hou wrote:
> Hi Brain,
> Thank you very much for your reply.
> I got the confusion matrix. How could I get the p value for each
> class?  Can I  just compare the classification accuracy for each class
> with the cvte.null_dist.ca.dist_samples?
> Best,
> Jian Hou
> 
> 
> 
> 
> 
> 
> 
> 
> At 2016-04-11 02:53:13, "Brian Murphy" <brian.murphy at qub.ac.uk> wrote:
> >Hi Jian,
> >
> >this tutorial page includes an example of multiclass analysis, and down
> >the end it shows how to get the confusion matrix out of a (in this case
> >cross-validated) classifier with cvte.ca.stats.matrix. Note that you
> >have to tell your classifier to retain that information, with
> >enable_ca=['stats'],
> >
> >best of luck,
> >
> >Brian
> >
> >
> >
> >
> >
> >On Sun, 2016-04-10 at 16:24 +0800, jian hou wrote:
> >> Hi all,
> >> I'd like to run a permutation test for a multi-class classification
> >> and I'm interested in the classification accuracy of each class rather
> >> than the overall accuracy. 
> >> Is there a way to get the null distribution of the accuracy of each
> >> class?
> >> Any suggestions? Thanks in advance.
> >> Best,
> >> Jian Hou
> >> 
> >> 
> >>  
> >> 
> >> _______________________________________________
> >> Pkg-ExpPsy-PyMVPA mailing list
> >> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> >> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
> >
> >-- 
> >Brian Murphy
> >Lecturer (Assistant Professor)
> >Knowledge and Data Engineering
> >School of EE Engineering and Computer Science
> >Queen's University Belfast
> >
> >
> >_______________________________________________
> >Pkg-ExpPsy-PyMVPA mailing list
> >Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> >http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
> 
> 
>  
> 
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa

-- 
Dr. Brian Murphy
Lecturer (Assistant Professor)
Knowledge & Data Engineering (EEECS)
Queen's University Belfast
brian.murphy at qub.ac.uk