That is the odd thing...cross-validation using linear SVM<br><br>clf = LinearCSVMC()<br>cvte = CrossValidation(clf, NFoldPartitioner(), enable_ca=['stats'])<br>cv_results=cvte(avgds)<br><br>gives a 64.6% accuracy, shouldn't RFE start at something like 36% error and improve from there? (SMLR, slightly lower, 58.3% acc, but still better than chance?)<br>
<br>Here are the dataset details:<br><br>Dataset: 48x135168@float64, <sa: chunks,targets,time_coords,time_indices>, <fa: voxel_indices>, <a: imghdr,imgtype,mapper,voxel_dim,voxel_eldim><br>stats: mean=0.00193459 std=1.16806 var=1.36437 min=-139.459 max=116.782<br>
<br>Counts of targets in each chunk:<br> chunks\targets nwrd word<br> --- ---<br> 0.0 1 1<br> 1.0 1 1<br> 2.0 1 1<br> 3.0 1 1<br> 4.0 1 1<br>
5.0 1 1<br> 6.0 1 1<br> 7.0 1 1<br> 8.0 1 1<br> 9.0 1 1<br> 10.0 1 1<br> 11.0 1 1<br> 12.0 1 1<br>
13.0 1 1<br> 14.0 1 1<br> 15.0 1 1<br> 16.0 1 1<br> 17.0 1 1<br> 18.0 1 1<br> 19.0 1 1<br> 20.0 1 1<br>
21.0 1 1<br> 22.0 1 1<br> 23.0 1 1<br><br>Summary for targets across chunks<br> targets mean std min max #chunks<br> nwrd 1 0 1 1 24<br> word 1 0 1 1 24<br>
<br>Summary for chunks across targets<br> chunks mean std min max #targets<br> 0 1 0 1 1 2<br> 1 1 0 1 1 2<br> 2 1 0 1 1 2<br> 3 1 0 1 1 2<br> 4 1 0 1 1 2<br>
5 1 0 1 1 2<br> 6 1 0 1 1 2<br> 7 1 0 1 1 2<br> 8 1 0 1 1 2<br> 9 1 0 1 1 2<br> 10 1 0 1 1 2<br> 11 1 0 1 1 2<br>
12 1 0 1 1 2<br> 13 1 0 1 1 2<br> 14 1 0 1 1 2<br> 15 1 0 1 1 2<br> 16 1 0 1 1 2<br> 17 1 0 1 1 2<br> 18 1 0 1 1 2<br>
19 1 0 1 1 2<br> 20 1 0 1 1 2<br> 21 1 0 1 1 2<br> 22 1 0 1 1 2<br> 23 1 0 1 1 2<br>Sequence statistics for 48 entries from set ['nwrd', 'word']<br>
Counter-balance table for orders up to 2:<br>Targets/Order O1 | O2 |<br> nwrd: 0 24 | 23 0 |<br> word: 23 0 | 0 23 |<br>Correlations: min=-1 max=1 mean=-0.021 sum(abs)=47<br><br>If the cross-validation performance is better than chance, I am guessing it must be RFE that is not working, and not the dataset. Would that be a correct assumption?<br>
<br>And of course, thank you very much for your time!<br> <br>Kimberly Zhou<br><br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Message: 2<br>
Date: Wed, 22 Jun 2011 20:32:57 -0400<br>
From: Yaroslav Halchenko <<a href="mailto:debian@onerussian.com">debian@onerussian.com</a>><br>
To: Development and support pf PyMVPA<br>
<<a href="mailto:pkg-exppsy-pymvpa@lists.alioth.debian.org">pkg-exppsy-pymvpa@lists.alioth.debian.org</a>><br>
Subject: Re: [pymvpa] RFE and dataset splits<br>
Message-ID: <<a href="mailto:20110623003257.GM17261@onerussian.com">20110623003257.GM17261@onerussian.com</a>><br>
Content-Type: text/plain; charset=us-ascii<br>
<br>
well, it is indeed weird that it is always 0.5 ... code looks ok<br>
<br>
what if you share the details of the dataset:<br>
<br>
print avgds.summary()<br>
<br>
what is the cross-validation performance on full featureset using e.g.<br>
SMLR? could it be indeed that there is no categorical information among<br>
the data or it is too hard to dig it out with RFE<br><br></blockquote></div>