[pymvpa] zscoremapper/zscore

[Kimberly Zhou]

Thanks again Yarik, for helping with RFE!
While working on this again, I ran the script for RFE and uncommented line
62:

zscore(ds, chunks_attr=None)

using chunks_attr=None as you suggested since I only have two samples per
chunk. However, I got much lower accuracies which is odd...
SMLR: 35.42% ACC
SVM: 50% ACC
RFE (overfit): 43.75% ACC
RFE (overfit less?): 43.75% ACC
RFE (half/splits+rank order): 50% ACC

In looking at the documentation for zscoremapper, I wonder if it's possible
for zscoremapper to zscore voxel-wise or per target per chunk/run? It seems
like it only zscores sample/target wise.

I appreciate your help!

Thank you,

Kimberly Zhou


On Fri, Jul 8, 2011 at 7:00 AM, <
pkg-exppsy-pymvpa-request at lists.alioth.debian.org> wrote:

> Send Pkg-ExpPsy-PyMVPA mailing list submissions to
>        pkg-exppsy-pymvpa at lists.alioth.debian.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
>
> or, via email, send a message with subject or body 'help' to
>        pkg-exppsy-pymvpa-request at lists.alioth.debian.org
>
> You can reach the person managing the list at
>        pkg-exppsy-pymvpa-owner at lists.alioth.debian.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Pkg-ExpPsy-PyMVPA digest..."
>
>
> Today's Topics:
>
>   1. Re: RFE and dataset splits (Yaroslav Halchenko)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 7 Jul 2011 11:04:08 -0400
> From: Yaroslav Halchenko <debian at onerussian.com>
> To: pkg-exppsy-pymvpa at lists.alioth.debian.org
> Subject: Re: [pymvpa] RFE and dataset splits
> Message-ID: <20110707150408.GJ26537 at onerussian.com>
> Content-Type: text/plain; charset="utf-8"
>
> just to catch up on things:
>
>  cons: indeed recipe was not correctly adopted from 0.4.x
>  pros: no changes needed in PyMVPA itself
>
> see attached a complete script with few ways for RFE, depending on what you
> take for the sensitivity and for the error to decide on when to stop.  I
> was
> going also implement a "biased" version, close to the original RFE version
> from
> Guyon, where stopping point is chosen by having a sneak view and the
> cross-validation held-out set, which would have negative consequences on
> significance testing, which is a corner-stone of machine-learning in
> neuroimaging... so I haven't done that -- only truly unbiased, although
> possibly slightly overfitting versions available ;-)
>
> we are yet to adjust the webpage describing the RFE in 0.6.x
>
> on the dataset you have provided, here is what I got with the script (d'oh
> --
> forgotten about zscoring being commented out):
>
> SMLR: 58.33% ACC
> SVM: 64.58% ACC
> RFE (overfit): 66.67% ACC
> RFE (overfit less?): 66.67% ACC
> RFE (half/splits+rank order): 64.58% ACC
>
> altogether -- results are comparable with SMLR and I bet with a simple
> nested
> cross-validation to choose lm parameter for SMLR you might get better
> results for lower computing cost
>
>
> On Wed, 06 Jul 2011, Yaroslav Halchenko wrote:
>
> > neh -- that was a good call from me but still not it -- just read it
> > nevertheless ;)
>
> --
> =------------------------------------------------------------------=
> Keep in touch                                     www.onerussian.com
> Yaroslav Halchenko                 www.ohloh.net/accounts/yarikoptic
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: dataset_rfe.py
> Type: text/x-python
> Size: 12162 bytes
> Desc: not available
> URL: <
> http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20110707/692e692a/attachment-0001.py
> >
>
> ------------------------------
>
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
>
> End of Pkg-ExpPsy-PyMVPA Digest, Vol 41, Issue 6
> ************************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20110902/1c205496/attachment.html>