[pymvpa] effect size (in lieu of zscore)

[Mike E. Klein]

OK, I think I'm starting to get this… after mightily confusing myself!

So my subjects had 9 experimental "runs," but I recoded this into 27 chunks
(by merely diving each run into 3 by time: 1st, 2nd and 3rd thirds).

The reason why I did this was, if I run-averaged over the entire "real"
runs, it would only leave the classifier with 18 comparisons to make. So
the classifier would have to perform incredibly well to surpass
significance thresholds in a binomial n=18 situation. 27 chunks (so 54
tests) seemed like a much better bet. I couldn't seem to get the software
to work without *any* run-averaging (162 tests), it would hang on
searchlight progress = 0%. (Though this seemed like not the best way to go
for other reasons as well.)

So you're absolutely right… I only have 3 baselines (per chunk) to zscore
against.

Would you recommend that I zscore within each of the 9 "real" runs, but
then run-average, test and do LOOCV with the larger number of chunks? (If
so, I'm not sure how to do this… it seems like it would take parallel
attribute file associations.)

Thanks again for all the help!

Mike

On Thu, Dec 22, 2011 at 11:26 PM, Yaroslav Halchenko
<debian at onerussian.com>wrote:

>
> On Thu, 22 Dec 2011, Mike E. Klein wrote:
> > got rid of 2 of my 3 "sound" conditions. The experiment involved 81
> > presentations of each of 3 sound conditions (243 total), and 81
> > presentations of of rest/baseline. After some averaging, this is reduced
> to
> > 27 of each.
> > ...
> > *The old line was:*
> > zscore(dataset, chunks_attr='chunks', param_est=('targets', ['rest']),
> > dtype='float32')
> > which I just ripped off from one of the website's tutorial pages.
> > The new line removed the "param_est=('targets', ['rest'])" and left
> > everything else the same.
>
> so  here is the catch ;-)  from above I deduce that you had only 81/27=3
> samples of 'rest' within each chunk... so mean/variance estimation
> was not really "good" (I think some time recently I even added a warning
> for such cases [1] -- which version are you using? ;-))
>
> [1]
> https://github.com/PyMVPA/PyMVPA/blob/master/mvpa2/mappers/zscore.py#L149
>
> > Just to be clear: 9 runs, each containing 39 volumes (this was slow
> > event-related sparse sampling). 3 of these were used for an orthogonal
> > behavioral task and thrown out. The remaining 36 (in each run) were
> > 9xsilence, 9xSound1, 9xSound2, and 9xSound3. For my "against silence"
> MVPA
> > sanity check above, I threw out 2 of the sound conditions, so the
> remaining
> > sound and silence should be balanced. For my more empirically interesting
> > analyses, I've been throwing out the rest and one of the sound
> conditions,
> > so the classifier looks at 2 balanced sound conditions.
>
> d'oh -- I wrote above first before reading this... but I guess ok -- so
> you used 3 volumes after each stimuli onset, that is why for 3 'rest'
> conditions in each run you had 9 volumes?  then pretty much the same
> logic applies on "why it didn't work", although the warning obviously
> wouldn't be triggered for such cases
>
>
> --
> =------------------------------------------------------------------=
> Keep in touch                                     www.onerussian.com
> Yaroslav Halchenko                 www.ohloh.net/accounts/yarikoptic
>
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20111222/92de8f8d/attachment.html>