[pymvpa] effect size (in lieu of zscore)

[Mike E. Klein]

Thanks for the response!

I just took a really quick first look (just a single 2-way comparison for a
single subject):

Looking between my experimental conditions, it looks like setting C=-5
and/or using *zscore(dataset, chunks_attr='chunks', dtype='float32') *leads
to accuracies that are a bit higher than before, but still worse than
without any zscoring at all. (I hope that I've done the zscoring against
the full time series correctly...)

Looking at sounds vs. silence:

*With old z-scoring, C=-5:*
Accuracies are around 60% with a heavy selection bias towards the rest
condition (chooses "rest" correctly 27/27 times, but also chooses rest for
21/27 sound conditions).

*With old z-scoring, C=-1:*
Same as above, except with accuracies around 54%

*With NO z-scoring, C=-5 or C=-1:*
Accuracies are 98-100%

*With C=-5 (or C=-1) and zscore(dataset, chunks_attr='chunks',
dtype='float32'):*
Accuracies are about 98%


So it looks like:

(a) Using C=-5 (as opposed to C=-1) helps a little with the zscore against
rest method. Although it might help across the board, but there's a ceiling
effect with the other combinations.
(b) There's a huge difference between whether I zscore against rest or with
the whole time series. I'm not sure what's up... running sounds > silence
GLMs in FSL show obvious responses in the expected brain regions.


Thanks again,
Mike





On Fri, Dec 16, 2011 at 11:14 PM, Yaroslav Halchenko
<debian at onerussian.com>wrote:

> before discussion kicks in -- out of curiosity... what happens if you
> either do nested cross-validation to choose C parameter or just set it
> a bit higher (e.g. C=-5 to still be scaled according  to the data), what
> if you do zscoring across full time series (not just baseline condition)
> -- for both of those there are explanations, I just wondered to get
> better idea of what might be happening in your case
>
>
> On Fri, 16 Dec 2011, Mike E. Klein wrote:
> >    where my baseline condition is silence. Without zscoring, SVMs can
> tell
> >    any of the sound conditions vs. the silence condition at 98-100%
> >    accuracy...which makes sense. With zscoring, this drops to the 80-90%
> >    level. The experiment has a good amount of samples, is well-balanced,
> >    motion-corrected, etc., so I can't find other obvious confounds.)�
> --
> =------------------------------------------------------------------=
> Keep in touch                                     www.onerussian.com
> Yaroslav Halchenko                 www.ohloh.net/accounts/yarikoptic
>
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20111216/b14e440a/attachment.html>