[pymvpa] clf generalization over a different set of classes

[Yaroslav Halchenko]

On Mon, 17 Aug 2009, Matthias Ekman wrote:
> this involves a random selection of some samples and as some samples
> might have more signal than others, the generalization depends on the
> sample selection.
you are perfectly correct! Though, if you have quite a large
number of samples and they do come from the same distribution, selection
of some smaller subset should still lead to the same result. Or even
more: you could even test if you wouldn't get the same "valuable"
samples in both cases of full and subsampled datasets, as support
vectors if you use SVM.

> Thats why I came up with the idea to do this several
> times and calculate the average... or even better to run a monte carlo
> simulation which is so comfortable with pymvpa.
btw -- splitters have that conveniently available to you, just look at
help of base class Splitter and parameters:
nperlabel
nrunspersplit

> ipython in one of your previous posts... this is really very helpful,
> especially in combination with matplotlob and scipy.
and pdb ;) if you get any crash/problem -- just get into 'automatic pdb'
fallover mode

> > Published Online May 7, 2009
> > Science DOI: 10.1126/science.1171599
> hehe .. cool idea, but they have "small" (chance-like) accuracies... and
> this in science :-)
;)
otherwise it would not be in science since would be "too easy" and
"straightforward" ;) just teasing... yeah -- science and nature are full
of those thought provoking articles, results in which present something
really intriguing but not really supported by the data/analysis -- most
of the time just "wishful" thinking

on discriminating between "what might be likely" and "statistically
significant" look at non-brain-imaging

"Of Beauty, Sex and Power"
http://www.americanscientist.org/issues/num2/2009/4/of-beauty-sex-and-power/1
or if you don't have access:
http://www.stat.columbia.edu/~gelman/research/published/power4r.pdf


> An other example of generalization between different classes is Seymour
> et al. (2009) The Coding of Color, Motion, and Their Conjunction in the
> Human Visual Cortex
> >...<
> doi:10.1016/j.cub.2008.12.050
Thanks! Didn't know it -- will check it out

-- 
                                  .-.
=------------------------------   /v\  ----------------------------=
Keep in touch                    // \\     (yoh@|www.)onerussian.com
Yaroslav Halchenko              /(   )\               ICQ#: 60653192
                   Linux User    ^^-^^    [175555]