[pymvpa] Train and test on different datasets using searchlights

[Yaroslav Halchenko]

Hi Michael,

Although your use case seems to be quite "reasonable to be common", apparently
we do not have an interface for efficient "1 liner"  solution for it...
thanks for bringing it up...

As for how to implement with PyMVPA 0.4 series, which I guess you are
using:

1. the easiest, but least efficient memory (and cpu) -wise

  join both datasets into a single one with two distinct chunk ids
  and then use CustomSplitter with a conventional flow of things for
  searchlight, i.e. something like (not tested -- be warned ;) )

  cv = CrossValidatedTransferError(
         TransferError(LinearCSVMC()),
         CustomSplitter([(0, 1)])) # 0 for training, 1 for testing

  sl = Searchlight(cv, radius=5)

  # assuming you have dstrain and dstest
  dstrain.chunks[:] = 0
  dstest.chunks[:] = 1
  
  sl_map = sl(dstrain + dstest)

2. Searchlight is just a simple loop (see mvpa/measures/searchlight.py),
   which you could simply embed in your code and there do
   .selectFeatures on both datasets and providing them to TransferError instance
   call.
3.
4. ...  well -- lets see first what you think about 1 (and 2) ;-)

Cheers,

On Thu, 28 Oct 2010, Michael W. Cole wrote:

>    Hello PyMVPAers,

>    I'm attempting to use the searchlight class to train with one dataset
>    and test the resulting classifiers on another dataset. Is there a
>    straightforward way to do this? The searchlight class seems to want a
>    splitter object, yet it doesn't appear that there is a splitter class
>    that can take two datasets and designate one as the
>    training dataset and the other as the testing dataset.

>    Thanks,

>    Michael

> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/mailman/listinfo/pkg-exppsy-pymvpa


-- 
                                  .-.
=------------------------------   /v\  ----------------------------=
Keep in touch                    // \\     (yoh@|www.)onerussian.com
Yaroslav Halchenko              /(   )\               ICQ#: 60653192
                   Linux User    ^^-^^    [175555]