[pymvpa] Sanity check

[Raúl Hernández]

Thanks your method is way better than my rudimentary way; I feel like a
newbie, I will definitely do it your way.


Just another thing after running the analysis, how can I get the
predictions for each trial?, I'm interested on the actual prediction, not
if it was accurate or not

On Wed, Sep 23, 2015 at 9:49 PM, Yaroslav Halchenko <debian at onerussian.com>
wrote:

>
> On Wed, 23 Sep 2015, Raúl Hernández wrote:
>
> > Hi, I’m trying to evaluate on trial by trial basis how well a region can
> > predict the stimulus being presented to compare it with the participant’s
> > judgment of the stimulus. So I’m training the classifier with data from
> all the
> > trials on all the runs except by the one that I want to predict.
>
>
> > I’m getting really good classifications better than when I was
> predicting one
> > run using all the others. Supposedly it should be a little better as I’m
> > training with a little more data but I’m worried I’m doing something
> wrong.
>
>
> > Could anyone let me know if I’m making some sort of mistake?
>
>
> > I know that there should be a more efficient way to do it but I wanted
> > something easy, this is my code:
>
>
> > predictions = [] #this is a vector that will contain the predictions of
> the
> > classifier
>
> > for i,dsTest in enumerate(ds): #go through all the trials on ds and
> separate
> > one to test
>
> >     clf = LinearCSVMC()
>
> >     fclf = FeatureSelectionClassifier(clf, fsel)
>
> >     dsTrain = []
>
> >     dsTrain.append(ds[0:i]) #separates the training data
>
> >     dsTrain.append(ds[i:-1])
>
> minor but note that :-1 would select all but last
>
> $> python -c 'print range(2)[:-1]'
> [0]
>
> you didn't have to do manual splitting but could've simply assigned some
> attribute like
>
> ds.sa['trials'] = np.arange(len(ds))
>
> and made use of NFoldPartitioner(attr='trials') and then
> CrossValidation... all standard stuff
>
> back to more optimistic results, as Jo pointed out, to carry out most
> trustworthy analysis you should have trained/cross-validated across runs.
> Also ds.summary() output last tables could provide  you some related
> information on trial orders ... which could also contribute to "optimistic"
> result (depending on the output of cause.. ;) )
> --
> Yaroslav O. Halchenko
> Center for Open Neuroscience     http://centerforopenneuroscience.org
> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
> Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
> WWW:   http://www.linkedin.com/in/yarik
>
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20150923/35f4b212/attachment.html>