[pymvpa] Sanity check

[Jo Etzel]

Do you mean that you're getting better performance when you're just 
leaving out one trial instead of one run?

If so, How many runs? How many examples per run? Is everything fully 
balanced (equal number of training examples in each class) under each 
cross-validation scheme?

Jo


On 9/23/2015 3:03 PM, Raúl Hernández wrote:
> Hi, I’m trying to evaluate on trial by trial basis how well a region can
> predict the stimulus being presented to compare it with the
> participant’s judgment of the stimulus. So I’m training the classifier
> with data from all the trials on all the runs except by the one that I
> want to predict.
>
> I’m getting really good classifications better than when I was
> predicting one run using all the others. Supposedly it should be a
> little better as I’m training with a little more data but I’m worried
> I’m doing something wrong.
>
>
> Could anyone let me know if I’m making some sort of mistake?
>
>
> I know that there should be a more efficient way to do it but I wanted
> something easy, this is my code:
>
>
> predictions = [] #this is a vector that will contain the predictions of
> the classifier
>
> for i,dsTest in enumerate(ds): #go through all the trials on ds and
> separate one to test
>
>      clf = LinearCSVMC()
>
>      fclf = FeatureSelectionClassifier(clf, fsel)
>
>      dsTrain = []
>
>      dsTrain.append(ds[0:i]) #separates the training data
>
>      dsTrain.append(ds[i:-1])
>
>      dsTrain = vstack(dsTrain) #stacks it
>
>      fclf.train(dsTrain)
>
>      predicted = fclf.predict(dsTest) #stores the prediction
>
>      predictions.append(dsTest.targets == predicted) #checks whether the
> prediction was correct
>
> print np.mean(predictions) #checks the mean -accuracy of all predictions
>
> I would really appreciate any feedback, thanks!
>
>
>
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
>

-- 
Joset A. Etzel, Ph.D.
Research Analyst
Cognitive Control & Psychopathology Lab
Washington University in St. Louis
http://mvpa.blogspot.com/