[pymvpa] What is the value of using errorfx when using Cross validation?

[Nick Oosterhof]

On 25 Feb 2015, at 16:55, gal star <gal.star3051 at gmail.com> wrote:

> > < Python Code for selecting only '0' chunk for train and '1' for test>
> 
> Can you provide this code please?
> 
> int_train = numpy.array([l in [0] for l in fds.sa.chunks])
> int_test = numpy.array([l in [1] for l in fds.sa.chunks])
>  
> train = fds[int_train]
> test = fds[int_test]
> 
> 
> > clf.train(train)
> > print clf.predict(test.samples)
> 
> How exactly do you determine the standard deviation among classification accuracies?
> 
> I am running this script code k times (each time, different part of the data input with '1' chunk).

If I understand correctly, you could have just one script and have a for-loop over the folds, something like:

for fold in xrange(nfolds):
  train=fds[fold!=fds.sa.chunks]
  train=fds[fold==fds.sa.chunks]

assuming the folds are in the range 0..(nfolds-1).

> In each time i get an accuracy result. then i'm averaging those k results and calculate standard divination.

With the above idea, do you get identical results?
>  
> Also, the topic of your email is about errorfx, but I didn’t see you using that function anywhere. Could you clarify?
> 
> Yes, i'm performing manual cross validation, but i've seen that 
> if i use the CrossValidation class there is an errrofx parameter.
> I'm trying to understand what is it contributes and how can i use it manually. 

errorfx is used to compute the accuracy, by computing, for each fold, how many samples have the same predicted label as the target label.