[pymvpa] get sensitivity in cross validation with n-fold partitioner

[Marius 't Hart]

Dear all,

I'm new to PyMVPA, and now trying to apply a Linear SVM to EEG data, in 
order to see when the EEG data starts to allow classification of 
different conditions. The strength of EEG is temporal resolution, so 
something akin to a temporal searchlight seems to be working right now. 
I don't expect the spatial information to be so interesting, but I want 
to check the sensitivity of features (electrodes) nonetheless. I found 
some instructions on how to do so on the FAQ page:

>>>sensitivities  =  []
>>>def  store_me(data,  node,  result):
...     sens  =  node.measure.get_sensitivity_analyzer(force_train=False)(data)
...     sensitivities.append(sens)
>>>
>>>cv  =  CrossValidation(SMLR(),  OddEvenPartitioner(),  callback=store_me)
>>>merror  =  cv(dataset)
>>>len(sensitivities)
2
>>>sensitivities[0].shape  ==  (len(dataset.uniquetargets),  dataset.nfeatures)
True


I have copied all this into my own script, but it doesn't seem to be 
working. That is, the sensitivity is calculated in the store_me callback 
function, but it doesn't seem to be stored in the sensitivities list. 
Here's my own code:

def store_me(data, node, result):
   sens = node.measure.get_sensitivity_analyzer(force_train=False)(data)
   sensitivities.append(sens)

sensitivities = []

def sensitivityMVPA():

   filenames = ['01-exp_AVGall', '02-exp_AVGall', '03-exp_AVGall', 
'04-exp_AVGall', '05-exp_AVGall', '06-exp_AVGall', '07-exp_AVGall', 
'08-exp_AVGall', '09_AVGall', '10-exp_AVGall', '11-exp_AVGall', '12_AVGall']
   electrodes = 
['Fp1','Fz','F3','F7','FC5','FC1','C3','T7','TP9','CP5','CP1','Pz','P3','P7','O1','Oz','O2','P4','P8','TP10','CP6','CP2','C4','T8','FC6','FC2','F4','F8','Fp2','Cz']
   epochlength = 200
   timepoints = 3100 + arange(0,3000,epochlength)

   results = zeros((len(filenames),len(timepoints)))
   all_sensitivities = {}

   for fileno in range(len(filenames)): # loop through subjects
     filename = filenames[fileno]
     print filename
     BVDS = BrainVisionDataset('../data/%s'%filename)

     cuetypes = classifySegments(BVDS)
     cuetypes = np.array(cuetypes)
     indices = sort(concatenate(( (cuetypes == 1).nonzero()[0][:90] , 
(cuetypes == 8).nonzero()[0][:90] ))) # first 90 trials from conditions 
1 and 8

     epoch_sensitivities = {}

     for epochno in range(len(timepoints)): # loop through 200 ms epochs
       sensitivities = [] # clear sensitivities
       data = mean(squeeze(BVDS.GetSegmentChannelData(list(indices+1), 
electrodes, 
baselinesamples=arange(3000,3100)))[:,:,timepoints[epochno]:timepoints[epochno]+epochlength-1], 
axis=2)

       ds = dataset_wizard(samples=data, targets=cuetypes[indices], 
chunks=range(len(indices)))
       cvte = CrossValidation(LinearCSVMC(), NFoldPartitioner(), 
errorfx=lambda p, t: np.mean(p == t), callback=store_me) # callback 
function is called, and sensitivities are calculated
       cv_results = cvte(ds)

       results[fileno,epochno] = np.mean(cv_results)
       epoch_sensitivities[epochno] = sensitivities # here, 
sensitivities is empty

     all_sensitivities[filename] = epoch_sensitivities

   return results, all_sensitivities


Any help would be appreciated!

Best,
Marius