<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi,<br>
<br>
I could not figure out why the SplitClassifier doesn't work with
RFE, but I found a solution with cross-validation (see code
below).<br>
Within the cross-validation a callback function saves the
sensitivity map of each iteration. The sensitivity maps could not
be mapped back to the original dataset, therefore I integrated a
little mapping function into the callback function. <br>
<br>
Cheers,<br>
Matthias<br>
<br>
# callback function<br>
def store_me(data, node, result):<br>
# get sensitivity map<br>
sens = node.measure.mapper.forward(dataset)<br>
# merge to max values of all maps<br>
sensmax = sens.get_mapped(maxofabs_sample())<br>
# make a list of selected voxel indices<br>
maxid = deepcopy(sensmax.fa.voxel_indices)<br>
maxid_list = maxid.tolist()<br>
# initialise sensitivity map with zeros<br>
sens_mapped = np.zeros((1,dataset.nfeatures))<br>
ind = 0<br>
# search for selected voxel<br>
for i, selvox in enumerate(maxid_list):<br>
# search for index between last index and end of list<br>
ind = dsId_list.index(selvox, ind, len(dsId_list))<br>
# write selecet voxel to correct position (as in
dataset)<br>
sens_mapped[0,ind] = abs(sensmax[0,i].samples)<br>
sensitivities.append(sens_mapped)<br>
<br>
rfesvm_split = SplitClassifier(LinearCSVMC(),
OddEvenPartitioner())<br>
<br>
rfe = RFE(rfesvm_split.get_sensitivity_analyzer(<br>
# take sensitivities per each split, L2 norm, mean,
abs them<br>
postproc=ChainMapper([ FxMapper('features',
l2_normed),<br>
FxMapper('samples', np.mean),<br>
FxMapper('samples', np.abs)])),<br>
# use the error stored in the confusion matrix
of split classifier<br>
ConfusionBasedError(rfesvm_split,
confusion_state='stats'),<br>
# we just extract error from confusion, so need
to split dataset<br>
Repeater(2),<br>
# select 80% of the best on each step<br>
fselector=FractionTailSelector(<br>
0.80,<br>
mode='select', tail='upper'),<br>
# and stop whenever error didn't improve for up
to 5 steps<br>
stopping_criterion=NBackHistoryStopCrit(BestDetector(), 5),<br>
# we just extract it from existing confusion<br>
train_pmeasure=False,<br>
# but we do want to update sensitivities on each
step<br>
update_sensitivity=True)<br>
<br>
clf = FeatureSelectionClassifier(<br>
LinearCSVMC(),<br>
# on features selected via RFE<br>
rfe,<br>
# custom description<br>
descr='LinSVM+RFE(splits_avg)' )<br>
<br>
# (chunks-1) fold cross validation<br>
# run callback function for each iteration to store
sensitivity maps <br>
cv = CrossValidation(clf, NFoldPartitioner(),
callback=store_me)<br>
error = cv(dataset)<br>
<br>
print 'mean error: %f' % np.mean(error)<br>
print 'min error: %f' % np.min(error)<br>
print 'max error: %f' % np.max(error)<br>
<br>
<br>
Am 24.07.2012 10:56, schrieb Matthias Hampel:<br>
</div>
<blockquote
cite="mid:6295_1343120241_q6O8vKgT022123_500E6353.7010508@uni-osnabrueck.de"
type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
Am 22.07.2012 10:33, schrieb Michael Hanke:
<blockquote cite="mid:20120722083352.GC6864@meiner" type="cite">
<pre wrap="">Hi,
On Fri, Jul 20, 2012 at 08:35:25AM +0200, <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:mhampel@uni-osnabrueck.de">mhampel@uni-osnabrueck.de</a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I'm currently working on my master thesis and using the PyMVPA toolbox for
the analysis of my fMRI data. My script for Recursive Feature Elimination
(RFE) is working with a CrossValidation but unfortunately not with a
SplitClassifier. Could you please give me some advice on that?
In my script (see below) I use the RFE example from the documentation. If
I add a CrossValidation I get an error value for each validation step. But
I'm also interested in the sensitivity maps of each step and I couldn't
figure out if that is possible with CrossValidation. Therefore, I tried to
use a SplittClassifier but I always get the same error message in
self.train(ds).
Could someone tell me the difference between SplitClassifier and
CrossValidation? I assumed that the SplitClassifier also does a
cross-validation internally. What do I have to change in my code to make
it work?
</pre>
</blockquote>
<pre wrap="">Your assumption is correct and your approach sounds appropriate. Could
you please provide some more information on your dataset (``print
ds.summary``) and the actual error message (incl. traceback) that you
are observing? At first glance, and without this additional information
I can't see an obvious problem.
Cheers,
Michael
</pre>
</blockquote>
<br>
It's good to know that I wasn't completely wrong. Here is the
error message and the whole output of ds.summary<br>
<br>
Best,<br>
Matthias<br>
<br>
<b>Error message:</b><br>
<br>
Traceback (most recent call last):<br>
File "RFE_Splitt1.py", line 114, in <module><br>
sens = cv_sensana(dataset)<br>
File "/usr/lib/pymodules/python2.6/mvpa2/base/learner.py", line
229, in __call__<br>
self.train(ds)<br>
File "/usr/lib/pymodules/python2.6/mvpa2/base/learner.py", line
119, in train<br>
result = self._train(ds)<br>
File "/usr/lib/pymodules/python2.6/mvpa2/measures/base.py", line
782, in _train<br>
return clf.train(dataset)<br>
File "/usr/lib/pymodules/python2.6/mvpa2/base/learner.py", line
119, in train<br>
result = self._train(ds)<br>
File "/usr/lib/pymodules/python2.6/mvpa2/clfs/meta.py", line
1211, in _train<br>
clf = clf_template.clone()<br>
File "/usr/lib/pymodules/python2.6/mvpa2/clfs/base.py", line
326, in clone<br>
return deepcopy(self)<br>
File "/usr/lib/python2.6/copy.py", line 189, in deepcopy<br>
y = _reconstruct(x, rv, 1, memo)<br>
File "/usr/lib/python2.6/copy.py", line 338, in _reconstruct<br>
state = deepcopy(state, memo)<br>
File "/usr/lib/python2.6/copy.py", line 162, in deepcopy<br>
y = copier(x, memo)<br>
File "/usr/lib/python2.6/copy.py", line 255, in _deepcopy_dict<br>
y[deepcopy(key, memo)] = deepcopy(value, memo)<br>
File "/usr/lib/python2.6/copy.py", line 189, in deepcopy<br>
y = _reconstruct(x, rv, 1, memo)<br>
File "/usr/lib/python2.6/copy.py", line 338, in _reconstruct<br>
state = deepcopy(state, memo)<br>
File "/usr/lib/python2.6/copy.py", line 162, in deepcopy<br>
y = copier(x, memo)<br>
File "/usr/lib/python2.6/copy.py", line 255, in _deepcopy_dict<br>
y[deepcopy(key, memo)] = deepcopy(value, memo)<br>
File "/usr/lib/python2.6/copy.py", line 189, in deepcopy<br>
y = _reconstruct(x, rv, 1, memo)<br>
File "/usr/lib/python2.6/copy.py", line 338, in _reconstruct<br>
state = deepcopy(state, memo)<br>
File "/usr/lib/python2.6/copy.py", line 162, in deepcopy<br>
y = copier(x, memo)<br>
File "/usr/lib/python2.6/copy.py", line 255, in _deepcopy_dict<br>
y[deepcopy(key, memo)] = deepcopy(value, memo)<br>
File "/usr/lib/python2.6/copy.py", line 189, in deepcopy<br>
y = _reconstruct(x, rv, 1, memo)<br>
File "/usr/lib/python2.6/copy.py", line 338, in _reconstruct<br>
state = deepcopy(state, memo)<br>
File "/usr/lib/python2.6/copy.py", line 162, in deepcopy<br>
y = copier(x, memo)<br>
File "/usr/lib/python2.6/copy.py", line 255, in _deepcopy_dict<br>
y[deepcopy(key, memo)] = deepcopy(value, memo)<br>
File "/usr/lib/python2.6/copy.py", line 162, in deepcopy<br>
y = copier(x, memo)<br>
File "/usr/lib/python2.6/copy.py", line 228, in _deepcopy_list<br>
y.append(deepcopy(a, memo))<br>
File "/usr/lib/python2.6/copy.py", line 189, in deepcopy<br>
y = _reconstruct(x, rv, 1, memo)<br>
File "/usr/lib/python2.6/copy.py", line 338, in _reconstruct<br>
state = deepcopy(state, memo)<br>
File "/usr/lib/python2.6/copy.py", line 162, in deepcopy<br>
y = copier(x, memo)<br>
File "/usr/lib/python2.6/copy.py", line 255, in _deepcopy_dict<br>
y[deepcopy(key, memo)] = deepcopy(value, memo)<br>
File "/usr/lib/python2.6/copy.py", line 189, in deepcopy<br>
y = _reconstruct(x, rv, 1, memo)<br>
File "/usr/lib/python2.6/copy.py", line 323, in _reconstruct<br>
y = callable(*args)<br>
File "/usr/lib/python2.6/copy_reg.py", line 93, in __newobj__<br>
return cls.__new__(cls, *args)<br>
TypeError: object.__new__(numpy.ufunc) is not safe, use
numpy.ufunc.__new__()<br>
<br>
<b><br>
ds.summary:</b><br>
<br>
<bound method Dataset.summary of Dataset(array([[ 1.66507124,
0.98552763, 0.99950778, ..., -0.80142293,<br>
-1.55295612, -0.11781561],<br>
[-1.2762681 , 1.06889722, -0.76036066, ..., -0.21888896,<br>
-1.43887115, -0.54421524],<br>
[ 0.06450247, 0.27004099, -0.85426304, ..., 0.79609439,<br>
0.36335411, 0.14502004],<br>
..., <br>
[-0.12649097, 0.53453771, 0.45816722, ..., -0.11964775,<br>
0.12668331, 0.10950945],<br>
[-1.07348862, -0.89344273, -0.7004796 , ..., 0.08824224,<br>
-1.11599424, 1.2118099 ],<br>
[ 0.75494889, 1.20532665, 1.58585258, ..., -0.63245843,<br>
0.89628655, -0.26900842]]),
sa=SampleAttributesCollection(items=[ArrayCollectable(name='chunks',
doc=None, value=array([ 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1.,<br>
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1.,<br>
1., 1., 1., 1., 2., 2., 2., 2., 2., 2., 2.,
2., 2.,<br>
2., 2., 2., 2., 2., 2., 2., 2., 2., 2., 2.,
2., 2.,<br>
2., 2., 2., 2., 2., 2., 2., 2., 3., 3., 3.,
3., 3.,<br>
3., 3., 3., 3., 3., 3., 3., 3., 3., 3., 3.,
3., 3.,<br>
3., 3., 3., 3., 3., 3., 3., 3., 3., 3., 3.,
3., 4.,<br>
4., 4., 4., 4., 4., 4., 4., 4., 4., 4., 4.,
4., 4.,<br>
4., 4., 4., 4., 4., 4., 4., 4., 4., 4., 4.,
4., 4.,<br>
4., 4., 4., 5., 5., 5., 5., 5., 5., 5., 5.,
5., 5.,<br>
5., 5., 5., 5., 5., 5., 5., 5., 5., 5., 5.,
5., 5.,<br>
5., 5., 5., 5., 5., 5., 5., 6., 6., 6., 6.,
6., 6.,<br>
6., 6., 6., 6., 6., 6., 6., 6., 6., 6., 6.,
6., 6.,<br>
6., 6., 6., 6., 6., 6., 6., 6., 6., 6., 6.,
7., 7.,<br>
7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.,
7., 7.,<br>
7., 7., 7., 7., 7., 7., 7., 7., 7., 7., 7.,
7., 7.,<br>
7., 7.]), length=210),
ArrayCollectable(name='time_indices', doc=None, value=array([
7, 12, 16, 20, 24, 29, 33, 38, 43, 48, 52,<br>
57, 62, 66, 70, 75, 80, 85, 89, 94,
98, 103,<br>
108, 112, 118, 123, 127, 132, 136, 141, 157,
162, 166,<br>
171, 175, 180, 185, 189, 194, 199, 204, 209,
213, 218,<br>
223, 228, 233, 238, 243, 248, 252, 257, 261,
266, 270,<br>
275, 279, 284, 289, 294, 309, 314, 319, 324,
329, 333,<br>
338, 342, 347, 351, 355, 360, 364, 369, 374,
378, 383,<br>
387, 392, 397, 401, 406, 411, 415, 420, 425,
429, 433,<br>
438, 443, 462, 467, 472, 476, 481, 485, 490,
495, 500,<br>
505, 510, 514, 519, 524, 528, 532, 538, 542,
547, 552,<br>
556, 561, 566, 571, 575, 580, 585, 589, 595,
599, 612,<br>
616, 621, 625, 630, 635, 640, 644, 649, 653,
658, 663,<br>
668, 672, 677, 682, 686, 691, 696, 700, 705,
710, 715,<br>
720, 725, 729, 734, 738, 742, 746, 759, 764,
769, 774,<br>
779, 783, 788, 793, 798, 802, 807, 812, 817,
821, 826,<br>
831, 836, 840, 846, 850, 855, 859, 865, 870,
874, 878,<br>
883, 888, 892, 897, 908, 913, 918, 922, 927,
931, 936,<br>
941, 946, 950, 955, 959, 965, 970, 974, 979,
984, 989,<br>
994, 998, 1003, 1008, 1012, 1017, 1022, 1026, 1031, 1036,
1041,<br>
1046]), length=210), ArrayCollectable(name='targets',
doc=None, value=array(['onsetNP', 'onsetNP', 'onsetNP', 'onsetP',
'onsetP', 'onsetNP',<br>
'onsetP', 'onsetNP', 'onsetNP', 'onsetP', 'onsetP',
'onsetP',<br>
'onsetP', 'onsetNP', 'onsetP', 'onsetP', 'onsetP',
'onsetNP',<br>
'onsetP', 'onsetNP', 'onsetNP', 'onsetNP', 'onsetP',
'onsetNP',<br>
'onsetP', 'onsetNP', 'onsetNP', 'onsetP', 'onsetP',
'onsetNP',<br>
'onsetP', 'onsetNP', 'onsetNP', 'onsetP', 'onsetNP',
'onsetP',<br>
'onsetNP', 'onsetNP', 'onsetP', 'onsetNP', 'onsetNP',
'onsetNP',<br>
'onsetP', 'onsetNP', 'onsetP', 'onsetNP', 'onsetNP',
'onsetP',<br>
'onsetP', 'onsetNP', 'onsetP', 'onsetP', 'onsetP',
'onsetNP',<br>
'onsetP', 'onsetP', 'onsetP', 'onsetNP', 'onsetNP',
'onsetP',<br>
'onsetNP', 'onsetP', 'onsetP', 'onsetNP', 'onsetP',
'onsetP',<br>
'onsetP', 'onsetNP', 'onsetNP', 'onsetP', 'onsetP',
'onsetNP',<br>
'onsetP', 'onsetNP', 'onsetNP', 'onsetNP', 'onsetP',
'onsetP',<br>
'onsetP', 'onsetNP', 'onsetNP', 'onsetNP', 'onsetP',
'onsetP',<br>
'onsetP', 'onsetNP', 'onsetNP', 'onsetNP', 'onsetP',
'onsetNP',<br>
'onsetNP', 'onsetP', 'onsetNP', 'onsetP', 'onsetP',
'onsetNP',<br>
'onsetP', 'onsetNP', 'onsetNP', 'onsetP', 'onsetP',
'onsetNP',<br>
'onsetP', 'onsetP', 'onsetP', 'onsetNP', 'onsetP',
'onsetNP',<br>
'onsetNP', 'onsetNP', 'onsetP', 'onsetNP', 'onsetP',
'onsetP',<br>
'onsetNP', 'onsetP', 'onsetNP', 'onsetNP', 'onsetP',
'onsetNP',<br>
'onsetNP', 'onsetNP', 'onsetNP', 'onsetNP', 'onsetNP',
'onsetP',<br>
'onsetP', 'onsetP', 'onsetP', 'onsetNP', 'onsetP',
'onsetP',<br>
'onsetNP', 'onsetP', 'onsetP', 'onsetP', 'onsetP',
'onsetP',<br>
'onsetNP', 'onsetNP', 'onsetNP', 'onsetP', 'onsetNP',
'onsetNP',<br>
'onsetP', 'onsetNP', 'onsetNP', 'onsetP', 'onsetNP',
'onsetP',<br>
'onsetP', 'onsetNP', 'onsetP', 'onsetNP', 'onsetNP',
'onsetP',<br>
'onsetNP', 'onsetP', 'onsetNP', 'onsetP', 'onsetNP',
'onsetP',<br>
'onsetNP', 'onsetP', 'onsetP', 'onsetNP', 'onsetP',
'onsetNP',<br>
'onsetNP', 'onsetP', 'onsetNP', 'onsetNP', 'onsetNP',
'onsetP',<br>
'onsetNP', 'onsetP', 'onsetP', 'onsetP', 'onsetNP',
'onsetP',<br>
'onsetP', 'onsetNP', 'onsetP', 'onsetP', 'onsetP',
'onsetNP',<br>
'onsetP', 'onsetNP', 'onsetP', 'onsetP', 'onsetP',
'onsetP',<br>
'onsetP', 'onsetP', 'onsetNP', 'onsetP', 'onsetNP',
'onsetNP',<br>
'onsetNP', 'onsetNP', 'onsetNP', 'onsetNP', 'onsetNP',
'onsetNP',<br>
'onsetNP', 'onsetP', 'onsetP', 'onsetNP', 'onsetP',
'onsetNP'], <br>
dtype='|S7'), length=210),
ArrayCollectable(name='time_coords', doc=None, value=array([ 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,<br>
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0.,<br>
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0.,<br>
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0.,<br>
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0.,<br>
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0.,<br>
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0.,<br>
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0.,<br>
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0.,<br>
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0.,<br>
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0.,<br>
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0.,<br>
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0.,<br>
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0.,<br>
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0.,<br>
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0.,<br>
0., 0.]), length=210)]),
fa=FeatureAttributesCollection(items=[ArrayCollectable(name='voxel_indices',
doc=None, value=array([[ 8, 29, 7],<br>
[ 8, 31, 11],<br>
[ 9, 23, 9],<br>
..., <br>
[54, 35, 9],<br>
[54, 35, 10],<br>
[54, 36, 9]]), length=39978)]),
a=DatasetAttributesCollection(items=[Collectable(name='mapper',
doc=None, value=ChainMapper(nodes=[FlattenMapper(shape=(64, 64,
37), auto_train=True, space='voxel_indices'),
StaticFeatureSelection(dshape=(151552,), slicearg=array([False,
False, False, ..., False, False, False], dtype=bool)),
PolyDetrendMapper(polyord=1, chunks_attr='chunks', opt_regs=None,
), ZScoreMapper(param_est=('targets', ['junk']),
chunks_attr='chunks', dtype='float32')])),
Collectable(name='imgtype', doc=None, value=<class
'nibabel.nifti1.Nifti1Image'>), Collectable(name='voxel_eldim',
doc=None, value=(3.0, 3.0, 3.3)), Collectable(name='voxel_dim',
doc=None, value=(64, 64, 37)), Collectable(name='imghdr',
doc=None, value=<nibabel.nifti1.Nifti1Header object at
0x4c48890>)]))><br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Pkg-ExpPsy-PyMVPA mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org">Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org</a>
<a class="moz-txt-link-freetext" href="http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa">http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa</a></pre>
</blockquote>
<br>
</body>
</html>