[pymvpa] mvpa noob questions...

Tue Oct 2 16:07:41 UTC 2012

Thanks Michael,

At 10:15 AM 10/1/2012, Michael Hanke wrote:

> > Errors:
> > 'gcc' is not recognized as an internal or external command,
> > operable program or batch file.
> > C:\Python27\lib\site-packages\scipy\integrate\quadpack.py:288:
>
>You don't have built pymvpa's extensions properly. How did you
>install it.

I got it from Christoph Gohlke's excellent build 
bot:  http://www.lfd.uci.edu/~gohlke/pythonlibs/
He did have a ticket for msvc compile errors earlier: 
https://github.com/PyMVPA/PyMVPA/pull/58
and might be amenable to debugging this build if we can help him in 
the right direction.
Incidentally, I added gcc on this system, and the only errors are now:
C:\Python27\lib\site-packages\scipy\integrate\quadpack.py:288: 
UserWarning: Extremely bad integrand behavior occurs at some points of the
   integration interval.
   warnings.warn(msg)
C:\Python27\lib\site-packages\mvpa2\misc\errorfx.py:106: 
RuntimeWarning: invalid value encountered in divide
   ([0], np.cumsum(~t)/(~t).sum(dtype=np.float), [1]))
C:\Python27\lib\site-packages\scipy\stats\stats.py:274: 
RuntimeWarning: invalid value encountered in double_scalars
   return np.mean(x,axis)/factor
C:\Python27\lib\site-packages\mvpa2\misc\errorfx.py:102: 
RuntimeWarning: invalid value encountered in divide
   ([0], np.cumsum(t)/t.sum(dtype=np.float), [1]))

>Here is also the usual advice: "Don't do this at home!" On
>windows
>it is a lot more convenient to get the NeuroDebian virtual machine and
>install PyMVPA inside -- shouldn't take you more than 5 min to set up

Downloading now for myself; the issue there is that I write code for 
a small group of Windows7 users here, and I'd like to incorporate 
mvpa with the usual Python exes I make for them. The 2nd best 
alternative is to run it on Amazon's *NIX cloud service we use.
I'll also try the py2.6 version next.

>.... <SNIP>
>stats['FP'] /
>(1.0*stats["P'"])
> > C:\Python27\lib\site-packages\mvpa2\measures\anova.py:111:
> > RuntimeWarning: invalid value encountered in divide
> >   msb = ssbn / float(dfbn)
> >
>
>Not sure why this happens -- could be specific to you input dataset.

attached, with a revised script
The runtime errors and the mvpa2 warning " Only 1 sets have estimates 
assigned from 2 sets" looks like my data setup (?)

> > WARNING: Obtained degenerate data with zero norm for training
>of
> > <LinearCSVMC>.  Scaling of C cannot be done.
>
>There seem to be a problem with the dataset. do you have invariant
>features in it?

I don't belive so, the features are FFT peaks' power and f.
I added
     mvpa2.datasets.miscfx.coarsen_chunks(ds, nchunks=4)
     mvpa2.datasets.miscfx.remove_invariant_features(ds)
to the code, with no apparent change.

I do note that CART (tested prior on this set) will _always_ split 
data even on junk, there is no guarantee that there are any useful 
correlations in the data presented...
I'm looking at mlpy right now as well.

Best,
Ray
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20121002/93679c32/attachment.html>
-------------- next part --------------
S001,Stroke,33175.5142,14408.18074,10849.84165,8059.24706,8010.452299,14,45,40,55,50,56060.79132,24908.80989,16687.6343,10154.6501,7901.745475,14,45,40,50,30,64268.60726,12620.57744,992.4884881,825.5158143,751.0024413,19,27,33,67,40,2170.966193,1879.560843,1741.498856,1340.718439,959.5283252,32,15,19,42,23
S002,Stroke,13949.87424,8982.503791,8263.746751,4618.995762,3205.907955,17,36,31,42,51,15045.29606,8688.322114,8664.694971,4438.756539,3296.550178,18,36,31,42,51,11284.75965,9245.968744,8812.669959,7737.298606,7602.351965,25,41,35,55,46,9929.52156,9798.211777,8717.782315,6653.911943,6554.729745,16,78,68,53,58
S003,Stroke,22259.75995,9902.411934,7439.217476,2316.205133,1373.566094,24,18,34,14,46,20182.57091,16574.93081,5772.954627,2805.685691,1541.999767,19,25,34,49,59,48799.93236,19789.56331,3481.015434,1353.448434,815.6284587,20,10,34,48,26,3909.159103,2219.365361,2189.082326,2035.110049,1449.020245,22,36,26,10,46
S004,Stroke,37536.41929,32305.35318,11464.6357,10796.15817,7807.086571,40,31,22,50,16,21399.50277,19871.71231,9017.82503,7784.927799,2343.456909,31,40,16,22,11,6559.376661,3505.384835,2164.53903,1241.298171,671.1796343,18,38,29,61,93,567.481842,457.0292878,324.9898472,282.1944614,234.0994137,40,31,45,16,22
S004post,Stroke,51234.10364,46358.25902,38817.93026,33053.87765,29915.75828,17,27,47,37,51,52452.32386,34325.30423,19123.11356,16212.36531,14487.34533,17,28,12,37,47,60498.54112,4243.573428,3751.950589,1778.858223,946.5012668,18,9,37,30,46,8335.496436,3313.786189,2521.593777,2502.74447,1662.166095,17,27,37,22,49
S005LML,Stroke,10746.79743,5303.973437,2034.794982,473.6322095,78.97424776,23,10,34,48,2,15715.94885,8341.165522,3485.590605,633.6094118,404.8661365,23,10,34,49,45,1165.497741,453.8466092,338.916284,196.7465767,177.5321536,9,20,34,39,29,299.5499831,90.62240694,71.00705757,52.63905869,44.51666875,22,33,38,48,43
S006,Stroke,4552.484308,733.6690994,352.6488736,158.7531598,102.0050764,14,31,6,41,37,4534.87792,128.9239303,96.45907745,80.97252839,69.14238924,13,26,78,90,63,9834.975013,5503.956576,468.3731553,407.7154245,294.1181097,8,15,49,54,45,135.2635065,65.27028845,43.00361014,25.35310559,22.18448923,19,43,55,7,50
S007,Stroke,7305.375598,6096.519862,1150.381843,1126.580301,140.1954075,12,24,44,40,54,8490.913398,4257.915467,2382.803335,688.4784055,439.1629974,24,10,15,44,39,18462.42628,3956.168385,727.9757684,498.7355164,265.3122482,13,20,45,6,73,135.072566,105.0928804,52.28150365,47.33419032,45.61565247,13,22,40,60,69
S008,Stroke,10270.52384,9878.954354,6504.924838,1263.123606,994.9116187,20,14,8,30,35,14058.63325,12187.7967,8111.279436,2166.452116,2051.831881,14,20,9,30,34,4136.238023,579.2636317,561.6605093,538.8065678,373.1796131,10,3,33,39,19,230.9554809,213.1305877,60.14892915,6.855848925,6.734007416,12,21,31,61,43
S009,Stroke,12530.14261,10608.84139,7863.63858,6868.797591,3570.929189,48,44,38,52,58,49685.92684,4678.298942,3793.519477,583.0235659,359.847283,21,32,12,45,89,46467.92152,6585.021456,4918.643354,1908.551985,1094.622933,21,33,37,12,48,17445.49114,3411.952393,2671.057033,2321.386523,2100.131389,21,38,34,49,43
S010,Stroke,4302.470354,2150.866607,1265.025748,580.7544947,253.0393021,15,20,30,36,42,5817.504704,4754.652414,2442.011992,1988.761945,1010.596622,15,22,36,30,42,4627.518445,1472.28931,1265.109614,929.8212217,854.8755832,15,74,37,68,80,294.1461433,289.329925,155.9588047,153.6713146,151.2488574,37,8,45,32,27
S011,Stroke,36556.85817,27118.08949,14632.22879,9934.91371,8005.745841,10,21,25,3,30,63151.81831,37843.56693,13390.04287,8360.172036,3587.293041,10,21,3,30,35,73494.36576,3321.681835,1547.739773,1367.53426,1012.560313,8,21,25,30,69,2177.258049,1947.587095,1466.520182,405.8248842,269.1912296,31,21,10,55,69
S012,Stroke,6566.953642,4247.90628,3917.684839,3742.10758,3385.801856,45,38,60,13,34,6576.190901,5442.042905,5043.867861,4802.808617,4011.830546,45,48,38,13,24,44593.48102,17024.87022,1139.401735,769.1183986,746.2300085,24,31,14,41,46,1254.548176,474.8511442,314.5532743,305.1359611,244.8470664,33,25,20,45,49
S013,Stroke,39891.94479,11830.45846,7605.629194,6159.563873,810.6924012,22,33,44,12,55,51971.08987,14202.144,7942.392446,7181.512247,1194.243922,22,33,12,43,6,5253.57125,4335.699153,2673.406772,1972.458966,1348.539154,17,25,10,33,43,830.6560654,397.6325044,303.930058,124.5087257,108.3331632,18,32,13,41,54
S014Post,Stroke,9514.824476,8821.326146,3818.859762,3733.5319,3321.824063,21,15,41,6,34,19479.46984,12422.11291,3930.530421,2022.545543,998.1853928,21,14,34,40,44,27435.42886,17914.96696,6044.069235,1487.99564,1390.208055,23,11,30,35,41,2017.420176,1577.770488,719.5467481,636.3215271,598.0320966,39,13,74,49,30
S015,Stroke,6711.779719,6220.788606,3031.422787,1978.557444,1673.31982,34,44,17,22,73,4575.858371,3286.254757,3155.790529,2003.531544,1522.977386,35,17,11,43,21,3938.812769,1853.794695,1778.942596,1027.981007,308.8947653,23,31,15,36,45,253.3214862,247.5429313,220.1422126,160.937178,145.5814569,34,17,11,22,44
S019,Stroke,9886.960278,5486.34135,4433.205173,2819.596611,2332.373017,29,21,16,40,11,10157.51616,5643.594368,3774.500815,3161.093417,1754.651944,30,16,21,11,40,69624.2951,5764.204884,3878.240006,652.4939506,245.8067308,18,25,11,40,7,392.1996492,346.8489059,129.187933,95.16441667,74.65035839,21,29,16,11,40
S020,Stroke,40951.521,24600.06732,15810.42189,11538.67122,10719.27762,11,17,33,37,28,67044.79452,43860.89803,25140.80614,19060.53443,16592.19452,11,17,33,28,38,91447.90868,3053.627382,1890.732664,1637.529745,1559.364614,20,9,32,60,64,2909.383005,2570.915448,2211.332394,2150.100084,1798.342233,19,64,59,69,24
SN01,Normal,10413.82019,4047.324918,3829.996938,2503.527822,1928.618566,30,45,40,36,58,8939.562031,3608.070952,2951.58035,1982.272174,1939.843423,29,45,40,35,49,7965.166108,7342.367421,892.3897452,721.2693191,494.7583049,25,13,58,54,50,7603.000285,5390.920269,625.3954696,495.7817855,338.3952769,13,24,43,58,63
SN02,Normal,9059.180131,5372.707086,3032.910216,1703.124053,748.250454,10,21,31,16,41,10183.4202,8630.937772,3624.958474,663.7729089,659.2431453,10,21,31,45,41,3826.044205,944.4286849,729.1723605,410.5789107,236.2846053,8,15,21,31,60,24607.99752,4813.86158,3535.420948,1458.768672,763.9359912,9,16,21,30,26
SN03,Normal,17333.85099,13624.69691,2190.532919,1428.992641,646.7420676,23,13,39,34,48,15616.16214,10158.93251,1041.421637,826.9497889,406.9449923,23,13,39,34,4,10460.45417,6059.502086,587.841526,511.5329675,492.2309364,8,17,52,42,47,17410.98007,9483.292668,595.8746231,571.0725655,474.7170481,9,16,63,68,42
SN04,Normal,10179.58678,6207.811913,1192.957189,257.9480385,203.6506683,24,36,16,10,50,6479.426335,3224.062557,1368.117893,74.72391868,68.60928324,24,36,15,77,54,5100.227923,1564.82399,103.7710168,87.8563791,29.17912685,10,21,3,34,82,4996.04795,2988.479318,945.0007119,618.8783778,537.9173771,10,22,16,36,32
SN05,Normal,7424.922839,5522.95347,2880.374718,2401.48268,712.4915079,22,10,15,32,45,6096.372688,3607.077294,2785.48376,1088.506535,485.7220903,22,9,15,32,46,5264.047427,4580.957971,1056.89431,788.9357442,714.294112,13,21,7,58,48,11792.0483,6212.159025,2335.268038,1056.914526,988.1251123,12,22,33,44,58
SN06,Normal,48388.60253,43594.01634,454.6620647,359.5235919,315.3222634,22,12,32,35,74,51838.76086,16888.40257,1399.219675,1277.260286,368.5047434,23,11,34,30,40,21913.90044,2761.321033,1316.226857,1256.073991,1010.628296,14,7,48,63,37,87284.50943,16373.23369,1377.207933,1130.245067,1109.343488,13,21,63,35,75
SN07,Normal,4293.547751,2655.321696,1652.741405,857.4020666,533.6334855,28,38,19,49,11,3467.383489,1962.17879,1523.593392,741.5941359,57.84590529,28,38,18,49,11,2165.754592,1634.265119,1611.515737,343.2434645,302.4449326,22,10,15,52,56,4441.441061,2733.474251,512.6397504,422.5834008,389.1201129,22,11,36,52,57
SN08,Normal,4261.111275,1703.438312,562.007216,222.3612023,80.03633625,25,10,16,35,41,4575.227692,1471.246582,653.0948942,253.599371,160.1954523,25,10,16,36,40,643.9162731,551.2331105,450.3032957,332.3854984,231.7802702,16,22,6,38,42,2330.170713,959.4526327,714.6930735,440.3421677,250.6423699,22,16,11,6,32
SN09,Normal,18601.70496,6923.77638,1881.611854,490.5702993,277.3549489,24,16,10,3,53,25486.75478,7936.383005,3001.98825,487.1176207,374.5441092,24,16,9,67,71,7998.276114,3718.235295,383.4302229,244.0255523,219.6934036,7,24,51,13,40,13600.98985,8779.79503,4251.282081,1205.260982,616.8816542,21,7,15,52,81
SN10,Normal,14325.43521,10300.90809,6290.130497,3697.155954,2941.405569,22,32,11,36,44,12267.43797,8067.236365,4575.375683,3100.670839,2692.550529,22,32,11,36,7,18863.96521,17463.28748,2077.654629,1296.674145,1274.163573,21,11,42,51,46,23284.20629,18318.07879,3244.982859,2020.140293,1870.180708,22,10,42,52,46
SN11,Normal,23278.8265,7571.761708,6626.117867,2573.606509,649.0780986,22,16,11,32,2,39221.34851,9945.990136,8276.514397,895.3326398,855.9316938,22,15,31,2,45,26304.49121,10121.89052,1944.122798,637.5965141,605.9835658,23,14,31,52,65,32374.68999,7278.977344,999.4318592,534.1920364,311.7532697,23,12,32,4,49
SN12,Normal,22785.78914,20258.49342,869.4863684,622.4815801,418.0048989,29,13,45,41,2,19536.2482,19349.13267,1148.946941,602.428797,482.8102572,30,16,46,41,2,8162.629901,5733.388616,953.5776749,784.0466136,659.7311598,16,29,45,40,7,24854.3651,6253.558241,5697.284034,2005.832802,565.7153993,13,27,23,7,2
SN13,Normal,58562.27748,21370.26619,13362.75056,10868.53839,1555.202737,25,15,19,34,53,34042.08535,19219.02303,10161.71461,2264.05825,1774.251171,26,15,34,49,53,41757.5277,3821.817933,2632.274536,2444.288069,2403.188654,23,13,54,34,50,92489.65746,25567.88926,21995.77448,8770.652009,2365.164146,23,14,11,34,6
SN14,Normal,12231.18027,8761.363911,3506.033663,2115.88277,657.6076009,19,9,31,36,46,8057.850018,5301.475334,2793.348533,1284.373693,485.7766882,20,9,31,36,46,5199.655905,3933.091259,3527.068766,2513.497412,1479.523147,18,9,37,32,26,11669.426,7232.495337,1682.2886,1541.553764,1358.41756,9,18,38,24,31
SN15,Normal,9048.362008,2308.421085,408.9271238,160.3851086,82.37352773,13,19,33,48,43,8248.692149,3148.66456,486.8420244,473.8733314,352.9882684,13,20,3,33,38,19668.54338,3988.617486,1197.389779,1008.185212,841.7652758,12,34,2,48,70,40872.49059,3282.539163,1640.500915,985.9437457,985.8431828,12,34,6,2,48
SN16,Normal,8108.318329,4588.290017,770.1372741,337.2399192,175.7336895,13,19,25,34,47,5299.300458,4134.073886,491.8658665,302.2979981,201.7964909,13,19,25,33,43,11691.42531,1344.355995,1307.291494,1169.568007,730.1374751,11,36,22,18,3,35844.36538,2183.55691,831.1601119,646.3655135,448.823016,12,19,37,2,41
SN17,Normal,16691.35673,10309.04442,955.6601008,860.2781283,34.81427648,21,16,33,2,46,11528.1972,8141.426908,2038.551397,328.5007505,271.5526015,21,16,7,33,37,2831.806986,2297.159125,1173.457458,649.212229,458.4737012,14,21,3,37,34,7169.7472,2576.875677,2389.103974,495.3016432,135.6323335,12,3,19,33,47
SN18,Normal,4319.323366,2762.24168,1785.658529,815.7001278,665.8870054,28,38,20,11,49,3340.178724,2003.722088,1495.402202,509.1106411,193.1125021,28,38,18,50,11,1942.409023,1355.560635,1147.915454,256.7677288,255.9024442,22,15,10,56,52,4282.504844,2300.568862,421.196901,310.8636376,304.5124522,22,11,38,57,53
SN19,Normal,3549.929004,2913.419711,1391.008337,1381.012864,842.5773525,35,39,29,44,49,4317.586406,2707.637979,1456.855688,1373.496036,1202.481747,35,40,25,30,44,2828.786053,2737.250114,793.531915,761.1667263,752.1336177,23,13,48,29,44,3725.869497,2768.016978,1493.645636,1198.765664,907.7526038,13,23,28,18,36
SN20,Normal,2515.798986,2462.686149,2271.438477,2052.165786,1644.380604,26,30,45,40,49,4736.779188,3524.186969,3394.963339,3049.692155,2328.004149,30,44,25,40,49,8276.826344,7238.918469,2869.583029,2535.953777,709.7072603,24,28,14,33,38,12233.11341,3164.139744,2078.375336,1999.32259,431.2510501,27,14,41,46,54
-------------- next part --------------
from mvpa2.tutorial_suite import *

d = file(r"C:\temp\test3_redo_trim.csv").readlines()
lol= [x[:-1].split(",") for x in d]
#print lol[0]
## the list of subject names
subjects = [r[0] for r in lol]
## the feature data
dat = [[float(c) for c in row[2:]] for row in lol]
## the truth values
labels = [r[1] for r in lol]

ds = Dataset(samples=dat)
ds.sa['subject'] = subjects
ds.sa['targets'] = labels
ds.sa['chunks'] = np.arange(len(ds))
mvpa2.datasets.miscfx.coarsen_chunks(ds, nchunks=4)
mvpa2.datasets.miscfx.remove_invariant_features(ds)
print mvpa2.datasets.miscfx.summary(ds), '\n'

clf = LinearCSVMC()
cvte = CrossValidation(clf, HalfPartitioner(count=2, 
    selection_strategy='random', attr='subject'), 
    errorfx=lambda p, t: np.mean(p == t), enable_ca=['stats'])
cv_results = cvte(ds)
#print 'cvte.ca.stats', cvte.ca.stats.as_string(description=True)
print cvte.ca.stats.matrix

clf = FeatureSelectionClassifier(
    kNN(k=5),
    SensitivityBasedFeatureSelection(
    SMLRWeights(SMLR(lm=1.0), postproc=maxofabs_sample()),
    FixedNElementTailSelector(1, tail='upper', mode='select')))
clf.train(ds)
print '\nkNN', len(clf.mapper.slicearg)
final_dataset = clf.mapper.forward(ds)
print final_dataset
#print 'cvte.ca.stats', cvte.ca.stats.as_string(description=True)