[pymvpa] Cross Validation Output

Paul Robinson prarobinson at gmail.com
Tue Apr 16 02:41:55 UTC 2013


'lo, again!

On Mon, Apr 15, 2013 at 6:16 PM, Yaroslav Halchenko
<debian at onerussian.com> wrote:
>
> On Mon, 15 Apr 2013, Paul Robinson wrote:
>
>> Hi, Yaroslav.
>
>> As follows:
>
>> >>> print fds.summary()
>> Dataset: 39x172800 at float32, <sa:
>> chunks,targets,time_coords,time_indices>, <fa: voxel_indices>, <a:
>> imghdr,imgtype,mapper,voxel_dim,voxel_eldim>
>> stats: mean=-0.00263811 std=0.566497 var=0.320918 min=-5.24132 max=5.29731
>> No details due to large number of targets or chunks. Increase maxc and
>> maxt if desired
>> Summary for targets across chunks
>>   targets  mean std min max #chunks
>>  Control  0.513 0.5  0   1     20
>>   Patient   0.487 0.5  0   1     19
>
> I suspect that you have placed each sample into a unique chunk
> (where they could actually rightfully belong since this is different
> subjects).

Yes, this is what I intended, so good to know I'm on the right track here.

> Then with N-Fold partitioning you are pretty much doing
> leave-1-out.  Hence your errors would be either 0 or 1 since you have
> only 1 sample you cross-validate into.

Here, too, I was going for leave-one-out, but didn't realize that
would lead to errors of 0 or 1...guess I thought the imbalance would
affect that.... :/

>
> print cvte.ca.stats
>
> should give you a better "picture"

Right. As I mentioned, this classifier is not doing so well. I get a
confusion matrix something like:

[[11, 13],
[9, 6]]

>
> but mention that you have only few samples, and lots of features, and
> disbalance between conditions, so classifier might just go for the one
> with more samples to reduce error etc...
>
>> Sequence statistics for 39 entries from set ['Control', 'Patient']
>> Counter-balance table for orders up to 2:
>> Targets/Order O1     |  O2     |
>>    Control:   19  1  |  18  2  |
>>     Patient:     0 18  |   0 17  |
>> Correlations: min=-0.95 max=0.9 mean=-0.026 sum(abs)=19
>
> this is irrelevant here since this is independent samples
>
>
> --
> Yaroslav O. Halchenko
> http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
> Senior Research Associate,     Psychological and Brain Sciences Dept.
> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
> Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
> WWW:   http://www.linkedin.com/in/yarik
>
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa

Again, thanks for the input!

Best,
Paul



More information about the Pkg-ExpPsy-PyMVPA mailing list