[pymvpa] Cross Validation Output
Paul Robinson
prarobinson at gmail.com
Tue Apr 16 02:41:55 UTC 2013
'lo, again!
On Mon, Apr 15, 2013 at 6:16 PM, Yaroslav Halchenko
<debian at onerussian.com> wrote:
>
> On Mon, 15 Apr 2013, Paul Robinson wrote:
>
>> Hi, Yaroslav.
>
>> As follows:
>
>> >>> print fds.summary()
>> Dataset: 39x172800 at float32, <sa:
>> chunks,targets,time_coords,time_indices>, <fa: voxel_indices>, <a:
>> imghdr,imgtype,mapper,voxel_dim,voxel_eldim>
>> stats: mean=-0.00263811 std=0.566497 var=0.320918 min=-5.24132 max=5.29731
>> No details due to large number of targets or chunks. Increase maxc and
>> maxt if desired
>> Summary for targets across chunks
>> targets mean std min max #chunks
>> Control 0.513 0.5 0 1 20
>> Patient 0.487 0.5 0 1 19
>
> I suspect that you have placed each sample into a unique chunk
> (where they could actually rightfully belong since this is different
> subjects).
Yes, this is what I intended, so good to know I'm on the right track here.
> Then with N-Fold partitioning you are pretty much doing
> leave-1-out. Hence your errors would be either 0 or 1 since you have
> only 1 sample you cross-validate into.
Here, too, I was going for leave-one-out, but didn't realize that
would lead to errors of 0 or 1...guess I thought the imbalance would
affect that.... :/
>
> print cvte.ca.stats
>
> should give you a better "picture"
Right. As I mentioned, this classifier is not doing so well. I get a
confusion matrix something like:
[[11, 13],
[9, 6]]
>
> but mention that you have only few samples, and lots of features, and
> disbalance between conditions, so classifier might just go for the one
> with more samples to reduce error etc...
>
>> Sequence statistics for 39 entries from set ['Control', 'Patient']
>> Counter-balance table for orders up to 2:
>> Targets/Order O1 | O2 |
>> Control: 19 1 | 18 2 |
>> Patient: 0 18 | 0 17 |
>> Correlations: min=-0.95 max=0.9 mean=-0.026 sum(abs)=19
>
> this is irrelevant here since this is independent samples
>
>
> --
> Yaroslav O. Halchenko
> http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
> Senior Research Associate, Psychological and Brain Sciences Dept.
> Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
> Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
> WWW: http://www.linkedin.com/in/yarik
>
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
Again, thanks for the input!
Best,
Paul
More information about the Pkg-ExpPsy-PyMVPA
mailing list