[pymvpa] RFE & Permutation

[Yaroslav Halchenko]

could you also enable storing all estimates from MC... i.e.

        cv = CrossValidatedTransferError(
            TransferError(clf),
            splitter,
            null_dist=MCNullDist(permutations=no_permutations,
                                 tail='left',
                                 enable_states=['dist_samples']),
            enable_states=['confusion'])

weird enough -- either I do not thing straight or smth is strange -- chance
distribution after permutation on our testdata is indeed quite biased into high
values (which are errors), although I would expect it to mean at chance
(i.e.  0.5 since I did binary classification).

how does your cv.states.null_dist.dist_samples would look? ;)

On Wed, 27 Jan 2010, Yaroslav Halchenko wrote:

> could you share more details about your data? number of labels/samples?
> what error are you getting from cv (may be confusion matrix).

> If your classifier manages actually to learn data then you can expect
> null_prob be actually very very small, so everything depends on your
> data and results.

-- 
                                  .-.
=------------------------------   /v\  ----------------------------=
Keep in touch                    // \\     (yoh@|www.)onerussian.com
Yaroslav Halchenko              /(   )\               ICQ#: 60653192
                   Linux User    ^^-^^    [175555]