[pymvpa] Returning trained classifiers generated during cross-validation
Yaroslav Halchenko
debian at onerussian.com
Mon Jan 9 03:03:19 UTC 2012
On Sun, 08 Jan 2012, Tyson Aflalo wrote:
> I happen to be using libsvm, so I am attempting to use option 2. From
> what I understand SplitClassifier is a meta-classifier, and so I can
> simply feed my previous classifier to SplitClassifier and feed that to
> CrossValidation. SplitClassifier than just provides a layer that can save
> stuff out over the folds... I have a tenuous grasp but hopefully this is
> basically correct.
seems to be 100% identical to my comprehension of that beast ;)
> Can you glance at the couple of lines below to verify
> that I am using SplitClassifier correctly?
I think it looks all right -- have you spot some oddity which lead you
to ask this question?
> Thanks for the help!
> baseclf = LinearCSVMC()
> svdmapper=SVDMapper()
> get_SVD_sliced = lambda x: ChainMapper([svdmapper,
> StaticFeatureSelection(x)])
> metaclf = MappedClassifier(baseclf, get_SVD_sliced(slice(0, 15)))
> sc = SplitClassifier(metaclf, enable_ca=['stats'])
> cv = CrossValidation(sc, NFoldPartitioner(),
> errorfx=mean_mismatch_error, enable_ca=['stats','datasets'])
> err = cv(ds)
> # now to test the novel dataset on an example classifier
> mean(sc.clfs[1].predict(ds2.samples) == ds2.targets)
> On Sun, Jan 8, 2012 at 4:14 PM, Yaroslav Halchenko
> <[1]debian at onerussian.com> wrote:
> there are 2 ways:
> 1. [available only in mvpa2]
> any RepeatedMeasure (including CrossValidation) takes argument
> 'callback':
> callback : functor
> Optional callback to extract information from inside the main
> loop of
> the measure. The callback is called with the input 'data', the
> 'node'
> instance that is evaluated repeatedly and the 'result' of a
> single
> evaluation -- passed as named arguments (see labels in quotes)
> for
> every iteration, directly after evaluating the node.
> so there you could access anything you care about in the 'node', which
> is
> classifier in this case
> BUT because the same classifier instance gets reused through the
> iterations,
> you can't just "store" the classifier. you can deepcopy some of them
> (e.g.
> the ones relying on swig-ed APIs, like libsvm, would not be
> deepcopy-able)
> 2. SplitClassifier
> That one behaves similarly to cross-validation (just access its
> .ca.stats to
> get results of cross-validation), but also operates on copies of the
> originally
> provided classifier, so you could access all of them via .clfs
> attribute.
> Helps?
> On Sun, 08 Jan 2012, Tyson Aflalo wrote:
> > Is there a means of accessing each trained classifier that is
> generated as
> > part of a cross-validation analysis?�
> > Thanks,
> > tyson
> > _______________________________________________
> > Pkg-ExpPsy-PyMVPA mailing list
> > [2]Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> [3]http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
--
=------------------------------------------------------------------=
Keep in touch www.onerussian.com
Yaroslav Halchenko www.ohloh.net/accounts/yarikoptic
More information about the Pkg-ExpPsy-PyMVPA
mailing list