[pymvpa] New Elastic Net classifier

[Per B. Sederberg]

It's actually pretty slow and the time it takes scales (non-linearly)
with the number of features.  I've had to do a feature selection step
first to get it in the range of acceptability.  You are likely running
into memory issues, too (i.e., swapping quite a bit), but I could be
wrong.  I'm not sure this speed issue is a problem of it being in R or
rather that the algorithm is slowish.

Can you try performing an easy feature selection of say 5-10K
features, first?  One to try might be the new feature stability
measure I added that looks for the most stable features across runs
for each label (different from ANOVA):

stab_enet = FeatureSelectionClassifier(
    ENET(lm=1.0,max_steps=500,trace=False,normalize=False),
    SensitivityBasedFeatureSelection(
    CorrStability(),
    FixedNElementTailSelector(5000,mode='select',tail='upper')),
    descr="ENET on 5K best(CorrStability) features")

Latros,
P



On Tue, Feb 24, 2009 at 1:20 PM, Michael Hanke <michael.hanke at gmail.com> wrote:
> Hi Per,
>
> On Wed, Feb 04, 2009 at 08:33:49AM +0100, Michael Hanke wrote:
>> On Tue, Feb 03, 2009 at 10:52:03PM -0500, Per B. Sederberg wrote:
>> > Hi Folks:
>> >
>> > I just wanted to let you know that I just added in a new
>> > regression-based classifier to PyMVPA that I think is going to be very
>> > effective: the elastic net (ENET).  It's a variant on least angle
>> > regression (LARS), but it has both L1 and L2 priors so that it
>> > sparsifies the features AND keeps redundant features, which helps with
>> > both generalizability and with sensitivity analysis.
>>
>> Sounds great -- I will take a look if it is friendly to my datasets ;-)
>
> Started playing (or intended to). I started a simple cross-validation
> (5-fold) on a 80000-feature dataset (not much useful signal in it). I
> was curious to see how ENET performs. However, I started it on Monday
> and it is still not done. What is the perceived speed of the R
> implementation in comparison to other classifiers?
>
>
> Cheers,
>
> Michael
>
>
> --
> GPG key:  1024D/3144BE0F Michael Hanke
> http://apsy.gse.uni-magdeburg.de/hanke
> ICQ: 48230050
>
> _______________________________________________
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA at lists.alioth.debian.org
> http://lists.alioth.debian.org/mailman/listinfo/pkg-exppsy-pymvpa
>