[pymvpa] multi-class major voting scheme paradox?

[Vadim Axel]

Hi guys,

I apply multi-class major voting scheme for three classes (all pairs
classification). I try to understand how the confusion matrix should look
like when two classes in a pair classification are not discriminated (chance
level). Consider pathological case where classes 1,2 and 2,3 are classified
with 100% and 1,3 are at chance level (50%). The confusion matrix I which
get looks like:
0.584    0.083    0.333
0    1    0
0.327    0.071    0.602

So, all of sudden it seems that classes 1 and 3 are discriminated. Isn't it
paradoxical?

When I checked out how I get this result, I have found that it indeed makes
sense. Consider class 1 as a correct label:
pair 1: the classification of classes 1,2 always results in '1' (we are at
100%, by definition)
pair 2: the classification of classes 1,3 results in half trials in '1' and
other half in '3' (we are at chance by definition).
pair 3: the classification of classes 2,3 results in half trials in '2' and
other in '3' (in case that classes are unrelated, the classifier should be
at chance here).

The bottom line: since all (1) pairs and half (2) pairs results in '1', I am
already at 50% hit rate for correct class.

What do you think about all this? Is there any flaw in my logic?
If someone is interested, I can send my matlab simulation.

Thanks for help,
Vadim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20100514/7fbc5b26/attachment.htm>