[pymvpa] General d-prime Question

[Hanson, Gavin Keith]

Hi All,

I have been seeing d’ used in place of accuracy more and more as a metric of classification performance, but it appears that no one feels the need to describe how they are calculating it. I am used to d’ in the context of ROC characteristics, i.e.
dprime=scipy.stats.norm.ppf(TPR) - scipy.stats.norm.ppf(FDR)
but I don’t see how that necessarily translates to a classification problem.  If I’m training a classifier to discriminate between patterns associated with animate and inanimate objects (as an example), then I will usually get 2 different d’ metrics depending on if I consider hits to be correct classification of animate objects or correct classification in inanimate objects, and this is underscored by the fact that the ca ‘stats’ gives me a different TPR and FDR for each class in the classification. How are these two d’ measures supposed to be combined to give me a single metric of classification performance? Is it just implied that the single reported d’ is the average of the two d' associated with the two-way classification, and everyone just knows that’s always the case already? Do I calculate d’ based on averaged TPR and FDR (which gives a different result)? I guess the biggest question is can someone please point me to where the standard way that singular d’ is to be calculated in a classification setting is spelled out to where no one includes the math in their published papers, because I can’t find it.

Thanks, and I hope no one gets too mad at me for posting this general question in the software support liserv.
- Gavin

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Gavin Hanson, B.S.
Research Assistant
Department of Psychology
University of Kansas
1415 Jayhawk Blvd., 534 Fraser Hall
Lawrence, KS 66045

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20150407/efa4faea/attachment.html>