[pymvpa] FDR adjustment after permutation testing

H.Y. Chan chan at rsm.nl
Fri Jun 19 10:09:04 UTC 2015


Hello,

Thank you for your very quick response. I'll need to take a look at the code and figure out what they do. 

Again not sure if it's the right venue for this, but I'd like to expand a bit on what I'm trying to do and see if it's statistically sound. 

I'm trying to find if inter-subject correlation (ISC) in different conditions correlates with an external measure (about the condition, but not obtained from the participants themselves), so what I do is:

(1) compute, for each condition, a whole-brain map of ISC measures using a pool of (MNI-normalized) subjects
(2) calculate the (Fisher-transformed) Spearman's correlations between the ISC maps and the single external measure (there are 17 conditions, i.e. 17 data points, at each voxel)
(3) compute the null distribution (10k samples) for each voxel by permutating the external measure 
(4) obtain the p value by comparing (2) against (3)

So what I have now is a vector [1 x 24k voxel] of ISC measures, and a matrix of [10k x 24k] raw null data. Is there a way to just plug the two things into a black box that would give out 
cluster-wise FDR-corrected / TFCE-ed p values? And does it make sense?

Thanks again!!

On 18 Jun 2015, at 14:10, H.Y. Chan <chan at rsm.nl> wrote:

> Maybe it's not related to the toolbox, but I'd be grateful if there's anyone who might have advice for me on multiple comparisons.
> 
> I've done permutation testing using the toolbox and got a whole brain map of p values. I wonder if the only FDR procedure possible is voxel-wise?

I consider FDR as pretty evil for whole-brain multiple comparison correction. A strong cluster in one part of the brain can make another, very weak cluster elsewhere, survive.

> Is it possible to do cluster-wise FDR or threshold-free cluster enhancement (TCFE) (I don't know how FSL implements it)? 

PyMVPA supports fixed-threshold Monte Carlo cluster-based correction in mvpa2/algorithms/group_clusterthr.py [1] for volumetric data.

If you are comfortable using Matlab / GNU Octave, consider CoSMoMPVA’s cosmo_montecarlo_cluster_stat [2,3] function, which supports both fixed-threshold and threshold-free cluster enhancement (TFCE) for volumetric and surface-based data.

[1] https://github.com/PyMVPA/PyMVPA/blob/master/mvpa2/algorithms/group_clusterthr.py
[2] https://github.com/CoSMoMVPA/CoSMoMVPA/blob/master/mvpa/cosmo_montecarlo_cluster_stat.m
[3] http://cosmomvpa.org/contents_demo.html#demo-surface-tfce



From: H.Y. Chan

Sent: 18 June 2015 14:10

To: pkg-exppsy-pymvpa at lists.alioth.debian.org

Subject: FDR adjustment after permutation testing









Hello gurus,





Maybe it's not related to the toolbox, but I'd be grateful if there's anyone who might have advice for me on multiple comparisons.





I've done permutation testing using the toolbox and got a whole brain map of p values. I wonder if the only FDR procedure possible is voxel-wise? Is it possible to do cluster-wise FDR or threshold-free cluster enhancement (TCFE) (I don't know how FSL implements
 it)? 





Thanks in advance!





Hang-yee

PhD candidate

Rotterdam School of Management

Erasmus University Rotterdam







More information about the Pkg-ExpPsy-PyMVPA mailing list