<div dir="ltr"><div><div>My main problem is that this isn't at all consistent with how the function for constructing fmri datasets works. Which way do you consider to be correct? I think the fmri way of handling it makes more sense. This way I have to understand that the mask needs to be flattened and I need to provide an array of nonzero indices. The other way does not work. No other way works. I can't just provide a flattened boolean mask, if by that you mean 1's and 0's. I can't provide a mask of the same shape as the original data. This is confusing because the original data will be automatically flattened but the mask won't. Why does that make any sense? Surely you would at least want the inputs to be consistent for the mask and the original data? It's neither consistent within the function itself nor is it consistent between other functions that do the same thing.<br><br></div>The fmri data provided for the tutorial does exactly the same thing. It hasn't got anything to do with the data. I can provide the code.<br><br></div>Anyway, I don't mean to non-constructively criticise but I found it confusing and frustrating and thought it might be nice to alleviate some of that.<br><div><br>#Open the main VBM data - some functions to help<br><br>import numpy as np<br>import nibabel as nb<br>import os<br>from functools import partial<br>from operator import mul<br>import pickle as p<br>from mvpa2.datasets.base import Dataset<br>from mvpa2.mappers.zscore import zscore<br>from mvpa2.datasets.mri import fmri_dataset<br><br>"""4ddata is fddata and 2ddata is tddata"""<br><br>def getImages(imagePath):<br> """load a nifti"""<br> return nb.load(imagePath)<br><br>def writeDatatoImages(writePath, Data):<br> """Write a numpy array to a nifti1 image"""<br> N = nb.Nifti1Image(Data, np.eye(4))<br> N.to_filename(writePath)<br><br>def getImageData(imagePath):<br> """load a nifti as numpy data"""<br> return getImages(imagePath).get_data()<br> <br>def rotateShape(FDData):<br> """we need to rotate the shape of the data as other wise the time dimension (here denoting the subjects) is actually last"""<br> if len(FDData.shape) == 4:<br> FDData = np.rollaxis(FDData, -1)<br> return FDData<br><br>def reshapeFlatten(FDData):<br> """Reshape the data and then flatten each member of the array"""<br> Data = rotateShape(FDData)<br> return Data.reshape(Data.shape[0],reduce(mul,Data.shape[1:],1))<br><br>#======================================Helper functions above=====================================<br><br>DATA_LOCATION = ./pymvpa/PyMVPA/tutorial_data/data/bold.nii.gz'<br><br>MASK = './pymvpa/PyMVPA/tutorial_data/data/mask_brain.nii.gz'<br><br><br><br>#Load the metadata pickle<br>#metaData = p.load(open('VBMMeta.pkl'))<br><br>#get image data and reshape<br>rotatedImageData = rotateShape(getImageData(DATA_LOCATION))<br><br>#Get mask data<br>mask = getImageData(MASK).flatten()#.nonzero()[0]#.flatten().nonzero()[0]<br>#mask = np.float32(mask)<br>#print type(mask)<br><br>#Load dataset<br>ds = Dataset.from_wizard(rotatedImageData, mask=mask)#targets=metaData['diagnosesT1'], mask=mask)<br>#ds = fmri_dataset(rotatedImageData)#, targets=metaData['diagnosesT1'], mask=mask)<br>#ds = fmri_dataset(DATA_LOCATION, mask=mask)#, targets=metaData['diagnosesT1'], mask=mask, chunks=np.array(range(133)))<br><br>#zscore normalise the data<br>#zscore(ds, chunks_attr=None) <br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Dec 10, 2014 at 3:38 PM, Yaroslav Halchenko <span dir="ltr"><<a href="mailto:debian@onerussian.com" target="_blank">debian@onerussian.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=""><br>
On Wed, 10 Dec 2014, Thomas Nickson wrote:<br>
<br>
> Ah, I see what I have done now, after I loaded the fmri dataset and<br>
> checked it seems that the from_wizard function wants as input for the mask<br>
> the set of indices from the flattened mask array and I was not doing this.<br>
<br>
</span>can't be that the only way -- boolean mask must work too.<br>
we even have a unittest ;)<br>
<br>
@reseed_rng()<br>
def test_feature_masking():<br>
mask = np.zeros((5, 3), dtype='bool')<br>
mask[2, 1] = True<br>
mask[4, 0] = True<br>
data = Dataset.from_wizard(np.arange(60).reshape((4, 5, 3)),<br>
targets=1, chunks=1, mask=mask)<br>
<br>
# check simple masking<br>
ok_(data.nfeatures == 2)<br>
...<br>
<span class=""><br>
<br>
> I wonder if anyone knows if the numpy array function .flatten() is<br>
> equivalent to how the data mapper handles the flattening or I should do<br>
> this some other way?<br>
<br>
</span>I believe so (test!) -- it doesn't use 'flatten' but 'reshape' though<br>
<span class=""><br>
> Also this function is confusingly constructed.<br>
<br>
</span>I am just still confused on what is the problem here since boolean masks<br>
should have worked. Once again -- if you just shared ds/mask -- we could look<br>
and resolve the mystery.<br>
<span class=""><br>
> Should I restructure it so<br>
> that it just accepts nifti files? Or are people happy with it this way?<br>
<br>
</span>why to limit applicability?<br>
<div class="HOEnZb"><div class="h5"><br>
--<br>
Yaroslav O. Halchenko, Ph.D.<br>
<a href="http://neuro.debian.net" target="_blank">http://neuro.debian.net</a> <a href="http://www.pymvpa.org" target="_blank">http://www.pymvpa.org</a> <a href="http://www.fail2ban.org" target="_blank">http://www.fail2ban.org</a><br>
Research Scientist, Psychological and Brain Sciences Dept.<br>
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755<br>
Phone: <a href="tel:%2B1%20%28603%29%20646-9834" value="+16036469834">+1 (603) 646-9834</a> Fax: <a href="tel:%2B1%20%28603%29%20646-1419" value="+16036461419">+1 (603) 646-1419</a><br>
WWW: <a href="http://www.linkedin.com/in/yarik" target="_blank">http://www.linkedin.com/in/yarik</a><br>
<br>
_______________________________________________<br>
Pkg-ExpPsy-PyMVPA mailing list<br>
<a href="mailto:Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org">Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org</a><br>
<a href="http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa" target="_blank">http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa</a><br>
</div></div></blockquote></div><br></div>