<div dir="ltr">If you don't want to do the "fancy dance", you can do just simply:<br><br>permutator = AttributePermutator(attr='targets')   
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">permuted_dataset = naive_permutation(original_dataset)</p><p style="margin:0px;text-indent:0px">print "Here are your new labels: ", permuted_dataset.sa.targets</p><p style="margin:0px;text-indent:0px"><br></p><p style="margin:0px;text-indent:0px">then you don't need to put permutator into your CV, just use permuted_dataset instead of orig. one<br></p><p style="margin:0px;text-indent:0px"><br></p><p style="margin:0px;text-indent:0px">or for the fancy part:<br></p><p style="margin:0px;text-indent:0px">
</p><p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"><br>
</p><p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">permutator = AttributePermutator('targets', limit={partition:1}, count=1)
</p><p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">partitioner_permutator = ChainNode([partitioner, permutator])</p>
<p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px">dataset_would_be_used_during_the_first_cv_itteration  = next(partitioner_permutator.generate(ds))</p><p style="margin-top:0px;margin-bottom:0px;margin-left:0px;margin-right:0px;text-indent:0px"><br></p><p style="margin:0px;text-indent:0px">but in this case, you would need to get creative if you would like to use the same labels also for CV.</p><p style="margin:0px;text-indent:0px"><br></p><p style="margin:0px;text-indent:0px">Regarding the importance of keeping the test labels intact, I think the state of the art knowledge is that no one really knows what should be done anyway.</p><p style="margin:0px;text-indent:0px"><br></p><p style="margin:0px;text-indent:0px">Best wishes,</p><p style="margin:0px;text-indent:0px">Richard<br></p></div>