[PATCH 09/15] Replace dictionary iteration methods

[Nicolas Sebrecht]

On Fri, May 13, 2016 at 08:58:11AM +0200, Łukasz Żarnowiecki wrote:

> Here is a good explanation[1].  We can safely remove wrapping with list
> call when looping if you want to.
> 
> > > @@ -1009,14 +1009,14 @@ class BaseFolder(object):
> > >                      delflaglist[flag] = []
> > >                  delflaglist[flag].append(uid)
> > >  
> > > -        for flag, uids in addflaglist.items():
> > > +        for flag, uids in list(addflaglist.items()):
> > 
> > Hmm, does this work as expected?
> 
> Yes, output is almost the same.  Underling hashing algorithm was
> probably changed and the data is differently ordered in memory.
> 
> Python2
> -------
> >>> addflaglist = a
> 
> >>> for flag, uids in list(addflaglist.items()):
> ...     print(flag)
> ...     print(uids)
> ...
> a
> 1
> c
> 3
> b
> 2
> >>> for flag, uids in addflaglist.items():
> ...     print(flag)
> ...     print(uids)
> ...
> a
> 1
> c
> 3
> b
> 2
> 
> Python3
> -------
> addflaglist = {'a': 1, 'b': 2, 'c': 3}
> >>> for flag, uids in addflaglist.items():
> ...     print(flag)
> ...     print(uids)
> ...
> c
> 3
> b
> 2
> a
> 1
> >>> for flag, uids in list(addflaglist.items()):
> ...     print(flag)
> ...     print(uids)
> ...
> c
> 3
> b
> 2
> a
> 1
> 
> 
> [1] http://stackoverflow.com/questions/17695456/why-python-3-needs-wrap-dict-items-with-list

Thank you much for this pointer.

In response to the first comment there is:

 Note: in general, it is not safe to ignore the warning if the loop
 modifies the dict. It breaks on Python 3 without list(). – J.F.
 Sebastian Jul 17 '13 at 18:28 

 This should never happen: a loop normally cannot modify the dictionary
 it is iterating over; this generally raises a RuntimeError: dictionary
 changed size during iteration (both in Python 2 and Python 3).
 Reference:
 docs.python.org/dev/whatsnew/2.7.html#pep-3106-dictionary-views – EOL
 Jul 18 '13 at 2:45 

 d.items() returns a list on Python 2 therefore the loop can modify the
 dictionary. Just try it: d = {1:2}; for k, v in d.items(): d[3]=4 -
 this code works on Python 2 but it breaks on Python 3 with the error
 that you've mentioned. – J.F. Sebastian Jul 18 '13 at 12:15

 Indeed. I had the likes of my_dict.iteritems() in mind (which does
 raise the exception I mentioned, even in Python 2): in the case you
 cite, one is not "iterating over the dictionary" but over a list, as
 you say. That said, now I see what you meant: that list() is necessary
 in Python 3 if one wants to modify the dictionary. I had missed this
 because (1) the warning is actually a little strange, as the for loop
 statement is perfect Python 3 code; and (2) this is a relatively rare
 situation (never modified a dictionary after items() in 7 years!). Good
 point. :) – EOL Jul 18 '13 at 13:22


I wonder the real error is to actually modify a dict when looping over
it. So, instead of changing all the loops I would only fix the loops
actually requiring the list() wrapper and add a comment on top of it:

  # Wrapping loop with list() because the dict is changed in this loop.
  # This is required for Python 3.

This way, further contributors are aware of why the list() wrapper when
usefull.

-- 
Nicolas Sebrecht