[PATCH] Remove unneeded workaround for Darwin

[Vincent Beffara]

Hi,

    SS> There is a clumsy workaround for Darwin that chunks reads into
    SS> 8kb blocks to avoid huge memory allocations.

I am the author of the workaround - sorry for the clumsiness !

    SS> First, this fix should not only be required but on FreeBSD2.6
    SS> too (see http://bugs.python.org/issue3531). Second, decent
    SS> python versions (I checked 2.6) already chunk in the SSL case
    SS> anyway, so there is no need to do that again. Remove that level
    SS> of indirection.

That's probably fine then. Initially I only patched the non-SSL version,
and that was not enough, but possibly things have changed since.
    
    SS> http://evanjones.ca/python-memory.html claims that this problem
    SS> has been fixed since python 2.5, so we might consider removing
    SS> the workaround completely even for the non-SSL case.

It is most definitely _not_ fixed on the version of python 2.5 shipped
by Apple in mac os 10.6 (which is 2.5.4 i.e. not the very last of the
branch). It is fixed in python 2.5.5 (the one in Fink) and in python 2.6
(the one coming with the system i.e. 2.6.1) as well.

By the way, 'python' now defaults to version 2.6 in snow leopard, so I
guess the problem will solve itself anyway eventually.

--8<---------------cut here---------------start------------->8---

vincent at elrond ~/Desktop > /usr/bin/python2.5 bug.py

python2.5(40154) malloc: *** mmap(size=10489856) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Traceback (most recent call last):
  File "bug.py", line 9, in <module>
    s = f.read(BLOCKSIZE)
MemoryError

vincent at elrond ~/Desktop > /usr/bin/python2.6 bug.py

vincent at elrond ~/Desktop > cat bug.py 

BLOCKSIZE = 10*1024*1024

f=open("empty.txt", "w")
f.close()

f=open("empty.txt")
data = []
for i in range(10000):
    s = f.read(BLOCKSIZE)
    assert len(s) == 0
    data.append(s)
    
--8<---------------cut here---------------end--------------->8---

Not sure about previous versions of the OS, but removing the patch would
effectively mean dropping python 2.5 compatibility on the mac, at least
until apple uprades it. Which if you ask me is 100% fine given that 2.6
ships with the system, but maybe someone will disagree.

    SS> Increase the chunk size on Mac from 8kb to 64kb. Even Macs should be
    SS> able to take that amount of memory usage nowadays.

Now _that_ is totaly fine, the problem was never the size of the window
but the number of calls to read() in the first place. Apparently with no
hint, read() ended up reading character by character, many times, but
with a largish buffer anyway, which was realloc()ed after the read - and
that is a noop on darwin, hence the crash.

Cheers,

        /v

-- 
|                 |   UMPA - ENS Lyon   | Mél: vbeffara at ens-lyon.fr |
| Vincent Beffara |  46 allée d'Italie  | Tél: (+33) 4 72 72 85 25  |
|                 | 69364 Lyon Cedex 07 | Fax: (+33) 4 72 72 84 80  |