Repository navigation
Pure-Python functools.reduce calls iter() twice, unlike the C implementation #161004
Description
Bug report
The pure-Python functools.reduce raises TypeError when iter(sequence) returns an iterator that defines __next__ but not __iter__, while the C version reduces it.
import operator
import sys
import functools as c_functools
del sys.modules['functools']
sys.modules['_functools'] = None # block the C accelerator
import functools as py_functools
class Iterator:
def __init__(self):
self.items = [1, 2, 3]
def __next__(self):
if not self.items:
raise StopIteration
return self.items.pop(0)
class Iterable:
def __iter__(self):
return Iterator()
print(c_functools.reduce(operator.add, Iterable()))
print(py_functools.reduce(operator.add, Iterable()))6
Traceback (most recent call last):
...
File ".../functools.py", line 262, in reduce
for element in it:
^^
TypeError: 'Iterator' object is not iterable
The C functools_reduce calls PyObject_GetIter(seq) once and then only PyIter_Next(it). The Python version calls iter(sequence) and then for element in it:, which calls iter() again on the iterator.
I know the glossary says an iterator must define __iter__, and that CPython does not apply this consistently. I am reporting it because PEP 399 asks the two implementations to behave the same, and PyPy, which uses this module as its reduce, was asked to get the fix here first (pypy/pypy#5701).
A possible fix is to replace the for loop with a while loop that calls next(it) until StopIteration.
CPython versions tested on:
3.14, 3.16, 3.12
Operating systems tested on:
macOS
Activity
picnixz commentedon Oct 31, 2026
Mmh. It's a bit hard to say that it's an issue here then. I consider this a broken class in this case. When you say "the two implementations to behave the same" I would expect that to be the case in non-degenerate cases. OTOH, one could imagine a legitimate stateful iterator for which re-calling
iteron it may change the semantics, and so we would be still "not behaving similarly".OOC, is there any performance impact using a while loop instead? I doubt so but always good to know how calling explicitly next() in a while loop and catching an exception matters.
rhettinger commentedon Oct 31, 2026
I think this is a bogus bug report. The iterator protocol specifies that iterators have an
__iter__method that returnsself.picnixz commentedon Oct 31, 2026
By any chance, does the protocol allow for
__iter__to have side-effects onself? (and thus calling it twice would be annoying). If not, then yes it's not an issue, but otherwise I guess it can be...rhettinger commentedon Oct 31, 2026
Calling
iteron an iterator just returnsself.