Overview
itertools provides building blocks for lazy iteration. All functions return iterators, not lists; they compute values on demand and compose without intermediate allocations. This makes them efficient for large data streams and functional-style pipelines. Wrap in list() or tuple() when you need a concrete sequence. This card covers the most-used functions with concise examples.
Chaining and flattening
These functions concatenate or flatten iterables without building intermediate lists.
| Function | Signature | Example | Output |
|---|---|---|---|
chain | chain(*iterables) | chain([1,2],[3,4]) | 1,2,3,4 |
chain.from_iterable | chain.from_iterable(iterable) | chain.from_iterable([[1,2],[3,4]]) | 1,2,3,4 |
zip_longest | zip_longest(*its, fillvalue=None) | zip_longest([1,2],[3]) | (1,3),(2,None) |
repeat | repeat(val, n) | list(repeat(0, 3)) | [0,0,0] |
cycle | cycle(iterable) | cycle([1,2,3]) | 1,2,3,1,2,3,... |
count | count(start, step) | count(0, 5) | 0,5,10,15,... |
Use chain.from_iterable instead of chain(*nested) when the number of sub-iterables is large; unpacking to *args has a stack limit.
Slicing iterators
| Function | Signature | Example | Output |
|---|---|---|---|
islice | islice(it, stop) or islice(it, start, stop, step) | islice(range(100), 5, 15, 2) | 5,7,9,11,13 |
takewhile | takewhile(pred, it) | takewhile(lambda x: x<5, [1,3,6,2]) | 1,3 |
dropwhile | dropwhile(pred, it) | dropwhile(lambda x: x<5, [1,3,6,2]) | 6,2 |
filterfalse | filterfalse(pred, it) | filterfalse(str.isdigit, "a1b2") | 'a','b' |
compress | compress(data, selectors) | compress("ABCD",[1,0,1,0]) | 'A','C' |
islice does not support negative indices. To get the last n items from an iterator use collections.deque(it, maxlen=n).
Combinatorics
All combinatoric functions return tuples.
| Function | Signature | Example | Output |
|---|---|---|---|
combinations | combinations(it, r) | combinations([1,2,3], 2) | (1,2),(1,3),(2,3) |
combinations_with_replacement | combinations_with_replacement(it, r) | combinations_with_replacement([1,2], 2) | (1,1),(1,2),(2,2) |
permutations | permutations(it, r=None) | permutations([1,2,3], 2) | (1,2),(1,3),(2,1),(2,3),(3,1),(3,2) |
product | product(*its, repeat=1) | product([0,1], repeat=2) | (0,0),(0,1),(1,0),(1,1) |
product is equivalent to nested for loops: product(A, B, C) = for a in A: for b in B: for c in C: yield (a,b,c).
Count of results:
combinations(n, r):n! / (r! * (n-r)!)permutations(n, r):n! / (n-r)!product(n, repeat=r):n ** r
groupby
groupby groups consecutive items in an iterable that have the same key. Sort first if you want all occurrences of a key together.
from itertools import groupby
data = sorted([
{"dept": "eng", "name": "Alice"},
{"dept": "eng", "name": "Bob"},
{"dept": "hr", "name": "Carol"},
], key=lambda x: x["dept"])
for dept, members in groupby(data, key=lambda x: x["dept"]):
print(dept, list(members))
# eng [{'dept': 'eng', 'name': 'Alice'}, {'dept': 'eng', 'name': 'Bob'}]
# hr [{'dept': 'hr', 'name': 'Carol'}]| Pattern | Notes |
|---|---|
groupby(sorted_it, key=fn) | Standard usage: sort by key first |
groupby(it) | Groups equal adjacent values; no key function |
| Consume the group iterator immediately | The group iterator becomes invalid once the outer iterator advances |
The most common bug: iterating the outer groupby iterator a second time without consuming the group iterators first causes empty groups. {k: list(v) for k, v in groupby(...)} is the safe one-liner.
accumulate and reduce
accumulate is a running reduction; it yields each intermediate result.
| Function | Signature | Example | Output |
|---|---|---|---|
accumulate | accumulate(it, func=add, initial=None) | accumulate([1,2,3,4]) | 1,3,6,10 |
accumulate with max | accumulate([3,1,4,1,5], max) | running max | 3,3,4,4,5 |
accumulate with initial | accumulate([1,2,3], initial=10) | 10,11,13,16 | includes initial |
functools.reduce | reduce(func, it, initial) | reduce(mul, [1,2,3,4], 1) | 24 (final only) |
Use accumulate when you need the running series; use functools.reduce when you only need the final value.
Common gotchas
- All
itertoolsfunctions return lazy iterators. Callinggroupby(...)orchain(...)does no work until you iterate. Forgetting this leads to subtle bugs when the source data is mutated between creation and consumption. groupbyonly groups adjacent equal keys. An unsorted input produces multiple groups for the same key. Always sort first.- The group iterator from
groupbyis invalidated when the outer iterator advances. Consume each group withlist(group)before callingnext()on the outer loop. product(*[large_list] * 3)generatesn**3tuples. Check cardinality before materializing combinatoric results.cycleloops forever. Always pair it withisliceortakewhileor a zip to limit output.countandrepeat(without n) are infinite. They must be consumed with a terminating combinator.