Generator Expressions and Iterators in Python
Posted by Mark on November 23, 2022 at 06:33 | Last modified: June 2, 2022 14:08After doing some further research, I have learned that my previous explanation of initializing multiple variables as empty lists was not very Pythonic. I want to correct some of that today.
The following is a generator expression:
( [] for i in range(3) )
The expression will generate three empty lists. Parentheses are needed around generator expressions like brackets are needed around list comprehensions:
A generator is a function that returns an object also known as an iterator.
An iterator is an object that contains a countable number of values. An iterator may be iterated over one value at a time until all values have been exhausted. Trying to iterate further will then raise a StopIteration exception (i.e. error or traceback).
A generator expression is like a list comprehension except the latter produces the entire list while the former produces one item at a time. A generator expression has “lazy execution” (producing items only when asked for), which makes it more memory efficient than a list comprehension.
Although the end result is to initialize three variables as empty lists, Pythonically speaking this is doing something else:
list_a, list_b, list_c = ( [] for i in range(3) )
The right side is an object containing three items: [], [], and []. The left side unpacks that object into three respective variables. I went into detail about unpacking with regard to the zip() method in these two posts.
Last time, I may have described things as if something magical was going on. This is not magic. The parentheses on the right represent a generator object and the commas on the left are to unpack it.
With regard to my previous discussion about making sure to use commas rather than equals signs, the latter…
a = b = c = (1, 2, 3)
…is basically doing:
c = (1, 2, 3)
b = c
a = b
b and c both get assigned (1, 2, 3).
With regard to the generator expression from above, this…
a = b = c = ( [] for i in range(3) )
…is basically doing:
c = ( [] for i in range(3) )
b = c
a = b
Because c cannot be assigned three different lists from the generator, Python assigns the generator to the variable a (and b):
<generator object <genexpr> at 0x0000019F45352200>
Lists, tuples, dictionaries, and strings are all iterable objects that have an iter() method used to return values. For example:
A for loop creates an iterator object, executes the next() method for each loop, and automatically stops at the end leaving us worry-free with regard to any potential [StopIteration] error:
