Skip to content

Generators generate modularity

February 5, 2014

Work has limited my time to post/do in depth work, but I wanted to write something about one of my favorite features of Python: generators. My secret goal in our current code base is to slowly make everything generators up and down. Maybe that is a touch facetious, but I do think generators are a great way to hide state and generate modularity.

Let’s start with an example. Can anyone tell me what is wrong with this code?

output_list = []
current_node = get_my_first_node()
while current_node:
  current_node = current_node.next_node

That’s a pretty straightforward example of iterating through a linked list and processing the data somehow. And you are correct, my dear reader, we are doing three logically distinct things with intermingled code. We are iterating over a linked list, processing the data, and accumulating the results of do_stuff() all together. We’ve written do_stuff to pull out a little bit of the complexity, but we’ve still kind of coded ourselves into a corner. What if we wanted to make this lazy? Why can’t I use my beloved list comprehension? What if I only wanted the first 5 items?

Ok, I went over the top a little there for a moment, but such is life. The solution is to abstract away the linked list with a generator:

def linked_list_generator(first_node):
  current_node = first_node
  while current_node:
    yield current_node
    current_node = current_node.next_node

Look at that! All of our linked list logic is hidden behind this interface. The yield statement means that the output of linked_list_generator(some_node) is going to be an iterable. More concretely, it means that we can write for node in linked_list_generator(some_node) and have that behave exactly as other iterables do. Now, we can replace our bad code about with a beautiful list comprehension:

first_node = get_my_first_node()
output_list = [do_something(
               for node in linked_list_generator(first_node)]

Isn’t that so much cleaner? We’ve now separated out our iterating logic, our processing logic and our accumulating logic. All without creating unnecessary mutable state.

If you’re looking to learn more about generators, you should probably look here.


From → Python tricks

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: