A simple trick to understand Ruby’s lazy enumerator

    Where you get to play with an interactive demo to visualize how Enumerator::Lazy helps avoid unnecessary work

    ruby-on-rails

    In this article, we'll explore a simple visual trick to help understand how lazy enumeration works in Ruby.

    Lazy enumeration may seem like an abstract concept at first. It might be difficult to conceptualize. But taking a moment to get familiar with the lazy enumerator pays dividends. Enumerator::Lazy is extremely useful in scenarios where you want to build complex pipelines of data transformations or when working with large datasets.

    Let’s see how.

    Link to heading Enumerable is eager by default

    Consider an Enumerable method chain.

    ruby
    7.times.map { |n| n + 1 }.select(&:even?).take(3)
# => [2, 4, 6]

    At each step of the chain, method calls are evaluated eagerly. Each element from the previous step must be processed before moving on to the next step.

    I think of this as "vertical" enumeration.

    To illustrate what I mean, I’ve included a visual demonstration below. Press Play/Pause/Reset to interact with the animation.

    See it?

    The collection of items is represented as a vertical column of objects. The map operation computes a new collection, represented by the second column. The select operation filters out some objects to produce the third column. The take operation picks the first 3 to yield the last column. Each column forms vertically one by one. Each intermediate collection is constructed before moving to the next.

    This represents the eagerness of default enumeration in Ruby.

    Link to heading Making enumeration lazy

    Now let’s consider laziness. First, we use the Enumerable#lazy method to produce a lazy enumerator.

    rb
    7.times.lazy
# => #<Enumerator::Lazy: ...>

    Methods like to_a or force convert a lazy enumerator back into a normal collection:

    rb
    7.times.lazy.force
# => [0, 1, 2, 3, 4, 5, 6]

    Lazy enumeration is useful for working with large collections or expensive operations. It is even necessary in some cases, like enumerating an infinite Ruby range:

    rb
    (1..).lazy.select(&:even?).take(3).force

    Lazy enumeration flips the order of operations on its side. I visualize this as "horizontal" enumeration. Try it:

    Did you spot the difference? In the lazy enumeration demo, each object moves across the method chain one at time before evaluation of the next item begins.

    The key insight: we avoid performing any operations on subsequent items after the required 3 items are "taken" at the end of the chain. Fewer operations are performed since only a subset of items are processed. Laziness reduces the overall amount of work being done.

    With this visual in mind, you may be able to see how lazy enumerator can be helpful when working with large datasets or expensive operations. Lazy doesn’t make the operations faster but it may be a strategy to avoid unnecessary work.

    Link to heading Ruby works hard so you can be lazy

    Ruby actually redefines Enumerable methods in Enumerator::Lazy so they return another lazy enumerator instead of intermediate arrays or hashes. Intermediate methods in the lazy chain, like map and select in our example, immediately yield their current value. Methods like take control how many items complete the enumeration chain.

    A deeper dive is beyond the scope of this article, but if you’re curious to learn more about how Ruby implements laziness, look no further than Pat Shaughnessy’s Ruby 2.0 Works Hard So You Can Be Lazy. Though written with Ruby 2 in mind, the concepts are still applicable to Ruby 3 today.

    Link to heading Reprise

    Here’s the demo again with some controls to play with the speed and style of enumeration. You can use the Eager/Lazy toggle to switch between strategies. Use the slider to move manually forward and backward through the animation.

    Next time you’re dealing with a large dataset or you encounter lazy in the wild, perhaps visualizing the "vertical" vs "horizontal" operation analogy will help you understand how your Ruby behaves.

    Let me know if you found this visual helpful!

    If you liked this article, please feel free to share it and subscribe to hear more from me and get notified of new articles by email.

    Did you find a mistake or do you have questions about the content? You can send me an email, connect with me on Twitter, Bluesky, Github, Mastodon, and/or Linkedin.

    Curious to peek behind the curtain and get a glimpse of the magic? Joy of Rails is open source on Github. Feel free to look through the code and contribute.

    Peas, please
    Peas, please

    More articles to enjoy