Some Smalltalk about Ruby Loops
29 points by stonecharioteer
29 points by stonecharioteer
If you're curious about some of the trade-offs between the imperative for loop approach to iteration and Smalltalk/Ruby's internal iterator style, you might like this old post of mine: https://journal.stuffwithstuff.com/2013/01/13/iteration-inside-and-out/
Russ Cox called them push iterators vs pull iterators when he added iterators to Go: https://research.swtch.com/coro
I enjoyed this post, though was very confused about this section. Ruby has .zip already built into it, and appears to have had it from the time of the post. Additionally, if you want to hand roll your own for some reason, it's pretty much as simple as this?
def interleave(arr1, arr2)
arr1.map.with_index do |e, idx|
next if arr2.size - 1 < idx
[e, arr2[idx]]
end.flatten.compact
end
Is the idea that you're supposed to do it only using .each? Because of course I could just as easily reimplement .map, .with_index, .flatten, and .compact using the tools at my disposal (all of them are also available as far back as Ruby 1.8.6 released in March 2007 according to my quick googling). At least one of the major perks of Ruby is that I don't have to do this and still get to benefit from its internal iterator pattern.
zip when not used on arrays uses Enumerator which uses Fibers which falls in the category of something like a thread or continuation
As far as I can tell, you simply can’t solve this problem using internal iterators unless you’re willing to reach for some heavy weaponry like threads or continuations.
Read part 2, which is linked at the bottom of the post :-)
Here's the link for everyone's convenience:
Iteration Inside and Out, Part 2
Long story short: Ruby iterators suspend their execution in place when returning values, preserving the call stack in the form of an iterator object. Iterators can be resumed at any time or not at all. This allows the user to compose iterators freely. Ruby builds this upon a lightweight threading primitive it calls fibers. It's related to generators and semicoroutines.
Go does something similar in order to create “pull” iterators from “push” iterators, an internal coroutine primitive was added to the runtime.
I'd like to thank you for these articles and the Crafting Interpreters book. They are such treasures. I don't think I'd have made much progress in my interpreter project without you. It's because of this exact article that I ended up implementing delimited continuations, even cited it in my own blog.
I'm always delighted when people experience Ruby and come across the whole iteration paradigm (and its luxurious collection of Enumerable methods). It feels like such a game changer when the thing that you do most often in code (iterating through collections and doing something about it) is such an intrinsic part of the language. I've always been curious: why does Python do it the way the way it does it? Why don't more dynamic languages do it the way Ruby does it?
It's at least partially historical: lambdas were only added to Python in 1.0, long after the syntax for for loops. And unlike Ruby, Python does not have multiline lambdas (or I guess blocks), which makes the OOP each impossible to use for anything serious.
An other historical component is that for object langages this require either implementing 20 billion methods or some sort of inheritance scheme, but old Python had a pretty severe separation between builtin (C) and userland (Python) classes, so this would have had to be entirely duplicated.
A non-historical component is that external iteration is a lot more flexible than internal, and you need significant runtime support in order to create an external iterator from an internal one and support a wider set of operations and compositions.
Enumerable methods
Is the idea that they are polymorphic on any collection type so you get a single and singular API?
There's a couple of parts to it.
First, both Array and Hash extend and override some of the Enumerable methods, which gets you most of what you need in your day to day.
Second, you can include Enumerable on any class and then so long as you implement (and therefore override) def each on that class, you automatically get access to every method available in Enumerable.
The luxury of it all is that the Enumerable methods (along with the Array- and Hash-specific implementations) covers like 99% of what you need on a daily basis through - yes - that common API.
Nice overview of looping with message-passing. I have only one footnote: what if the iterator and iteratee don't trust each other? In that case, we can still get by with single-use delimited continuations, which are efficiently implemented as stack-unwinding exceptions, but only with external iteration. I can't find a good erights.org page on this, but I can show a small E/Monte example. Here's a simple for-loop with a simplified version of Monte's iteration protocol:
for x in (xs) { f(x) }
How do we expand it when xs and f are mutually untrusting? We have to write it in terms of a trusted loop helper which mediates their interaction:
_loop(xs, f)
Where _loop is morally something nice-looking:
def _loop(xs, f):
def iterator := xs._makeIterator()
escape ej { while (true) { f(iterator.next(ej)) } }
In practice, making a VM that absorbs looping requires a native loop helper. The actual _loop object is more legible to a JIT compiler than a human!