Zig's new plan for asynchronous programs
55 points by val
55 points by val
It looks good for “do N things possibly in parallel then join” but that’s the simplest case. I want to see if and how std.Io would express more complicated patterns like you need in servers. I filed an issue about it here: https://github.com/ziglang/zig/issues/26056
If your server tasks are simple, you won't even need async. Thread pool will get you there just fine and will have very predictable scaling. Async is useful when you have tasks that also have to wait doing nothing (thus wasting precious computation time). These kind of applications might be user-interactive GUI apps, or apps that heavily depend on networking (pending on many network responses).
I’m just talking about using std.Io for a server, I don’t care if it uses a thread pool or stackful/stackless coroutines underneath. That seems to be the whole point of std.Io, that you don’t have to decide up front. You can’t (or at least you’re encouraged not to) just avoid it because all the relevant stdlib functions now take an io parameter.
After rereading Loris' article on asynchrony and listening to his reaction of this article, I feel like I have a better understanding of the distinction Zig's API is trying to make, but I'm unconvinced on the nomenclature.
Something feels off when I'm surprised by the definition of io.async having a catch that runs the function inline. For me, and I would bet many other programmers, "async" has be conflated with "concurrent" for too long to make any semantic distinction viable. This API becomes much more intuitive to me when renamed:
io.asyncConcurrent => io.async
io.async => io.maybeAsync
Where here we've ditched the separate definition of "async" and it now just means "concurrent". I think this renaming is justified solely on how many times Loris' reaction contains "asynchrony is not enough, it also needs to be concurrent". LWN funnily enough commits to the conflation that I'm used to while trying to explain io.async:
When using an Io.Threaded instance, the async() function isn't actually required to do anything asynchronously [although the actual implementation may dispatch the function to a separate thread, depending on how it was configured] — it can just run the provided function right away.
In Zig's nomenclature, this is incorrect and should be: 'async() isn't required to do anything concurrently'. It possibly being concurrent is enough to fulfill Zig's definition of "asynchronously". This feels like a major footgun for teaching Zig, requiring you to detach these words solely for this language. Which could be fine, but I must ask, what is the value in doing so? The precision feels unnecessary.
This feels like a major footgun for teaching Zig, requiring you to detach these words solely for this language. Which could be fine, but I must ask, what is the value in doing so? The precision feels unnecessary.
Let's imagine that I write a library, let's call it robust-file-saver, which saves a user-provided string to two files (hence the robust part of the name).
You could write it like this:
fn robustFileSaver(data: []const u8) !void {
try writeToFile("fileA.txt", data);
try writeToFile("fileB.txt", data);
}
This works, but when run in a context where concurrency is available (e.g. multithreaded process), you won't be able to fully exploit it because you didn't express in your code that the second call to writeToFile doesn't have to wait for the first to be done before starting.
Here's how you can express the extra flexibility in Zig:
fn robustFileSaver(data: []const u8) !void {
var a_future = io.async(writeToFile, .{"fileA.txt", data});
var b_future = io.async(writeToFile, .{"fileB.txt", data});
try a_future.await();
try b_future.await();
}
This code expresses that the two tasks (writing to a file) are asynchronous (they don't care about each other, roughly speaking) and so in a concurrent context they can be run "at the same time" (i.e. concurrently).
When this program runs over Io.Threaded the two calls will run on two different threads in the general case.
That said, even more importantly, io.async doesn't mean that the program must have concurrency support so if we use Io.Threaded but make a single-threaded build of the program (zig build-exe -fsingle-threaded), then the two calls will happen sequentially one after the other, because that's fine too.
When running two tasks sequentially would instead produce wrong behavior, you have to use io.concurrent. The article has a good example: if you want to make a program that both reads from a socket and also reacts to user input (e.g. a chat application), then you have to use io.concurrent.
io.async(readNetwork, .{});
io.async(handleUserInput, .{})
This code is wrong because you are saying that it's also fine to first read from the network and, only once the connection cloces, to start handling user input.
try io.concurrent(readNetwork, .{});
try io.concurrent(handleUserInput, .{})
This code is correct because you need both things to happen concurrently.
And this is also what allows us to prevent surprise deadlocks when you make single-threaded builds of applications that use Io.Threaded: if you can't spawn threads (which are the unit of concurrency of Io.Threaded), then it's impossible to progress both readNetwork and handleUserInput at the same time, and so you get an error right away.
Without the distinction between io.async and io.concurrent we wouldn't know which code is safe to run sequentially and which will misbehave instead.
Other languages don't make that distinction because any usage of async/await immediately opts you into a concurrent execution context, which is also why languages end up with an ecosystem split of async vs non-async versions of essentially the same libraries.
In Zig you can decide to create a program that uses Io.Threaded, which uses exclusively blocking I/O syscalls, build in single-threaded mode, and you will still be able to use a library that internally uses io.async, like robust-file-saver from before.
At the same time users that do want to opt into Io.Evented or multithreaded Io.Threaded will also be able to use robust-file-saver and they won't get a performance penalty because io.async calls will correctly leverage the concurrent execution context.
Without the distinction between io.async and io.concurrent we wouldn't know which code is safe to run sequentially and which will misbehave instead.
I understand why the distinction exists, my issue is with the taxonomy used for the distinction, and that it reuses established words with unintuitive definitions.
If I say something is asymmetrical, I don't mean that it can possibly be symmetrical. When most people are talking about Asynchronous Execution, they are speaking of execution that is not synchronous, not that it is possibly synchronous. Your original article quoted the wikipedia pages for concurrency and parallel, but you skipped asynchrony. The first example it gives is:
Asynchronous procedure call, a method to run a procedure concurrently, a lightweight alternative to threads.
Wikipedia doesn't have to be authoritative and Zig can do whatever it wishes, but I'm struggling to find any usage of async that doesn't immediately imply or explicitly state we're that we're talking about non-synchronous execution. This definition of asynchronous being 'possibly out of order' seems novel?
This is why my io.maybeAsync feels more natural. It communicates the intent better for those that have assumed asynchronous implies concurrency (which I would argue is the whole industry).
Other languages don't make that distinction because any usage of async/await immediately opts you into a concurrent execution context, which is also why languages end up with an ecosystem split of async vs non-async versions of essentially the same libraries.
This feels overstated. Weirdly enough this feels like 3 colors: always blocking (no io), possibly blocking (using io.async only), not blocking (using io.concurrent). I still have to opt into io semantics when building out my library, but now I get the extra tool to make functions that don't require concurrency. The only difference is you guys had the foresight to make every fs function async before tagging 1.0, while Rust of course didn't.
The only difference is you guys had the foresight to make every fs function async before tagging 1.0, while Rust of course didn't.
You're mixing up two things:
All functions can be called asynchronously, whether or not they accept an I/O interface parameter.
I understand why the distinction exists, my issue is with the taxonomy used for the distinction, and that it reuses established words with unintuitive definitions.
If I say something is asymmetrical, I don't mean that it can possibly be symmetrical. When most people are talking about Asynchronous Execution, they are speaking of execution that is not synchronous, not that it is possibly synchronous.
I don't think that these terms are well defined nor established. People use the word 'concurrency' a lot and they say 'async' as an extremely general term that almost always means 'stackless coroutines'.
asynchrony means something along th lines of: 'a' - "lack of" 'synchrony' - "relationship in time" / "happening at the same time"
When two things operate in synchrony it means that their execution is in "lockstep". When two things are asynchronous it means that they don't have to do that and thus can execute independently.
From this perspective, saveFileA and saveFileB (from my previous example) are still asynchronous to one another regardless of how you schedule them. This is in opposition to synchronous code, which is code that has to run in a precise order for it to work correctly.
So if you have foo(); bar();, in normal imperative language semantics, bar cannot start until foo is done, that's what your code is saying. If instead you write (in pseudo-zig) async(foo); async(bar);, you're saying that order doesn't matter.
Once you have identified two tasks that are asynchronous to one another, then you decide to schedule them concurrently, if you want/can.
This feels overstated. Weirdly enough this feels like 3 colors: always blocking (no io), possibly blocking (using io.async only), not blocking (using io.concurrent). I still have to opt into io semantics when building out my library, but now I get the extra tool to make functions that don't require concurrency. The only difference is you guys had the foresight to make every fs function async before tagging 1.0, while Rust of course didn't.
No that's not really the point. The Io interface is a piece of the puzzle but it's just a normal function argument and most importantly by itself it doesn't make any function async / concurrent or whatnot. What I'm getting at is that if you want to use a library that uses async in Python / Rust / etc, you must also opt into stackless coroutines (which make you program a nightmare to debug) and have an event loop at runtime because asynchrony and concurrency are conflated in the language.
In Zig you can take a library that uses async and use it in your single-threaded program that has no event loop running at runtime and that does exclusively blocking I/O. That's the key practical distinction.
In Zig you can take a library that uses async and use it in your single-threaded program that has no event loop running at runtime and that does exclusively blocking I/O. That's the key practical distinction.
What does io.concurrent in a single-threaded program?
Depends on the implementation of Io that you're using.
It's possible to have single-threaded concurrency with Io.Evented because of non-blocking I/O + green threads and in the future we'll probably also have a stackless coroutine implementation.
If you're using Io.Threaded, which uses OS threads as the unit of concurrency, all calls to io.concurrent will error out since it's not possible to perform concurrent execution in single-threaded mode.
Note also that the expectation is that it's fairly uncommon for a library to use io.concurrent, and when that happens, then it will be obvious to the user of such library that they need to provide a different execution context.
Yes! IMHO dropping the entire unclear async terminology would be an improvement:
io.concurrent schedules a task that might be evented or might be threadedio.parallel schedules a task that pretty much must be threadedThat loses them the property that they want sequential programs to be able to use the io.async libraries as blocking io without overhead.
I missed that goal. But, how would the current io.concurrent work in that context?
I think the point is that you can't compile something that requires io.concurrent without something that enables that (event loop or multithreading or whatever).
If you only use io.async then you keep the choice about whether to stay sequential or not.
Threads are not parallelism, they're a unit of concurrency!
The only difference is that threads use preemptive scheduling, while usually green threads and stackless coroutines are cooperative. Preemptive-ness can make a difference when it comes to the runtime properties of the program, but that has nothing to do with what we're doing with io.async and io.concurrent.
From the perspective of being able to express asynchrony, while still being able to run code in single-threaded blocking I/O mode, all units of concurrency are the same.
Can a Future returned from io.async be made to run concurrently? That is, can I do metaphorically something like:
fn reportLiveliness(io: Io, task: Future) !void {
var reporter = io.concurrent(reporter, .{io});
defer reporter.cancel();
try task.await();
}
(Please excuse my syntax, I don't know Zig!)
I'm not 100% sure I understood your intent, so I'll try to rephrase is based on my understanding:
If the question is "can a function run from io.async call io.concurrent?" then the answer is yes.
Ugh, my wifi must have eaten my code.
AIUI .async creates a Future. The function associated with said future might run immediately (single threaded w/ threads) or it might be ran concurrently (multi-threaded or evented). It might even run later— say when .await is called?
Meanwhile, .concurrent creates a Future and then runs it concurrently. That is, it does the equivalent of .async and then schedules the Future that was created.
In that way, it seems like .async is the more fundamental operation. It makes a Future. So, is there a function that lets me take a Future already created by .async and then run it concurrently as opposed to it being ran… whenever?
My understanding of how they use terminology is as follows:
If something is "async", it can be run independently/concurrently, but not necessarily in parallel. "The catch that runs the function inline" is because if allocation fails, running things serially still results in correct behavior.
If something is "concurrent", it must be run in parallel for correct behavior, and the canonical example is a client server in one process:
try io.asyncConcurrent(Server.accept, .{server, io});
io.async(Client.connect, .{client, io});
They've renamed asyncConcurrent to concurrent. Running that code serially will deadlock, so falling back to serial execution on allocation failure is not acceptable, therefore concurrent has different semantics than async.
"Parallel" seems incorrect here. If something is "async" it can but doesn't have to be run concurrently. "Concurrent" need not be parallel at all, such as green threads running in a single core / os thread. That code will work as long as the scheduler is interleaving execution and making progress towards each task.
I missed the io.asyncConcurrent to io.concurrent rename. That's definitely better and emphasizes the distinction they're making, but I still question the usefulness of it.
"Concurrent" need not be parallel at all, such as green threads running in a single core / os thread. That code will work as long as the scheduler is interleaving execution and making progress towards each task.
Yes that's consistent with what I said.
That's definitely better and emphasizes the distinction they're making, but I still question the usefulness of it.
It is necessary for correctness to make that distinction.
That is an interesting explanation, thanks. I like the separation from io.async and io.concurrent, like another commenter said it can be confusing since people seem to conflate those 2 terms together, but I think if you start writing Zig code it is something that you will eventually learn, and having 2 different concepts is good because it make things more explicitly in the end.
One questions that I have is why io.Threaded.async couldn't have an option where it would run whatever is async in a thread? I understand that you want to enforce the separation for development so you don't mistankely use async instead of concurrent with io.Evented and have a silent bug that only happens if someone switches the io implementation, but I would expect it to be some kind of io.Debug like we have with std.heap.debug_allocator for memory allocation.
Indeed, std.Io.Threaded dispatches async operations into a configurably sized thread pool. However, async is infallible, which means the callsite must be able to tolerate the function being called directly in case a thread or closure cannot be allocated.