Hey guys, I've had some time to think about both aproaches Jotschi and I are pursuing and have come to this conclusion:
Each approach has flaws but combined can be something really special. First, I made a list of the the pros/cons.
Async-Await: In this approach, when async(X) is called, we offload X to a virtual thread that executes on the same event-loop (platform) thread.
Pros:
- Works with the existing netty event-loop on platform threads.
- Since await returns Future, it can be integrated into existing Vertx codebases incrementally.
- You can call any plain blocking library within async() without blocking the event loop threads
Cons:
- While async-await is a lot nicer then Future-like apis, you still have to remember to await() futures.
- Within an async scope, Vertx.currentContext() returns null.
- Users only really need 1 virtual thread per request, but if every method spawns an async() scope, way too many virtual threads will be created. There's no way to signal that a method uses async-await. In Kotlin, you do this with the suspend keyword, and likewise in Scala with Context Functions. Both of those add complexity to the language.
Event-Loop Running on Virtual Threads: This is Jotschi's approach.
Pros:
- You can call any blocking code, anywhere.
- The virtual thread creation is managed by Vertx, not the user.
- It's 'invisible'; there's no added complexity or need to understand new concepts.
Cons:
- You still need a Virtual Thread per request, Otherwise you'll still block the event loop unless you program in a non-blocking style.
- Netty threads are 'heavy', and have a lot of potential issues running on virtual threads, such as: thread locals, JNI calls, synchronized blocks, etc. From Jotschi's experiment even the JDK selector has compatibility problems with Virtual Threads, 'pinning' them and spawning ManagedBlockers.
- Architecturally, there's somewhat of a conflict between the idea of a event loop which should never be blocked, and virtual threads which are designed to be blocked.
With those points in mind, here's the big idea: Keep the netty event loop unchanged and offload to virtual threads (ala async-await), but in a way that users never have to use Future, or even async-await (ala Jotschi).
This is how we could do it:
- We make a vertx-loom project that uses vertx-gen or manually wraps all the current Vertx project APIs.
- For a method like
Route::respond(Function<RoutingContext, Future<T>> function)
, we translate it to be
package io.vertx.loom.ext.web;
public class Route {
private final io.vertx.ext.web.Route underlying;
public Route(io.vertx.ext.web.Route underlying) {
this.underlying = underlying;
}
public Route respond(Function<RoutingContext, T> function) {
underlying.respond(ctx -> async(() -> function(ctx)));
return this;
}
// Likewise for handler, but we can keep the Handler<T> interface.
public Route handler(Handler<RoutingContext> requestHandler) {
underlying.handler(ctx -> async(() -> requestHandler(ctx)));
return this;
}
}
And, for methods that return Future, just return T instead, calling await() on the underlying Future result.
Finally, we need to make sure Vertx.currentContext() returns the context while on the virtual thread. There's probably a smart way we can do thi.
I think this has the benefits of both approaches: Users never need to use Future or async-await again; they can call blocking code without ever stopping the event loop; netty continues to run on platform threads, yet the virtual threads are managed by Vertx itself.