HTTP Concurrency Model
Thread-per-request, reactive WebFlux, virtual threads and Go’s goroutine-per-request: how each model handles slow I/O, and what that means for your code.
Section 25 of 31 4 min read
How a server handles thousands of concurrent requests that each wait on a database shapes how you write every handler. The Java world has had three answers over the years. Go has had one since day one.
Four concurrency models
Spring MVC on platform threads: each request occupies a Tomcat worker thread (200 by default) until the response is written, including all the time spent waiting on I/O. Simple code; thread count caps concurrency.
Spring WebFlux (reactive): a few Netty event-loop threads serve every request with non-blocking I/O. High concurrency, but every layer must return Mono/Flux and nothing may block.
Spring MVC on virtual threads (Java 21+): set spring.threads.virtual.enabled=true and each request runs on a cheap virtual thread that unmounts while blocked. Blocking code, high concurrency.
Go net/http: every request is served by its own goroutine. Blocking calls park the goroutine and free the OS thread. Blocking code, high concurrency, and the only model the ecosystem has ever used.
Spring MVC: a thread per request
@RestController
@RequestMapping("/api")
class MessageController {
private final MessageService service;
MessageController(MessageService service) {
this.service = service;
}
@GetMapping("/messages/{id}")
Message get(@PathVariable long id) {
// On platform threads, this worker sits idle while the
// database answers. With server.tomcat.threads.max=200,
// request 201 waits in the worker executor's queue.
return service.findById(id);
}
}Spring WebFlux: reactive all the way down
@RestController
@RequestMapping("/api")
class MessageController {
private final ReactiveMessageService service;
MessageController(ReactiveMessageService service) {
this.service = service;
}
@GetMapping("/messages/{id}")
Mono<ResponseEntity<Message>> get(@PathVariable long id) {
return service.findById(id) // R2DBC, not JDBC
.map(ResponseEntity::ok)
.defaultIfEmpty(ResponseEntity.notFound().build());
}
// One accidental blocking call inside the chain stalls an
// event-loop thread, and every request queued behind it.
}Go: a goroutine per request
type MessageHandler struct {
messages *message.Service
}
// net/http runs this in a goroutine of its own: one per connection
// on HTTP/1.x, one per request stream on HTTP/2.
// While Find waits on Postgres, the goroutine is parked and its
// OS thread runs other goroutines. No callbacks, no Mono.
func (h *MessageHandler) Get(w http.ResponseWriter, r *http.Request) {
msg, err := h.messages.Find(r.Context(), r.PathValue("id"))
if err != nil {
writeError(w, statusFor(err), "could not load message")
return
}
writeJSON(w, http.StatusOK, msg)
}
func main() {
mux := http.NewServeMux()
mux.HandleFunc("GET /api/messages/{id}", handler.Get)
// ... server with timeouts, graceful shutdown (see Lifecycle)
}Why goroutines stay cheap: the M:N scheduler
The runtime multiplexes many goroutines (G) onto a few OS threads (M) through GOMAXPROCS logical processors (P), by default one per available CPU core.
When a goroutine waits on the network, it is parked and its thread picks up another runnable goroutine. The runtime’s network poller (epoll on Linux, kqueue on macOS) wakes it when data arrives. When a goroutine makes a blocking system call, the runtime hands its processor to another thread so the rest keep running.
Idle goroutines cost only their stack, a few kilobytes. Tens of thousands of in-flight requests fit comfortably in memory.
Spring MVC, platform threads
WebFlux: a few event-loop threads
Spring MVC, virtual threads
Go: a goroutine per request
Comparing the models
| Spring MVC + platform threads | Spring MVC + virtual threads | Go net/http | |
|---|---|---|---|
| Unit of work | Spring MVC + platform threadsOS thread (~1 MB stack reserved) | Spring MVC + virtual threadsVirtual thread (heap-allocated stack) | Go net/httpGoroutine (few-KB growable stack) |
| While waiting on I/O | Spring MVC + platform threadsThread blocked | Spring MVC + virtual threadsUnmounted from its carrier | Go net/httpParked; thread runs others |
| Concurrency limit | Spring MVC + platform threadsThread pool size (200) | Spring MVC + virtual threadsMemory, and downstream pools | Go net/httpMemory, and downstream pools |
| Code style | Spring MVC + platform threadsBlocking | Spring MVC + virtual threadsBlocking | Go net/httpBlocking |
| Cancellation | Spring MVC + platform threadsInterrupts, per-library timeouts | Spring MVC + virtual threadsInterrupts, per-library timeouts | Go net/httpcontext.Context everywhere |
| Caveats | Spring MVC + platform threadsTune pool sizes | Spring MVC + virtual threadsPinning in native calls, ThreadLocal cost, library readiness | Go net/httpBound fan-out yourself |