Skip to content
JavaGo
Part VI · Building Services

HTTP Concurrency Model

Thread-per-request, reactive WebFlux, virtual threads and Go’s goroutine-per-request: how each model handles slow I/O, and what that means for your code.

Section 25 of 31 4 min read

How a server handles thousands of concurrent requests that each wait on a database shapes how you write every handler. The Java world has had three answers over the years. Go has had one since day one.

Four concurrency models

Spring MVC on platform threads: each request occupies a Tomcat worker thread (200 by default) until the response is written, including all the time spent waiting on I/O. Simple code; thread count caps concurrency.

Spring WebFlux (reactive): a few Netty event-loop threads serve every request with non-blocking I/O. High concurrency, but every layer must return Mono/Flux and nothing may block.

Spring MVC on virtual threads (Java 21+): set spring.threads.virtual.enabled=true and each request runs on a cheap virtual thread that unmounts while blocked. Blocking code, high concurrency.

Go net/http: every request is served by its own goroutine. Blocking calls park the goroutine and free the OS thread. Blocking code, high concurrency, and the only model the ecosystem has ever used.

Spring MVC: a thread per request

each request holds a Tomcat worker for its whole lifetime
@RestController
@RequestMapping("/api")
class MessageController {
    private final MessageService service;

    MessageController(MessageService service) {
        this.service = service;
    }

    @GetMapping("/messages/{id}")
    Message get(@PathVariable long id) {
        // On platform threads, this worker sits idle while the
        // database answers. With server.tomcat.threads.max=200,
        // request 201 waits in the worker executor's queue.
        return service.findById(id);
    }
}

Spring WebFlux: reactive all the way down

non-blocking, but every layer must be reactive
@RestController
@RequestMapping("/api")
class MessageController {
    private final ReactiveMessageService service;

    MessageController(ReactiveMessageService service) {
        this.service = service;
    }

    @GetMapping("/messages/{id}")
    Mono<ResponseEntity<Message>> get(@PathVariable long id) {
        return service.findById(id)          // R2DBC, not JDBC
            .map(ResponseEntity::ok)
            .defaultIfEmpty(ResponseEntity.notFound().build());
    }
    // One accidental blocking call inside the chain stalls an
    // event-loop thread, and every request queued behind it.
}

Go: a goroutine per request

plain blocking code, served concurrently
type MessageHandler struct {
    messages *message.Service
}

// net/http runs this in a goroutine of its own: one per connection
// on HTTP/1.x, one per request stream on HTTP/2.
// While Find waits on Postgres, the goroutine is parked and its
// OS thread runs other goroutines. No callbacks, no Mono.
func (h *MessageHandler) Get(w http.ResponseWriter, r *http.Request) {
    msg, err := h.messages.Find(r.Context(), r.PathValue("id"))
    if err != nil {
        writeError(w, statusFor(err), "could not load message")
        return
    }
    writeJSON(w, http.StatusOK, msg)
}

func main() {
    mux := http.NewServeMux()
    mux.HandleFunc("GET /api/messages/{id}", handler.Get)
    // ... server with timeouts, graceful shutdown (see Lifecycle)
}

Why goroutines stay cheap: the M:N scheduler

The runtime multiplexes many goroutines (G) onto a few OS threads (M) through GOMAXPROCS logical processors (P), by default one per available CPU core.

When a goroutine waits on the network, it is parked and its thread picks up another runnable goroutine. The runtime’s network poller (epoll on Linux, kqueue on macOS) wakes it when data arrives. When a goroutine makes a blocking system call, the runtime hands its processor to another thread so the rest keep running.

Idle goroutines cost only their stack, a few kilobytes. Tens of thousands of in-flight requests fit comfortably in memory.

Spring MVC, platform threads

WebFlux: a few event-loop threads

Spring MVC, virtual threads

Go: a goroutine per request

Fig. 25.1Java's three answers, and Go's one, to thousands of requests waiting on a database. Spring MVC on platform threads holds a worker thread for the whole wait. WebFlux keeps a few event-loop threads busy with callbacks, so nothing may block. Spring MVC on virtual threads and Go both park the waiting request and let the thread run another.

Comparing the models

Spring MVC + platform threadsSpring MVC + virtual threadsGo net/http
Unit of workSpring MVC + platform threadsOS thread (~1 MB stack reserved)Spring MVC + virtual threadsVirtual thread (heap-allocated stack)Go net/httpGoroutine (few-KB growable stack)
While waiting on I/OSpring MVC + platform threadsThread blockedSpring MVC + virtual threadsUnmounted from its carrierGo net/httpParked; thread runs others
Concurrency limitSpring MVC + platform threadsThread pool size (200)Spring MVC + virtual threadsMemory, and downstream poolsGo net/httpMemory, and downstream pools
Code styleSpring MVC + platform threadsBlockingSpring MVC + virtual threadsBlockingGo net/httpBlocking
CancellationSpring MVC + platform threadsInterrupts, per-library timeoutsSpring MVC + virtual threadsInterrupts, per-library timeoutsGo net/httpcontext.Context everywhere
CaveatsSpring MVC + platform threadsTune pool sizesSpring MVC + virtual threadsPinning in native calls, ThreadLocal cost, library readinessGo net/httpBound fan-out yourself
↑ ↓ to navigateEnter to openEsc to close