[repost ]Jetty and Netty: I can’t get more than 2.5K requests/sec


I’m trying to find a Java webserver that can handle 10 requests/sec from 1,000 clients. Doesn’t need to be good at (or even able to) serve files–I just want to be able to respond to request with some Java code that says “hello world”.

I’ve been assuming that it has to be a NIO based framework that does not have a thread-per-request concurrency model so that I can hanging-GETs with a lot of clients.

So far, I’ve tried Jetty 7 (which supports a kind of asynchronous Servlet framework for hanging-GETs) and Netty 3 (a NIO framework with a bare-bones webserver). I’ve tried running both on my iMac and using JMeter on a laptop (same network, but over WiFi) to hammer the server. I can’t seem to get more than about 2.5K requests/second out of either.
I suspect I’m doing something wrong.
Things I’ve tried:
  • Playing with VM options. There’s lots more to try here. So far I’m just using: -XX:+UserParallelGC -server
  • Binding to multiple ports in the server and balancing between them on the client. (Tried this on Netty based on this thread about a bottleneck in the accept thread)
  • Increasing the max open files on client and server (ulimit didn’t work as expected on the Mac, used “sysctl -w kern.maxfiles=XXXX sysctl -w kern.maxfilesperproc=XXXX” instead)
  • Changing the number of clients in the load test; over 300 seems to get lots of errors in both cases. Under 100 hurts the aggregate throughput (no client seems able to get more than 20-30 serial requests/second)
Apache (the default web server on the Mac) can server about 1.5K requests/sec untweaked. I’d expect a “hello world” server that does non-blocking IO to come out farther ahead of Apache than what I’m getting.
I thought maybe JMeter was the bottleneck so I installed Tsung, an Erlang-based, load test tool. It was too difficult to understand their config environment and I wasn’t able to get a simple number (aggregate requests/sec) out of it. I’m sure it does this, I just couldn’t figure it out and JMeter is so much easier I stuck with it.
Dear Lazy Web, do you have any hints?