tomcat-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Preißer <kpreis...@apache.org>
Subject RE: Possible IIS SPDY Redirector for Tomcat
Date Fri, 18 Oct 2013 18:04:40 GMT
Hi all,

I would like to let you know some updates on this:

1) Results of performance measurement with various configurations and the SPDY-Redirector
2) Switching to Async I/O for the implementation of the IIS module


1) Performance

> -----Original Message-----
> From: Christopher Schultz [mailto:chris@christopherschultz.net]
> Sent: Monday, September 30, 2013 8:53 PM
> To: Tomcat Developers List
> Subject: Re: Possible IIS SPDY Redirector for Tomcat
> 
> Konstantin,
> 
> > So it seems that the main performance problem is IIS when using
> > managed code/ASP.Net to write to the response, but I need to do
> > additional testing.
> 
> Wow. Are you sure the thread burning the CPU is the one running the
> above code? Seems ... unfortunate.

I have now done additional testing with various configurations and I think my previous statement
was wrong - IIS is not the culprit here. I guess it's the multiplexing that the SPDY Redirector
is doing (multiplexing multiple HTTP requests on a single TCP connection) that causes the
CPU usage. However, I don't think that there is a real problem here comparing the performance/CPU
usage to Jetty.

To measure performance (speed and CPU usage), I set up a 2-CPU VM with IIS 8.5 (Windows Server
2012 R2 x64), Tomcat 7.0.42 (for ISAPI Redirector) and Jetty 8.1.13 (for SPDY Redirector).
For IIS, I created one web application using the SPDY Redirector (connecting to Jetty) and
one web application using the ISAPI Redirector (connecting to Tomcat).
I then created a Servlet 3.0 webapp with a Servlet that produces ~ 700 MB of random data,
sending it in 32 KB chunks to the client, using the following code:

    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws
ServletException, IOException {
        response.setContentType("text/plain");
        response.setCharacterEncoding("UTF-8");
        response.addHeader("Content-Disposition", "attachment; filename=\"myTextfile.txt\"");

        byte[] bytes = new byte[32768];
        for (int i = 0; i < bytes.length; i++) {
            bytes[i] = (byte)'a';
        }
        
        long bytesWritten = 0;
        long lastDisplay = 0;
        try (OutputStream out = response.getOutputStream()) {
            while (true) {
                out.write(bytes);
                bytesWritten += bytes.length;
                
                if (bytesWritten - lastDisplay > 500 * 1024) {
                    lastDisplay = bytesWritten + 500 * 1024;
                    System.out.println(bytesWritten + " Bytes written (completely).");
                }
                
                if (bytesWritten > 700 * 1024 * 1024)
                    break;
            }
        }
    }


Then, I created a C# application that creates a TCP connection and sending the following raw
request:
GET /TestSpdyServer/BigFileProducerServlet HTTP/1.1
Host: localhost
Connection: close

and reads from the TCP connection until it is closed, counting the bytes and milliseconds
to calculate the average speed.
I then tested the speed when 1) connecting directly to Jetty (HTTP), 2) connecting directly
to Tomcat (HTTP), 3)+4) connecting to IIS with SPDY-Redirector, 5) connecting to IIS with
ISAPI redirector, 6) connecting to IIS using a ASP.Net handler producing 700 MB of random
data, equally to the Servlet above.
This is what I got:

1) JettyHttp-Direct: 260 MB/s (Jetty: 47% CPU)
2) TomcatHttpBio-Direct: 275 MB/s (Tomcat: 45% CPU)
3) Iis-SpdyRedirector(Async)-JettySpdy (64 KB initial window size): 170 MB/s (Jetty: 35% CPU,
IIS: 43% CPU)
4) Iis-SpdyRedirector(Async)-JettySpdy (128 KB initial window size): 205 MB/s (Jetty: 40%
CPU, IIS: 50% CPU)
5) Iis-IsapiRedirector-TomcatAjpBio: 190 MB/s (Tomcat: 10% CPU, IIS: 20% CPU)
6) Iis-Asp.Net-Handler-Direct: 400 MB/s, 50% CPU

As you can see, with the SPDY redirector, both Jetty and IIS have a high amount of CPU usage
(probably the costs of multiplexing) whereas the ISAPI redirector had a low CPU usage. However,
I'm pretty happy with the current performance of the SPDY redirector.


2) Switching the implementation of the IIS module to Async I/O - see [1]: "Scalable Apps with
Asynchronous Programming in ASP.NET"

.Net 4.5 introduces new "async" and "await" keywords for C# that allow you to easily create
asynchronous code. E.g. you can write code just as you would with synchronous operations,
and the compiler will create the Async State Machine for you.

As the previous implementation of the SPDY Redirector used blocking/synchronous IO for the
IIS module, it meant that 1 thread was used per request. Now imagine you have 200 clients
that simultaneously do requests to a resource that produces the response body very slowly,
it would mean that IIS had to create 200 threads for serving the requests. This can be a problem
also e.g. for long-polling comet applications (where it may take some time until the HTTP
response is sent) or Websocket connections (however WebSockets with IIS 8 can only be used
with async IO so I had to do the switch to Async anyway for being able to support Websocket).

For example, with Servlet 3.1 you can use non-blocking I/O so that you don't need to acquire
a thread for the complete duration of a request.

Therefore, I now switched the IIS module implementation of the SPDY redirector to async IO
(with the "async" and "await" keywords this was rather simple). The result is that if a request
takes a long time to complete, IIS does not need to acquire a thread for the complete duration
of the request - instead it uses a thread from its Threadpool only if there is new data to
be forwarded. (The underlying SPDY connector still uses blocking I/O for the SPDY connection,
but since one SPDY connection is intended to multiplex a huge amount of HTTP requests, atm
there is no need for the SPDY connector to use async I/O for the SPDY TCP connection).


For example, the ISAPI redirector seems to use blocking I/O when forwarding HTTP requests
to Tomcat. I tested this by creating a servlet that writes 100 lines of text and waiting 10
seconds after writing each line. Then, I made a C# application creating 170 TCP connections
and simultaneously sending a request on each connection.

Then I monitored the number of threads of the IIS worker processes (w3wp.exe) before and after
sending the requests:
IIS with SPDY Redirector: 29 Threads -> 35 Threads
IIS with ISAPI Redirector: 26 Threads -> 195 Threads (the additional threads are also created
very slowly - I saw 1 thread per second - which means there is additional waiting time for
the clients)

As you can see, with the async implementation of the IIS module of the SPDY redirector the
number of threads did not significantly increase when doing 170 concurrent requests (initially
I wanted to try more, but Jetty does not allow this - after about 180 requests are created
on the SPDY connection, Jetty simply sends a GOAWAY frame and closes the TCP connection, effectively
aborting all running HTTP requests).

Also, as AJP needs one TCP connection for each request, doing 170 concurrent requests means
the ISAPI redirector has to establish 170 separate TCP connections to Tomcat, whereas the
SPDY redirector only needs a single TCP connection where it can multiplex the concurrent requests.


The SPDY Redirector does not yet support redirecting WebSocket connections, but I think I
will look into the "WebSocket Layering over SPDY/3" [1] document (and the Websocket protocol
spec) to implement redirecting Websocket connections (Jetty 8.1 seem to support Websockets
while it is only Servlet 3.0, whereas Jetty 9.1 should support Servlet 3.1 but I was not yet
able to create a non-SSL SPDY/3 connector with it).


If you would like to test the SPDY redirector with a downloadable version of IIS 8 (that runs
also on client Windows editions), you can find the instructions for setting it up at [2].
I would appreciate if someone could test the SPDY redirector for some feedback.


What do you think?


Thanks!

Regards,
Konstantin Preißer


[1] http://msdn.microsoft.com/en-us/magazine/cc163463.aspx
[2] http://markmail.org/message/atwky5b4qpm52qq3


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


Mime
View raw message