tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Preißer <kpreis...@apache.org>
Subject Status of the current IIS ISAPI Redirector for Tomcat
Date Fri, 24 Jan 2014 19:06:01 GMT
Hi all,

for my Java Servlet web applications which run on Tomcat (currently 8.0.0-RC 10) on various
Windows Server OSes (currently Windows Server 2012 R2), I use the ISAPI Redirector to forward
requests from IIS to Tomcat over AJP. I use IIS as primary web server because I also host
other websites that use different technologies like ASP.Net and PHP (and because IIS allows
to run web applications as different processes as different user accounts, and because I can
configure the SSL settings over IIS, and so on).

The ISAPI Redirector has its job done well in the past and currently I'm still using it. Note
that I'm only using it to forward requests from a single IIS instance to a single Tomcat instance,
but not for load-balancing or other features.


However, over the time I found some issues which seem to result from the changes that Win
Server, IIS and other components have experienced over time, which I wanted to list here and
see how these could be changed.

A possibility that I see is to use an ASP.Net (C#) based redirector instead of an ISAPI based
redirector as that will have a number of advantages - see below.


1.

The ISAPI Redirector seems to be quite complicate to configure. You have to:
1) place the ISAPI redirector DLL in some arbitrary path (the docs suggest to place them in
your Tomcat\bin directory)
2) create a virtual directory in your IIS web application which points to this path
3) change the handler settings for the virtual directory to allow to execute ISAPI dlls
4) add the ISAPI redirector DLL to the list of CGI and ISAPI restrictions in IIS
5) add the ISAPI redirector DLL to your web app as ISAPI filter
6) create some registry entries at HKLM\Software\Apache Software Foundation\Jakarta Isapi
Redirector\1.0 to specify the path of the virtual directory, path to configuration files etc.
7) create configuration files (uriworkersmap.properties, worker.properties) and but them in
some arbitrary path (the docs suggest to place them in your Tomcat\conf directory)

I see a few problems here.
First, you have to place the ISAPI redirector DLL in some external arbitrary path. This can
introduce additional maintenance issues as you always have to remember this when e.g. moving
the server. Because the docs suggest to place them in your Tomcat\bin directory, you might
delete that file by mistake when you delete your Tomcat installation and create a new one.
The same is true for the config files - if you place them in your Tomcat directory, you might
delete them when you change your Tomcat config.

Normally, these files do not belong to Tomcat, but to the ISAPI redirector, so I would expect
to place them somewhere in your IIS web application.

E.g, ASP.Net web applications have a "web.config" file in their root directory for configuration,
and a "bin" directory where .Net assemblies can be placed. If you were using an ASP.Net based
redirector for example (implemented as a managed module), you can place the binary into the
"bin" directory of your IIS webapp and configure it by adding it to the web.config file. This
would also mean that you don't have to create a virtual directory any more.

I also got problems with the system-wide registry keys that you need to set up for the ISAPI
redirector, as they don't allow separate configs for different IIS webapps. I once tried to
create a "isapi_redirect.properties" with the configuration (instead of using the registry)
like it is described on the Tomcat Connectors IIS reference page [1], but I didn't have success,
so I reverted back to the registry settings.

Note also that Microsoft has deprecated ISAPI filters and extensions in favor of native http
modules [2].


2.

When using the ISAPI redirector with IIS 7, I got a few problems with response buffering -
it seemed that IIS buffered the complete response body before starting to send it to the client.
However, when I tried this again on IIS 8.5 today, then IIS only buffered a few MB before
sending it to the client, so I think this is not a problem any more.
Note that you can manually specify the amount that IIS should buffer, by adding the following
in web.config (in <configuration> , <system.webServer>):

        <handlers>
            <remove name="ISAPI-dll" />
            <add name="ISAPI-dll" path="*.dll" verb="*" modules="IsapiModule" resourceType="File"
requireAccess="Execute" allowPathInfo="true" responseBufferLimit="1" />
        </handlers>


3.

Chunked encoding is disabled by default.
Personally I think every web server should use chunked encoding if the client specifies connection:
keep-alive and the content-length is unknown, so that the server doesn't have to close the
connection to signal EOF. It seems that this is a bit difficult in an ISAPI extension as this
feature was being considered experimental until a few years ago.

However, for example if you code a managed HTTP module using C#, you can just write bytes
to the response without having to think about chunked encoding, as IIS will do automatically
 - it is pretty much like writing to the response in a Java Servlet. However I don't know
how this would be done in native HTTP modules for IIS.


4.

By default, IIS applies a limit to POST requests that specify a Content-Length and its value
exceeds a specific size. This means IIS will reject such POST requests before they reach Tomcat
(which might allow such a POST request). You can probably change the POST limit in IIS to
allow bigger request bodys. 
E.g. In a managed module, since .Net 4.5 you can call Request.GetBufferlessInputStream(true)
[3] to retrieve a bufferless InputStream to read the request body, and disable the request
length limit.


5.

Sometimes, the ISAPI redirector stops to work - a request times out and IIS is not able to
shutdown the worker process (w3wp.exe) that uses the ISAPI redirector.
Normally, this does not happen when no configuration is changed on the server.

However, in my case, I have multiple IIS web applications that use the ISAPI redirector (using
the same AppPool), and one web application that uses another app pool and uses PHP via FastCGI.
Now, everything works. However, if I now open the IIS manager and go to the FastCGI settings,
then double-click on  the php-cgi.exe entry and change e.g. the maximum number of instances,
then the ISAPI redirector suddenly stops working... This means IIS will not answer a request
that goes to this web application (or even to another app with the same apppool - I haven’t
tested this). There is no CPU usage.
Sending a request to a webapp with another apppool (the one with PHP) still works.

I have no idea why this happens (and how changing settings for a FastCGI application can impact
the ISAPI redirector), but the only way to get things working again is to kill the w3wp.exe
process so that IIS starts a new one - then everything will work again.

I think with managed or native IIS modules, such strange issues shouldn't happen.   ;-)


6.

As far as I can see, the ISAPI redirector uses blocking I/O when forwarding requests to Tomcat.

This means when a slow client sends a request to IIS which gets forwarded to Tomcat, and Tomcat
starts to send the response, in the IIS worker process at least 1 Thread will be dedicated
to this request as long as it is not finished. This means if 500 slow clients concurrently
send requests to IIS (which get forwarded), IIS has to create 500 threads in the w3wp.exe.
This can be a problem in terms of scalability, if lots of clients send concurrent requests
to your server.

The recently released Servlet 3.1 spec allows to use non-blocking I/O when reading the request
body or writing the response body, so that Tomcat doesn't have to dedicate a thread for the
lifetime of the request, but only take one from a Therad pool when further data can be sent
to (or read from) the client. IIS however would still need to dedicate a thread for the request.
Another problem is that at least IIS 8 only seems to create additional threads very slowly
(1 Thread per second) when its thread pool is exhausted, which can happen if a few slow clients
concurrently send requests handled by the ISAPI redirector. So some of the clients would have
to wait very long until a thread is created which can handle their request.

Now, e.g. .Net 4.5 introduces a new async programming model that allows you to write code
in blocking/synchronous style, but actually behave as async code (using non-blocking I/O if
possible). If you use such coding for a managed module to forward requests (and use an async
managed module), then IIS only needs to take a thread when there's actually code to run for
sending additionally data. This means that most of the time the request doesn't need a thread,
so it should be much more scalable than the blocking model.

I once wrote a SPDY redirector for IIS based on C# (ASP.Net) that forwards request over an
SPDY connection, and after switching it to Async I/O, for 170 concurrent requests the w3wp.exe
only had ~ 35 threads, instead of ~195 threads with blocking I/O [4].

SPDY allows to multiplex requests on a single TCP connection so this combination should allow
for a high number of concurrent requests without needing a huge number of threads.
The SPDY redirector already works with Jetty's SPDY/3 implementation, but for Tomcat it seems
that it does not yet support SPDY/3 (Costin Manolache once wrote that he wanted to switch
Tomcat's SPDY implementation to SPDY/3, but I didn't see activity in this arey so I don't
know what the state of SPDY/3 for Tomcat is.

For AJP, however, a TCP connection can only server one request at a time, so while an async
AJP redirector would not require one thread per request, it still would require one TCP connection
per (concurrent) request.


7.

The ISAPI redirector handles requests to directories like WEB-INF by itself (rejecting it
with 404, or just disconnecting the client). However, if it is just forwarding requests, then
my point of view would be that it should not reject requests by itself, but instead let this
do Tomcat (or whatever AJP server is behind), because Tomcat needs to also do this if you
directly connect to it over HTTP.
I can see a reason for rejecting requests to WEB-INF at IIS level if IIS it is configured
to serve static resources directly, and only forward requests to Tomcat. However, personally
I never to this due to the security implications, and the performance benefit for IIS serving
static resources directly instead of letting Tomcat serve them was rather not noticeable for
me.


8.

In the past there have been reports of problems with the ISAPI redirector when using IIS features
like "web garden" (when one app pool has multiple worker processes) or apppool recycling.
I can remember (but I do not have references for this atm) that Mladen Turk once said, one
should disable web garden (so only 1 process is used per app pool) and disable the apppool
recycling to avoid problems with the ISAPI redirector.
I think such problems shouldn't happen when using a managed or native IIS module.


9.

The 32-bit version of the ISAPI redirector does not work on 64-bit versions of Windows. Even
when you set the Apppool to allow to execute 32-bit applications (this means that the 32-bit
version of w3wp.exe will be executed), the 32-bit version of ISAPI redirector does not work.
I do not know why this is so, but e.g. as .Net is platform-independent, you can use the same
assembly (dll) for 32-bit as well as for 64-bit apppools.


10.

Sometimes I see in the Windows event log erros that IIS rejected a request due to bad characters,
but the URL that is logged is actually a good one (e.g. http://myserver.com/myurl?param=value).
If I request this URL, it is correctly server, and no new entry is created in the event log.
If I look in the IIS logs for this URL and the logged time, I only find "200 OK" entries so
it seems this URL was correctly served.

My guess is that it was actually a different URL that failed, but the ISAPI redirector somehow
had overwritten some structure containing the URL so that IIS would log this one instead.
However this is only very rarely so I cannot verify what actually happens there.

(Fin)



Now, I think a future-proof redirector can be one that uses SPDY (or a similar protocol) to
redirects the requests instead of AJP, because
1) it allows multiple concurrent requests per TCP connection, which in conjunction with Async
I/O should allow for high scalability, and
2) it allows to forward WebSocket requests.
However, there are some issues with SPDY/3 (NPN and header compression) that put a few barriers
for writing a simple request redirector.

Some time ago, I have written such an SPDY redirector based in C# on a managed IIS module
[4]. It already works well with Jetty (but Jetty does not yet support WebSocket over SPDY/3).
However, as Tomcat seems not yet to support SPDY/3, it cannot be used with Tomcat. Another
problem is that SPDY/3 is still a draft of the SPDY spec which probably change a few times
in the future (Google is already working on SPDY/4). For a real usable SPDY redirector, one
would probably have to wait until SPDY is final to have a common basis for using it with different
SPDY servers.


Before the SPDY redirector, I also wrote a very simple AJP redirector in C# (but still with
blocking I/O etc.) for which I only needed ~ 1 day so it should be very easy to write a AJP
based replacement for the current ISAPI redirector.

Personally, I would like to get rid of the ISAPI based redirector for Tomcat in the near future
due to the above mentioned issues. My preference would be to continue implementing the C#-based
AJP redirector (but it will probably need a rewrite), or if Tomcat already supports SPDY/3
in that time, use the SPDY redirector.

However, I'm wondering about the ISAPI redirector which is available at the Tomcat website.
From the docs one can see that the ISAPI redirector is already a bit old as they still talk
about Windows NT 4.0 and Windows 98 (and PWS). What do you think about this? Should the ISAPI
redirector still stay? Is SPDY the right way for forwarding requests to Tomcat in the future?

Thanks!


Regards,
Konstantin Preißer



[1] http://tomcat.apache.org/connectors-doc/reference/iis.html
[2] http://forums.iis.net/t/1189728.aspx
[3] http://msdn.microsoft.com/de-de/library/hh195568.aspx
[4] http://markmail.org/message/ojknieujkcchxqla
[5] http://markmail.org/message/livcbkoa4b7bl7yq


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Mime
View raw message