tomcat-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frances Wang <Frances.W...@asg.com>
Subject need help with isapi and ntdll problem for windows 2008 R2/tomcat 6
Date Tue, 01 Feb 2011 22:42:48 GMT
Hello,

We are running Tomcat 6.0.26 on Windows Server 2008 R2 Enterprise 64 Bit using IIS and isapi
redirector version 1.2.30 (just upgraded to 1.2.31).  The 6 months or so we have been using
Tomcat  have been plagued with intermittent periods crashes and instability.  Specifically,
tomcat requests are resulting in crashes and the application log shows numerous (over 1200)
instances of error 1000:
Faulting application name: w3wp.exe, version: 7.5.7600.16385, time stamp: 0x4a5bd0eb
Faulting module name: ntdll.dll, version: 6.1.7600.16385, time stamp: 0x4a5be02b
Exception code: 0xc0000005
Fault offset: 0x000000000004d174
Faulting process id: 0x1254
Faulting application start time: 0x01cbc19aceedc9cd
Faulting application path: c:\windows\system32\inetsrv\w3wp.exe
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
Report Id: 0d413cc2-2d8e-11e0-92d0-005056340288

We installed debug diag on the machine and opened a case with Microsoft since it seemed like
it was a MS problem.  After opening a case and sending MS several dump files for analysis
we were told:

We are failing during a call into isapi_redirect64.  It appears to be passing a bad value
for a critical section to NTDLL (0x00000000`00000003) which in turn is causing the Access
Violation to occur.


Here is the complete email response from MS with some of the data from the dumps:

All of the dumps show the same issue:



Child-SP          RetAddr           Call Site

00000000`019bd040 00000000`7734d0a2 ntdll!RtlpWaitOnCriticalSection+0xb4

00000000`019bd0f0 00000000`6a6d1c41 ntdll!RtlEnterCriticalSection+0xab

00000000`019bd120 00000000`6a6bd40c isapi_redirect64!jk_md5+0x1c1 00000000`019bd150 00000000`6a6ebbb7
isapi_redirect64!ajp_service+0xb4c

00000000`019bd1e0 00000000`6a6c679a isapi_redirect64!uri_worker_map_open+0x2c7

00000000`019bd240 000007fe`f31bc540 isapi_redirect64!read_registry_init_data+0x89a

00000000`019be520 000007fe`f31c4007 isapi!ProcessIsapiRequest+0x13c 00000000`019be580 000007fe`f31bb421
isapi!W3_ISAPI_HANDLER::DoWork+0x4fb

00000000`019be690 000007fe`f31bb964 isapi!RequestDoWork+0x40d 00000000`019be720 000007fe`f3db2fa7
isapi!CIISHttpModule::OnExecuteRequestHandler+0x1c

00000000`019be760 000007fe`f3db464e iiscore!NOTIFICATION_CONTEXT::RequestDoWork+0x233

00000000`019be7a0 000007fe`f3db4465 iiscore!NOTIFICATION_CONTEXT::CallModulesInternal+0x17f

00000000`019be890 000007fe`f3db5e56 iiscore!NOTIFICATION_CONTEXT::CallModules+0x25

00000000`019be8e0 000007fe`f3de51bd iiscore!W3_CONTEXT::DoWork+0x3a6 00000000`019bec00 000007fe`f3de55df
iiscore!W3_MAIN_CONTEXT::StartNotificationLoop+0x5d

00000000`019bec30 000007fe`f319551e iiscore!W3_CONTEXT::ExecuteRequest+0x2bf

00000000`019becb0 000007fe`f3191060 filter!RequestDoWork+0x40be 00000000`019bf360 000007fe`f3db6ac1
filter!CIISHttpModule::OnPostAuthenticateRequest+0x20

00000000`019bf3a0 000007fe`f3db464e iiscore!NOTIFICATION_CONTEXT::RequestDoWork+0x20a

00000000`019bf3e0 000007fe`f3db4465 iiscore!NOTIFICATION_CONTEXT::CallModulesInternal+0x17f

00000000`019bf4d0 000007fe`f3db593d iiscore!NOTIFICATION_CONTEXT::CallModules+0x25

00000000`019bf520 000007fe`f3db8ede iiscore!W3_CONTEXT::DoWork+0x326 00000000`019bf840 000007fe`f3da10d2
iiscore!W3_MAIN_CONTEXT::OnNewRequest+0x1ea

00000000`019bf870 000007fe`f3da109c w3dt!UL_NATIVE_REQUEST::DoWork+0x126

00000000`019bf8d0 000007fe`f4581fba w3dt!OverlappedCompletionRoutine+0x1c

00000000`019bf900 000007fe`f4582024 w3tp!THREAD_POOL_DATA::ThreadPoolThread+0x7a

00000000`019bf950 000007fe`f45820a1 w3tp!THREAD_POOL_DATA::ThreadPoolThread+0x34

00000000`019bf980 00000000`770ff56d w3tp!THREAD_MANAGER::ThreadManagerThread+0x61

00000000`019bf9b0 00000000`77333281 kernel32!BaseThreadInitThunk+0xd 00000000`019bf9e0 00000000`00000000
ntdll!RtlUserThreadStart+0x1d



We are failing during a call into isapi_redirect64.  It appears to be passing a bad value
for a critical section to NTDLL (0x00000000`00000003) which in turn is causing the Access
Violation to occur.



This is a TomCat redirector from the Apache Foundation:



0:004> lmvm isapi_redirect64

start             end                 module name

00000000`6a6b0000 00000000`6a714000   isapi_redirect64   (export symbols)       isapi_redirect64.dll

    Loaded symbol image file: isapi_redirect64.dll

    Image path: D:\isapiRedirect\isapi_redirect64.dll

    Image name: isapi_redirect64.dll

    Timestamp:        Thu Sep 09 18:04:53 2010 (4C895A05)

    CheckSum:         00062701

    ImageSize:        00064000

    File version:     1.2.30.0

    Product version:  1.2.30.0

    File flags:       0 (Mask 3F)

    File OS:          40004 NT Win32

    File type:        1.0 App

    File date:        00000000.00000000

    Translations:     0409.04b0

    CompanyName:      Apache Software Foundation

    ProductName:      Apache Tomcat Connectors project

    InternalName:     isapi_redirect-1.2.30

    OriginalFilename: isapi_redirect-1.2.30.dll

    ProductVersion:   1.2.30

    FileVersion:      1.2.30

    FileDescription:  Apache Tomcat IIS Redirector



You will need to follow up with the vendor of that product for further details on what is
causing the error.
----end MS response-----------

We have not been able to figure out what is causing this problem.  It is intermittent - we
have gone as long as a month between incidents and had it happen 5 or 6 times in a day.  
 Searching the list archives we found several instances of people with similar problems, but
none with solutions.  We also found an one issue in ASF Bugzilla which appears similar:
https://issues.apache.org/bugzilla/show_bug.cgi?id=45063
However, we are using the same identity ("ApplicationPoolIdentity") to run all of our app
pools, so that one doesn't quite match.  We are also on different version of windows/IIS.

We have just upgraded the redirector to version 1.2.31.  Based on the change log for that
version we don't expect that it will fix the problem but we will monitor and report if it
does.

Any help would be appreciated.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message