lucenenet-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alastair Todd <alastair.t...@gmail.com>
Subject Re: IIS Crash
Date Mon, 23 Apr 2012 13:56:04 GMT
Well we have a 4-node farm and one of these is currently taken out of the
farm. This one is currently running in "broken" mode.

I will see if Process Explorer throws anything up.

Thanks

On 23 April 2012 14:49, Simon Svensson <sisve@devhost.se> wrote:

> I believe you've configured your debugger to break at first exception, so
> the dump won't show anything else.
>
> I would wait until the problem appears again and look into different
> performance counters. I've seen similar scenarios with memory leaks where
> "% time in gc" (category .net clr memory) indicates that it spends a lot of
> time collecting garbage. This would, depending on the gc implementation in
> use (I don't recall which one IIS uses by default) either slow down your
> application, or completely pause it while collecting occurs, which would
> match what you describe.
>
> You should also look into cpu usage and memory usage. The kind of memory
> leak I'm talking about would cause high amount of cpu usage (garbage
> collection) and high amount of memory usage, usually enough to start
> swapping away like crazy.
>
> You could use a tool like Process Explorer (from Sysinternals) to generate
> a dump from a running program, and windbg to dive into the generated dump.
>
> There's nothing so far that indicates that Lucene is the culprit. Are you
> using any other non-bcl libraries?
>
> On 23 apr 2012, at 15:37, Alastair Todd wrote:
>
> > OK< sorry for the double post - wasnt sure if the first one went through
> as
> > it was before I'd subscribed.
> >
> > Anyway, I don't think its a prob posting the dmp - if i do in fact have
> > this. Is this the .dmp file generated by the debug diagnostic tool? If so
> > its like 186Mb zipped.
> >
> > The site stops responding to requests eventually (browser sits
> > there indefinitely loading)  and only an app pool recycle fixes it. And
> > we've no set of actions to manually trigger the "crash".
> >
> >
> >
> > On 23 April 2012 14:26, Simon Svensson <sisve@devhost.se> wrote:
> >
> >> Hi,
> >>
> >> This is a common exception which is handled within the QueryParser
> itself.
> >> You're seeing a first chance exception which does not cause the crashes
> >> you're experiencing.
> >>
> >> Is this from a production system? Can you share the raw memory dump, or
> >> does it contain sensitive information?
> >>
> >> // Simon
> >>
> >> On 23 apr 2012, at 15:21, Alastair Todd wrote:
> >>
> >>> I have been investigating an IIS 7.5 crashing website which only gets
> >> fixed
> >>> by an app pool recycle.
> >>>
> >>> I have used DebugDiag to catch crash exceptions. The problem appears to
> >> be
> >>> manifesting itself in Lucene.Net.
> >>>
> >>> I am confused as to why this would fatally crash IIS. Should we be
> >> wrapping
> >>> a try catch around the call and throwing our own exception?
> >>>
> >>> Any "normal" website exception would normally be bubbled up and logged
> by
> >>> ELMAH, but this error never finds its way into the ELMAH log (makes
> sense
> >>> since it crashes IIS).
> >>>
> >>> How would you go about keeping this site up?
> >>>
> >>> Here's the report:
> >>>
> >>> Report for W3WP__~1.DMP
> >>> Type of Analysis Performed   Crash Analysis
> >>> Machine Name   WEB01
> >>> Operating System   Windows Server 2008 Service Pack 2
> >>> Number Of Processors   8
> >>> Process ID   20048
> >>> Process Image   C:\Windows\SysWOW64\inetsrv\w3wp.exe
> >>> System Up-Time   214 day(s) 23:40:43
> >>> Process Up-Time   00:44:32
> >>>
> >>>
> >>> Thread 34 - System ID 22368
> >>> Entry point   mscorwks!Thread::intermediateThreadProc
> >>> Create time   23/04/2012 11:49:25
> >>> Time spent in user mode   0 Days 0:0:16.582
> >>> Time spent in kernel mode   0 Days 0:0:1.622
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> .NET Call Stack
> >>>
> >>>
> >>>
> >>> Function
> >>> Lucene.Net.QueryParsers.FastCharStream.Refill()
> >>> Lucene.Net.QueryParsers.FastCharStream.ReadChar()
> >>> Lucene.Net.QueryParsers.QueryParserTokenManager.JjMoveNfa_3(Int32,
> Int32)
> >>>
> >>
> Lucene.Net.QueryParsers.QueryParserTokenManager.JjMoveStringLiteralDfa0_3()
> >>> Lucene.Net.QueryParsers.QueryParserTokenManager.GetNextToken()
> >>> Lucene.Net.QueryParsers.QueryParser.Jj_scan_token(Int32)
> >>> Lucene.Net.QueryParsers.QueryParser.Jj_3R_2()
> >>> Lucene.Net.QueryParsers.QueryParser.Jj_3_1()
> >>> Lucene.Net.QueryParsers.QueryParser.Jj_2_1(Int32)
> >>> Lucene.Net.QueryParsers.QueryParser.Clause(System.String)
> >>> Lucene.Net.QueryParsers.QueryParser.Query(System.String)
> >>> Lucene.Net.QueryParsers.QueryParser.TopLevelQuery(System.String)
> >>> Lucene.Net.QueryParsers.QueryParser.Parse(System.String)
> >>>
> >>
> OurProject.Content.Search.SearchManager.GetModuleSearchResults(System.String)
> >>> Search.Page_Load(System.Object, System.EventArgs)
> >>> System.Web.Util.CalliHelper.EventArgFunctionCaller(IntPtr,
> >>> System.Object, System.Object, System.EventArgs)
> >>> System.Web.Util.CalliEventHandlerDelegateProxy.Callback(System.Object,
> >>> System.EventArgs)
> >>> System.Web.UI.Control.OnLoad(System.EventArgs)
> >>> System.Web.UI.Control.LoadRecursive()
> >>> System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)
> >>> System.Web.UI.Page.ProcessRequest(Boolean, Boolean)
> >>> System.Web.UI.Page.ProcessRequest()
> >>> System.Web.UI.Page.ProcessRequestWithNoAssert(System.Web.HttpContext)
> >>> System.Web.UI.Page.ProcessRequest(System.Web.HttpContext)
> >>> ASP.search_aspx.ProcessRequest(System.Web.HttpContext)
> >>>
> >>
> System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
> >>> System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)
> >>>
> >>
> System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)
> >>>
> >>
> System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest,
> >>> System.Web.HttpContext)
> >>>
> >>
> System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr,
> >>> IntPtr, IntPtr, Int32)
> >>> System.Web.Hosting.UnsafeIISMethods.MgdIndicateCompletion(IntPtr,
> >>> System.Web.RequestNotificationStatus ByRef)
> >>>
> >>
> System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr,
> >>> IntPtr, IntPtr, Int32)
> >>> System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr,
> >>> IntPtr, IntPtr, Int32)
> >>>
> >>>
> >>> Full Call Stack
> >>>
> >>>
> >>>
> >>> Function     Arg 1     Arg 2     Arg 3     Arg 4   Source
> >>> kernel32!RaiseException+58     e0434f4d     00000001     00000001
> >>> 1cc0ec08
> >>> mscorwks!RaiseTheExceptionInternalOnly+2a8     29af23c4     00000000
> >>> 00000000     c22aa5e5
> >>> mscorwks!JIT_Throw+fc     0000000f     29aefc04     29af12f0
> 7fffffff
> >>> Lucene.Net.QueryParsers.FastCharStream.Refill()     c22aa4c1
> >>> 00000000     29af12f0     00000000
> >>> mscorwks!JIT_NewArr1+1e3     00000000     29aef878     1cc0ee20
> >> 1cc0ede8
> >>> Lucene.Net.QueryParsers.QueryParser.Jj_scan_token(Int32)     29aef878
> >>>  1cc0edf8     017167c5     29af1310
> >>> Lucene.Net.QueryParsers.QueryParser.Jj_3R_2()     29af1310
> >>> 29aef878     1cc0ee30     0171672c
> >>> Lucene.Net.QueryParsers.QueryParser.Jj_3_1()     29af1310     29aef878
> >>>   00000000     00000002
> >>> Lucene.Net.QueryParsers.QueryParser.Jj_2_1(Int32)     29aef878
> >>> 29af1310     00000000     00000000
> >>> Lucene.Net.QueryParsers.QueryParser.Clause(System.String)     29aef1e8
> >>>   29ab4cf0     29aeb9a8     1cc0ef70
> >>> System.Web.UI.StateBag.get_Item(System.String)     0c9955e8
> >>> 1cc0ef94     6ff33d94     0c9955e8
> >>> System.Web.Util.CalliHelper.EventArgFunctionCaller(IntPtr,
> >>> System.Object, System.Object, System.EventArgs)     0c9955e8
> >>> 29ab4cf0     29ab4cf0     29adf078
> >>> System.Web.Util.CalliEventHandlerDelegateProxy.Callback(System.Object,
> >>> System.EventArgs)     0c9955e8     29ab4cf0     29ab4cf0     1cc0f108
> >>> System.Web.UI.Control.OnLoad(System.EventArgs)     01eda6a4
> >>> 29ab4cf0     29ab4cf0     1cc0f108
> >>> System.Web.UI.Control.LoadRecursive()     00000000     0a9901d0
> >>> 00000000     29a6600c
> >>> System.Web.UI.Page.ProcessRequestMain(Boolean, Boolean)     00000001
> >>> 29ab4cf0     00000001     00000000
> >>> System.Web.UI.Page.ProcessRequest(Boolean, Boolean)     00000001
> >>> 0e9e76bc     0e9e76bc     0c990f4c
> >>> System.Web.UI.Page.ProcessRequest()     29ab4cf0     1cc0f1ac
> >>> 6ff28e32     00000000
> >>> System.Web.UI.Page.ProcessRequestWithNoAssert(System.Web.HttpContext)
> >>>  00000000     29ab4cf0     29a6ac0c     1cc0f1e4
> >>> System.Web.UI.Page.ProcessRequest(System.Web.HttpContext)     0250af40
> >>>   00000000     29a6600c     00000000
> >>>
> >>
> System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
> >>>   00000000     00000000     00000000     29a6ac0c
> >>> System.Web.HttpApplication.ExecuteStep(IExecutionStep, Boolean ByRef)
> >>>  1cc0f280     8c16dcd2     72e149e8     1cc0f2d8
> >>>
> >>
> System.Web.HttpApplication+PipelineStepManager.ResumeSteps(System.Exception)
> >>>   00000000     00000000     1cc0f2d8     1cc0f334
> >>>
> >>
> System.Web.HttpApplication.BeginProcessRequestNotification(System.Web.HttpContext,
> >>> System.AsyncCallback)     0e99b5e8     8c16dcd2     72e149e8
> >>> 1cc0f344
> >>>
> >>
> System.Web.HttpRuntime.ProcessRequestNotificationPrivate(System.Web.Hosting.IIS7WorkerRequest,
> >>> System.Web.HttpContext)     29a6600c     8c16dcd2     72e149e8
> >>> 1cc0f898
> >>>
> >>
> System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr,
> >>> IntPtr, IntPtr, Int32)     00000000     00000009     00000000
> >>> 00000000
> >>> System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr,
> >>> IntPtr, IntPtr, Int32)     01a6119c     023d15a0     00000009
> >>> 00000000
> >>> webengine!W3_MGD_HANDLER::ProcessNotification+59     023d0ab8
> >>> 023d15a0     00000000     1cc0f46c
> >>> webengine!W3_MGD_HANDLER::DoWork+235     00000080     00000000
> >>> 1b7233bc     023d0b10
> >>> webengine!RequestDoWork+2c3     00000080     00000000     1b7233bc
> >>> 023d0ab8
> >>> webengine!CMgdEngHttpModule::OnExecuteRequestHandler+18     023d0ab8
> >>> 023d0b10     023d0abc     1cc0f518
> >>> iiscore!NOTIFICATION_CONTEXT::RequestDoWork+128     1b7233bc
> >>> 023d0ab8     00000000     023d0b10
> >>> iiscore!NOTIFICATION_CONTEXT::CallModulesInternal+2ec     00000000
> >>> 00000000     00000000     023d0ab8
> >>> iiscore!NOTIFICATION_CONTEXT::CallModules+28     00000000     00000000
> >>>   00000000     023d0ab8
> >>> iiscore!W3_CONTEXT::DoStateRequestExecuteHandler+36     00000000
> >>> 00000000     00000000     00000000
> >>> iiscore!W3_CONTEXT::DoWork+d7     00000000     00000000     00000001
> >>> 00000000
> >>> iiscore!W3_MAIN_CONTEXT::ContinueNotificationLoop+1f     00000000
> >>> 00000000     023d0ab8     1cc0f83c
> >>> iiscore!W3_MAIN_CONTEXT::ProcessIndicateCompletion+1f     00000000
> >>> 023d15a0     767516b3     023d0ab8
> >>> iiscore!W3_CONTEXT::IndicateCompletion+63     00000000     1cc0f8e8
> >>> 1cc0f8ac     1cc0f860
> >>> webengine!W3_MGD_HANDLER::IndicateCompletion+3b     00000000
> >>> 1cc0f898     1cc0f880     01771671
> >>> webengine!MgdIndicateCompletion+22     023d15a0     1cc0f8e8
> >>> 8c16dcd2     00000000
> >>> 0x01771671     29a6600c     29a65dc4     00000000     1cc0f91c
> >>>
> >>
> System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr,
> >>> IntPtr, IntPtr, Int32)     00000004     00000003     00000000
> >>> 00000000
> >>> System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr,
> >>> IntPtr, IntPtr, Int32)     00000000     023d15a0     00000003
> >>> 00000004
> >>> mscorwks!UM2MThunk_WrapperHelper+1f     1cc0fb30     00000010
> >>> 01682388     016bee48
> >>> mscorwks!UM2MThunk_WrapperWorker+60     1cc0facc     0250af40
> >>> 0250af40     1cc0faa8
> >>> mscorwks!UM2MThunk_Wrapper+14     1cc0facc     1cc0fa9c     72ef7e43
> >>> c22ab3b9
> >>> mscorwks!Thread::DoADCallBack+da     00000003     7308e32d
> >>> 1cc0facc     c22ab211
> >>> mscorwks!UM2MDoADCallBack+7c     ffffffff     0250af40     1cc0fb30
> >>> 00000010
> >>> 0x01682372     00000000     023d15a0     00000003     00000004
> >>> webengine!W3_MGD_HANDLER::ProcessNotification+59     7336d32c
> >>> 1cf27800     1cc0fba8     72e54404
> >>> webengine!ProcessNotificationCallback+2d     023d15a0     c22ab2b9
> >>> 72e53d1c     00040000
> >>> mscorwks!UnManagedPerAppDomainTPCount::DispatchWorkItem+9a
> >>> 00000000     00000000     76751110     00000001
> >>> mscorwks!ThreadpoolMgr::ExecuteWorkRequest+af     c22ab505
> >>> 00000000     72e53d69     0250c220
> >>> mscorwks!ThreadpoolMgr::WorkerThreadStart+20b     00000000
> >>> fffde000     77c70094     1cc0fbb8
> >>> mscorwks!Thread::intermediateThreadProc+49     0250c220     1cc0ff7c
> >>> 77c0d80d     0250c220
> >>> kernel32!BaseThreadInitThunk+e     0250c220     39fb7237     00000000
> >>>  00000000
> >>> ntdll!__RtlUserThreadStart+23     72f76e83     0250c220     ffffffff
> >>> 77c2e348
> >>> ntdll!_RtlUserThreadStart+1b     72f76e83     0250c220     00000000
> >>> 00000000
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> Exception Information
> >>> In W3WP__~1.DMP the assembly instruction at kernel32!RaiseException+58
> >>> in C:\Windows\SysWOW64\kernel32.dll from Microsoft Corporation has
> >>> caused a CLR Exception on thread 34 with the following error
> >>> information:
> >>>
> >>>   Type:   System.IO.IOException
> >>>   Message:   read past eof
> >>>
> >>>
> >>> This exception originated from
> >> Lucene.Net.QueryParsers.FastCharStream.Refill().
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message