lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <karl.wri...@nokia.com>
Subject RE: Doppleganger threads after ingestion completed
Date Sun, 20 Jun 2010 23:50:11 GMT
So far they've stayed around for 72 hours and counting.
Also, I don't care what the stack trace says - CPU is listing as 500%.  So it may be momentarily
"blocked" but then it must loop.

Karl
________________________________________
From: ext Lance Norskog [goksron@gmail.com]
Sent: Saturday, June 19, 2010 8:51 PM
To: dev@lucene.apache.org
Subject: Re: Doppleganger threads after ingestion completed

"Chewing up cpu" or "blocked". The stack trace says it's blocked.

The sockets are abandoned by the program, yes, but TCP/IP itself has a
complex sequence for shutting down sockets that takes a few minutes.
If these sockets stay around for hours, then there's a real problem.
(In fact, there is a bug in the TCP/IP specification, 40 years old,
that causes zombie sockets that never shut down.)

The HTTP solr server really needs a socket close() method.

On Thu, Jun 17, 2010 at 6:08 AM,  <karl.wright@nokia.com> wrote:
> Folks,
>
> I ran 20,000,000 records into Solr via the extractingUpdateRequestHandler
> under jetty.  The previous problems with resources have apparently been
> resolved by using Http1.1 with keep-alive, rather than creating and
> destroying 20,000,000 sockets. ;-)  However, after the client terminates, I
> still find the Solr process chewing away CPU – indeed, there were 5 threads
> doing this.
>
> A thread dump yields the following partial trace for all 5 threads:
>
> "btpool0-13" prio=10 tid=0x0000000041391000 nid=0xe7c runnable
> [0x00007f4a8c789000]
>    java.lang.Thread.State: RUNNABLE
>         at
> org.mortbay.jetty.HttpParser$Input.blockForContent(HttpParser.java:925)
>         at org.mortbay.jetty.HttpParser$Input.read(HttpParser.java:897)
>         at
> org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:977)
>         at
> org.apache.commons.fileupload.MultipartStream$ItemInputStream.close(MultipartStream.java:924)
>         at
> org.apache.commons.fileupload.MultipartStream$ItemInputStream.close(MultipartStream.java:904)
>         at org.apache.commons.fileupload.util.Streams.copy(Streams.java:119)
>         at org.apache.commons.fileupload.util.Streams.copy(Streams.java:64)
>         at
> org.apache.commons.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:362)
>         at
> org.apache.commons.fileupload.servlet.ServletFileUpload.parseRequest(ServletFileUpload.java:126)
>         at
> org.apache.solr.servlet.MultipartRequestParser.parseParamsAndFillStreams(SolrRequestParsers.java:343)
>         at
> org.apache.solr.servlet.StandardRequestParser.parseParamsAndFillStreams(SolrRequestParsers.java:396)
>         at
> org.apache.solr.servlet.SolrRequestParsers.parse(SolrRequestParsers.java:114)
>         at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
>         at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
> …
>
> I could be wrong, but it looks to me like either jetty or fileupload may
> have a problem here.  I have not looked at the jetty source code, but
> infinitely spinning processes even after the socket has been abandoned do
> not seem reasonable to me.  Thoughts?
>
> Karl
>
>



--
Lance Norskog
goksron@gmail.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message