lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: solr stuck in writing to inexisting sockets
Date Mon, 01 Nov 2010 15:44:20 GMT
I'm going to nudge you in the direction of understanding why the queries
take so long in the first place rather than going toward the blunt approach
of cutting them off after some time. The fact that you don't control the
queries submitted doesn't prevent you from trying to understand what
is taking so long.

The first thing I'd look for is whether the system is memory starved. What
JVM are you using and what memory parameters are you giving it? What
version of Solr are you using? Have you tried any performance monitoring
to determine what is happening?

The reason I'm pushing in this direction is that 5 minute searches are
pathological. Once you're up in that range, virtually any fix you come up
with will simply mask the underlying problems, and you'll be forever
chasing the next manifestation of the underlying problem.

Besides, I don't know how you'd stop Solr processing a query mid-way
through,
I don't know of any way to make that happen.

Best
Erick

On Mon, Nov 1, 2010 at 9:30 AM, Roxana Angheluta <anghelutar@yahoo.com>wrote:

> Hi,
>
> Yes, sometimes it takes >5 minutes for a query. I agree this is not
> desirable. However, if the application has no control over the input queries
> other that closing the socket after a while, solr should not continue
> writing the response, but terminate the thread.
>
> In general, is there a way to quantify the complexity of a given query on a
> certain index? Some general guidelines which can be used by non-technical
> people?
>
> Thanks a lot,
> roxana
>
> --- On Sun, 10/31/10, Erick Erickson <erickerickson@gmail.com> wrote:
>
> > From: Erick Erickson <erickerickson@gmail.com>
> > Subject: Re: solr stuck in writing to inexisting sockets
> > To: solr-user@lucene.apache.org
> > Date: Sunday, October 31, 2010, 2:29 AM
> > Are you saying that your Solr server
> > is at times taking 5 minutes to
> > complete? If so,
> > I'd get to the bottom of that first off. My first guess
> > would be you're
> > either hitting
> > memory issues and swapping horribly or..well, that would be
> > my first guess.
> >
> > Best
> > Erick
> >
> > On Thu, Oct 28, 2010 at 5:23 AM, Roxana Angheluta <anghelutar@yahoo.com
> >wrote:
> >
> > > Hi all,
> > >
> > > We are using Solr over Jetty with a large index,
> > sharded and distributed
> > > over multiple machines. Our queries are quite long,
> > involving boolean and
> > > proximity operators. We cut the connection at the
> > client side after 5
> > > minutes. Also, we are using parameter timeAllowed to
> > stop executing it on
> > > the server after a while.
> > > We quite often run into situations when solr "blocks".
> > The load on the
> > > server increases and a thread dump on the solr process
> > shows many threads
> > > like below:
> > >
> > >
> > > "btpool0-49" prio=10 tid=0x00007f73afe1d000 nid=0x3581
> > runnable
> > > [0x00000000451a0000]
> > >   java.lang.Thread.State: RUNNABLE
> > >        at
> > java.io.PrintWriter.write(PrintWriter.java:362)
> > >        at
> > org.apache.solr.common.util.XML.escape(XML.java:206)
> > >        at
> > org.apache.solr.common.util.XML.escapeCharData(XML.java:79)
> > >        at
> > org.apache.solr.request.XMLWriter.writePrim(XMLWriter.java:832)
> > >        at
> > org.apache.solr.request.XMLWriter.writeStr(XMLWriter.java:684)
> > >        at
> > org.apache.solr.request.XMLWriter.writeVal(XMLWriter.java:564)
> > >        at
> > org.apache.solr.request.XMLWriter.writeDoc(XMLWriter.java:435)
> > >        at
> > org.apache.solr.request.XMLWriter$2.writeDocs(XMLWriter.java:514)
> > >        at
> > >
> > org.apache.solr.request.XMLWriter.writeDocuments(XMLWriter.java:485)
> > >        at
> > >
> >
> org.apache.solr.request.XMLWriter.writeSolrDocumentList(XMLWriter.java:494)
> > >        at
> > org.apache.solr.request.XMLWriter.writeVal(XMLWriter.java:588)
> > >        at
> > >
> > org.apache.solr.request.XMLWriter.writeResponse(XMLWriter.java:130)
> > >        at
> > >
> >
> org.apache.solr.request.XMLResponseWriter.write(XMLResponseWriter.java:34)
> > >        at
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:325)
> > >        at
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
> > >        at
> > >
> >
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
> > >        at
> > >
> > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
> > >        at
> > >
> >
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> > >        at
> > >
> > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
> > >        at
> > >
> > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
> > > ......................................
> > >
> > >
> > > A netstat on the machine shows sockets in state
> > CLOSE_WAIT. However, they
> > > are fewer than the number of RUNNABLE threads as the
> > above.
> > >
> > > Why is this happening? Is there anything we can do to
> > avoid getting in
> > > these situations?
> > >
> > > Thanks,
> > > roxana
> > >
> > >
> > >
> > >
> >
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message