lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergiu Bivol <sbi...@blackberry.com>
Subject RE: Why Solr Returns Error for a Search that has rows parameter smaller than numFound?
Date Tue, 14 May 2013 18:25:35 GMT
Forgot to mention that formdataUploadLimitInKB must be increased on all solr shards.

-----Original Message-----
From: Sergiu Bivol [mailto:sbivol@blackberry.com] 
Sent: Tuesday, May 14, 2013 1:55 PM
To: solr-user@lucene.apache.org
Subject: RE: Why Solr Returns Error for a Search that has rows parameter smaller than numFound?

I spent some time chasing this as well; in our case only sharded requests requesting >500K
rows were failing.

If you enable FINE logging on the main (coordinating) shard you will notice a number of "400
Bad Request" errors, complaining about an upload limit being exceeded. Increasing formdataUploadLimitInKB
below in solrconfig.xml made the difference, and now even larger queries succeed (~1.5mln
rows); default value is 2048KB (I think).
<config>
  ...
  <requestDispatcher ... >
    <requestParsers enableRemoteStreaming="true" 
                    multipartUploadLimitInKB="2048000"
                    formdataUploadLimitInKB="2048000"/>
  </requestDispatcher>
  ...
</config>

It would great if the "Upload limit exceeded" error message would be logged at WARNING or
ERROR level, instead of the very confusing "Incorrect Version". Would've saved us quite a
few hours of painful reading of FINE logs.

-----Original Message-----
From: Furkan KAMACI [mailto:furkankamaci@gmail.com]
Sent: Tuesday, May 14, 2013 11:52 AM
To: solr-user@lucene.apache.org
Subject: Re: Why Solr Returns Error for a Search that has rows parameter smaller than numFound?

I entered that URL from my browser, not used SolrJ.

2013/5/14 Upayavira <uv@odoko.co.uk>

> That error looks like a SolrJ version incompatibility, not because of 
> your rows param. However, your rows param is extremely high, and will 
> likely cause Solr to blow up in other ways. Solr isn't designed to 
> return all documents like that, only the n best matches, where n is a 
> low number.
>
> If you need to retrieve all docs, you should sort by some static ID, 
> and grab them in batches, using range queries on that ID.
>
> Upayavira
>
> On Tue, May 14, 2013, at 04:34 PM, Furkan KAMACI wrote:
> > When I make a query:
> >
> > :8983/solr/select?q=*:*
> >
> > I have numFound: 4264579
> >
> > and then when I make that query:
> >
> > :8983/solr/select?q=*:*&rows=4264579
> >
> > or even that:
> >
> > :8983/solr/select?q=*:*&rows=4264578
> >
> > <lst name="error">
> > <str name="msg">
> > org.apache.solr.client.solrj.SolrServerException:
> > java.lang.RuntimeException: Invalid version (expected 2, but 60) or 
> > the data in not in 'javabin' format </str> <str name="trace">
> > org.apache.solr.common.SolrException:
> > org.apache.solr.client.solrj.SolrServerException:
> > java.lang.RuntimeException: Invalid version (expected 2, but 60) or 
> > the data in not in 'javabin' format at
> >
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(Sear
> chHandler.java:302)
> > at
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandle
> rBase.java:135)
> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1812) at
> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.
> java:639)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter
> .java:345)
> > at
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter
> .java:141)
> > at
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletH
> andler.java:1307)
> > at
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:
> 453)
> > at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.ja
> va:137)
> > at
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java
> :560)
> > at
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandle
> r.java:231)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandle
> r.java:1072)
> > at
> > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java
> > :382)
> > at
> >
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler
> .java:193)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler
> .java:1006)
> > at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.ja
> va:135)
> > at
> >
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(Conte
> xtHandlerCollection.java:255)
> > at
> >
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerColle
> ction.java:154)
> > at
> >
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.
> java:116)
> > at org.eclipse.jetty.server.Server.handle(Server.java:365) at
> >
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(Abstract
> HttpConnection.java:485)
> > at
> >
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(Blocking
> HttpConnection.java:53)
> > at
> >
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(Abstrac
> tHttpConnection.java:926)
> > at
> >
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerC
> omplete(AbstractHttpConnection.java:988)
> > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
> > at
> > org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235
> > ) at
> >
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpCon
> nection.java:72)
> > at
> >
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(Soc
> ketConnector.java:264)
> > at
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool
> .java:608)
> > at
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.
> java:543)
> > at java.lang.Thread.run(Thread.java:722) Caused by:
> > org.apache.solr.client.solrj.SolrServerException:
> > java.lang.RuntimeException: Invalid version (expected 2, but 60) or 
> > the data in not in 'javabin' format at
> >
> org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrS
> erver.java:291)
> > at
> >
> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHan
> dler.java:174)
> > at
> >
> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHan
> dler.java:135)
> > at
> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:166) at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:4
> > 71)
> > at
> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> > at
> > java.util.concurrent.FutureTask.run(FutureTask.java:166) at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.j
> ava:1145)
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
> java:615)
> > ... 1 more Caused by: java.lang.RuntimeException: Invalid version 
> > (expected 2, but 60) or the data in not in 'javabin' format at 
> > org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java
> > :109)
> > at
> >
> org.apache.solr.client.solrj.impl.BinaryResponseParser.processResponse
> (BinaryResponseParser.java:41)
> > at
> >
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServe
> r.java:387)
> > at
> >
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServe
> r.java:181)
> > at
> >
> org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrS
> erver.java:264)
> > ... 10 more
> > </str>
> > <int name="code">
> > 500
> > </int>
> > </lst>
> >
> >
> > Why I get that error and how can I return all the results (just for 
> > a test purpose)?
>

---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged
material (including material protected by the solicitor-client or other applicable privileges),
or constitute non-public information. Any use of this information by anyone other than the
intended recipient is prohibited. If you have received this transmission in error, please
immediately reply to the sender and delete this information from your system. Use, dissemination,
distribution, or reproduction of this transmission by unintended recipients is not authorized
and may be unlawful.

---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged
material (including material protected by the solicitor-client or other applicable privileges),
or constitute non-public information. Any use of this information by anyone other than the
intended recipient is prohibited. If you have received this transmission in error, please
immediately reply to the sender and delete this information from your system. Use, dissemination,
distribution, or reproduction of this transmission by unintended recipients is not authorized
and may be unlawful.

Mime
View raw message