lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Problems for indexing large documents on SolrCloud
Date Wed, 10 Sep 2014 20:05:19 GMT
bq: org.apache.solr.common.SolrException: Unexpected end of input
block; expected an identifier

This is very often an indication that your packets are being
truncated by "something in the chain". In your case, make sure
that Tomcat is configured to handle inputs of the size that you're sending.

This may be happening before things get to Solr, in which case your settings
in solrconfig.xml aren't germane, the problem is earlier than than.

A "semi-smoking-gun" here is that there's a size of your multivalued
field that seems to break things... That doesn't rule out time problems
of course.

But I'd look at the Tomcat settings for maximum packet size first.

Best,
Erick

On Wed, Sep 10, 2014 at 9:11 AM, Olivier <olivauron@gmail.com> wrote:
> Hi,
>
> I have some problems for indexing large documents in a SolrCloud cluster of
> 3 servers  (Solr 4.8.1) with 3 shards and 2 replicas for each shard on
> Tomcat 7.
> For a specific document (with 300 K values in a  multivalued field), I
> couldn't index it on SolrCloud but I could do it in a single instance of
> Solr on my own PC.
>
> The indexation is done with Solarium from a database. The data indexed are
> e-commerce products with classic fields like name, price, description,
> instock, etc... The large field (type int) is constitued of other products
> ids.
> The only difference with other documents well-indexed on Solr  is the size
> of that multivalued field. Indeed, other documents well-indexed have all
> between 100K values and 200 K values for that field.
> The index size is 11 Mb for 20 documents.
>
> To solve it, I tried to change several parameters including ZKTimeout in
> solr.xml  :
>
> In solrcloud section :
>
> <int name="zkClientTimeout">60000</int>
>
> <int name="distribUpdateConnTimeout">100000</int>
>
> <int name="distribUpdateSoTimeout">100000</int>
>
>
>
>  In shardHandlerFactory section  :
>
>
>
> <int name="socketTimeout">${socketTimeout:100000}</int>
>
> <int name="connTimeout">${connTimeout:100000}</int>
>
>
> I also tried to increase these values in solrconfig.xml :
>
> <requestParsers enableRemoteStreaming="true"
>
>                     multipartUploadLimitInKB="100000000"
>
>                     formdataUploadLimitInKB="100000"
>
>                     addHttpRequestToContext="false"/>
>
>
>
>
> I also tried to increase the quantity of RAM (there are VMs) : each server
> has 4 Gb of RAM with 3Gb for the JVM.
>
> Are there other settings which can solve the problem that I would have
> forgotten ?
>
>
> The error messages are :
>
> ERROR
>
> SolrDispatchFilter
>
> null:java.lang.RuntimeException: [was class java.net.SocketException]
> Connection reset
>
> ERROR
>
> SolrDispatchFilter
>
> null:ClientAbortException:
>
> java.net.SocketException:
> broken pipe
>
> ERROR
>
> SolrDispatchFilter
>
> null:ClientAbortException:
>
> java.net.SocketException:
> broken pipe
>
> ERROR
>
> SolrCore
>
> org.apache.solr.common.SolrException:
>                                                   Unexpected end of input
> block; expected an identifier
>
> ERROR
>
> SolrCore
>
> org.apache.solr.common.SolrException:
>                                                   Unexpected end of input
> block; expected an identifier
>
> ERROR
>
> SolrCore
>
> org.apache.solr.common.SolrException:
>                                                   Unexpected end of input
> block; expected an identifier
>
> ERROR
>
> SolrCore
>
> org.apache.solr.common.SolrException:
>                                                   Unexpected EOF in
> attribute value
>
>
>
>
>
>
>
>
> Thanks,
>
> Olivier
>
> SolrCore
>
> org.apache.solr.common.SolrException:
>                                                   Unexpected end of input
> block in start tag

Mime
View raw message