lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: Performance vs. maxBufferedAddsPerServer=10
Date Mon, 29 Jul 2013 11:58:50 GMT
SOLR-4816 won't address this - it will just speed up *different* parts. There are other things
that will need to be done to speed up that part.

- Mark

On Jul 26, 2013, at 3:53 PM, Erick Erickson <erickerickson@gmail.com> wrote:

> This is current a hard-coded limit from what I've understood. From what
> I remember, Mark said Yonik said that there are reasons to make the
> packets that size. But whether this is empirically a Good Thing I don't know.
> 
> SOLR-4816 will address this a different way by making SolrJ batch up
> the docs and send them to the right leader, which should pretty much remove
> any performance consideration here.
> 
> There's some anecdotal evidence that changing that in the code might
> improve throughput, but I don't remember the details.
> 
> FWIW
> Erick
> 
> On Thu, Jul 25, 2013 at 7:09 AM, Otis Gospodnetic
> <otis.gospodnetic@gmail.com> wrote:
>> Hi,
>> 
>> Context:
>> * https://issues.apache.org/jira/browse/SOLR-4956
>> * http://search-lucene.com/c/Solr:/core/src/java/org/apache/solr/update/SolrCmdDistributor.java%7C%7CmaxBufferedAddsPerServer
>> 
>> As you can see, maxBufferedAddsPerServer = 10.
>> 
>> We have an app that sends 20K docs to SolrCloud using CloudSolrServer.
>> We batch 20K docs for performance reasons. But then the receiving node
>> ends up sending VERY small batches of just 10 docs around for indexing
>> and we lose the benefit of batching those 20K docs in the first place.
>> 
>> Our app is "add only".
>> 
>> Is there anything one can do to avoid performance loss associated with
>> maxBufferedAddsPerServer=10?
>> 
>> Thanks,
>> Otis
>> --
>> Solr & ElasticSearch Support -- http://sematext.com/
>> Performance Monitoring -- http://sematext.com/spm


Mime
View raw message