lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Santosh Narayan <santosh.narayan....@gmail.com>
Subject Re: Solrj : ConcurrentUpdateSolrClient based on QueueSize and Time
Date Wed, 21 Feb 2018 14:41:23 GMT
Hi Shawn,
May be it is my understanding of the documentation. As per the
JavaDoc, ConcurrentUpdateSolrClient
buffers all added documents and writes them into open HTTP connections.

So I thought that this class would buffer documents in the client side
itself till the QueueSize is reached and then send all the cached documents
together in one HTTP request. Is this not the case?

On Wed, Feb 21, 2018 at 7:26 PM, Shawn Heisey <elyograg@elyograg.org> wrote:

> On 2/21/2018 1:21 AM, Santosh Narayan wrote:
>
>> I'm using ConcurrentUpdateSolrClient to push data into Solr. Currently,
>> I'm
>> initializing it as follows:
>>
>> ConcurrentUpdateSolrClientclient = new
>> ConcurrentUpdateSolrClient.Builder(serverUrl).withThreadCoun
>> t(100).withQueueSize(50).build();
>>
>> This works fine when there are 50 requests coming in a short span of
>> time(a
>> few seconds). The problem is when there aren't many requests. If there are
>> not many requests for say 5 minutes, then the queue size may not touch 50
>> and the data is not sent to the Solr Server. Is there a way I can add
>> another condition to this, where I can say either the QueueSize if 50 or a
>> timeout of 60 seconds, whichever is earliest? This way, in case there are
>> less requests, the records that came in would get pushed to the server
>> every 60 seconds.
>>
>
> The client should begin processing requests as soon as they are added, not
> when the queue fills up.  If you're seeing something different, then either
> there's a bug in ConcurrentUpdateSolrClient or your code is doing something
> very unusual.  Can you share the rest of the code using that client
> object?  What version of SolrJ are you using?
>
> Thanks,
> Shawn
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message