lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Solr cloud and batched updates
Date Sun, 21 Apr 2013 17:54:25 GMT
I'm pretty sure there's a JIRA to do just that, it just hasn't been
implemented yet.

I guess it's one of those things that would undoubtedly be more efficient, but
whether it would really be noticeable or not is an open question. At any rate,
there are more important fish to fry but if you'd like to submit a patch.....

Best
Erick

On Sun, Apr 21, 2013 at 11:57 AM, Timothy Potter <thelabdude@gmail.com> wrote:
> There's no problem here, but I'm curious about how batches of updates
> are handled on the Solr server side in Solr cloud?
>
> Going over the code for DistributedUpdateProcessor and
> SolrCmdDistributor, it appears that the batch is broken down and docs
> are processed one-by-one. By processed, I mean that each doc in the
> batch from the client is sent to replicas individually.
>
> This makes sense but I wonder if the forwarding on to replicas could
> be done in sub-batches? For instance, if the client sends a batch of
> 100 documents to a cluster with 4 shards, I wonder if it would be more
> efficient to calculate the shard assignments to create 4 sub-batches
> and then forward those 4 sub-batches on to their respective leaders?
> Maybe I'm overthinking it too ;-)
>
> Cheers,
> Tim

Mime
View raw message