lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Cowell <>
Subject Re: Distributed Indexing
Date Thu, 03 Feb 2011 15:12:25 GMT
Hi all,

Just a couple of questions that have arisen.

1. For handling non-distributed update requests (shards param is not present
or is invalid), our code currently

   - assumes the user would like the data indexed, so gets the request
   handler assigned to "/update"
   - executes the request using core.execute() for the SolrCore associated
   with the original request

Is this what we want it to do and is using core.execute() from within a
request handler a valid method of passing on the update request?

2. We have partially implemented an update processor which actually
generates and sends the split update requests to each specified shard (as
designated by the policy). As it stands, the code shares a lot in common
with the HttpCommComponent class used for distributed search. Should we look
at "opening up" the HttpCommComponent class so it could be used by our
request handler as well or should we continue with our current
implementation and worry about that later?

3. Our update processor uses a MultiThreadedHttpConnectionManager to send
parallel updates to shards, can anyone give some appropriate values to be
used for the defaultMaxConnectionsPerHost and maxTotalConnections params?
Won't the  values used for distributed search be a little high for
distributed indexing?



View raw message