lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Lea <ian....@gmail.com>
Subject Re: upgrading from 3.0.3 to 3.5.0
Date Wed, 01 Feb 2012 10:17:51 GMT
The javadocs for ParallelReader say that all indexes must have same
number of docs and all be created and modified the same way.  Doesn't
sound like your shards.

I think you need to create a MultiReader on top of the readers for
your individual shards and pass that to the IndexSearcher constructor
that takes a reader and an ExecutorService.

IndexReader ir1 = ... shard1
IndexReader ir2 = ... shard2
etc.
MultiReader mr = new MultiReader(ir1, ir2, ...)
IndexSearcher searcher = new IndexSearcher(mr, execservice)


Unless you are taking advantage of shards being on different drives or
something like that, I'd guess that one index would be as quick to
search, and easier to maintain.  Try it and see - there is no
substitute for real world tests on your data/indexes on your hardware.


--
Ian.


On Wed, Feb 1, 2012 at 9:56 AM, Ganesh <emailgane@yahoo.co.in> wrote:
> Thanks Ian.
>
>>>The deprecation warning in the javadocs says "Please pass an ExecutorService to
IndexSearcher, instead" so I'd do that.
> I may need to use IndexSearcher(Reader, ExecutorService). I have sharded my index. Say
if i have 10 indexes then i will have 10 IndexSearchers. How to use this API. It takes one
Reader and ExecutorService.
>
> What about using IndexSearcher(ParallelReader)?
>
>>>Regarding optimization?
> I have sharded to multiple small indexes (in a single system). This helps to do faster
indexing. I have updates every couple of minutes and the updates will go in latest shard.
All shards will be optimized per day. If lucene performs well in searching multiple segment
then i don't need to go for shards. I could very well use single index and don't optimize
it. Whether it will impact search? Previous releases perform better in optimized indexes.
What about 3.5.0?
>
> Regards
> Ganesh
>
>
>
> ----- Original Message -----
> From: "Ian Lea" <ian.lea@gmail.com>
> To: <java-user@lucene.apache.org>
> Sent: Wednesday, February 01, 2012 3:07 PM
> Subject: Re: upgrading from 3.0.3 to 3.5.0
>
>
>> I am upgrading from 3.0.3 to 3.5.0.
>>
>> 1) NumberTools is deprecated. I am converting long to string and storing it in Index.
Now this is deprecated. If i replace this API with NumericUtils / NumericField, will it work
for existing index? Whether i need to rebuild the index?
>
> You will need to rebuild if you switch to using NumericField.
>
>> 2) I am using ParallelMultiSearcher. This is now deprecated. Whether i should use
ParallelReader / IndexSearcher (reader, subreaders)?
>
> The deprecation warning in the javadocs says "Please pass an
> ExecutorService to IndexSearcher, instead" so I'd do that.
>
>> 3) Optimize method is deprecated. The help says "This method has been deprecated,
as it is horribly inefficient and very rarely justified. Lucene's multi-segment search performance
has improved over time, and the default TieredMergePolicy now targets segments with deletions."
>> This means usage of this method is not justified any more. I am doing optimization
once per day. Is that not required now.
>
> No.
>
>
> --
> Ian.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message