lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Willnauer <simon.willna...@googlemail.com>
Subject Re: Comparing Indexing Speed of Lucene 3.5 and 4.0
Date Tue, 03 Jan 2012 20:43:23 GMT
hey Peter,

as far as I can see you are comparing apples and pears. Your
comparison is waiting for merges to finish and if you are using
multiple threads lucene 4.0 will flush more segments to disk than 3.5
so what you are seeing is likely a merge that is still trying to merge
small segments. can you rerun and only measure the time until the last
commit finishes (not the close)

one more thing, you are indexing always the more or less same document
and the text is very very short. You should add some more randomness
or reality to your test.

simon

On Tue, Jan 3, 2012 at 5:56 PM, Peter K <peathal@yahoo.de> wrote:
> Hi,
>
> I recently switched an experimental project from Lucene 3.5 to 4.0 from
> 6th Dec 2011
> and my indexing time increased by nearly 20% on my local machine*.
> It seems to me that two simple StringField's could cause this slow down:
> Field uIdField = new Field("_uid", "" + id, StringField.TYPE_STORED);
> Field typeField = new Field("_type", "test", StringField.TYPE_STORED);
>
> Without them Lucene 4 is faster**. Here is a recreation using different
> branches for every lucene version:
> https://github.com/karussell/lucene-tmp
> Or is there something wrong with my too simplistic scenario?
>
> Furthermore: How could I further improve Lucene 4.0 indexing speed?
> (I already read through the performance list on the wiki)
>
> Regards,
> Peter.
>
> *
> open jdk 1.6.0_20  (but also confirmed with latest java6 from oracle)
> ubuntu/10.10 linux/2.6.35-31 i686, 2GB ram
>
> **
> lucene 3.5
> 23.5sec index all three fields: _id, _uid, type
> 19.0sec index only the _id field
>
> lucene 4
> 29.5sec index _id, _uid, type
> 16.5sec index only the _id
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message