lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhang, Lisheng" <Lisheng.Zh...@BroadVision.com>
Subject RE: lucene 4.3 seems to be much slower in indexing than lucene 3.6?
Date Tue, 30 Jul 2013 22:13:32 GMT
Hi Mike,

I did more tests with realistic text from different languages (typical
text for 8 different languages, English one is novel "Animal Farm").

What I found seems to be:

## Indexing: 
36 and 43 comparable (your previous comment is very correct).

## Search: 
43 seems to be slower (30%), checking details, it seems it all due to 
initial searcher creation and first search (warming), as if 43 did much
more in warming?

Is above statement OK, if so we have a problem: we have so many collections
(in solr called cores), so we cannot afford "warming", we wish we can each
time create searcher on the fly, but seems lucene goes further away from
that?

Your guidance would be very appreciated,

Lisheng 


-----Original Message-----
From: Zhang, Lisheng [mailto:Lisheng.Zhang@broadvision.com]
Sent: Saturday, July 27, 2013 11:06 PM
To: java-user@lucene.apache.org
Subject: RE: lucene 4.3 seems to be much slower in indexing than lucene
3.6?


Hi,

That's a very good point, I will test with a more realistic text
(like a novel).

Thanks very much for helps, Lisheng

-----Original Message-----
From: Michael McCandless [mailto:lucene@mikemccandless.com]
Sent: Saturday, July 27, 2013 3:42 AM
To: Lucene Users
Subject: Re: lucene 4.3 seems to be much slower in indexing than lucene
3.6?


It's also possible 4.x is slower than 3.x for purely random terms: the
terms dictionary is completely different.

But purely random terms is a poor test since it doesn't match the
reality of a typical search index.

Mike McCandless

http://blog.mikemccandless.com


On Fri, Jul 26, 2013 at 2:55 PM, Zhang, Lisheng
<Lisheng.Zhang@broadvision.com> wrote:
>
> Hi,
>
> I did some basic performance testing, just use random number to generate
> text for indexing,
> below I attached source java code. The command I used are:
>
> java TestReal43 index -docCount 500 -start 1 -optimize true -luceneDir mmap
> java TestReal36 index -docCount 500 -start 1 -optimize true -luceneDir mmap
>
> The difference is NOT in optimization, I changed directory within (Simple,
> MMap, NIOFS),
> it does not make much difference, 43 is about 30~40% slower. I conducted
> test in Linux:
>
> Linux ec2usevmsstgamq 3.2.0-49-virtual #75-Ubuntu SMP Tue Jun 18 17:59:38
> UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
>
> Thanks very much for helps, Lisheng
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message