lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kiwi clive <kiwi_cl...@yahoo.com>
Subject Re: Lucene 3.6.0 high CPU usage
Date Fri, 09 Nov 2012 15:41:49 GMT
Hi Ian,
Yes I/O ->CPU, but read on...

The throughput is about the same with similar i/o and 20% higher CPU. We have plenty of CPU
and I would rather be cpu bound, thean i/o bound.


However, we have increased the number of indexing threads in our application. On Lucene 2.3.2,
this gave no performance improvement as the I/O just increased. However,  lucene 3.6.0 ramps
up the CPU (user) usage while I/O only increases marginally. 


Our throughput has trebled !! - amazing improvement, and all in CPU space.


So it appears there have been a lot of performance improvements made in the new version that
I was not previously using, the Ferrari was in first gear and now we are taking advantage
of these features.

This is truly phenominal step-change in throughput, thank you Lucene developers !

Clive



________________________________
 From: Ian Lea <ian.lea@gmail.com>
To: java-user@lucene.apache.org; kiwi clive <kiwi_clive@yahoo.com> 
Sent: Friday, November 9, 2012 10:04 AM
Subject: Re: Lucene 3.6.0 high CPU usage
 
Are you getting the same, improved or worse performance/throughput?
Has the bottleneck switched from IO to CPU?


--
Ian.


On Thu, Nov 8, 2012 at 12:40 PM, kiwi clive <kiwi_clive@yahoo.com> wrote:
> Having played with merge parameters and various index parameters, it seems possible to
change the I/O usage at the cost of the number of index files. However, it does appear that
this version of lucene is using more CPU.  Is there any reason for this ? is it normal ?
We can push a large amount of documents through the indexer and it seams to work admirably
although I would be happier if the load average came down.
>
> Any lucene devs out there who could shed some light on this behaviour ?
>
>
> Thanks,
> Clive
>
>
> ________________________________
>  From: kiwi clive <kiwi_clive@yahoo.com>
> To: "java-user@lucene.apache.org" <java-user@lucene.apache.org>
> Sent: Wednesday, November 7, 2012 10:58 AM
> Subject: Lucene 3.6.0 high CPU usage
>
> Hi Guys,
>
> Having upgraded from lucene2.3.2 to lucene3.6.0 and jumping through a few hoops, we note
that the CPU usage on the new version is perhaps 20-30% higher. The i/o wait appears about
the same on both versions.
>
> Can anyone suggest why this should be so or is the newer lucene version just more CPU
hungry? If not, any tuning suggestions to reduce CPU usage would be gratefully received !
>
> Thanks
> Clive

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message