lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From eks dev <eks...@yahoo.co.uk>
Subject Re: [jira] Created: (LUCENE-1172) Small speedups to DocumentsWriter
Date Mon, 11 Feb 2008 08:32:03 GMT
Robert, 

you may or may not be right, I do not know. The only way to prove it would be to show you
can do it better, no?
If you are so convinced this is wrong, you could, much better than quoting textbooks:
 
a) write better patch, get attention with something you think is "better bottleneck" 
b) provide realistic "performance tests" as you dispute the measurement provided here

It has to be that concrete, academic discussions are cool, but at the end of a day, it is
the code that executes that counts.

cheers, 
eks

----- Original Message ----
From: robert engels <rengels@ix.netcom.com>
To: java-dev@lucene.apache.org
Sent: Sunday, 10 February, 2008 9:15:30 PM
Subject: Re: [jira] Created: (LUCENE-1172) Small speedups to DocumentsWriter

I am not sure these numbers matter. I think they are skewed because  
you are probably running too short a test, and the index is in memory  
(or OS cache).

Once you use a real index that needs to read/write from the disk, the  
percentage change will be negligible.

This is the problem with many of these "performance changes" - they  
just aren't real world enough.  Even if they were, I would argue that  
code simplicity/maintainability is worth more than 6 seconds on a  
operation that takes 4 minutes to run...

There are many people that believe micro benchmarks are next to  
worthless. A good rule of thumb is that if the optimization doesn't  
result in 2x speedup, it probably shouldn't be done. In most cases  
any efficiency gains are later lost in maintainability issues.

See http://en.wikipedia.org/wiki/Optimization_(computer_science)

Almost always there is a better bottleneck somewhere.

On Feb 10, 2008, at 1:37 PM, Michael McCandless wrote:

>
> Yonik Seeley wrote:
>
>> I wonder how well a single generic quickSort(Object[] arr, int low,
>> int high) would perform vs the type-specific ones?  I guess the main
>> overhead would be a cast from Object to the specific class to do the
>> compare?  Too bad Java doesn't have true generics/templates.
>
>
> OK I tested this.
>
> Starting from the patch on LUCENE-1172, which has 3 quickSort methods
> (one per type), I created a single quickSort method on Object[] that
> takes a Comparator, and made 3 Comparators instead.
>
> Mac OS X 10.4 (JVM 1.5):
>
>     original patch --> 247.1
>   simplified patch --> 254.9 (3.2% slower)
>
> Windows Server 2003 R64 (JVM 1.6):
>
>     original patch --> 440.6
>   simplified patch --> 452.7 (2.7% slower)
>
> The times are best in 10 runs.  I'm running all tests with these JVM
> args:
>
>   -Xms1024M -Xmx1024M -Xbatch -server
>
> I think this is a big enough difference in performance that it's
> worth keeping 3 separate quickSorts in DocumentsWriter.
>
> Mike
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org






      __________________________________________________________
Sent from Yahoo! Mail - a smarter inbox http://uk.mail.yahoo.com



---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message