lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Porting benchmark suite
Date Sat, 07 Feb 2009 16:00:43 GMT

You'll also need at least some of the *QueryMaker under feeds.

You might also want to make an improvment: change the QueryMaker API
to include both the query and the "arrival time" of that query.  And
then fix all ReadTask (and Search*Task) so that queries are executed
at their scheduled time (assuming enough threads & hardware).

This way one could play back a true search log and measure "realistic"
query latencies, or, one could concoct synthetic difficult cases (4
very hard queries suddenly running at once) and understand how
performance degrades.

Another thing I miss (which I've worked around w/ Python scripts on
top) is to be able to save a set of runs, and then use it as a
baseline when comparing to another set of runs, with the ability to
print out resulting tables in Jira's markup.

Mike

Grant Ingersoll wrote:

> The build file in the benchmarker has a "run" target that shows how  
> to run it.  The important part to port is the "by task" stuff: http://lucene.apache.org/java/2_4_0/api/contrib-benchmark/org/apache/lucene/benchmark/byTask/package-summary.html
>
>
>
>
> On Feb 6, 2009, at 10:11 PM, Marvin Humphrey wrote:
>
>> Greets,
>>
>> Lucy needs sophisticated search-time benchmarking.  The obvious  
>> approach is to
>> port the Lucene contrib benchmark suite.
>>
>> However, contrib benchmark has a large number of classes, the  
>> documentation is
>> sparse and occasionally wrong ("Usage: java Benchmark algorithm- 
>> file"),
>> there's no howto or Wiki page (just package.html) ... and one  
>> obvious starting
>> point, the "Benchmarker" class, is deprecated.
>>
>> What's actually important in the benchmark suite?  Besides  
>> "Benchmarker" being
>> deprecated, there look to be multiple "stats" and "utils"  
>> directories. Are
>> there large chunks of obsolete code that can be safely ignored?
>>
>> Marvin Humphrey
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika) using  
> Solr/Lucene:
> http://www.lucidimagination.com/search
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message