lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <>
Subject Re: Porting benchmark suite
Date Sat, 07 Feb 2009 16:00:43 GMT

You'll also need at least some of the *QueryMaker under feeds.

You might also want to make an improvment: change the QueryMaker API
to include both the query and the "arrival time" of that query.  And
then fix all ReadTask (and Search*Task) so that queries are executed
at their scheduled time (assuming enough threads & hardware).

This way one could play back a true search log and measure "realistic"
query latencies, or, one could concoct synthetic difficult cases (4
very hard queries suddenly running at once) and understand how
performance degrades.

Another thing I miss (which I've worked around w/ Python scripts on
top) is to be able to save a set of runs, and then use it as a
baseline when comparing to another set of runs, with the ability to
print out resulting tables in Jira's markup.


Grant Ingersoll wrote:

> The build file in the benchmarker has a "run" target that shows how  
> to run it.  The important part to port is the "by task" stuff:
> On Feb 6, 2009, at 10:11 PM, Marvin Humphrey wrote:
>> Greets,
>> Lucy needs sophisticated search-time benchmarking.  The obvious  
>> approach is to
>> port the Lucene contrib benchmark suite.
>> However, contrib benchmark has a large number of classes, the  
>> documentation is
>> sparse and occasionally wrong ("Usage: java Benchmark algorithm- 
>> file"),
>> there's no howto or Wiki page (just package.html) ... and one  
>> obvious starting
>> point, the "Benchmarker" class, is deprecated.
>> What's actually important in the benchmark suite?  Besides  
>> "Benchmarker" being
>> deprecated, there look to be multiple "stats" and "utils"  
>> directories. Are
>> there large chunks of obsolete code that can be safely ignored?
>> Marvin Humphrey
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:
> --------------------------
> Grant Ingersoll
> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika) using  
> Solr/Lucene:
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message