lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Created: (LUCENE-947) Some improvements to contrib/benchmark
Date Mon, 02 Jul 2007 13:42:04 GMT
Some improvements to contrib/benchmark
--------------------------------------

                 Key: LUCENE-947
                 URL: https://issues.apache.org/jira/browse/LUCENE-947
             Project: Lucene - Java
          Issue Type: Improvement
          Components: contrib/benchmark
            Reporter: Michael McCandless
            Assignee: Michael McCandless
            Priority: Minor


I've made some small improvements to the contrib/benchmark, mostly
merging in the ad-hoc benchmarking code I've been using in LUCENE-843:

  - Fixed thread safety of DirDocMaker's usage of SimpleDateFormat

  - Print the props in sorted order

  - Added new config "autocommit=true|false" to CreateIndexTask

  - Added new config "ram.flush.mb=int" to AddDocTask

  - Added new configs "doc.term.vector.positions=true|false" and
    "doc.term.vector.offsets=true|false" to BasicDocMaker

  - Added WriteLineDocTask.java, so you can make an alg that uses this
    to build up a single file containing one document per line in a
    single file.  EG this alg converts the reuters-out tree into a
    single file that has ~1000 bytes per body field, saved to
    work/reuters.1000.txt:

      docs.dir=reuters-out
      doc.maker=org.apache.lucene.benchmark.byTask.feeds.DirDocMaker
      line.file.out=work/reuters.1000.txt
      doc.maker.forever=false
      {WriteLineDoc(1000)}: *

    Each line has tab-separted TITLE, DATE, BODY fields.

  - Created feeds/LineDocMaker.java that creates documents read from
    the file created by WriteLineDocTask.java.  EG this alg indexes
    all documents created above:

      analyzer=org.apache.lucene.analysis.SimpleAnalyzer
      directory=FSDirectory
      doc.add.log.step=500

      docs.file=work/reuters.1000.txt
      doc.maker=org.apache.lucene.benchmark.byTask.feeds.LineDocMaker
      doc.tokenized=true
      doc.maker.forever=false

      ResetSystemErase
      CreateIndex
      {AddDoc}: *
      CloseIndex

      RepSumByPref AddDoc

I'll attach initial patch shortly.


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message