lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erick Erickson" <erickerick...@gmail.com>
Subject Re: Reading Performance
Date Thu, 07 Dec 2006 18:31:04 GMT
Well, the performance isn't bad considering you're executing the *search*
around 1,000 times.......

One of the characteristics of a Hits object is that it's optimized for
getting the top 100 docs or so. To get the next 100 docs it re-executes the
query. Repeatedly <G>. I'd try using a HitCollector or TopDocs instead of a
Hits object....

Hope this helps
Erick

On 12/7/06, Aigner, Thomas <TAigner@wescodist.com> wrote:
>
>
>
>
>
> Howdy all,
>
>
>
>       I have a question on reading many documents and time to do this.
> I have a loop on the hits object reading a record, then writing it to a
> file.  When there is only 1 user on the Index Searcher, this process to
> read say 100,000 takes around 3 seconds.  This is slow, but can be
> acceptable.  When a few more users do searchers, this time to just read
> from the hits object becomes well over 10 seconds, sometimes even 30+
> seconds.  Is there a better way to read through and do something with
> the hits information?  And yes, I have to read all of them to do this
> particular task.
>
>
>
> for (int i = 0;(i <= hits.length() - 1); i++)
>
> {
>
>
>
>       if (fw == null)
>
>       {
>
>             fw = new BufferedWriter( new FileWriter( searchWriteSpec ),
> 8196) ;
>
>       }
>
>
>
>       //Write Out records
>
>       String tmpHold = "";
>
> tmpHold = hits.doc(i).get("somefield1") + hits.doc(i).get("somefield2");
>
>
>
>       fw.write(tmpHold + "\n" );
>
>
>
> }
>
>
>
> Any ideas on how to speed this up especially with multiple users?  Each
> user gets their own class which has the above code in it.
>
>
>
> Thanks,
>
> Tom
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message