lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anthony Vito <v...@mnis.com>
Subject Re: Is there any way to Collect all the searched documents at once?
Date Tue, 29 Jun 2004 20:15:28 GMT

> Is there any way to fetch the fields of the searched
> documents at once rather than looping through the
> found documents one by one....because it consumes lots
> of time and time increases by magnitude if the no. of
> documents found are more.
> 
> This is how I am fetching the found documents ..
> 
> count=1500000 (no. of documents found) 
> 
> for(int i=0 ; i < count;i++ )
>                         {
>                                 Document doc =
> hits.doc(i);
>                                 fields[i] =
> Long.parseLong(doc.get("FIELD"));
>                         }
> 
> This takes around 2.5 minutes  for fetching 1500000
> records which is unacceptable.
> 
> Regards
> Karan

1.5 million in 2.5 minutes gives me 100 microseconds per iteration. (
correct me if I messed up the math. ) I'd be willing to bet the
Long.parseLong() is a significant part of that time because it iterates
through each character. You should profile your code, a simple
-Xrunhprof:cpu=samples should suffice. If the parse is a significant
percentage of total run time, > 30% or so. You can save iterations in
the parse by encoding and decoding using a very high radix, probably 36,
so it would be alphanumeric.


-vito


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message