Mailing-List: contact lucene-user-help@jakarta.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Lucene Users List" <lucene-user@jakarta.apache.org>
Subject: Re: Is there any way to Collect all the searched documents at once?
From: Anthony Vito <vito@mnis.com>
To: Lucene Users List <lucene-user@jakarta.apache.org>
In-Reply-To: <20040629090845.41616.qmail@web8301.mail.in.yahoo.com>
References: <20040629090845.41616.qmail@web8301.mail.in.yahoo.com>
Content-Type: text/plain
Organization: PPI
Message-Id: <1088540127.5024.145.camel@mortal>
Mime-Version: 1.0
Date: Tue, 29 Jun 2004 16:15:28 -0400
Content-Transfer-Encoding: 7bit


> Is there any way to fetch the fields of the searched
> documents at once rather than looping through the
> found documents one by one....because it consumes lots
> of time and time increases by magnitude if the no. of
> documents found are more.
> 
> This is how I am fetching the found documents ..
> 
> count=1500000 (no. of documents found) 
> 
> for(int i=0 ; i < count;i++ )
>                         {
>                                 Document doc =
> hits.doc(i);
>                                 fields[i] =
> Long.parseLong(doc.get("FIELD"));
>                         }
> 
> This takes around 2.5 minutes  for fetching 1500000
> records which is unacceptable.
> 
> Regards
> Karan

1.5 million in 2.5 minutes gives me 100 microseconds per iteration. (
correct me if I messed up the math. ) I'd be willing to bet the
Long.parseLong() is a significant part of that time because it iterates
through each character. You should profile your code, a simple
-Xrunhprof:cpu=samples should suffice. If the parse is a significant
percentage of total run time, > 30% or so. You can save iterations in
the parse by encoding and decoding using a very high radix, probably 36,
so it would be alphanumeric.


-vito


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org