lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ganesh" <emailg...@yahoo.co.in>
Subject Re: [Bulk] RE: [Bulk] RE: Exception while adding document in 3.0
Date Tue, 02 Feb 2010 09:18:22 GMT
Yes. I am using the objects across threads. Thanks for pointing out.

But still i didn't see much increase in performance by reusing documents and field objects.


Regards
Ganesh

----- Original Message ----- 
From: "Uwe Schindler" <uwe@thetaphi.de>
To: <java-user@lucene.apache.org>
Sent: Tuesday, February 02, 2010 2:02 PM
Subject: [Bulk] RE: [Bulk] RE: Exception while adding document in 3.0


They can be reused, but the exception and looking into the code shows that you are doing it
wrong. You can reuse Documents but only under two conditions:

a) In one thread, not one document in multithreaded app - one document per thread! And using
one document in more than one thread is what you are likely doing. This exception cannot occur
in one thread (just look into the lucene source code): What it does: it iterates over all
fields of the document using a for-loop. A RangeCheck exception can only occur, if the document
is modified during this iteration.
b) Only reuse the Document instance after the IndexWriter.xxxDocumentis finished! So don't
prepare the document and the do the IndexWriter stuffs in another task, this would exactly
produce this error.

Your problem is exactly the same as when you create an ArrayList, then iterate over the ArrayList
using a for-loop and remove elements from the ArrayList in a parallel thread.

Both of above conditions are mentioned in the FAQ (indirectly).

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Ganesh [mailto:emailgane@yahoo.co.in]
> Sent: Tuesday, February 02, 2010 9:24 AM
> To: java-user@lucene.apache.org
> Subject: Re: [Bulk] RE: Exception while adding document in 3.0
> 
> I tried My App with v2.9.0 without reusing Documents and it is working
> fine.
> 
> Based on the suggestion from group (refer attached mail), I modified
> the code to use v3.0.0 and in order to increase performance, I am
> reusing the Documents and Fields objects as suggessted. I am facing
> problems with this approach.
> 
> Documents cannot be re-used in v3.0?
> 
> Regards
> Ganesh
> 
> ----- Original Message -----
> From: "Uwe Schindler" <uwe@thetaphi.de>
> To: <java-user@lucene.apache.org>
> Sent: Tuesday, February 02, 2010 1:41 PM
> Subject: [Bulk] RE: Exception while adding document in 3.0
> 
> 
> > They can.
> >
> > -----
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: uwe@thetaphi.de
> >
> >> -----Original Message-----
> >> From: Glen Newton [mailto:glen.newton@gmail.com]
> >> Sent: Tuesday, February 02, 2010 9:03 AM
> >> To: java-user@lucene.apache.org
> >> Subject: Re: Exception while adding document in 3.0
> >>
> >> Documents cannot be re-used in v3.0?
> >>  http://wiki.apache.org/lucene-java/ImproveIndexingSpeed
> >>
> >> -glen
> >> http://zzzoot.blogspot.com/
> >>
> >> On 2 February 2010 02:55, Simon Willnauer
> >> <simon.willnauer@googlemail.com> wrote:
> >> > Ganesh,
> >> >
> >> > do you reuse your Document instances in any way or do you create
> new
> >> > docs for each add?
> >> >
> >> > simon
> >> >
> >> > On Tue, Feb 2, 2010 at 7:18 AM, Ganesh <emailgane@yahoo.co.in>
> wrote:
> >> >> I am getting below exception, while adding documents. I am adding
> >> documents continously and at some point, i am getting the below
> >> exception. This exception is not occuring with v2.9.0
> >> >>
> >> >>  Exception: Index: 21, Size: 2
> >> >>  java.util.ArrayList.RangeCheck(Unknown Source)
> >> >>  java.util.ArrayList.get(Unknown Source)
> >> >>
> >>
> org.apache.lucene.index.DocFieldProcessorPerThread.processDocument(Doc
> >> FieldProcessorPerThread.java:175)
> >> >>
> >>
> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter
> >> .java:779)
> >> >>
> >>
> org.apache.lucene.index.DocumentsWriter.addDocument(DocumentsWriter.ja
> >> va:757)
> >> >>
> >>
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:2472)
> >> >>
> >>
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:2446)
> >> >>
> >> >> Regards
> >> >> Ganesh
> >> >> Send instant messages to your online friends
> >> http://in.messenger.yahoo.com
> >> >>
> >> >> -----------------------------------------------------------------
> ---
> >> -
> >> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >>
> >> >>
> >> >
> >> > ------------------------------------------------------------------
> ---
> >> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >
> >> >
> >>
> >>
> >>
> >> --
> >>
> >> -
> >>
> >> --------------------------------------------------------------------
> -
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Send instant messages to your online friends http://in.messenger.yahoo.com 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message