lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: IllegalArgumentException when IndexWriter.addDocument
Date Thu, 14 Jan 2010 21:55:11 GMT
This problem occurs, if you have a Tokenizer or TokenFilter that produces new tokens but does
not call clearAttributes(). What TokenStreams do you use in your analyzer?

If you not call clearAttributes() (see javadocs of Tokenizer!) whenever you produce new tokens
(in any type of TokenStream), the positionIncrement is never reset back to 1 (and when in
adiition the analyzer reuses the tokenstream, it is really never reset even across multiple
documents). So whenever a stopword occurs it get larger...

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Chris Lu [mailto:chris.lu@gmail.com]
> Sent: Thursday, January 14, 2010 10:41 PM
> To: java-user@lucene.apache.org
> Subject: Re: IllegalArgumentException when IndexWriter.addDocument
> 
> Notes: I am using Lucene 3.0
> > Seems a integer overflow problem?
> >
> > java.lang.IllegalArgumentException: Increment must be zero or
> greater:
> > -472893952
> >  at
> >
> org.apache.lucene.analysis.tokenattributes.PositionIncrementAttributeIm
> pl.setPositionIncrement(PositionIncrementAttributeImpl.java:58)
> >
> >  at
> >
> org.apache.lucene.analysis.StopFilter.incrementToken(StopFilter.java:25
> 6)
> >  at
> >
> org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPe
> rField.java:188)
> >
> >  at
> >
> org.apache.lucene.index.DocFieldProcessorPerThread.processDocument(DocF
> ieldProcessorPerThread.java:246)
> >
> >  at
> >
> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.
> java:774)
> >
> >  at
> >
> org.apache.lucene.index.DocumentsWriter.addDocument(DocumentsWriter.jav
> a:752)
> >
> >  at
> >
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1931)
> >  at
> >
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1905)
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message