lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Lu <chris...@gmail.com>
Subject Re: IllegalArgumentException when IndexWriter.addDocument
Date Thu, 14 Jan 2010 22:16:33 GMT
I am using a custom analyzer upgrading from Lucene 2.x.
Need to get more familiar with Lucene 3.0 behavior.
I think this is one of the upgrade pitfalls.

Thanks for the help!

Chris

Uwe Schindler wrote:
> This problem occurs, if you have a Tokenizer or TokenFilter that produces new tokens
but does not call clearAttributes(). What TokenStreams do you use in your analyzer?
>
> If you not call clearAttributes() (see javadocs of Tokenizer!) whenever you produce new
tokens (in any type of TokenStream), the positionIncrement is never reset back to 1 (and when
in adiition the analyzer reuses the tokenstream, it is really never reset even across multiple
documents). So whenever a stopword occurs it get larger...
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>   
>> -----Original Message-----
>> From: Chris Lu [mailto:chris.lu@gmail.com]
>> Sent: Thursday, January 14, 2010 10:41 PM
>> To: java-user@lucene.apache.org
>> Subject: Re: IllegalArgumentException when IndexWriter.addDocument
>>
>> Notes: I am using Lucene 3.0
>>     
>>> Seems a integer overflow problem?
>>>
>>> java.lang.IllegalArgumentException: Increment must be zero or
>>>       
>> greater:
>>     
>>> -472893952
>>>  at
>>>
>>>       
>> org.apache.lucene.analysis.tokenattributes.PositionIncrementAttributeIm
>> pl.setPositionIncrement(PositionIncrementAttributeImpl.java:58)
>>     
>>>  at
>>>
>>>       
>> org.apache.lucene.analysis.StopFilter.incrementToken(StopFilter.java:25
>> 6)
>>     
>>>  at
>>>
>>>       
>> org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPe
>> rField.java:188)
>>     
>>>  at
>>>
>>>       
>> org.apache.lucene.index.DocFieldProcessorPerThread.processDocument(DocF
>> ieldProcessorPerThread.java:246)
>>     
>>>  at
>>>
>>>       
>> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.
>> java:774)
>>     
>>>  at
>>>
>>>       
>> org.apache.lucene.index.DocumentsWriter.addDocument(DocumentsWriter.jav
>> a:752)
>>     
>>>  at
>>>
>>>       
>> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1931)
>>     
>>>  at
>>>
>>>       
>> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1905)
>>     
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>     
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>   

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message