lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From andi rexha <a_re...@hotmail.com>
Subject RE: WhitespaceTokenizer, incrementToke() ArrayOutOfBoundException
Date Mon, 15 Apr 2013 14:31:13 GMT
Thank you, 
that was the reason.

> From: jack@basetechnology.com
> To: java-user@lucene.apache.org
> Subject: Re: WhitespaceTokenizer, incrementToke() ArrayOutOfBoundException
> Date: Mon, 15 Apr 2013 10:25:26 -0400
> 
> I didn't read your code, but do you have the "reset" that is now mandatory 
> and throws AIOOBE if not present?
> 
> -- Jack Krupansky
> 
> -----Original Message----- 
> From: andi rexha
> Sent: Monday, April 15, 2013 10:21 AM
> To: java-user@lucene.apache.org
> Subject: WhitespaceTokenizer, incrementToke() ArrayOutOfBoundException
> 
> Hi,
> I have tryed to get all the tokens from a TokenStream in the same way as I 
> was doing in the 3.x version of Lucene, but now (at least with 
> WhitespaceTokenizer) I get an exception:
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1
>     at java.lang.Character.codePointAtImpl(Character.java:2405)
>     at java.lang.Character.codePointAt(Character.java:2369)
>     at 
> org.apache.lucene.analysis.util.CharacterUtils$Java5CharacterUtils.codePointAt(CharacterUtils.java:164)
>     at 
> org.apache.lucene.analysis.util.CharTokenizer.incrementToken(CharTokenizer.java:166)
> 
> 
> 
> The code is quite simple, and I thought that it could have worked, but 
> obviously it doesn't (unless I have made some mistakes).
> 
> Here is the code, in case you spot some bugs on it (although it is trivial):
> String str = "this is a test";
>         Reader reader = new StringReader(str);
>         TokenStream tokenStream = new WhitespaceTokenizer(Version.LUCENE_42, 
> reader);  //tokenStreamAnalyzer.tokenStream("test", reader);
>         CharTermAttribute attribute = 
> tokenStream.getAttribute(CharTermAttribute.class);
>         while (tokenStream.incrementToken()) {
>             System.out.println(new String(attribute.buffer(), 0, 
> attribute.length()));
>         }
> 
> Hope you have any idea of why it is happening.
> Regards,
> Andi
>  
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message