lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: java.lang.IllegalStateException: TokenStream contract violation: close() call missing
Date Tue, 07 Feb 2017 09:12:26 GMT
Hi,

The issue is: Your Analyzer's createComponents method returns the same instance on every call,
this method has to _create_ a new instance on every call. Change your code to build the TokenStream
*inside* createComponents, not outside.

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: sumitj25 [mailto:sumitjain91@gmail.com]
> Sent: Tuesday, February 7, 2017 9:35 AM
> To: java-user@lucene.apache.org
> Subject: java.lang.IllegalStateException: TokenStream contract violation:
> close() call missing
> 
> Hi,
> 
> I am getting IllegalStateException even though I am following the
> TokenStream guidelines:
> 
>         public List<String> tokenize(String text){
> 		List<String> tokens = new ArrayList<>();
> 		try (TokenStream ts = analyzer.tokenStream("",text)){   //note
> that this
> is implicit finally with close
> 			ts.reset();
> 			while (ts.incrementToken()) {
> 
> 	tokens.add(ts.getAttribute(CharTermAttribute.class).toString());
> 			}
> 			ts.end();
> 			return tokens;
> 		}catch(Exception e){
> 			throw new RuntimeException(e);
> 		}
> 	}
> 
> Here's my analyzer creation code:
> 
>                 TokenizerFactory tokenizerFactory = (TokenizerFactory)
> obj(config.getTokenizer());
> 		Tokenizer tokenizer = tokenizerFactory.create();
> 		TokenStream stream = new StandardFilter(tokenizer);
> 		return new Analyzer() {
> 			protected TokenStreamComponents
> createComponents(String fieldName) {
> 				return new
> TokenStreamComponents(tokenizer, stream);
> 			}
> 		};
> 
> 
> 
> 
> 
> 
> 
> --
> View this message in context: http://lucene.472066.n3.nabble.com/java-
> lang-IllegalStateException-TokenStream-contract-violation-close-call-missing-
> tp4319109.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message