lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From William Koscho <wkos...@gmail.com>
Subject Can I omit ShingleFilter's filler tokens
Date Wed, 11 May 2011 04:04:56 GMT
Hi,

Can I remove the filler token _ from the n-gram-tokens that are generated by
a ShingleFilter?

I'm using a chain of filters: ClassicFilter, StopFilter, LowerCaseFilter,
and ShingleFilter to create phrase n-grams.  The ShingleFilter inserts
FILLER_TOKENs in place of the stopwords, but I don't want them.

How can I omit the filler tokens?

thanks
Bill

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message