Position increments are for relative token positions. A position
increment of zero means that a token is logically at the same position
as the previous token. A position increment of one means that a token
immediately follows the preceding token in the stream, it's the next
token to the right (in a left-to-right language). A position increment
of two means that it is two tokens past the previous token, that there's
a "phantom" token between them, inhibiting exact phrase matches.
You're setting the position increment to things based on the number of
characters in the token's text. That makes no sense. Token positions
are not character positions. I think what you want to do is use a
positionIncrement of zero, so the tokens lie at the same position.
Doug
Allen Atamer wrote:
> The FAQ describes implementing a TokenFilter for applying aliases. I have a
> trouble accomplishing this.
>
> This is the code that I have so far for the next Method within AliasFilter.
> After reading some posts, I also got the idea to call
> setPositionIncrement(). Neither way works, because when I search for the
> alias, no search results come back.
>
> Thank you for your help,
>
> Allen Atamer
>
> ----
>
> public Token next() throws java.io.IOException {
> Token token = tokenStream.next();
>
> if (aliasMap == null || token == null) {
> return token;
> }
>
> TermData t = (TermData)aliasMap.get(token.termText());
>
> if (t == null) {
> return token;
> }
>
> String tokenText = AliasManager.replaceIgnoreCase(
> token.termText(), t.getTerm(), t.getTeach());
>
> int increment = tokenText.length() - token.termText().length();
> if (increment > 0) {
> token.setPositionIncrement(increment);
> }
>
> return new Token(tokenText, token.startOffset(), token.endOffset());
> }
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
|