lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (Updated) (JIRA)" <>
Subject [jira] [Updated] (LUCENE-3942) SynonymFilter should set pos length att
Date Mon, 02 Apr 2012 18:55:23 GMT


Michael McCandless updated LUCENE-3942:

    Attachment: LUCENE-3942.patch

Patch to set pos length > 1 when appropriate... I think it's ready.

Note that SynFilter still cannot *consume* a graph, so eg you cannot apply it after WDF or
after Kuromoji... we need to separately fix that.
> SynonymFilter should set pos length att
> ---------------------------------------
>                 Key: LUCENE-3942
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: 4.0
>         Attachments: LUCENE-3942.patch
> Tokenizers/Filters can now produce graphs instead of a single linear
> chain of tokens, by setting the PositionLengthAttribute, expressing
> where (how many positions ahead) this token "ends".
> The default is 1, meaning it ends at the next position, to be
> backwards compatible.
> SynonymFilter produces graph output tokens, as long as the output is a
> single token, but currently never sets the pos length to express this.
> EG for the rule "wifi network -> hotspot", the hotspot token should
> have pos length = 2.  With LUCENE-3940 this will allow us to verify
> that the offsets for such tokens are correct...

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message