lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-3942) SynonymFilter should set pos length att
Date Mon, 02 Apr 2012 18:55:23 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Michael McCandless updated LUCENE-3942:
---------------------------------------

    Attachment: LUCENE-3942.patch

Patch to set pos length > 1 when appropriate... I think it's ready.

Note that SynFilter still cannot *consume* a graph, so eg you cannot apply it after WDF or
after Kuromoji... we need to separately fix that.
                
> SynonymFilter should set pos length att
> ---------------------------------------
>
>                 Key: LUCENE-3942
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3942
>             Project: Lucene - Java
>          Issue Type: Improvement
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: 4.0
>
>         Attachments: LUCENE-3942.patch
>
>
> Tokenizers/Filters can now produce graphs instead of a single linear
> chain of tokens, by setting the PositionLengthAttribute, expressing
> where (how many positions ahead) this token "ends".
> The default is 1, meaning it ends at the next position, to be
> backwards compatible.
> SynonymFilter produces graph output tokens, as long as the output is a
> single token, but currently never sets the pos length to express this.
> EG for the rule "wifi network -> hotspot", the hotspot token should
> have pos length = 2.  With LUCENE-3940 this will allow us to verify
> that the offsets for such tokens are correct...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message