lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-3508) Decompounders based on CompoundWordTokenFilterBase cannot be used with custom attributes
Date Sun, 23 Oct 2011 15:52:32 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13133674#comment-13133674
] 

Robert Muir commented on LUCENE-3508:
-------------------------------------

I agree, lets fix this! 

Are there any other tokenstreams that don't yet work correctly with the attributes API? 
I was surprised there was no warning in the javadocs for this one.
I thought I remembered us adding a warning, maybe that was ShingleMatrix though.

                
> Decompounders based on CompoundWordTokenFilterBase cannot be used with custom attributes
> ----------------------------------------------------------------------------------------
>
>                 Key: LUCENE-3508
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3508
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/analysis
>    Affects Versions: 3.4, 4.0
>            Reporter: Spyros Kapnissis
>            Assignee: Uwe Schindler
>             Fix For: 3.5, 4.0
>
>         Attachments: LUCENE-3508.patch
>
>
> The CompoundWordTokenFilterBase.setToken method will call clearAttributes() and then
will reset only the default Token attributes (term, position, flags, etc) resulting in any
custom attributes losing their value. Commenting out clearAttributes() seems to do the trick,
but will fail the TestCompoundWordTokenFilter tests..

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message