lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Woodward (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-8273) Add a ConditionalTokenFilter
Date Fri, 11 May 2018 13:29:00 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471900#comment-16471900
] 

Alan Woodward commented on LUCENE-8273:
---------------------------------------

The elasticsearch CI had some failures due to this:
{code}
ant test  -Dtestcase=TestRandomChains -Dtests.method=testRandomChains -Dtests.seed=EF8BCF910EB1138C
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es-CR -Dtests.timezone=Asia/Ashgabat
-Dtests.asserts=true -Dtests.file.encoding=UTF8
{code}

They look to be caused by FingerprintFilter being wrapped in a ConditionalTokenStream, which
I don't think makes any sense?  So the simplest solution is probably to blacklist it.

> Add a ConditionalTokenFilter
> ----------------------------
>
>                 Key: LUCENE-8273
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8273
>             Project: Lucene - Core
>          Issue Type: New Feature
>            Reporter: Alan Woodward
>            Assignee: Alan Woodward
>            Priority: Major
>             Fix For: 7.4
>
>         Attachments: LUCENE-8273.patch, LUCENE-8273.patch, LUCENE-8273.patch, LUCENE-8273.patch,
LUCENE-8273.patch, LUCENE-8273.patch, LUCENE-8273.patch, LUCENE-8273.patch
>
>
> Spinoff of LUCENE-8265.  It would be useful to be able to wrap a TokenFilter in such
a way that it could optionally be bypassed based on the current state of the TokenStream.
 This could be used to, for example, only apply WordDelimiterFilter to terms that contain
hyphens.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message