lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (Commented) (JIRA)" <>
Subject [jira] [Commented] (LUCENE-3971) MappingCharFilter rarely has wrong correctOffset (for finalOffset)
Date Wed, 11 Apr 2012 14:19:16 GMT


Robert Muir commented on LUCENE-3971:

I agree Dawid: what do you think about the difficulty of LUCENE-3830? 

I feel like with an FST, the logic would probably be easier, and
the filter would probably be faster (and we have pretty good tests,
in general this thing works, this is just a corner case).

On the other hand if there is a simple way we can fix the bug in the
existing code it could be nice, e.g. for a future 3.6.1 or something
like that.

But I'll take any solutions anyone has :)
> MappingCharFilter rarely has wrong correctOffset (for finalOffset) 
> -------------------------------------------------------------------
>                 Key: LUCENE-3971
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/analysis
>            Reporter: Robert Muir
>             Fix For: 4.0
>         Attachments: LUCENE-3971_test.patch
> Found this bug over on LUCENE-3969, but I'm currently tracking a ton of bugs, so
> I figure I would open an issue and see if this one is obvious to anyone:
> Consider this input string: "gzw f quaxot" (length = 12) with a WhitespaceTokenizer.
> If i have mapping rules like this, then it works!:
> {noformat}
> "t" => ""
> {noformat}
> But if I have mapping rules like this:
> {noformat}
> "t" => ""
> "tmakdbl" => "c"
> {noformat}
> Then it will compute final offset wrong:
> {noformat}
>     [junit] junit.framework.AssertionFailedError: finalOffset  expected:<12> but
> {noformat}
> Looks like some logic/recursion bug in the correctOffset method? The second rule is not
even "used" for this string,
> it just happens to also start with 't'

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message