lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <>
Subject [jira] [Commented] (LUCENE-3113) fix analyzer bugs found by MockTokenizer
Date Tue, 17 May 2011 15:33:47 GMT


Uwe Schindler commented on LUCENE-3113:

A quick check on the fixes in the implementations: all fine. I was just confused about PrefixAndSuffixAwareTF,
but thats fine (Robert explained it to me - this Filters are very complicated from the code/class
hierarchy design *g*).

I did not verify the Tests, I assume its just dumb search-replacements.

> fix analyzer bugs found by MockTokenizer
> ----------------------------------------
>                 Key: LUCENE-3113
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/analysis
>            Reporter: Robert Muir
>             Fix For: 3.2, 4.0
>         Attachments: LUCENE-3113.patch, LUCENE-3113.patch
> In LUCENE-3064, we beefed up MockTokenizer with assertions, and I've switched over the
analysis tests to use MockTokenizer for better coverage.
> However, this found a few bugs (one of which is LUCENE-3106):
> * incrementToken() after it returns false in CommonGramsQueryFilter, HyphenatedWordsFilter,
ShingleFilter, SynonymFilter
> * missing end() implementation for PrefixAwareTokenFilter
> * double reset() in QueryAutoStopWordAnalyzer and ReusableAnalyzerBase
> * missing correctOffset()s in MockTokenizer itself.
> I think it would be nice to just fix all the bugs on one issue... I've fixed everything
except Shingle and Synonym

This message is automatically generated by JIRA.
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message