lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (JIRA)" <>
Subject [jira] [Updated] (LUCENE-4489) improve LimitTokenCountFilter and/or it's tests
Date Thu, 18 Oct 2012 04:54:03 GMT


Robert Muir updated LUCENE-4489:

    Attachment: LUCENE-4489.patch

Some combination of the above, attached is an updated patch:

* previous patch had a test bug where it reused the same directory in the loop, so the docFreq()
would be wrong as it kept adding documents.
* there was another test bug where it looped form 0..limit and then added "x" but checked
docFreq=1, this won't happen as the limit was exceeded (I changed the loop to 1..limit).
* previous assertions about finalOffset from end() had wrong values that depended upon implementation
details: thats the whole bug here! so these assertions are correct now: if you pass consumeAll
= true, the finalOffset is correct, highlighting on multivalued fields with limiting will
work correctly and so on. If you pass consumeAll=false, its impl dependent (and likely will
be wrong, just as before). p.s. would be better to still improve the test to actually not
assert endOffset at all if consumeAll=false, because my "fix" still keeps the test fragile
and dependent on MockTokenizer impl in the "wrong" case.
* previous patch also had a real bug: if you passed consumeAllTokens=true, but the stream
had less tokens than the limit, it would incorrectly call incrementToken() after it returned
* fixes for javadocs errors (see references needed #) and typos.

> improve LimitTokenCountFilter and/or it's tests
> -----------------------------------------------
>                 Key: LUCENE-4489
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Hoss Man
>         Attachments: LUCENE-4489.patch, LUCENE-4489.patch
> spinning off a discussion about LimitTokenCountFilter  and it's tests from SOLR-3961
(which was about a specific bug in the LimitTokenCountFilterFactory)

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message