lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (JIRA)" <>
Subject [jira] [Commented] (LUCENE-6529) NumericFields + SlowCompositeReaderWrapper + UninvertedReader + -Dtests.codec=random can results in incorrect SortedSetDocValues
Date Mon, 08 Jun 2015 23:12:00 GMT


Hoss Man commented on LUCENE-6529:

Well, ok ... that may be obvious to you -- but my point, as someone completley unfamiliar
with that code, is that the new test in this issue only fails when SlowCompositReaderWrapper
is used around an index with multiple segments -- any other LeafReader (either Slow wrapper
arround single segment index, or direct on the individual segments of multi segment index)
don't cause the same failures regardless of codec used.

Hence my comment that if you are wondering why BasePostingsFormatTestCase isn't triggering
similar failures, maybe there is a coreleation?

if you know definitively that he two things have nothing to do with eachother - great, i'll
take your word for it.  

* Do you have any specific suggestions for fixing this?
* Do you have any suggestions for why BasePostingsFormatTestCase isn't catching this? and/or
what should be added to BasePostingsFormatTestCase in order to start catching this?
* Do you have opinions on my suggestion to remove this optimization? ...

My inclination would be that we should remove this optimization for 5.2.1, commit these tests,
and open a new issue to re-add the optimization if/when if can be done in such a way that
these tests pass reliably.
what do folks think?

> NumericFields + SlowCompositeReaderWrapper + UninvertedReader + -Dtests.codec=random
can results in incorrect SortedSetDocValues 
> ---------------------------------------------------------------------------------------------------------------------------------
>                 Key: LUCENE-6529
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Hoss Man
>         Attachments: LUCENE-6529.patch, LUCENE-6529.patch
> Digging into SOLR-7631 and SOLR-7605 I became fairly confident that the only explanation
of the behavior i was seeing was some sort of bug in either the randomized codec/postings-format
or the UninvertedReader, that was only evident when two were combined and used on a multivalued
Numeric Field using precision steps.  But since i couldn't find any -Dtests.codec or -Dtests.postings.format
options that would cause the bug 100% regardless of seed, I switched tactices and focused
on reproducing the problem using UninvertedReader directly and checking the SortedSetDocValues.getValueCount().
> I now have a test that fails frequently (and consistently for any seed i find), but only
with -Dtests.codec=random -- override it with -Dtests.codec=default and everything works fine
(based on the exhaustive testing I did in the linked issues, i suspect every named codec works
fine - but i didn't re-do that testing here)
> The failures only seem to happen when checking the SortedSetDocValues.getValueCount()
of a SlowCompositeReaderWrapper around the UninvertedReader -- which suggests the root bug
may actually be in SlowCompositeReaderWrapper? (but still has some dependency on the random

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message