lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shai Erera <ser...@gmail.com>
Subject Re: API Semantics and Backwards
Date Mon, 06 Dec 2010 09:32:15 GMT
I've hit another backwards tests problem.

Over in LUCENE-2790 we've changed LogMergePolicy.useCompoundFile's behavior
to factor in the newly added noCFSRatio. After some discussion we decided
that even though it breaks back-compat's runtime behavior, it's ok in this
case because how Lucene manages the internal representation of segments
(compound or not) is up to it. And you can override it by disabling the
CFSRatio setting.

And indeed some tests failed (backwards as well as core stream) and the way
to fix them was to force CFS creation. However, on backwards this is not
doable because the tests are compiled against 3.0's source, where
setNoCFSRatio does not exist on LogMergePolicy, even though we agree that
this change is allowed back-compat wise.

I ended up fixing it by querying for the method using reflection and the
tests now pass.

Now, regardless of this change (whether it's ok or not), I think this shows
another problem with how we maintain backwards tests. Internal changes like
this, especially for @experimental / @internal classes are allowed, but we
need to revert to reflection hacks to resolve the tests.

So either we delete the offending tests, because like Uwe says - they
duplicate the test efforts, or we maintain a source for backwards.

I personally am in favor of removing all "non backwards" tests, and keep
those that do actually test backwards behavior. But I know the opinions are
divided here.

Shai

On Wed, Dec 1, 2010 at 4:48 PM, Shai Erera <serera@gmail.com> wrote:

> While I'm not against going back towards a checkout backwards that we can
> modify, I wonder if all the tests there should be there and how much do we
> actually duplicate.
>
> Lucene 3x should include all of 3.0 tests + new ones that test new
> functionality, or assert bug fixes etc. There shouldn't be a test in 3.0
> that does not exist in 3x, unless the missing test/feature was an agreed
> upon backwards break.
>
> So I think it would be really nice if backwards tested exactly what it
> should. For example, testing index format backcompat is done twice today, in
> "test-core" and "test-backwards", while it should only be run by backwards.
> There are a bunch of test classes I've created once that impl/extend
> 'search' related classes, for back-compat compilation only. They should also
> be run in backwards only.
>
> The downside of this is that maintenance is going to be difficult - it's
> much easier to copy tests over to backwards, then decide which ones should
> go there and which shouldn't. Also, adding new API requires a matching
> backwards test etc. Not non doable, but difficult - requires discipline.
>
> Shai
>
>
> On Tue, Nov 30, 2010 at 2:02 PM, Robert Muir <rcmuir@gmail.com> wrote:
>
>> On Tue, Nov 30, 2010 at 4:47 AM, Shai Erera <serera@gmail.com> wrote:
>> >
>> > Like you said, the rest of the tests just increase the test running
>> time.
>> >
>>
>> I'm not completely sure about this: do we always switch over our tests
>> to do the equivalent checks both against the new API and the old API
>> when we make API changes? There could be bugs in our 'backwards
>> handling' that are actually logic bugs that the new tests dont detect.
>>
>> So I'm a little concerned about only running "pure" simplistic API
>> tests in backwards.
>>
>> On the other hand, I'm really worried about what Shai brings up here:
>> we are doing some refactoring of the tests system and there is more
>> shared code at the moment: similar to MockRAMDirectory.
>> Because we worry about preventing things like index corruption, its my
>> opinion we need things like MockRAMDirectory, and they should be able
>> to break all the rules/etc (use pkg-private APIS) if we can prevent
>> bugs.
>> Just look at our trunk or 3.x tests and imagine them as backwards
>> tests... these sort of "utilities" like RandomIndexWriter will be more
>> fragile to internal/experimental/pkg-private changes, but as mentioned
>> above i think these are good to have in backwards tests.
>>
>> So, I think at the moment I'm leaning towards the idea of going back
>> towards a checkout that we can modify, in combination with us all
>> soliciting more reviews / longer time for any backports to stable
>> branches that require backwards tests modifications?  I understand
>> Uwe's point too, its dangerous to modify the code and seems to defeat
>> the purpose of backwards, but I think this is going to be a more
>> serious problem after releasing 3.1!
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>

Mime
View raw message