Mailing-List: contact lucy-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: lucy-dev@incubator.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Date: Sat, 30 Apr 2011 21:37:34 -0700
From: Marvin Humphrey <marvin@rectangular.com>
To: lucy-dev@incubator.apache.org, peter@peknet.com
Message-ID: <20110501043734.GA9947@rectangular.com>
References: <4DBCCCFC.9080200@peknet.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4DBCCCFC.9080200@peknet.com>
User-Agent: Mutt/1.5.18 (2008-05-17)
Subject: Re: [lucy-dev] FType_Equals

On Sat, Apr 30, 2011 at 10:01:16PM -0500, Peter Karman wrote:
> This change:
> ================================
> r1085159 | marvin | 2011-03-24 17:02:51 -0500 (Thu, 24 Mar 2011) | 4 lines
> 
> LUCY-138 ftype_equals.patch
> Have FType_Equals() return false whenever classes for "self" and "other" don't
> match.
> ================================
> 
> is causing some of my existing code to break.

Mmm, yes, I can see why this might be happening.  Some of our backwards
compatibility stubs are not providing enough backwards compatibility.

That changeset implements an important bugfix, though.  We discovered the
problem it addresses after corrupting an index because we didn't detect that a
field type had changed from one subclass to another and throw an error before
the damage was unrecoverable. :(

Without that patch, Lucy will let you change a field to a different type --
say one that uses a different posting format -- so long as the other tests
within FType_Equals return true.  That's bad, because as soon as the indexing
session completes, you start trying to read from old segments using the wrong
posting codec and Lucy crashes.

> 'foo' cannot be Lucy::FieldType::FullTextType@0x000000010190d870 because already
> assigned conflicting FieldType Lucy::Plan::FullTextType@0x000000010193bfe0
> 
> 
> 
> All of which makes me think that the comparison being done isn't really between
> classes, as the commit message claims, because the objects compared are, in
> fact, instances of the same class.
> 
> What am I missing?

They aren't quite the same -- they both start with "Lucy" and end with
"FullTextType", but in between, one has "FieldType" and the other has "Plan".

When we moved FullTextType from KinoSearch::FieldType::FullTextType to
KinoSearch::Plan::FullTextType, we installed a compatibility stub subclass at
the old location so that code like this would continue to work:

    my $type = KinoSearch::FieldType::FullTextType->new(%args);

However, the classes aren't exactly the same -- the FieldType variant now
subclasses the Plan variant -- and after the commit in question,
FType_Equals() won't treat them as equivalent.

Probably the best solution for Lucy is to rip out all the compatibility stub
subclasses that were inherited from KinoSearch.  You should make sure that all
new code uses the Lucy::Plan::* variant.

If you have conflicting field type classes in your active code, you should fix
that and hopefully the problem will go away.

If you don't, then you're hitting a problem that arises because within the
schema file, the class name for stock FullTextType fields is not recorded --
they are just stored as "type":"fulltext".  This is a little complicated to
explain and it will go away if we remove the compat stubs, so I'll hold off
on the long version until we see whether Plan A works for you.

Marvin Humphrey