incubator-lucy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marvin Humphrey <mar...@rectangular.com>
Subject Re: [lucy-dev] FieldType: no default properties
Date Wed, 22 Sep 2010 00:39:39 GMT
On Tue, Sep 21, 2010 at 07:17:20PM +0200, Simon Willnauer wrote:
> I don't know if that is a really good usecase for flags integers though. For
> something high level as FieldType I would guess there is more than just
> boolean flags - maybe not now but in the future. 

I think we agree in that the constructors for FieldType subclasses (such as
TextType) will have to take more than just a flags integer.  For instance,
TextType's constructor has to take an Analyzer, and I think it should take a
Similarity as well.

    TextType*
    TextType_new(Analyzer *analyzer, Similarity *similarity, uint32_t flags);

However, we do have to bear in mind that in C, we don't have signature-based
overloading or optional parameters -- so every time we add a new parameter to
the constructor, the C API will have a compat break.  That's acceptable under
our "unstable trunk" policy, but it's not desirable.  

A flags integer allows us to add new boolean properties (so long as they can
default to false) without breaking back compat on the C API, up until we run
out of flag bits.  It's also highly idiomatic C, and it translates easily to
the named parameters we'd use in Perl, Python, Ruby, etc.

> I have a whole bunch of ideas for FieldType since I work on something
> similar in lucene land and I am happy to share those ideas.

What did you have in mind?  I know of
<https://issues.apache.org/jira/browse/LUCENE-2308>, but I didn't see you
there.

> While I see your point I think we should not try to maintain bw compat
> to kino search. I had the impression that this is a fresh start please
> correct me if I am wrong. If we maintain BW compat (what a pain man!)
> then +1

This would be index file-format compat only.  API compatibility is impossible
because of the namespace change from KinoSearch to Lucy.  

I'm not wedded to the idea of maintaining compat -- it just seemed like the
obvious thing to do.  Releasing early and often was a fundamental goal we set
for ourselves in our incubation proposal.  Making Lucy read KinoSearch indexes
wouldn't be that hard unless we get really ambitious between code import and
the first release.

Deprecating KinoSearch is easier if we can provide a clear migration path for
KinoSearch's users.  Here at Eventful, we have several dozen multi-gigabyte,
fast-moving indexes; I'm sure you can understand why it would be nice for us
to just switch in some code based on Lucy, restart the apps, and declare
migration done. :)  If that's not to be, it's not to be, and we'll just deal
with it.  However, the sooner that Lucy can persuade KinoSearch's users to
switch, the better off we'll be.

> Are we already that far to talk about something like Field Type?

FieldType is already deeply embedded in the codebase.  Unlike Lucene, we
already have per-index field-name-to-field-type associations.

Marvin Humphrey


Mime
View raw message