lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Busch <>
Subject Re: Flexible indexing
Date Tue, 13 Mar 2007 09:38:17 GMT
Marvin Humphrey wrote:
> It uses global field semantics, which Hoss won't be happy about.  ;)  
> However, I'm grateful to Hoss for past critiques, as they've helped me 
> to refine and improve how Schema works.  For instance, as of KS 
> 0.20_02 you can introduce new field_name => FieldSpec associations to 
> KS at any time during indexing.
> It may be that adapting Lucene to use something like what KS uses 
> would be too radical a change.  However, I believe that one reason 
> that flexible indexing has been in incubation so long is that the 
> current mechanism for attaching semantics to field names does not 
> scale as well as it might.
> For instance, the logical extension of the current FieldInfos system 
> is to add booleans as described at 
> <>.  However, 
> conflict resolution during segment merging is going to present 
> challenges.  What happens when in one segment 'content' has freq and 
> in another segment it doesn't?  Things are so much easier if the 
> posting format, once set, never changes.
I was thinking in the same direction. Global field semantics make our 
life with FI much easier in a single index. But even with global field 
semantics we would have the same problem with the 
IndexWriter.addIndexes() method, no? I'm curious about how you solved 
that conflict in KinoSearch? Btw, I like it that you don't force the 
user to define all fields up front but rather allow to add fields at any 
time. I think if we implement global field semantics in Lucene we should 
go the same way.

I'm going to respond in more detail to your other points in this email 
tomorrow. I want to read the KinoSearch specs first but it's already 
kind of late....

- Michael

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message