lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marvin Humphrey <>
Subject Re: Flexible index format / Payloads Cont'd
Date Tue, 04 Jul 2006 21:49:52 GMT

On Jul 4, 2006, at 3:35 AM, Doug Cutting wrote:

> Marvin Humphrey wrote:
>> IMO, this should wait.  It's going to be freakishly difficult to  
>> get this stuff to work and maintain the commitments that Doug has  
>> laid out for backwards compatibility.
> Perhaps we can implement an all-new index format, in a new package.

/me whistles low and grins.

org.apache.lucene.invindex?  As in inverted index, InvIndexer, and  

org.apache.lucene.ix? As in IxWriter and IxReader?

> An implementation of IndexReader can be provided to integrate with  
> existing search code.  And the ability to add an IndexReader to an  
> index can be provided to upgrade existing indexes to the new  
> format.  So the new code would not need to be able to process an  
> old index: the old code can continue to do that.  Does that make  
> sense?  Is that "freakishly difficult"?

It's labor-intensive -- that's a lot of code, to write and to test!   
But it would be a lot of code regardless, and it probably introduces  
fewer bugs and complications putting everything in a new package than  
interweaving so much new stuff into the existing code base.

The difficulty of keeping two packages afloat simultaneously will  
depend on how loose the coupling is between org.apache.lucene.index  
and the rest of Lucene.

> We'll need the ability to sniff a directory and tell which version  
> of index it contains, but that should not be too hard.

As simple as touching a meaningless file, if need be.  But I'll be  
arguing for the introduction of a global field definition file, which  
would serve just fine for that purpose.  ;)

Marvin Humphrey
Rectangular Research

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message