lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Busch <>
Subject Re: Flexible indexing
Date Mon, 12 Mar 2007 21:11:40 GMT
Marvin Humphrey wrote:
> On Mar 10, 2007, at 3:27 PM, Michael Busch wrote:
> I'm going to respond to this over several mails (: and possibly days 
> :) because there's an awful lot here, and I've already implemented a 
> lot of it in KS.
>> We should also make this public, so that users can store their own 
>> index metadata.
>> (Remark: LUCENE-783 is also a neat idea, we can write one xml parser 
>> for both items)
> There's a significant downside to allowing users to store arbitrary 
> data in an XML index file: you can't use a bare-bones parser, 
> hand-coded for a tiny, controlled subset of XML syntax and a limited 
> set of data structures.  You'd need a full-on XML encoder/decoder, 
> presumably an existing one that would be added as a dependency.
> The only reason that the KinoSearch's YAML codec requires only 600 
> lines of C is that it's a closed system.  No multi-line strings.  No 
> objects.  No nulls.  You get the picture.
That's a good point, Marvin. The parser would be much simpler if we had 
no open XML file. I think our best option here is to have a closed XML 
file for the index format/configuration (something like you sent in your 
other mail) plus a binary file for custom index-level metadata like 
Grant suggested.

Btw, I'm not really familiar with YAML. Maybe you could explain briefly 
why you chose YAML over XML in KinoSearch?

- Michael

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message