lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Lalevée <>
Subject Re: Flexible index format / Payloads Cont'd
Date Fri, 21 Jul 2006 08:23:53 GMT
Le Jeudi 20 Juillet 2006 22:18, Marvin Humphrey a écrit :
> On Jul 19, 2006, at 10:26 AM, Nicolas Lalevée wrote:
> > Then I looked deeper in the Lucene file format, and I manage to
> > introduce some
> > generic field metadata without breaking the file format
> > compatibility. I just
> > used another bit of the "Bits" to mark that there is or not some
> > metadata on
> > the field. And the metadata is stored next to it :
> > DocFieldData --> FieldCount, <FieldNum, Bits, FieldMetadata,
> > Value>^FieldCount
> > FieldMetadata --> ValueSize, <Byte>^ValueSize
> My thought is instead of providing an ever-lengthening fixed menu of
> field-types to choose from, that the menu should be per-index and the
> codec should be indicated by an integer pointing to a spot on that menu.

In fact, that was my first implementaion. The problem with that is you can 
only store one value. But thinking a little more about it, storing one or 
more value is not an issue, because with the solution I proposed, no space is 
saved at all.
In fact, when I thought about this format of field metadata, I was thinking 
about a way to make the Lucene user specify how to store it in the Lucene 
index format. For instance, the simple one would specify that it's a pointeur 
on some metadata (as you proposed), another one would specify that there are 
two pointeurs (in my use case, one for type, the other one for the language), 
and another one whould specify that it will be store directly as it is 
actually an integer (so no need to make a pointer on integer. But it was just 
a thought, I don't know if it is possible. WDYT ?

> > Does this feature interest the Lucene commiters ? Should I provide
> > a patch in
> > Jira? If not, is there any common place where to provide some patch
> > for some
> > Lucene hackers (ie not necessaraily commiters) ?
> >
> > So, Marvin, could you provide your patch about payload ?
> I'm totally slammed this month because I got a talk accepted at OSCON
> late and so I'm taking an unexpected week off in the midst of a very
> busy time.

So, have a nice OSCON ! ;)

> There is not a patch per se, in any case. 

Oh yes of course. In fact Michael have already done something, I have switched 
the names, sorry.
So, Michael, could you provide your patch about payload ?

> > And is there a wiki page where there is a starting point about
> > defining the
> > future index format ?

ok thank you.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message