lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley" <>
Subject Re: new Token API
Date Sun, 18 Nov 2007 20:01:59 GMT
On Nov 18, 2007 6:07 AM, Michael McCandless <> wrote:
> "Yonik Seeley" <> wrote:
> > 1) If we are deprecating some methods like String termText(), how
> > about at the same time deprecating "String type"?  If we want
> > lightweight per-token metadata for communication between filters, an
> > int or a long used as a bitvector (32 or 64 independent boolean vars
> > per token) would be much more useful than a single String.
> You mean replace String type with a series of booleans stored as
> bit-flags (and also allowing room for custom bit flags for the
> application)?

Yeah, I figured something like 16 for user, 16 for core in an int.

> This sounds nice but is there a compelling reason to do
> this now?

Now seemed a more natural time to consider it because we are already
making all these changes to Token, and the "String type" is almost
never used.  Adding an int for metadata is a related but separate
decision... we could deprecate "type" w/o adding anything (but then
StandardFilter would need to figure out type.)

>  Eg, I don't think "String type" costs us much performance
> loss now?

If it's unused, it's another field that clear() needs to take care of.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message