lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless" <luc...@mikemccandless.com>
Subject Re: Payload API
Date Mon, 19 Nov 2007 19:03:06 GMT
"Yonik Seeley" <yonik@apache.org> wrote:
> On Nov 19, 2007 11:38 AM, Michael McCandless <lucene@mikemccandless.com>
> wrote:
> > > If we opt to treat payload like termBuffer and copy the bytes, then we
> > > need no offset member.
> >
> > I think I'd lean towards leaving payload "by reference"?
> 
> It seems difficult to allow the payload setter to reuse their byte[],
> unless we break back compatibility with other token filters.  Do you
> have a solution in mind?

I think I must be missing something.

The payload API is marked as experimental?  And, since we are changing
it anyway (& removing or deprecating the current experimental one),
how would using the "by reference" approach break backwards
compatibility?

Maybe you mean that each Token must be fully independent because there
are plenty of filters that hold onto each Token long after next() is
called again, and then serve them up again later, eg Grant's new
LUCENE-1058?

But this is why we have the Token next() method (which returns a
"stable" Token that the callee can't change later) vs the Token
next(Token) method which allows the callee to re-use the passed in
Token on the next call.

So eg in TokenStream.next() base implementation we would have to make
a new copy of the payload byte[] and set it in the copied token.  This
way when a caller calls next() they get a full private copy, even of
the by-reference payload.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message