lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <gsing...@apache.org>
Subject Re: attribute thoughts
Date Thu, 13 Aug 2009 15:21:13 GMT

On Aug 13, 2009, at 10:29 AM, Yonik Seeley wrote:

> I'm liking the new attribute based analysis (in conjunction with
> reusability), but I'm running into some questions...
>
> Is it valid for tokenizers or token filters add new attributes after
> their constructor (after they have processed some tokens)?

I don't know if it explicitly prohibits it, but I believe the docs  
discourage it for performance reasons.


>
> Should restoreState() be able to add attributes (it currently throws
> an exception)?  If not, does that mean that it's not supported/advised
> to use state across different TokenStreams?

Not sure

>
> We've previously seen that the native java clone() can be much slower
> than implementing it ourselves in Java.  Should we have our own
> clone() method on Attribute?  Or just implement clone() ourselves and
> require that subclasses override if needed?  This is inner-loop
> per-token stuff, and a single captureState() will invoke many clone
> operations (6 attributes make up the legacy Token object).


Sounds reasonable, but I haven't benchmarked.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message