lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Muir <rcm...@gmail.com>
Subject Re: Analyzer forcing tokenStream and reusableTokenStream to be final
Date Tue, 19 Oct 2010 15:59:57 GMT
On Tue, Oct 19, 2010 at 11:52 AM, Shai Erera <serera@gmail.com> wrote:
> I still don't understand how not declaring my tokenStream and
> reusableTokenStream final can break anything. The methods are there (in my
> analyzers), and if I risk overriding them somewhere else it's my problem.
>

Well it is your problem, but we created it with our confusing APIs :)

So if you subclass your analyzer but only implement tokenStream and
not also reusableTokenStream, you get very terrible performance like
https://issues.apache.org/jira/browse/LUCENE-2279

By enforcing these to be final we prevent the trap where you subclass
and don't implement reusableTokenStream and get bad performance, but
its still not completely solved.
There is still the trap (especially with the attributes-based API,
even more overhead), that you just implement an Analyzer with only
tokenStream and get bad performance.

If we only had one of these methods, lets say called "tokenStream",
and it was reusable, we could remove these final checks completely.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message