lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Earwin Burrfoot <>
Subject Re: IndexReader plugins
Date Tue, 14 Apr 2009 18:22:00 GMT
> The original example justification was to avoid putting a ValueSource in the
> IndexReader (I guess avoiding the funky init code? valueSource = new
> CachingValueSource(this, new UninversionValueSource(this))
That was a bit of drama for the sake of drama, I couldn't restrain myself :)
My justification is that besides ValueSource, I want to bind a pair of
my own classes to segments, and other lucene developers/users will
probably have similar needs later.
Lucene developers will add references to their components directly on
readers, users will produce ugly code to keep track of segments or
work off patched Lucene, what a mess!

> Filter caches, Query caches, etc seem easy enough to implement external to Lucene just
> using the Reader instance? I guess it helps with sharing (possible merging
> in RAM)?
Yes, I currently have a bunch of caches bound to reader, and they do a
full warmup after each reopen, before this reader starts being used
for searching.
I get ~0-30ms search time for ugly requests with ranges and lots of
and/or/notted filters with nontrivial sorts and clustering, but pay
with ~10-40s reopens.
Per-segment warmup will definetly help here, especially considering
the layout of my segments.

I'm not saying I can't do it without reader plugins. They will simply
save me (and, hopefully, not only me) from writing strange
segment-tracking code.

> Mike appears to have driven it towards modularizing the current pieces in
> SegmentReader now. Thats not so flexible though (as a generic plugin
> framework), and it seems like a bit of a leap over the original proposal.
I believe the api could handle both concerns - modularizing Lucene
innards and allowing external extensibility. Splitting two concerns
will produce two instances of code that does essentially the same

> The gain if we get this is possibly sharing the components and being able to
> plug in new impls (do we really want to support that, would it make sense
> based on current customization needs?)?
More like new types, not impls, I made several examples up there.
Switching impls for core components is nice for experimentation, but I
don't see absolutely winning cases right now.

> And I don't understand how initing vs inflight destroys the whole reason for
> the API - if you don't necessarily care that its reconfigurable inflight,
> whats the problem? The static constructors are not beautiful, but a leaky
> API would be worse - I'd rather unsuspecting Lucene users were not hit by
> trains :)
initing vs inflight is largely a matter of preference. Initing, I must
admit, is way more kosher, but inflight looks better :) I'm currently
thinking of a way to take the best of both.
What destroys the whole reason is the factory Michael offered - it
hardcodes component types that you can possibly bind to the reader.

> So Marvin has mentioned that that leads us to a bit of a hybrid approach,
> but Earwin doesn't like that?
I just don't see any compelling reason to go hybrid here. If you have
a method to bind 'any' plugin, why do you want to have a special
method to bind ABC plugin, no matter how 'core' is it?

> Hopefully I have not muddied the water further, but I can't get a clear grasp on this.
Happy to wait for example code, but if someone can unify the discussion in a bit more a succinct
way, I wouldnt complain ;)
I 'hope' to have some time this weekend, so I can provide examples.

Kirill Zakharenko/Кирилл Захаренко (
Home / Mobile: +7 (495) 683-567-4 / +7 (903) 5-888-423
ICQ: 104465785

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message