lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Willnauer <>
Subject Re: modularization discussion
Date Sat, 07 May 2011 11:10:38 GMT
On Sat, May 7, 2011 at 1:02 PM, Michael McCandless
<> wrote:
> OK I opened:

> Mike
> On Sat, May 7, 2011 at 6:46 AM, Michael McCandless
> <> wrote:
>> I agree!  And I think you're saying the same thing as Grant.
>> Ie, others are fully free to refactor stuff, as long as they don't
>> hurt Solr/Lucene (functionality, performance).
>> But you are tempering that with a nice dose of reality (successfully
>> factoring out faceting will be insanely hard).
>> I very much agree with that.
>> And, I (and other refactor-itchers) very much want to hear the
>> specific technical skepticism/concerns on a given module: that
>> assessment is awesome and very useful.  In fact, I love your
>> enumeration of how faceting is so well integrated into Solr so much
>> that I'll go open an issue (to factor out faceting), and put your list
>> in!
>> I think this will mean, in practice, that the refactoring should
>> itself proceed in baby steps.  Ie, birthing a new faceting module,
>> iterating on it, etc., and then at some point cutting Solr over to it,
>> are two events likely spread out substantially in time.
>> Freedom to refactor/poach is the bread and butter of open source.
>> Mike
>> On Fri, May 6, 2011 at 4:35 PM, Chris Hostetter
>> <> wrote:
>>> : To me, the third camp is just saying the proof is in the pudding.  If
>>> : you want to refactor, then go for it.  Just make sure everything still
>>> : works, which of course I know people will (but part of that means
>>> : actually running Solr, IMO).  Perhaps, more importantly don't get mad
>>> : that if I have only one day a week to work on Lucene/Solr that I spend
>>> : it putting a specific feature in a specific place.  Just because
>>> : something can/should be modularized, doesn't mean that a person working
>>> : in that area must do it before they add whatever they were working on.
>>> : For instance, if and when function queries are a module, I will add to
>>> : them there and be happy to do so.  In the meantime, I will likely add to
>>> : them in Solr if that is something I happen to be interested in at that
>>> : time b/c I can certainly add a new function in a day, but I can't
>>> : refactor the whole module _and_ add my new function in a day.
>>> +1
>>> I want to get that printed on a t-shirt
>>> the corrolarry issue in my mind...
>>> I am happily in favor of code reuse and modularization in the abstract,
>>> and when it works in practice i'm plesantly delighted.
>>> But when people talk about modularization as a goal, and make a laundry
>>> list things in solr that people think should be refactored into modules
>>> (w/o showing specifics of what that module would look like) then i have a
>>> hard time buying into some of these ideas panning out in a way that:
>>>  a) is a useful module to people in and of itself
>>>  b) doesn't hamstring the evolution/performance in solr.
>>> To look at "faceting" as a concrete example, there are big the reasons
>>> faceting works so well in Solr: Solr has total control over the
>>> index, knows exactly when the index has changed to rebuild caches, has a
>>> strict schema so it can make sense of field types and
>>> pick faceting algos accordingly, has multi-phase distributed search
>>> approach to get exact counts efficiently across multiple shards, etc...
>>> (and there are still a lot of additional enhancements and improvements
>>> that can be made to take even more advantage of knowledge solr has because
>>> it "owns" the index that we no one has had time to tackle)
>>> I find it really hard to picture a way that this code could be refactored
>>> into a reusable module in such a way that it could have an API that would
>>> be easily usable outside of Solr -- and when i do get a glimmer of an
>>> inkling of what that might look like, that vision scares me because of how
>>> that API might then "hobble" Solr's ability to leverage it's total control
>>> of the underlying index to add additional performance/features.
>>> To be crystal clear: I recognize that this is *my* hangup -- I am not
>>> suggesting that "I am short sighted and have little imagination
>>> therefore this code should never be modularized."
>>> I'm trying to explain why i *personally* am hesitant and sceptical of how
>>> well modularizations of features like like this might actually work in
>>> practice, and why i'm not eager to jump in and contribute on a goal whose
>>> end result is something that i can't fully picture (and when i can picture
>>> it, i'm a little scared by what i see)
>>> That doesn't mean i'm opposed to it happening -- i would love to live in
>>> the land of candy where houses are made of ginger bread and sugar plums
>>> grow on trees, I'm just too skeptical that such a land exists (or is as
>>> great as legend describes) to go slogging along on an epic journey to try
>>> and reach it -- i'm too old for that shit.
>>> I'm certainly not going to stop anyone else fro going on that quest -- but
>>> i am entitled to voice my skepticism and concerns, just as adventursome
>>> folks are entitled to ignore me.
>>> -Hoss
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>>> For additional commands, e-mail:
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message