lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <>
Subject Re: modularization discussion
Date Fri, 06 May 2011 20:45:19 GMT
+1, +1, +1, +1, +1, +1 - This is just what I've been trying (and seemingly failing) to express.

On May 6, 2011, at 4:35 PM, Chris Hostetter wrote:

> : To me, the third camp is just saying the proof is in the pudding.  If 
> : you want to refactor, then go for it.  Just make sure everything still 
> : works, which of course I know people will (but part of that means 
> : actually running Solr, IMO).  Perhaps, more importantly don't get mad 
> : that if I have only one day a week to work on Lucene/Solr that I spend 
> : it putting a specific feature in a specific place.  Just because 
> : something can/should be modularized, doesn't mean that a person working 
> : in that area must do it before they add whatever they were working on.  
> : For instance, if and when function queries are a module, I will add to 
> : them there and be happy to do so.  In the meantime, I will likely add to 
> : them in Solr if that is something I happen to be interested in at that 
> : time b/c I can certainly add a new function in a day, but I can't 
> : refactor the whole module _and_ add my new function in a day.
> +1
> I want to get that printed on a t-shirt
> the corrolarry issue in my mind...
> I am happily in favor of code reuse and modularization in the abstract, 
> and when it works in practice i'm plesantly delighted.
> But when people talk about modularization as a goal, and make a laundry 
> list things in solr that people think should be refactored into modules 
> (w/o showing specifics of what that module would look like) then i have a 
> hard time buying into some of these ideas panning out in a way that:
>  a) is a useful module to people in and of itself
>  b) doesn't hamstring the evolution/performance in solr.
> To look at "faceting" as a concrete example, there are big the reasons 
> faceting works so well in Solr: Solr has total control over the 
> index, knows exactly when the index has changed to rebuild caches, has a 
> strict schema so it can make sense of field types and 
> pick faceting algos accordingly, has multi-phase distributed search 
> approach to get exact counts efficiently across multiple shards, etc...
> (and there are still a lot of additional enhancements and improvements 
> that can be made to take even more advantage of knowledge solr has because 
> it "owns" the index that we no one has had time to tackle)
> I find it really hard to picture a way that this code could be refactored 
> into a reusable module in such a way that it could have an API that would 
> be easily usable outside of Solr -- and when i do get a glimmer of an 
> inkling of what that might look like, that vision scares me because of how 
> that API might then "hobble" Solr's ability to leverage it's total control 
> of the underlying index to add additional performance/features.
> To be crystal clear: I recognize that this is *my* hangup -- I am not 
> suggesting that "I am short sighted and have little imagination 
> therefore this code should never be modularized."
> I'm trying to explain why i *personally* am hesitant and sceptical of how 
> well modularizations of features like like this might actually work in 
> practice, and why i'm not eager to jump in and contribute on a goal whose 
> end result is something that i can't fully picture (and when i can picture 
> it, i'm a little scared by what i see)
> That doesn't mean i'm opposed to it happening -- i would love to live in 
> the land of candy where houses are made of ginger bread and sugar plums 
> grow on trees, I'm just too skeptical that such a land exists (or is as 
> great as legend describes) to go slogging along on an epic journey to try 
> and reach it -- i'm too old for that shit.
> I'm certainly not going to stop anyone else fro going on that quest -- but 
> i am entitled to voice my skepticism and concerns, just as adventursome 
> folks are entitled to ignore me.
> -Hoss
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

- Mark Miller

Lucene/Solr User Conference
May 25-26, San Francisco

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message