lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <>
Subject Re: modularization discussion
Date Fri, 06 May 2011 20:35:48 GMT

: To me, the third camp is just saying the proof is in the pudding.  If 
: you want to refactor, then go for it.  Just make sure everything still 
: works, which of course I know people will (but part of that means 
: actually running Solr, IMO).  Perhaps, more importantly don't get mad 
: that if I have only one day a week to work on Lucene/Solr that I spend 
: it putting a specific feature in a specific place.  Just because 
: something can/should be modularized, doesn't mean that a person working 
: in that area must do it before they add whatever they were working on.  
: For instance, if and when function queries are a module, I will add to 
: them there and be happy to do so.  In the meantime, I will likely add to 
: them in Solr if that is something I happen to be interested in at that 
: time b/c I can certainly add a new function in a day, but I can't 
: refactor the whole module _and_ add my new function in a day.


I want to get that printed on a t-shirt

the corrolarry issue in my mind...

I am happily in favor of code reuse and modularization in the abstract, 
and when it works in practice i'm plesantly delighted.

But when people talk about modularization as a goal, and make a laundry 
list things in solr that people think should be refactored into modules 
(w/o showing specifics of what that module would look like) then i have a 
hard time buying into some of these ideas panning out in a way that:
  a) is a useful module to people in and of itself
  b) doesn't hamstring the evolution/performance in solr.

To look at "faceting" as a concrete example, there are big the reasons 
faceting works so well in Solr: Solr has total control over the 
index, knows exactly when the index has changed to rebuild caches, has a 
strict schema so it can make sense of field types and 
pick faceting algos accordingly, has multi-phase distributed search 
approach to get exact counts efficiently across multiple shards, etc...
(and there are still a lot of additional enhancements and improvements 
that can be made to take even more advantage of knowledge solr has because 
it "owns" the index that we no one has had time to tackle)

I find it really hard to picture a way that this code could be refactored 
into a reusable module in such a way that it could have an API that would 
be easily usable outside of Solr -- and when i do get a glimmer of an 
inkling of what that might look like, that vision scares me because of how 
that API might then "hobble" Solr's ability to leverage it's total control 
of the underlying index to add additional performance/features.

To be crystal clear: I recognize that this is *my* hangup -- I am not 
suggesting that "I am short sighted and have little imagination 
therefore this code should never be modularized."

I'm trying to explain why i *personally* am hesitant and sceptical of how 
well modularizations of features like like this might actually work in 
practice, and why i'm not eager to jump in and contribute on a goal whose 
end result is something that i can't fully picture (and when i can picture 
it, i'm a little scared by what i see)

That doesn't mean i'm opposed to it happening -- i would love to live in 
the land of candy where houses are made of ginger bread and sugar plums 
grow on trees, I'm just too skeptical that such a land exists (or is as 
great as legend describes) to go slogging along on an epic journey to try 
and reach it -- i'm too old for that shit.

I'm certainly not going to stop anyone else fro going on that quest -- but 
i am entitled to voice my skepticism and concerns, just as adventursome 
folks are entitled to ignore me.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message