Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of simon.willnauer@googlemail.com
 designates 74.125.82.48 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=googlemail.com; s=gamma;
        h=mime-version:reply-to:in-reply-to:references:date:message-id
         :subject:from:to:content-type:content-transfer-encoding;
        b=uuxvA0+0O3x0Ktm53P123wBlkrRl7pq+j0YVRVE/hLHLtzlOla15YPmHTrvkQWlBQb
         qD1EpoXUEPJ/KI15WjnLo+N1cn516K0OI7m1ZR75OEfI9TsqeFPIWAFZfktLy0oE1EtL
         nPs5ad4JehyLKpT2/4gWTw2MSQaIai3YKVNX0=
MIME-Version: 1.0
Reply-To: simon.willnauer@gmail.com
In-Reply-To: <BANLkTinZEc7N6fA0FFwLStLGr50_ydc_KA@mail.gmail.com>
References: <BANLkTikk++8ge7hihkUWSkTspiuvg58icA@mail.gmail.com>
	<BANLkTikaPMeTXMLwge6Lbi27TVxP8u0fJA@mail.gmail.com>
	<BANLkTi=3+n8hmrPrLJRx4hZjGC=ppRv15g@mail.gmail.com>
	<BANLkTim1Ys+2UJQN+L7k-yk28kZcOutKuA@mail.gmail.com>
	<BANLkTiks95ek5WyzZ-vay_MAH4WmXj1MZw@mail.gmail.com>
	<BANLkTin+fvqGf3oi4Fs=Zj1sTsxO9ALy0w@mail.gmail.com>
	<BANLkTi=5nw4_w_NvJUq_DZZxzBpB-mec3A@mail.gmail.com>
	<BANLkTimwZJ0s1jBR7Gm1hqSBu7xF3vJxYA@mail.gmail.com>
	<E2363694-98EE-4DCB-B03B-4BD9AA73751E@apache.org>
	<25CE504A-0E2E-4CF3-A3B4-42EC88F10DF4@gmail.com>
	<BANLkTim7-J35=oLUJL1mj2vtLNam_SiHJg@mail.gmail.com>
	<5695D270-424F-4324-8214-37543F9BC6AF@apache.org>
	<alpine.DEB.2.00.1105061307200.5457@bester>
	<BANLkTiku=4ki6DyntyEV7Sn2LW_-OhhEyg@mail.gmail.com>
	<BANLkTinZEc7N6fA0FFwLStLGr50_ydc_KA@mail.gmail.com>
Date: Sat, 7 May 2011 13:10:38 +0200
Message-ID: <BANLkTikcYdx7X6pqm8TybRoQ1W9-1a4KuQ@mail.gmail.com>
Subject: Re: modularization discussion
From: Simon Willnauer <simon.willnauer@googlemail.com>
To: dev@lucene.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Sat, May 7, 2011 at 1:02 PM, Michael McCandless
<lucene@mikemccandless.com> wrote:
> OK I opened:
>
> =C2=A0 =C2=A0https://issues.apache.org/jira/browse/LUCENE-3079
awesome!

+1
>
> Mike
>
> http://blog.mikemccandless.com
>
> On Sat, May 7, 2011 at 6:46 AM, Michael McCandless
> <lucene@mikemccandless.com> wrote:
>> I agree! =C2=A0And I think you're saying the same thing as Grant.
>>
>> Ie, others are fully free to refactor stuff, as long as they don't
>> hurt Solr/Lucene (functionality, performance).
>>
>> But you are tempering that with a nice dose of reality (successfully
>> factoring out faceting will be insanely hard).
>>
>> I very much agree with that.
>>
>> And, I (and other refactor-itchers) very much want to hear the
>> specific technical skepticism/concerns on a given module: that
>> assessment is awesome and very useful. =C2=A0In fact, I love your
>> enumeration of how faceting is so well integrated into Solr so much
>> that I'll go open an issue (to factor out faceting), and put your list
>> in!
>>
>> I think this will mean, in practice, that the refactoring should
>> itself proceed in baby steps. =C2=A0Ie, birthing a new faceting module,
>> iterating on it, etc., and then at some point cutting Solr over to it,
>> are two events likely spread out substantially in time.
>>
>> Freedom to refactor/poach is the bread and butter of open source.
>>
>> Mike
>>
>> http://blog.mikemccandless.com
>>
>> On Fri, May 6, 2011 at 4:35 PM, Chris Hostetter
>> <hossman_lucene@fucit.org> wrote:
>>>
>>> : To me, the third camp is just saying the proof is in the pudding. =C2=
=A0If
>>> : you want to refactor, then go for it. =C2=A0Just make sure everything=
 still
>>> : works, which of course I know people will (but part of that means
>>> : actually running Solr, IMO). =C2=A0Perhaps, more importantly don't ge=
t mad
>>> : that if I have only one day a week to work on Lucene/Solr that I spen=
d
>>> : it putting a specific feature in a specific place. =C2=A0Just because
>>> : something can/should be modularized, doesn't mean that a person worki=
ng
>>> : in that area must do it before they add whatever they were working on=
.
>>> : For instance, if and when function queries are a module, I will add t=
o
>>> : them there and be happy to do so. =C2=A0In the meantime, I will likel=
y add to
>>> : them in Solr if that is something I happen to be interested in at tha=
t
>>> : time b/c I can certainly add a new function in a day, but I can't
>>> : refactor the whole module _and_ add my new function in a day.
>>>
>>> +1
>>>
>>> I want to get that printed on a t-shirt
>>>
>>> the corrolarry issue in my mind...
>>>
>>> I am happily in favor of code reuse and modularization in the abstract,
>>> and when it works in practice i'm plesantly delighted.
>>>
>>> But when people talk about modularization as a goal, and make a laundry
>>> list things in solr that people think should be refactored into modules
>>> (w/o showing specifics of what that module would look like) then i have=
 a
>>> hard time buying into some of these ideas panning out in a way that:
>>> =C2=A0a) is a useful module to people in and of itself
>>> =C2=A0b) doesn't hamstring the evolution/performance in solr.
>>>
>>> To look at "faceting" as a concrete example, there are big the reasons
>>> faceting works so well in Solr: Solr has total control over the
>>> index, knows exactly when the index has changed to rebuild caches, has =
a
>>> strict schema so it can make sense of field types and
>>> pick faceting algos accordingly, has multi-phase distributed search
>>> approach to get exact counts efficiently across multiple shards, etc...
>>> (and there are still a lot of additional enhancements and improvements
>>> that can be made to take even more advantage of knowledge solr has beca=
use
>>> it "owns" the index that we no one has had time to tackle)
>>>
>>> I find it really hard to picture a way that this code could be refactor=
ed
>>> into a reusable module in such a way that it could have an API that wou=
ld
>>> be easily usable outside of Solr -- and when i do get a glimmer of an
>>> inkling of what that might look like, that vision scares me because of =
how
>>> that API might then "hobble" Solr's ability to leverage it's total cont=
rol
>>> of the underlying index to add additional performance/features.
>>>
>>> To be crystal clear: I recognize that this is *my* hangup -- I am not
>>> suggesting that "I am short sighted and have little imagination
>>> therefore this code should never be modularized."
>>>
>>> I'm trying to explain why i *personally* am hesitant and sceptical of h=
ow
>>> well modularizations of features like like this might actually work in
>>> practice, and why i'm not eager to jump in and contribute on a goal who=
se
>>> end result is something that i can't fully picture (and when i can pict=
ure
>>> it, i'm a little scared by what i see)
>>>
>>> That doesn't mean i'm opposed to it happening -- i would love to live i=
n
>>> the land of candy where houses are made of ginger bread and sugar plums
>>> grow on trees, I'm just too skeptical that such a land exists (or is as
>>> great as legend describes) to go slogging along on an epic journey to t=
ry
>>> and reach it -- i'm too old for that shit.
>>>
>>> I'm certainly not going to stop anyone else fro going on that quest -- =
but
>>> i am entitled to voice my skepticism and concerns, just as adventursome
>>> folks are entitled to ignore me.
>>>
>>>
>>> -Hoss
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org