Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of nrdufour@gmail.com designates
 209.85.216.52 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc:content-type:content-transfer-encoding;
        b=UXg3kLHPtAyWRB1b4fOwAfGWsSQJOc6Oqb4oV3XCMafzhnPtishmXWwXlUE9PQfww9
         m26t8auZz6cwl/pwoxXcS2Soo7cwsoBxQFvkZTfxa3aTWK4YUFu8KqiYaLeVBAnP0uD5
         Urq4CmJfzOdMajHU3F6F93OwRP4UhRTyc74JE=
MIME-Version: 1.0
In-Reply-To: <4C6E67B6.3050509@gmail.com>
References: <AANLkTinQDa=BePucd4-UWnxbLJyDbhrPbb4EdG+c32fV@mail.gmail.com>
	<95D3E102-B5DF-4815-BC3C-628EE4AB4C19@dionne-associates.com>
	<4C6E67B6.3050509@gmail.com>
Date: Fri, 20 Aug 2010 12:47:28 -0400
Message-ID: <AANLkTimeXacHE2tOdnmE6ADkkBQhUUDYDrPp5j6+bJB9@mail.gmail.com>
Subject: Re: splitting the code in different apps or rewrite httpd layer
From: Nicolas Dufour <nrdufour@gmail.com>
To: dev@couchdb.apache.org
Cc: Robert Dionne <dionne@dionne-associates.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hello,

I remember following your great tutorial on how to create a new
indexer (numidx).
A very good way to build a new one, but it did show the internal
architecture needs a serious refactor.
You have the impression everything is spread wherever it might fit.

Also I saw that Riak did such transformative change in their source
code and seemed to obtain a nice result with it. A clean way to
separate concerns.

I'm not an official couchdb dev but such change would really help us all.
(I can help on it too ;) ).

Thank you,

Nicolas Dufour
nrdufour@gmail.com

--
=E2=80=9CInvestment in knowledge pays the best interest.=E2=80=9D
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0 =E2=80=94Benjamin Franklin


On Fri, Aug 20, 2010 at 7:32 AM, Volker Mische <volker.mische@gmail.com> wr=
ote:
> +1 for a refactor.
>
> GeoCouch duplicates a lot of code. I tried to keep the names in as simila=
r
> (though meaningful) to the original ones as possible to see where the
> duplicated code is.
>
> I would love to see that everyone who wants a new kind of indexer just ne=
ed
> to provide the data structure and all the design document handling, updat=
er
> (group) handling, list functions etc is done automatically.
>
> Cheers,
> =C2=A0Volker
>
> On 20.08.2010 13:06, Robert Dionne wrote:
>>
>> +1
>>
>> I would change the or in the subject line to and, .ie. do both :)
>>
>> I think this is an excellent idea and a good time to start this. At a
>> conceptual level CouchDB is dirt simple internally. This fact and it's u=
se
>> of Erlang in my opinion should be seen as it's main advantage. One way t=
o
>> leverage that advantage is to enable programmers who want to extend couc=
h. I
>> know of at least three projects [1,2,3] that have done this. A good meas=
ure
>> of a successful refactor would be how much code these projects could thr=
ow
>> away.
>>
>> In my terminology prototype [3] I'm currently using bitcask for
>> persistence so I basically only extend the HTTP front end piece and need
>> programmatic access to the b-tree storage layer. All this needs to be is
>> some sort of mapping that let's one run a function over the b-tree, supp=
ort
>> for ranges, and access to changes.
>>
>> Doing this is a thankless task, anyone already deeply familiar with the
>> internals would likely have little *interest* (academic, financial, etc.=
.)
>> in it. CouchDB runs on phones now and in the cloud which is awesome and =
of
>> course a strong argument to maintain the simple design. As the complexit=
y of
>> the code base increases however, the use of Erlang becomes a barrier to
>> entry.
>>
>> Best,
>>
>> Bob
>>
>> [1] http://github.com/normanb/couchdb-multiview
>> [2] http://github.com/vmx/couchdb
>> [3] http://github.com/bdionne/bitstore
>>
>>
>>
>>
>> On Aug 20, 2010, at 5:09 AM, Benoit Chesneau wrote:
>>
>>> Hi all,
>>>
>>> I work a lot these days around the httpd code and the more I work on
>>> the more I think we should refactor it to make it easier to hack and
>>> extend. =C2=A0There is indeed a lot of code in one module (couch_httpd_=
db)
>>> and recent issue like vhost and location rewriting could be easier to
>>> solve if we had an http layer more organized in my opinion.
>>>
>>> Actually we do (in 1.0.1 or trunk) :
>>>
>>> request -> =C2=A0couch_httpd loop -> =C2=A0request_handler -> =C2=A0che=
ck vhost and
>>> eventually rewrite url -> =C2=A0request_int -> =C2=A0request_db -> =C2=
=A0request
>>> doc|request _design | request attachment | request global handler |
>>> request misc handler
>>>
>>> with extra level : request_design -> =C2=A0rewrite handler|
>>> show|lists|update\lview ... and request_int that catch all errors and
>>> has the responsibility to send errors if anything happend and wasn't
>>> catched on other layers.
>>>
>>> It could be easier. We could do it more resource oriented for example
>>> than it is. 1 module, 1 resource. Refactoring httpd code would also
>>> allow us to reuse more code than we do actually maybe by wrapping api.
>>>
>>> How :
>>>
>>> - Some times ago we started to port it using webmachine with davisp,
>>> but we didn't finish. Maybe it's a good time ? Or do we want to follow
>>> another way ?
>>>
>>> - If we go on this refactoring it could be also a good time to split
>>> couchdb in different apps : couchdb-core and couchdb foe example
>>> (maybe couchdb-appengine ?) so we could develop independantly each
>>> levels and make code history cleaner.
>>>
>>>
>>> Thoughts ?
>>>
>>>
>>> - benoit
>>
>
>