couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Dufour <>
Subject Re: splitting the code in different apps or rewrite httpd layer
Date Fri, 20 Aug 2010 16:47:28 GMT

I remember following your great tutorial on how to create a new
indexer (numidx).
A very good way to build a new one, but it did show the internal
architecture needs a serious refactor.
You have the impression everything is spread wherever it might fit.

Also I saw that Riak did such transformative change in their source
code and seemed to obtain a nice result with it. A clean way to
separate concerns.

I'm not an official couchdb dev but such change would really help us all.
(I can help on it too ;) ).

Thank you,

Nicolas Dufour

“Investment in knowledge pays the best interest.”
                               —Benjamin Franklin

On Fri, Aug 20, 2010 at 7:32 AM, Volker Mische <> wrote:
> +1 for a refactor.
> GeoCouch duplicates a lot of code. I tried to keep the names in as similar
> (though meaningful) to the original ones as possible to see where the
> duplicated code is.
> I would love to see that everyone who wants a new kind of indexer just need
> to provide the data structure and all the design document handling, updater
> (group) handling, list functions etc is done automatically.
> Cheers,
>  Volker
> On 20.08.2010 13:06, Robert Dionne wrote:
>> +1
>> I would change the or in the subject line to and, .ie. do both :)
>> I think this is an excellent idea and a good time to start this. At a
>> conceptual level CouchDB is dirt simple internally. This fact and it's use
>> of Erlang in my opinion should be seen as it's main advantage. One way to
>> leverage that advantage is to enable programmers who want to extend couch. I
>> know of at least three projects [1,2,3] that have done this. A good measure
>> of a successful refactor would be how much code these projects could throw
>> away.
>> In my terminology prototype [3] I'm currently using bitcask for
>> persistence so I basically only extend the HTTP front end piece and need
>> programmatic access to the b-tree storage layer. All this needs to be is
>> some sort of mapping that let's one run a function over the b-tree, support
>> for ranges, and access to changes.
>> Doing this is a thankless task, anyone already deeply familiar with the
>> internals would likely have little *interest* (academic, financial, etc..)
>> in it. CouchDB runs on phones now and in the cloud which is awesome and of
>> course a strong argument to maintain the simple design. As the complexity of
>> the code base increases however, the use of Erlang becomes a barrier to
>> entry.
>> Best,
>> Bob
>> [1]
>> [2]
>> [3]
>> On Aug 20, 2010, at 5:09 AM, Benoit Chesneau wrote:
>>> Hi all,
>>> I work a lot these days around the httpd code and the more I work on
>>> the more I think we should refactor it to make it easier to hack and
>>> extend.  There is indeed a lot of code in one module (couch_httpd_db)
>>> and recent issue like vhost and location rewriting could be easier to
>>> solve if we had an http layer more organized in my opinion.
>>> Actually we do (in 1.0.1 or trunk) :
>>> request ->  couch_httpd loop ->  request_handler ->  check vhost
>>> eventually rewrite url ->  request_int ->  request_db ->  request
>>> doc|request _design | request attachment | request global handler |
>>> request misc handler
>>> with extra level : request_design ->  rewrite handler|
>>> show|lists|update\lview ... and request_int that catch all errors and
>>> has the responsibility to send errors if anything happend and wasn't
>>> catched on other layers.
>>> It could be easier. We could do it more resource oriented for example
>>> than it is. 1 module, 1 resource. Refactoring httpd code would also
>>> allow us to reuse more code than we do actually maybe by wrapping api.
>>> How :
>>> - Some times ago we started to port it using webmachine with davisp,
>>> but we didn't finish. Maybe it's a good time ? Or do we want to follow
>>> another way ?
>>> - If we go on this refactoring it could be also a good time to split
>>> couchdb in different apps : couchdb-core and couchdb foe example
>>> (maybe couchdb-appengine ?) so we could develop independantly each
>>> levels and make code history cleaner.
>>> Thoughts ?
>>> - benoit

View raw message