couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Dufour <nrduf...@gmail.com>
Subject Re: splitting the code in different apps or rewrite httpd layer
Date Fri, 20 Aug 2010 16:47:28 GMT
Hello,

I remember following your great tutorial on how to create a new
indexer (numidx).
A very good way to build a new one, but it did show the internal
architecture needs a serious refactor.
You have the impression everything is spread wherever it might fit.

Also I saw that Riak did such transformative change in their source
code and seemed to obtain a nice result with it. A clean way to
separate concerns.

I'm not an official couchdb dev but such change would really help us all.
(I can help on it too ;) ).

Thank you,

Nicolas Dufour
nrdufour@gmail.com

--
“Investment in knowledge pays the best interest.”
                               —Benjamin Franklin



On Fri, Aug 20, 2010 at 7:32 AM, Volker Mische <volker.mische@gmail.com> wrote:
> +1 for a refactor.
>
> GeoCouch duplicates a lot of code. I tried to keep the names in as similar
> (though meaningful) to the original ones as possible to see where the
> duplicated code is.
>
> I would love to see that everyone who wants a new kind of indexer just need
> to provide the data structure and all the design document handling, updater
> (group) handling, list functions etc is done automatically.
>
> Cheers,
>  Volker
>
> On 20.08.2010 13:06, Robert Dionne wrote:
>>
>> +1
>>
>> I would change the or in the subject line to and, .ie. do both :)
>>
>> I think this is an excellent idea and a good time to start this. At a
>> conceptual level CouchDB is dirt simple internally. This fact and it's use
>> of Erlang in my opinion should be seen as it's main advantage. One way to
>> leverage that advantage is to enable programmers who want to extend couch. I
>> know of at least three projects [1,2,3] that have done this. A good measure
>> of a successful refactor would be how much code these projects could throw
>> away.
>>
>> In my terminology prototype [3] I'm currently using bitcask for
>> persistence so I basically only extend the HTTP front end piece and need
>> programmatic access to the b-tree storage layer. All this needs to be is
>> some sort of mapping that let's one run a function over the b-tree, support
>> for ranges, and access to changes.
>>
>> Doing this is a thankless task, anyone already deeply familiar with the
>> internals would likely have little *interest* (academic, financial, etc..)
>> in it. CouchDB runs on phones now and in the cloud which is awesome and of
>> course a strong argument to maintain the simple design. As the complexity of
>> the code base increases however, the use of Erlang becomes a barrier to
>> entry.
>>
>> Best,
>>
>> Bob
>>
>> [1] http://github.com/normanb/couchdb-multiview
>> [2] http://github.com/vmx/couchdb
>> [3] http://github.com/bdionne/bitstore
>>
>>
>>
>>
>> On Aug 20, 2010, at 5:09 AM, Benoit Chesneau wrote:
>>
>>> Hi all,
>>>
>>> I work a lot these days around the httpd code and the more I work on
>>> the more I think we should refactor it to make it easier to hack and
>>> extend.  There is indeed a lot of code in one module (couch_httpd_db)
>>> and recent issue like vhost and location rewriting could be easier to
>>> solve if we had an http layer more organized in my opinion.
>>>
>>> Actually we do (in 1.0.1 or trunk) :
>>>
>>> request ->  couch_httpd loop ->  request_handler ->  check vhost
and
>>> eventually rewrite url ->  request_int ->  request_db ->  request
>>> doc|request _design | request attachment | request global handler |
>>> request misc handler
>>>
>>> with extra level : request_design ->  rewrite handler|
>>> show|lists|update\lview ... and request_int that catch all errors and
>>> has the responsibility to send errors if anything happend and wasn't
>>> catched on other layers.
>>>
>>> It could be easier. We could do it more resource oriented for example
>>> than it is. 1 module, 1 resource. Refactoring httpd code would also
>>> allow us to reuse more code than we do actually maybe by wrapping api.
>>>
>>> How :
>>>
>>> - Some times ago we started to port it using webmachine with davisp,
>>> but we didn't finish. Maybe it's a good time ? Or do we want to follow
>>> another way ?
>>>
>>> - If we go on this refactoring it could be also a good time to split
>>> couchdb in different apps : couchdb-core and couchdb foe example
>>> (maybe couchdb-appengine ?) so we could develop independantly each
>>> levels and make code history cleaner.
>>>
>>>
>>> Thoughts ?
>>>
>>>
>>> - benoit
>>
>
>

Mime
View raw message