couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Russell Branca <chewbra...@apache.org>
Subject Re: rcouch merge: integration with the bigcouch branch questions
Date Mon, 07 Jul 2014 21:44:49 GMT
I switched to using chttpd in couch_mrview for delayed responses as it
has more robust error handling logic (in particular it fixes a nasty
bug where error messages were sent as full responses with headers in
the stream) .

Benoit, I don't understand what you mean by "Which makes this
functions unsable with the standalone HTTP api." The change in
couch_mrview to use chttpd should _not_ prevent the single node
interface from operating, if you're encountering that then it's a bug.

IMO the logical next step for all this is to refactor the separate
httpd layers into a single app and address the issues there, but I
think that can wait for 2.x.

Also, for "port the couch_mrview http modules to chttpd for the layer
part *and* couch_httpd" I don't think this is the correct approach. I
think we should define the core http layer in c(ouch_)httpd and then
continue to use the pattern of the separate applications defining the
relevant http bits.

I've already done a lot of legwork to make the http view layer usable
in both couch_httpd and chttpd with the long term game plan of
consolidating down to only one http layer. For instance, the clustered
view logic uses the existing couch_mrview_http code for actually
sending the responses:
https://github.com/apache/couchdb-chttpd/blob/master/src/chttpd_view.erl#L57.

Compare these two functions:

https://github.com/apache/couchdb-chttpd/blob/master/src/chttpd_view.erl#L49-L62
https://github.com/apache/couchdb-couch-mrview/blob/master/src/couch_mrview_http.erl#L191-L204

You can see they're almost identical, aside from
`couch_mrview:query_view` vs `fabric:query_view`. I wanted to use the
view layer as an example of removing unnecessary duplication in the
two http layers, and to jump start the process of consolidation.
Hopefully that gives a good idea on approach and the parts that we
actually need to abstract.


-Russell

On Mon, Jul 7, 2014 at 2:04 PM, Benoit Chesneau <bchesneau@gmail.com> wrote:
> On Fri, Jul 4, 2014 at 6:38 PM, Paul Davis <paul.joseph.davis@gmail.com> wrote:
>> On Fri, Jul 4, 2014 at 5:01 AM, Benoit Chesneau <bchesneau@gmail.com> wrote:
>>>
>>> Some blocking points/questions I have about the integration of rcouch
>>> and bigcouch.
>>>
>>>
>>> - OTP release: the bigcouch and the rcouch branches are quite similar
>>> but I still wonder about the usage of this configure script. Is this
>>> really nedded? Can't we just rely on a makefile?
>>
>>
>> I'd think it'd be possible to figure out something for this. I really
>> don't consider it a "configure" script so much as a "bootstrap"
>> script. Could very well be possible to put this into a Makefile. I
>> don't think this is a merge blocker though as its a relatively minor
>> change regardless of what we decide.
>>
>>> - HTTP api. some recent additions have been made in couch_mrview that
>>> added chttpd usage. Which makes this functions unsable with the
>>> standalone HTTP api. Also the couch application  in the bigcouch
>>> branch still contains couch_httpd* modules. Rcouch extracted them in a
>>> full erlang app: couch_httpd. My question is do we still need the
>>> legacy api (my understanding is that it is still used as standalone
>>> node frontend in bigcouch) ? If yes I propose to have a better
>>> separation in the api. There maybe 2 ways:
>>>     - port the couch_mrview http modules to chttpd for the layer part
>>> *and* couch_httpd
>>>     -  have different functions or better modules for chttpd and
>>> couch_httpd in couch_mrview (couch_mrview_httpd_*.erl and
>>> (couch_mrview_chttpd_*.erl))
>>
>>
>> The chttpd/couch_httpd_* split is a bit historical. The merge plan has
>> been to maintain the separation through the merge and then have
>> post-merge work that will parameterize them at runtime to work with
>> either single-node or clustered APIs underneath. As to extracting them
>> into a separate app that's not impossible but not something we've done
>> in the BigCouch branch because it hasn't happened on master. I think
>> it'd be a good idea to do post bigcouch merge when we're bringing in
>> rcouch.
>
> The couch_httpd extraction is already done in rcouch. The question is
> more having a mix of chttpd and couch_httpd in the couch_mrview app.
>>
>>> - couch_collate nif to replace couch_drv + ejson_compare nif . Is this
>>> OK for you? It's extensively tested there.
>>
>>
>> We don't use ejson_compare at all so from the BigCouch point of view
>> it's just a matter of replacing couch_drv which would be fine by me.
>> The only thing I remember about couch_collate was wanting to
>> investigate the way that the collation contexts are moved around as it
>> looked fairly complicated. But mostly I think that comes down to
>> measuring how heavy weight of an operation it is to create and destroy
>> those contexts.
>>
>>> - how the fauxton build is handled right now in the bigcouch branch.
>>> What are requirements to make it automatic? (no manual installation).
>>> Is there a way to disable it
>>
>>
>> We haven't spent too much time on Fauxton yet so I don't have much to
>> add here. I was under the impression that its build system was
>> divorced from the autotools build enough that it'd be easy to slip
>> into a rebar build at some step. I'm not sure what you mean about
>> manual installation or disabling it.
>>
>>> - I don't see any unittests about the sharding api. How to test the
>>> view changes is correctly working in the sharding api? How does
>>> cloudant?
>>
>>
>> We certainly don't have the best test coverage for mem3/fabric.
>> Internally there are some Python based integration tests that are used
>> as part of our build system. We've certainly thought about refactoring
>> this layer to provide better testability but have never had the time
>> or priorities to make it happen. We've considered open sourcing some
>> of our integration test suites but so far the priority has been to get
>> the bigcouch merge done before we go trying to extract all of that
>> code.
>>
>>> Hopefully someone can answer to them. I plan to merge both branches ASAP.
>>>
>>> - benoit

Mime
View raw message