couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Samuel Newson <rnew...@apache.org>
Subject Re: question: mix between the sharding level and couch_* applications
Date Fri, 04 Jul 2014 10:12:11 GMT
HI,

These are great questions, thank you.

Yes, single node couchdb still works, that’s a critical goal for the merge. Port 5986 points
directly to the couch_http*.erl modules we all know. Where they might make calls to fabric/mem3
they do so conditionally. If any couchdb 1.6 API is missing or broken on port 5986, then that’s
a regression that needs fixing before the release.

As for "preventing completely the sharding level" there are two answers. Initially, for the
merge and the 2.0 release, yes, absolutely, we should include the couch_http* modules. Going
forward, it’s important to remove that. A single node installation of couchdb 2.0 should
run through the full codebase (chttpd to fabric to rexi, calling mem3 as appropriate) so that
there are no surprises when moving to multiple nodes.

As you’ve seen, the core of couchdb has been extracted into couchdb-couch.git. What I hope
to see after the 2.0 merge is the further whittling of that repo down to its essentials (which
would be a similar thing that you’ve done in cowdb). Perhaps couchdb-couch becomes multiple
smaller repositories, extracting the {couch_file, couch_btree, couch_db, couch_db_updater}
heart of couchdb.

I can’t speak to what work is required to add clustering to new rcouch features but I hope
I’ve not made it harder to merge them into single-node (couchdb-couch.git) couch.

To take one example, the couch_replicator_manager has custom work for this merge specifically
to allow it work correctly for clustered and non-clustered _replicator databases (indeed,
both at once). It does not call fabric (the original bigcouch version did) and uses mem3 to
determine local ownership of shards (but only for clustered databases).

Summary, we (Cloudant as a whole, and myself as primary actor) are taking great pains to ensure
unclustered couchdb works for this release, for at least the reason that it helps everyone
verify that we’ve forgotten nothing in this really quite complicated code merging process.
All the existing tests should still pass, for what that’s worth.

B.

On 4 Jul 2014, at 10:34, Benoit Chesneau <bchesneau@gmail.com> wrote:

> Hi all,
> 
> While doing a review on the bigcouch branch (for the rcouch merge) I
> found that the sharding level is mixed with the current applications
> and I wonder if we could skip some of it.
> 
> The one that could probably easily be done is the usage of chttpd in
> couch_mrview:
> 
> src/couch_mrview/src/couch_mrview_http.erl
> src/couch_mrview/src/couch_mrview_show.erl
> 
> Why not having these changes directly in  chttpd. I could port the in
> the couch_httpd application (which contains the HTTP standalone api)
> in rcouch. It would also ease that merge.
> 
> Others I am not sure, but  it may be implemented differently. Or at
> least can someone can provides the detail for each? Why they are
> needed there?
> 
> mem3 is used in couch, couch_index, couch_mrview and couch_replicator:
> 
> src/couch/src/couch_changes.erl
> src/couch/src/couch_db.erl
> src/couch/src/couch_db_updater.erl
> src/couch/src/couch_server.erl
> src/couch/src/couch_util.erl
> src/couch_index/src/couch_index_server.erl
> src/couch_mrview/src/couch_mrview.erl
> src/couch_replicator/src/couch_replicator.app.src
> src/couch_replicator/src/couch_replicator_manager.erl
> 
> fabric is the other one:
> 
> src/couch/src/couch_changes.erl
> src/couch/src/couch_db.erl
> src/couch/src/couch_db_updater.erl
> src/couch_index/src/couch_index_server.erl
> src/couch_mrview/src/couch_mrview_show.erl
> 
> 
> I am not sure that fabric and mem3 need to be there. I didn't test it
> yet but do they prevent the standalone usage of couch ? Ie. can we
> prevent completely the sharding level?
> 
> Just some questions. I am really interested in them. I am asking that
> in the view of the rcouch merge. Also I think it's interesting to have
> a clear understanding of the new architecture and how things need to
> be articulated all together. It would also help anyone that want to
> ship internally custom versions of the apache couchdb. Maybe people
> from cloudant and some others already thought about that?
> 
> 
> 
> - benoit


Mime
View raw message