couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: question: mix between the sharding level and couch_* applications
Date Fri, 04 Jul 2014 16:52:45 GMT
On Fri, Jul 4, 2014 at 4:34 AM, Benoit Chesneau <bchesneau@gmail.com> wrote:
> Hi all,
>
> While doing a review on the bigcouch branch (for the rcouch merge) I
> found that the sharding level is mixed with the current applications
> and I wonder if we could skip some of it.
>
> The one that could probably easily be done is the usage of chttpd in
> couch_mrview:
>
> src/couch_mrview/src/couch_mrview_http.erl
> src/couch_mrview/src/couch_mrview_show.erl
>
> Why not having these changes directly in  chttpd. I could port the in
> the couch_httpd application (which contains the HTTP standalone api)
> in rcouch. It would also ease that merge.
>

These are referencing chttpd because that's where we implemented that
delayed response code. I think the answer in this case is to move the
chttpd function definitions to couch_httpd and then have chttpd proxy
through to them as it does for most of its library functions of this
nature.

> Others I am not sure, but  it may be implemented differently. Or at
> least can someone can provides the detail for each? Why they are
> needed there?
>
> mem3 is used in couch, couch_index, couch_mrview and couch_replicator:
>
> src/couch/src/couch_changes.erl
> src/couch/src/couch_db.erl
> src/couch/src/couch_db_updater.erl
> src/couch/src/couch_server.erl
> src/couch/src/couch_util.erl
> src/couch_index/src/couch_index_server.erl
> src/couch_mrview/src/couch_mrview.erl
> src/couch_replicator/src/couch_replicator.app.src
> src/couch_replicator/src/couch_replicator_manager.erl
>
> fabric is the other one:
>
> src/couch/src/couch_changes.erl
> src/couch/src/couch_db.erl
> src/couch/src/couch_db_updater.erl
> src/couch_index/src/couch_index_server.erl
> src/couch_mrview/src/couch_mrview_show.erl
>
>
> I am not sure that fabric and mem3 need to be there. I didn't test it
> yet but do they prevent the standalone usage of couch ? Ie. can we
> prevent completely the sharding level?
>

Most of this is that CouchDB is not 100% designed to work in a
cluster. For instance, everywhere that there's an assumption that it
can just grab all design documents we need to insert shims so that
it'll be a global call instead of trying to just check a local file. I
definitely consider each of these a wart and would like to be able to
remove them but unfortunately that can be quite difficult in some
cases. In the future I think it'd be a good idea on figuring out how
to pull these out as much as possible.

> Just some questions. I am really interested in them. I am asking that
> in the view of the rcouch merge. Also I think it's interesting to have
> a clear understanding of the new architecture and how things need to
> be articulated all together. It would also help anyone that want to
> ship internally custom versions of the apache couchdb. Maybe people
> from cloudant and some others already thought about that?
>
>
>
> - benoit

Bob's reply I think gives the general overview. For the merge we're
quite focused on making sure that the single node interface to CouchDB
has zero regressions. And internally we use the single-node interface
quite often even in the case of a cluster for working at the node
level which can be required for some operations.

On the other hand I think the question Benoit is really asking is if
we can ship a working single node without any of the clustering code
at all. I honestly don't know. Most of the cases I think would work
fine but it may be possible that we have something in there that'd
prevent it. In a perfect world that'd be the case but I would wager
we'd need to exert some effort to make it a reality. Though I don't
think its a case we had really considered happening though perhaps its
something we should be considering.

I would say I don't really think that requiring fabric/mem3 etc to
ship for single node CouchDB should be a blocker to merging seeing as
we've never had that sort of separation before. Though I would agree
its work we should be looking into doing to enable it after the merge.

Mime
View raw message