Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of paul.joseph.davis@gmail.com
 designates 74.125.46.28 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=nEeSOnlihTWtMjEnUmtf5lu3tjTt7DSI2rk/qqZ9Qe2wW6+n8ZQ6dOIRHoElUCBsdU
         iHDiUk+pDoE5G4oK7FgJsXUNxosMUxMz7HgfMBewnL5ygxDC+aGGS0ENokm0DOLxeolr
         UzM9FpU/VZCSlANeFhDZuQ7Fs0Rs4sKG0ikDs=
MIME-Version: 1.0
Date: Sat, 6 Jun 2009 01:06:17 -0400
Message-ID: <e2111bbb0906052206v186c3e52l24122e38c917cf1f@mail.gmail.com>
Subject: OTP-ification
From: Paul Davis <paul.joseph.davis@gmail.com>
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Hey,

I was just sitting around thinking about
http://issues.apache.org/jira/browse/COUCHDB-375 when I figured I'd
try and sketch out an OTP-ification layout for CouchDB. This is all
hand waving and basically I'm just trying to minimize the amount of
bi-directional dependencies through the stack. This would obviously
require some refactoring to make it so that everything in the list
only depends on things above it. Most notably would probably be that
we'd need to beef up couch_db_update_notifier.erl to allow for things
like stats and tasks to exist not inside the core.

Some random rationale for this layout:

* CouchDB - Should be runnable and installable without HTTP stuff for
use in other Erlang apps etc.
* CouchHTTPd - Next logical thing to add
* CouchRep - Ideally, I'd like to keep this in core, but I don't know
the logic in it to know if we can split it into an Erlang core version
and add an HTTP layer on top. Or something. Anyway, that's all I got.
* CouchView - Next logical step. Next on the agenda would be to
refactor this into CouchView and CouchMapReduce, where CouchView is
the mechanics of keeping all the gen servers alive and feeding
documents to an update function.
* Couch Tasks, Stats, Batch, External, and App - These seem like
they're 'feature' modules. I think that as we expand and possibly add
contirbuted features, we should promote them to maintain a self
contained directory which would allow people to disable all but the
bare essentials for whatever they need. Some side effects of this
would be how things like couch_query_server would need to be split up
and refactored. IMO this is a good thing as the split up main.js
indicates. Plus, splitting things up it seems like we'd have the
oppurtunity to add more efficient styles of transmission for each use
of couch_query_server instead of forcing a sync io access to
processes.

The awesomeness of this is that it allows us to start plugging things
like CouchShard in (ideally) completely transparently. I'm fairly
certain that as new bits start coming around we'll see how things
could be slightly more generic and refactor bits here and there to
accommodate, but that's how this sort of thing generally goes.


REALLY BIG NOTE: I just thought that i should mention, I'm in no way
at all what so ever even remotely hinting at the possibility that we
start distributing these bits separately. All of the things here are
part of the single distribution that is CouchDB, this is merely source
code and Erlang mechanics separations to help enforce non-spaghetti
dependencies so that we maintain flexibility, #buzzword, and
#otherbuzzword.


Erlang note: Each of these directories would contain a complete Erlang
app which has the src/ priv/ ebin/ and include/ directories. This
would fix up the original motivation for COUCHDB-375.

CouchDB
=======
couch_btree.erl
couch_config.erl
couch_config_writer.erl
couch_db.erl
couch_db_update_notifier.erl
couch_db_update_notifier_sup.erl
couch_db_updater.erl
couch_doc.erl
couch_event_sup.erl
couch_file.erl
couch_key_tree.erl
couch_log.erl
couch_os_process.erl
couch_ref_counter.erl
couch_server.erl
couch_server_sup.erl
couch_stream.erl
couch_util.erl

CouchHTTPd
==========
couch_httpd.erl
couch_httpd_db.erl
couch_httpd_misc_handlers.erl

CouchRep
========
couch_rep.erl
couch_rep_sup.erl

CouchView
=========
couch_httpd_view.erl
couch_query_servers.erl
couch_view.erl
couch_view_compactor.erl
couch_view_group.erl
couch_view_updater.erl

CouchTasks
==========
couch_task_status.erl

CouchStats
==========
couch_stats_aggregator.erl
couch_stats_collector.erl
couch_httpd_stats_handlers.erl

CouchBatch
==========
couch_batch_save.erl
couch_batch_save_sup.erl

CouchExternal
=============
couch_external_manager.erl
couch_external_server.erl
couch_httpd_external.erl

CouchApp
========
couch_httpd_show.erl


Paul Davis