couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <>
Subject Re: Using rebar to install couchdb
Date Sun, 17 Oct 2010 19:20:52 GMT
Ace, Paul :)

Just some bikeshedding as I agree with most that is said here.

I'm wondering if the OS process handling could be its own module or
part of couch_core. While the View Server is the primary user, it is
not the only one and having OS process management in the view server
module just feels wrong to me :)


On 14 Oct 2010, at 20:54, Paul Davis wrote:

> On Wed, Oct 13, 2010 at 5:23 PM, Benoit Chesneau <> wrote:
>> In an attempt to start some merging with cloudant I would like to
>> start by using rebar in our install process.
>> Like i see it, we could continue to use autotools to create the
>> rebar.config files and other templates an then rebar for the final
>> build and dependencies management. This changes as noticed by @davisp
>> also imply we make our tree a little more OTP compliant. I would like
>> to start this work asap.
>> Thoughts ?
>> - benoit
> So there's a couple issues at hand here which seem to be motivated by
> the desire to start using tools like rebar.
> Our current source tree is not compliant with some of the basic
> Erlang/OTP conventions. This is both bad technically and socially.
> Technically, it prevents us from easily integrating tools like rebar
> that would help advanced users with things like making Erlang reltools
> packages. Socially, it doesn't reflect well on us to members of the
> Erlang community that may have otherwise become contributors. All
> languages have a standard package layout and Erlang is no different.
> The current CouchDB Erlang app has grown considerably. There's been
> general consensus that we need to start splitting it up into smaller
> applications that encompass specific functionality. There's been a bit
> of effort in this direction, but its such a major change to source
> file location it needs to have a community consensus to really start
> working on seriously.
> I don't think we should focus directly on the issue of integrating
> rebar. It should definitely be a goal, but not at the cost of our
> current situation. Noah Slater has maintained an excellent build
> system for us as is shown by the number of people building CouchDB
> from source and the number of packages available. While I have argued
> with him on numerous occasions about details, I have come to the
> conclusion that it is not possible for him to be wrong. I personally
> attribute this to the fact that he's most likely an advanced robot
> from the future. That said, Noah has voiced concerns to various ideas
> and we should make sure that any of his concerns are fully addressed.
> We should attempt to make sure that any tool support doesn't morph
> into tool requirement. For instance, I think we should make sure that
> its possible to keep compiling CouchDB without rebar and not come to
> rely on it.
> While I'd be more than happy to start in on this and handle all of the
> build system refactoring to make this happen, I'm not going to start
> until there's a community consensus on what needs to be done. There
> are a couple paths that I could see us taking to make this happen. We
> could just make the current source tree be rebar compatible and figure
> out the build system to do the optional rebar build or we could also
> take this chance to split the source code into multiple applications.
> Personally, I'd prefer to take this opportunity to organize the code
> with multiple erlang apps.
> Too get the conversation rolling here's a first pass at a new app proposal:
> etap:
>    Nick Gerakines now releases etap as a single .erl file that can be
> dropped into the test directory. This app should be removed in favor
> of that method.
> erlang-oauth:
>    Should be renamed to just oauth. That erlang- prefix has bugged me
> fore entirely too long.
> mochiweb, ibrowse, oauth:
>    Refactored to use standard src, include, ebin, priv directories to
> be OTP compliant. This results in directories like
>        ./src/$APP/ebin
>        ./src/$APP/incldue
>        ./src/$APP/priv
>        ./src/$APP/src
> couchdb:
>    Each proposed app will be structured as described above. Proposed apps:
>    couch_core: The core Erlang modules for storing docs and managing
> "internal infrastructure"
>    couch_view: The view engine as well as the holder for managing OS processes.
>    couch_rep: couch_rep*.erl
>    couch_externals: couch_external*.erl
>    couch_httpd: couch_http*.erl
> At the bottom of this email I made an initial pass through the
> ./src/couchdb tree to classify file by file into the described apps.
> There are also some minor warts in this split. Things like the core
> couchdb classes currently require calling into the couch_view app and
> what not. Given the "interesting" call graph for CouchDb internals, I
> think we should avoid addressing this and aim to minimize the actual
> change to source file contents during this move. After modules have
> been moved we can then start on teasing out the various cyclic
> dependencies as necessary.
> Also, on a last note, seeing as this is a major change to the source
> tree (and ideally 0 change to source code) it might be a good idea to
> start this work just after we release 1.1.0.
> What do you guys think?
> Paul Davis
> List of files by per app:
> couch_core:
>    ./src/couchdb/couch.erl
>    ./src/couchdb/couch_app.erl
>    ./src/couchdb/couch_auth_cache.erl
>    ./src/couchdb/couch_btree.erl
>    ./src/couchdb/couch_changes.erl
>    ./src/couchdb/couch_config.erl
>    ./src/couchdb/couch_config_writer.erl
>    ./src/couchdb/couch_db.erl
>    ./src/couchdb/couch_db.hrl
>    ./src/couchdb/couch_db_update_notifier.erl
>    ./src/couchdb/couch_db_update_notifier_sup.erl
>    ./src/couchdb/couch_db_updater.erl
>    ./src/couchdb/couch_doc.erl
>    ./src/couchdb/couch_event_sup.erl
>    ./src/couchdb/couch_file.erl
>    ./src/couchdb/couch_key_tree.erl
>    ./src/couchdb/couch_log.erl
>    ./src/couchdb/couch_ref_counter.erl
>    ./src/couchdb/couch_server.erl
>    ./src/couchdb/couch_server_sup.erl
>    ./src/couchdb/couch_stats_aggregator.erl
>    ./src/couchdb/couch_stats_collector.erl
>    ./src/couchdb/couch_stream.erl
>    ./src/couchdb/couch_task_status.erl
>    ./src/couchdb/couch_util.erl
>    ./src/couchdb/couch_uuids.erl
>    ./src/couchdb/priv/icu_driver/couch_icu_driver.c
>    ./src/couchdb/priv/
> couch_view:
>    ./src/couchdb/couch_native_process.erl
>    ./src/couchdb/couch_os_process.erl
>    ./src/couchdb/couch_query_servers.erl
>    ./src/couchdb/couch_view.erl
>    ./src/couchdb/couch_view_compactor.erl
>    ./src/couchdb/couch_view_group.erl
>    ./src/couchdb/couch_view_updater.erl
>    ./src/couchdb/couch_work_queue.erl
>    ./src/couchdb/priv/couch_js/http.c
>    ./src/couchdb/priv/couch_js/http.h
>    ./src/couchdb/priv/couch_js/main.c
>    ./src/couchdb/priv/couch_js/utf8.c
>    ./src/couchdb/priv/couch_js/utf8.h
>    ./src/couchdb/priv/spawnkillable/
>    ./src/couchdb/priv/spawnkillable/couchspawnkillable_win.c
> couch_rep:
>    ./src/couchdb/couch_js_functions.hrl
>    ./src/couchdb/couch_rep.erl
>    ./src/couchdb/couch_rep_att.erl
>    ./src/couchdb/couch_rep_changes_feed.erl
>    ./src/couchdb/couch_rep_db_listener.erl
>    ./src/couchdb/couch_rep_httpc.erl
>    ./src/couchdb/couch_rep_missing_revs.erl
>    ./src/couchdb/couch_rep_reader.erl
>    ./src/couchdb/couch_rep_sup.erl
>    ./src/couchdb/couch_rep_writer.erl
> couch_externals:
>    ./src/couchdb/couch_external_manager.erl
>    ./src/couchdb/couch_external_server.erl
> couch_httpd:
>    ./src/couchdb/couch_httpd.erl
>    ./src/couchdb/couch_httpd_auth.erl
>    ./src/couchdb/couch_httpd_db.erl
>    ./src/couchdb/couch_httpd_external.erl
>    ./src/couchdb/couch_httpd_misc_handlers.erl
>    ./src/couchdb/couch_httpd_oauth.erl
>    ./src/couchdb/couch_httpd_rewrite.erl
>    ./src/couchdb/couch_httpd_show.erl
>    ./src/couchdb/couch_httpd_stats_handlers.erl
>    ./src/couchdb/couch_httpd_vhost.erl
>    ./src/couchdb/couch_httpd_view.erl

View raw message