couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: Using rebar to install couchdb
Date Thu, 14 Oct 2010 18:54:26 GMT
On Wed, Oct 13, 2010 at 5:23 PM, Benoit Chesneau <bchesneau@gmail.com> wrote:
> In an attempt to start some merging with cloudant I would like to
> start by using rebar in our install process.
>
> Like i see it, we could continue to use autotools to create the
> rebar.config files and other templates an then rebar for the final
> build and dependencies management. This changes as noticed by @davisp
> also imply we make our tree a little more OTP compliant. I would like
> to start this work asap.
>
> Thoughts ?
>
> - benoit
>

So there's a couple issues at hand here which seem to be motivated by
the desire to start using tools like rebar.

Our current source tree is not compliant with some of the basic
Erlang/OTP conventions. This is both bad technically and socially.
Technically, it prevents us from easily integrating tools like rebar
that would help advanced users with things like making Erlang reltools
packages. Socially, it doesn't reflect well on us to members of the
Erlang community that may have otherwise become contributors. All
languages have a standard package layout and Erlang is no different.

The current CouchDB Erlang app has grown considerably. There's been
general consensus that we need to start splitting it up into smaller
applications that encompass specific functionality. There's been a bit
of effort in this direction, but its such a major change to source
file location it needs to have a community consensus to really start
working on seriously.

I don't think we should focus directly on the issue of integrating
rebar. It should definitely be a goal, but not at the cost of our
current situation. Noah Slater has maintained an excellent build
system for us as is shown by the number of people building CouchDB
from source and the number of packages available. While I have argued
with him on numerous occasions about details, I have come to the
conclusion that it is not possible for him to be wrong. I personally
attribute this to the fact that he's most likely an advanced robot
from the future. That said, Noah has voiced concerns to various ideas
and we should make sure that any of his concerns are fully addressed.

We should attempt to make sure that any tool support doesn't morph
into tool requirement. For instance, I think we should make sure that
its possible to keep compiling CouchDB without rebar and not come to
rely on it.

While I'd be more than happy to start in on this and handle all of the
build system refactoring to make this happen, I'm not going to start
until there's a community consensus on what needs to be done. There
are a couple paths that I could see us taking to make this happen. We
could just make the current source tree be rebar compatible and figure
out the build system to do the optional rebar build or we could also
take this chance to split the source code into multiple applications.
Personally, I'd prefer to take this opportunity to organize the code
with multiple erlang apps.

Too get the conversation rolling here's a first pass at a new app proposal:

etap:

    Nick Gerakines now releases etap as a single .erl file that can be
dropped into the test directory. This app should be removed in favor
of that method.

erlang-oauth:

    Should be renamed to just oauth. That erlang- prefix has bugged me
fore entirely too long.

mochiweb, ibrowse, oauth:

    Refactored to use standard src, include, ebin, priv directories to
be OTP compliant. This results in directories like

        ./src/$APP/ebin
        ./src/$APP/incldue
        ./src/$APP/priv
        ./src/$APP/src

couchdb:

    Each proposed app will be structured as described above. Proposed apps:

    couch_core: The core Erlang modules for storing docs and managing
"internal infrastructure"
    couch_view: The view engine as well as the holder for managing OS processes.
    couch_rep: couch_rep*.erl
    couch_externals: couch_external*.erl
    couch_httpd: couch_http*.erl

At the bottom of this email I made an initial pass through the
./src/couchdb tree to classify file by file into the described apps.
There are also some minor warts in this split. Things like the core
couchdb classes currently require calling into the couch_view app and
what not. Given the "interesting" call graph for CouchDb internals, I
think we should avoid addressing this and aim to minimize the actual
change to source file contents during this move. After modules have
been moved we can then start on teasing out the various cyclic
dependencies as necessary.

Also, on a last note, seeing as this is a major change to the source
tree (and ideally 0 change to source code) it might be a good idea to
start this work just after we release 1.1.0.


What do you guys think?

Paul Davis


List of files by per app:

couch_core:

    ./src/couchdb/couch.erl
    ./src/couchdb/couch_app.erl
    ./src/couchdb/couch_auth_cache.erl
    ./src/couchdb/couch_btree.erl
    ./src/couchdb/couch_changes.erl
    ./src/couchdb/couch_config.erl
    ./src/couchdb/couch_config_writer.erl
    ./src/couchdb/couch_db.erl
    ./src/couchdb/couch_db.hrl
    ./src/couchdb/couch_db_update_notifier.erl
    ./src/couchdb/couch_db_update_notifier_sup.erl
    ./src/couchdb/couch_db_updater.erl
    ./src/couchdb/couch_doc.erl
    ./src/couchdb/couch_event_sup.erl
    ./src/couchdb/couch_file.erl
    ./src/couchdb/couch_key_tree.erl
    ./src/couchdb/couch_log.erl
    ./src/couchdb/couch_ref_counter.erl
    ./src/couchdb/couch_server.erl
    ./src/couchdb/couch_server_sup.erl
    ./src/couchdb/couch_stats_aggregator.erl
    ./src/couchdb/couch_stats_collector.erl
    ./src/couchdb/couch_stream.erl
    ./src/couchdb/couch_task_status.erl
    ./src/couchdb/couch_util.erl
    ./src/couchdb/couch_uuids.erl
    ./src/couchdb/priv/icu_driver/couch_icu_driver.c
    ./src/couchdb/priv/stat_descriptions.cfg.in

couch_view:

    ./src/couchdb/couch_native_process.erl
    ./src/couchdb/couch_os_process.erl
    ./src/couchdb/couch_query_servers.erl
    ./src/couchdb/couch_view.erl
    ./src/couchdb/couch_view_compactor.erl
    ./src/couchdb/couch_view_group.erl
    ./src/couchdb/couch_view_updater.erl
    ./src/couchdb/couch_work_queue.erl
    ./src/couchdb/priv/couch_js/http.c
    ./src/couchdb/priv/couch_js/http.h
    ./src/couchdb/priv/couch_js/main.c
    ./src/couchdb/priv/couch_js/utf8.c
    ./src/couchdb/priv/couch_js/utf8.h
    ./src/couchdb/priv/spawnkillable/couchspawnkillable.sh
    ./src/couchdb/priv/spawnkillable/couchspawnkillable_win.c

couch_rep:

    ./src/couchdb/couch_js_functions.hrl
    ./src/couchdb/couch_rep.erl
    ./src/couchdb/couch_rep_att.erl
    ./src/couchdb/couch_rep_changes_feed.erl
    ./src/couchdb/couch_rep_db_listener.erl
    ./src/couchdb/couch_rep_httpc.erl
    ./src/couchdb/couch_rep_missing_revs.erl
    ./src/couchdb/couch_rep_reader.erl
    ./src/couchdb/couch_rep_sup.erl
    ./src/couchdb/couch_rep_writer.erl

couch_externals:

    ./src/couchdb/couch_external_manager.erl
    ./src/couchdb/couch_external_server.erl

couch_httpd:

    ./src/couchdb/couch_httpd.erl
    ./src/couchdb/couch_httpd_auth.erl
    ./src/couchdb/couch_httpd_db.erl
    ./src/couchdb/couch_httpd_external.erl
    ./src/couchdb/couch_httpd_misc_handlers.erl
    ./src/couchdb/couch_httpd_oauth.erl
    ./src/couchdb/couch_httpd_rewrite.erl
    ./src/couchdb/couch_httpd_show.erl
    ./src/couchdb/couch_httpd_stats_handlers.erl
    ./src/couchdb/couch_httpd_vhost.erl
    ./src/couchdb/couch_httpd_view.erl

Mime
View raw message