couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Dionne <dio...@dionne-associates.com>
Subject Re: Using rebar to install couchdb
Date Thu, 14 Oct 2010 22:52:12 GMT
+1 also

I think the convention is

./apps/couch_core/ebin
./apps/couch_core/src
./apps/couch_core/include
./apps/couch_core/priv
./apps/couch_http/ebin
....

rather than ./src/....

I like the idea of still using the existing build, which is awesome, and have it feed into
rebar so we can make use of reltools, etc, and templates for parameterizing the various .ini
files

doing it after the next release will be a good time to break everything

On Oct 14, 2010, at 4:03 PM, Robert Newson wrote:

> Paul,
> 
> Brilliant writeup and proposal. I'd like to see all those things
> happen pretty much as you said. Cleaning the cycles out will be much
> easier once things are broken out in that style.
> 
> +1
> 
> B.
> 
> On Thu, Oct 14, 2010 at 7:54 PM, Paul Davis <paul.joseph.davis@gmail.com> wrote:
>> On Wed, Oct 13, 2010 at 5:23 PM, Benoit Chesneau <bchesneau@gmail.com> wrote:
>>> In an attempt to start some merging with cloudant I would like to
>>> start by using rebar in our install process.
>>> 
>>> Like i see it, we could continue to use autotools to create the
>>> rebar.config files and other templates an then rebar for the final
>>> build and dependencies management. This changes as noticed by @davisp
>>> also imply we make our tree a little more OTP compliant. I would like
>>> to start this work asap.
>>> 
>>> Thoughts ?
>>> 
>>> - benoit
>>> 
>> 
>> So there's a couple issues at hand here which seem to be motivated by
>> the desire to start using tools like rebar.
>> 
>> Our current source tree is not compliant with some of the basic
>> Erlang/OTP conventions. This is both bad technically and socially.
>> Technically, it prevents us from easily integrating tools like rebar
>> that would help advanced users with things like making Erlang reltools
>> packages. Socially, it doesn't reflect well on us to members of the
>> Erlang community that may have otherwise become contributors. All
>> languages have a standard package layout and Erlang is no different.
>> 
>> The current CouchDB Erlang app has grown considerably. There's been
>> general consensus that we need to start splitting it up into smaller
>> applications that encompass specific functionality. There's been a bit
>> of effort in this direction, but its such a major change to source
>> file location it needs to have a community consensus to really start
>> working on seriously.
>> 
>> I don't think we should focus directly on the issue of integrating
>> rebar. It should definitely be a goal, but not at the cost of our
>> current situation. Noah Slater has maintained an excellent build
>> system for us as is shown by the number of people building CouchDB
>> from source and the number of packages available. While I have argued
>> with him on numerous occasions about details, I have come to the
>> conclusion that it is not possible for him to be wrong. I personally
>> attribute this to the fact that he's most likely an advanced robot
>> from the future. That said, Noah has voiced concerns to various ideas
>> and we should make sure that any of his concerns are fully addressed.
>> 
>> We should attempt to make sure that any tool support doesn't morph
>> into tool requirement. For instance, I think we should make sure that
>> its possible to keep compiling CouchDB without rebar and not come to
>> rely on it.
>> 
>> While I'd be more than happy to start in on this and handle all of the
>> build system refactoring to make this happen, I'm not going to start
>> until there's a community consensus on what needs to be done. There
>> are a couple paths that I could see us taking to make this happen. We
>> could just make the current source tree be rebar compatible and figure
>> out the build system to do the optional rebar build or we could also
>> take this chance to split the source code into multiple applications.
>> Personally, I'd prefer to take this opportunity to organize the code
>> with multiple erlang apps.
>> 
>> Too get the conversation rolling here's a first pass at a new app proposal:
>> 
>> etap:
>> 
>>    Nick Gerakines now releases etap as a single .erl file that can be
>> dropped into the test directory. This app should be removed in favor
>> of that method.
>> 
>> erlang-oauth:
>> 
>>    Should be renamed to just oauth. That erlang- prefix has bugged me
>> fore entirely too long.
>> 
>> mochiweb, ibrowse, oauth:
>> 
>>    Refactored to use standard src, include, ebin, priv directories to
>> be OTP compliant. This results in directories like
>> 
>>        ./src/$APP/ebin
>>        ./src/$APP/incldue
>>        ./src/$APP/priv
>>        ./src/$APP/src
>> 
>> couchdb:
>> 
>>    Each proposed app will be structured as described above. Proposed apps:
>> 
>>    couch_core: The core Erlang modules for storing docs and managing
>> "internal infrastructure"
>>    couch_view: The view engine as well as the holder for managing OS processes.
>>    couch_rep: couch_rep*.erl
>>    couch_externals: couch_external*.erl
>>    couch_httpd: couch_http*.erl
>> 
>> At the bottom of this email I made an initial pass through the
>> ./src/couchdb tree to classify file by file into the described apps.
>> There are also some minor warts in this split. Things like the core
>> couchdb classes currently require calling into the couch_view app and
>> what not. Given the "interesting" call graph for CouchDb internals, I
>> think we should avoid addressing this and aim to minimize the actual
>> change to source file contents during this move. After modules have
>> been moved we can then start on teasing out the various cyclic
>> dependencies as necessary.
>> 
>> Also, on a last note, seeing as this is a major change to the source
>> tree (and ideally 0 change to source code) it might be a good idea to
>> start this work just after we release 1.1.0.
>> 
>> 
>> What do you guys think?
>> 
>> Paul Davis
>> 
>> 
>> List of files by per app:
>> 
>> couch_core:
>> 
>>    ./src/couchdb/couch.erl
>>    ./src/couchdb/couch_app.erl
>>    ./src/couchdb/couch_auth_cache.erl
>>    ./src/couchdb/couch_btree.erl
>>    ./src/couchdb/couch_changes.erl
>>    ./src/couchdb/couch_config.erl
>>    ./src/couchdb/couch_config_writer.erl
>>    ./src/couchdb/couch_db.erl
>>    ./src/couchdb/couch_db.hrl
>>    ./src/couchdb/couch_db_update_notifier.erl
>>    ./src/couchdb/couch_db_update_notifier_sup.erl
>>    ./src/couchdb/couch_db_updater.erl
>>    ./src/couchdb/couch_doc.erl
>>    ./src/couchdb/couch_event_sup.erl
>>    ./src/couchdb/couch_file.erl
>>    ./src/couchdb/couch_key_tree.erl
>>    ./src/couchdb/couch_log.erl
>>    ./src/couchdb/couch_ref_counter.erl
>>    ./src/couchdb/couch_server.erl
>>    ./src/couchdb/couch_server_sup.erl
>>    ./src/couchdb/couch_stats_aggregator.erl
>>    ./src/couchdb/couch_stats_collector.erl
>>    ./src/couchdb/couch_stream.erl
>>    ./src/couchdb/couch_task_status.erl
>>    ./src/couchdb/couch_util.erl
>>    ./src/couchdb/couch_uuids.erl
>>    ./src/couchdb/priv/icu_driver/couch_icu_driver.c
>>    ./src/couchdb/priv/stat_descriptions.cfg.in
>> 
>> couch_view:
>> 
>>    ./src/couchdb/couch_native_process.erl
>>    ./src/couchdb/couch_os_process.erl
>>    ./src/couchdb/couch_query_servers.erl
>>    ./src/couchdb/couch_view.erl
>>    ./src/couchdb/couch_view_compactor.erl
>>    ./src/couchdb/couch_view_group.erl
>>    ./src/couchdb/couch_view_updater.erl
>>    ./src/couchdb/couch_work_queue.erl
>>    ./src/couchdb/priv/couch_js/http.c
>>    ./src/couchdb/priv/couch_js/http.h
>>    ./src/couchdb/priv/couch_js/main.c
>>    ./src/couchdb/priv/couch_js/utf8.c
>>    ./src/couchdb/priv/couch_js/utf8.h
>>    ./src/couchdb/priv/spawnkillable/couchspawnkillable.sh
>>    ./src/couchdb/priv/spawnkillable/couchspawnkillable_win.c
>> 
>> couch_rep:
>> 
>>    ./src/couchdb/couch_js_functions.hrl
>>    ./src/couchdb/couch_rep.erl
>>    ./src/couchdb/couch_rep_att.erl
>>    ./src/couchdb/couch_rep_changes_feed.erl
>>    ./src/couchdb/couch_rep_db_listener.erl
>>    ./src/couchdb/couch_rep_httpc.erl
>>    ./src/couchdb/couch_rep_missing_revs.erl
>>    ./src/couchdb/couch_rep_reader.erl
>>    ./src/couchdb/couch_rep_sup.erl
>>    ./src/couchdb/couch_rep_writer.erl
>> 
>> couch_externals:
>> 
>>    ./src/couchdb/couch_external_manager.erl
>>    ./src/couchdb/couch_external_server.erl
>> 
>> couch_httpd:
>> 
>>    ./src/couchdb/couch_httpd.erl
>>    ./src/couchdb/couch_httpd_auth.erl
>>    ./src/couchdb/couch_httpd_db.erl
>>    ./src/couchdb/couch_httpd_external.erl
>>    ./src/couchdb/couch_httpd_misc_handlers.erl
>>    ./src/couchdb/couch_httpd_oauth.erl
>>    ./src/couchdb/couch_httpd_rewrite.erl
>>    ./src/couchdb/couch_httpd_show.erl
>>    ./src/couchdb/couch_httpd_stats_handlers.erl
>>    ./src/couchdb/couch_httpd_vhost.erl
>>    ./src/couchdb/couch_httpd_view.erl
>> 


Mime
View raw message