couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hans J Schroeder ...@cloudno.de>
Subject Re: OTP releases and autotools
Date Sat, 29 Sep 2012 11:43:49 GMT

Hi,

It would be great to have a universal, standardized build environment for CouchDB and its
variants. Additionally we should offer prebuilt packages because the average user likes to
start using CouchDB without having to hassle with the build.

As CouchDB is so great for syncing with mobile apps we should also support iOS and Android
out of the box (if that is possible with the proposed build environment).

- Hans   

On Sep 29, 2012, at 10:48 AM, Benoit Chesneau wrote:

> Hi all,
> 
> This mail try to summarize the current build process and propose a
> solution to integrate all requirements. Maybe it can be posted on the
> wiki as a specification proposal ?
> 
> # Build summary:
> 
> ## Apache CouchDB build:
> 
> - based on autotools
> - install couchdb on the system
> - use and require shared libraries installed on the system:
>  spidermonkey, openssl, curl, icu
> - require Erlang installed on the target system
> - Embedding couchdb is possible but difficult.
> - no hot upgrade possible
> - build on linux*, unix*, bsd*, macosx, windows on i386, amd64
>  platforms. Building on other platorms is possible modulo hacks on the
> libraries and erlang. Windows requires a specific environment [1]
> 
> 
> ## Rcouch build
> 
> - makefile based
> - use rebar [2]
> - build as an OTP release [3] : the runtime is distributed as part of
>  the release and custom Erlang application can be added to the build
> easily, rebar is used for it.
> - all libraries are fetched and built as static module that will be
>  statically linked: no need to install the libraries on the target
> system once the release is built. The libraries are automatically
> patched and built for each supported platforms.
> - build on linux*, unix*, bsd*, solaris, macosx, windows possible but need to be
>  adapted
> - can be embedded in other Erlang applications easily: propose a
>  couch_core [4] that can be used by any other platforms
> - an rcouch build can be distributed and installed on the same platform
>  it has be been built for without requiring any other installation.
> - can build a source release with all dependencies integrated.
> - propose release templates to build custom release [5]
> - hot upgrade possible
> 
> 
> ## Bigcouch build
> 
> - makefile based
> - use rebar
> - build an OTP release, rebar is used for it
> - build couchjs using scons to find spidermonkey on the system
> - libraries are not statically linked and path to find libs are set in
>  rebar.config
> - build on linux*, unix*, bsd*, solaris
> - can be embedded in other erlang apps, but requires libs installed in
>  fixed paths. Also the couch application is integrated on the release
> so specific build require to fork this release.
> - hot upgrade possible
> 
> # New build system proposal
> 
> 
> This proposal is based on the following requirements:
> 
> - Possibility to embed couchdb in other erlang apps
> - Create a full Erlang OTP release
> - Support rebar
> - Possibility to ship a static build of apache couchdb for each
>  supported platforms
> - Possibility to reuse installed libraries on targeted systems (good for
>  distributions packagers)
> - Erlang dependencies should be always available and shipped with the
>  source release.
> 
> The first 2 points requires in my opinion that we can build a full couchdb
> core that can be included easily in other erlangs apps. It should also
> be fetched without apache-couchdb build system. The perfect way to
> handle it in the Erlang world today is by using rebar and create a
> `rebar.config` eventually fetching needed dependencies etc. Also some will
> want to build their release statically and not care about dependencies
> to install when they distribute or build their own code.
> 
> A rebar build need to know where to find shared or static libs. Some
> paths can be set using env. It can be also possible to call external
> scripts that will be used to build external C code and eventually set
> some makefiles. rebar proposes a system of pluging to extend its
> possibilities.
> 
> To use shared libraries the 2 majors and working systems today are
> autotools and cmake. While cmake is interresting I think that some of us
> are pefectly fluent with autotools so it may be better to use them.
> 
> I propose to mix autotools and rebar in the build process. Also I
> propose to fetch erlang dependencies on build or on release instead of
> having them in our sources like now. Erlang dependencies are for now:
> mochiweb, oauth, ibrowse, snappy & ejson.
> 
> 
> Imo the new build process could be the following:
> 
> - autootools used to boostrap the build: create make file and set rebar
>  config files using templates.
> - 2 rebar config files will be generated :
> - 1 allowing static build against libraries installed using the
>  apache-couchdb-sdk, this file is generated on bootstrap. It will be
> used by other erlangs code if needed.
> - The others will be created by running ./configure with needed options
> - The release will be built using reltools and rebar. Using either
>  dynamically generated rebar files or the static one using the
> apache-couchdb-sdk
> - A source release will be generated using the bootstrapped makefiled. It
>  will fetch dependencies and then a signed tar.gz willl be created from
> it.
> 
> 
> The apache-couchdb-sdk is something like the android sdk or other sdk
> around. It proposes a build environment for each platforms supported by
> apache couchdb with all the dependencies already statically builded
> (erlang, openssl, spidermonkey, icu, curl) . Something we already do for
> windows and which is done somehow by rcouch too. (Note that couchbase did
> the same for ios & android platforms.)
> 
> It may require some change in our current code structure. Generally an
> Erlang application is organized like this:
> 
> deps/
> ebin/
> src/
> include/
> rebar.config
> 
> `deps/` is the folder where rebar put all dependencies fetches. It's
> configurable now.
> `ebin/` is the folder where all binaries (.beam & .app) files are put
> during compilation.
> 
> 
> When multiple apps are handled by the same project they can be
> integrated eithers as dependencies or as included apps and generally the
> project became:
> 
> apps/
> deps/
> rebar.config
> 
> See for example the couch_core repository [4] or th ebigcouch one [6].
> The `apps` folder is also configurable.
> 
> For releases there are 2 ways to handled it. One is to handle it in the
> same sources. So above become
> 
> 
> apps/
> deps/
> rel/
> rebar.config
> 
> Where the `rel` folder contains all the files needed to generate a
> release. (generally a reltool.config file and a files/ folder) . This is
> the way chosen by bigcouch. But generally it's better to put the release
> generation in its own repository so people can integrate the core
> applications they need (here couch_core) in their own releases
> templates. See the `rcouch` repository for example [7]. Lot of Erlang
> developers are doing this way now. My preference go for this one too but
> I'm not sure how it can be done in apache. If we follow this way it will
> imply to have 2 or more subprojects
> 
> 1 subproject : the apache couchdb release (doc, OTP release generation)
> 1 subproject for the couch_core
> 
> Eventually we could split the couch_core in different apps:
> 
> - couch_core (the K/V api)
> - couch_index
> - couch_http
> - couch_replicator
> 
> But that's can be discussed in another thread.
> 
> Voilà, hopefully this thread can be a good start for merging rcouch,
> bigcouch & apache couchdb and will ease merge of the other features imo.
> 
> - benoît
> 
> 
> [1] http://wiki.apache.org/couchdb/Installing_on_Windows
> [2] https://github.com/basho/rebar/wiki
> [3] http://www.erlang.org/doc/design_principles/release_structure.html#id75442
> [4] https://github.com/refuge/couch_core
> [5] https://github.com/refuge/rcouch_template
> [6] https://github.com/cloudant/bigcouch
> [7] [5] https://github.com/refuge/rcouch


Mime
View raw message