couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benoit Chesneau <bchesn...@gmail.com>
Subject Re: windows support, cross compilation, and build reuqirements
Date Sat, 15 Dec 2012 07:54:24 GMT
On Thu, Dec 13, 2012 at 8:47 PM, Noah Slater <nslater@apache.org> wrote:

> Okay, here's my take on it.
>
> TL;DR: Let's switch to rebar and scrap Autotools completely. Probably.
>
> Firstly, let's clear up this mess about VPATH builds. Basically, VPATH is
> not that important in the grand scheme of things. But it is required for an
> Autotools build. Simply because Autotools builds provide VPATH builds. Not
> having one would be like... I dunno, having a CouchDB that didn't support
> replication. You know, VPATH is just part of what we mean when we say "this
> project uses Autotools."
>
> What is it though? Like, seriously, WTF is VPATH anyway?
>
> I can demonstrate it in a few commands:
>
> $ mkdir -p /tmp/couchdb && cd /tmp/couchdb
> $ wget
>
> http://www.apache.org/dist/couchdb/releases/1.2.0/apache-couchdb-1.2.0.tar.gz
> $ tar -xvzf apache-couchdb-1.2.0.tar.gz
> $ ./apache-couchdb-1.2.0/configure
> $ make
>
> Go on. Run those commands. The results will be very interesting for you if
> you don't know what VPATH means.
>
> Have you run them? Cool. Now, inspect your working directory. Thoroughly.
> ;) Can you see what has happened?
>
> Autotools is very strict about separating the source tree and build tree.
> All the source files remain in the apache-couchdb-1.2.0 directory, and all
> the files you just built are in your working directory. This means that the
> source tree, for example, could be mounted read-only. This sort of thing,
> you imagine, is more useful in very tightly controlled situations, when
> installing from a network share, or whatever.
>
> So. Is it important? Well, if we're doing Autotools, yes, because it's just
> part of what you mean when you say you support Autotools. Various
> downstream packaging systems rely on this sort of functionality.
>
> However, in the grand scheme of things, will CouchDB survive if we have a
> build system that doesn't support VPATH builds?
>
> Yep.
>


OK it's more clear now. Thanks for the explanation. Even if it's not
totally needed , it may not be ou of the scope with rebar. I will have a
closer look on that topic.



>
> So. Let's talk about rebar.
>
> I think rebar looks great. And I think that if we use it, we will attract
> more Erlang developers. And I think that is probably more important than
> any other technical argument that we can come up with. Infinitely more
> important than supporting VPATH builds.
>
> We should definitely use rebar with CouchDB.
>
> But how should we use it?
>
> There are two paths forward, I think.
>
> Path 1: Scrap Autotools and replace the whole build system with rebar.
>
> Path 2: Turn CouchDB core into a rebar project, and embed it into
> Autotools.
>
> Path 1 is obviously preferable. I only have one concern. And my concern is
> the same concern that leads me to think that Path 2 is even on the table.
> Namely, Autotools actually does a bunch of really cool shit for us.
>
> Namely:
>
> * Checks dependencies before the build takes place
>

This is the real part that will miss somehow. Though if it we rely on
pkg-config that will works well on most systems.


> * Handles compilation of C code, etc (waves hands, don't understand C,
> seems like magic)
>

Latest versions of rebar gave us all the magie we need for that and is
really integrated with Erlang.



> * Customises various files (setting values, etc) before installing them
>

can be done with rebar templates, ex:

https://github.com/refuge/rcouch/blob/master/rel/files/couch.ini#L10


> * Installs CouchDB locally in a way that properly integrates it with the
> operating system
>
> Now, pay close attention to this last point. I think it's important.
>
> The last time I checked out MongoDB, when you unzip the download, or
> whatever, you have like a binary file. You can run it, and Mongo starts up.
> And I was like, WTF? Where am I meant to put this file. I guess I am
> supposed to move it to /usr/local/bin or something.
>
> Like, I was really expected to just take this pre-built binary, and sort
> of, figure it out myself. You know. Set up the config files, set up the
> data directory, set up the log file directory, set up a log rotation
> policy, all that sort of stuff.
>
> Perhaps that's changed? I dunno. It's been a while since I checked any of
> that stuff out
>
> CouchDB does all that for you. Well, as much of it as seems sensible. I
> always thought that was a great thing. You know. We install CouchDB
> properly, in a way that, with a few additional commands to set up a user,
> gives you the level of system integration you might expect from APT.
>
> But you know, MongoDB is hugely popular. So maybe it's not that important
> after all?
>
> What about Riak? Well, I just spent some time checking Riak out. And
> basically, it does none of this stuff for you either. It builds an Erlang
> release into a folder, and you run it from that. If you're deploying to a
> production machine, it is assumed you move the release on to the system,
> and take care of everything.
>

That's untrue, riak does it for its packages. This all the point of
vars.config or more precisely the overlay config which set the layout and
other vars you need to build and kater install the release. See for example
the debian package:

https://github.com/basho/riak/blob/master/package/deb/vars.config#L5-L10

Same for RPM:

https://github.com/basho/riak/blob/master/package/rpm/SPECS/riak.spec#L42-L49


In short a maintainer can set the path where riak will be run. I do similar
things for rcouch as well so you can build it as a relocatable binary or as
a binary  following the layout of the system where it is installed. We can
even make it easier with a shell script building this overlay. Just like
Cloudant does  with bigcouch.



>
> So. This seems sloppy to me. But hey. If it works for Riak, and it works
> for Mongo, and it makes it easier to hack on the code, and it helps us
> attract more Erlang devs, then maybe we should do it also?
>
> Not sure how we do packaging, but I guess we can just copy what Riak does.
>
> So. If we do this. What do we loose? What's holding us back?
>
> Benoit, how soon do you think we can merge rcouch, wholesale, back into
> CouchDB?
>

sometimes before the end of the year. I will be busy until the middle of
the next week working on releasing camp.io

>
> What work needs to be done before we can replace one for the other?
>
>
Since rcouch is more than just rebar (it has also performances patch, new
features and such) , the hard work is to split each changes in their
commits so it can be added atomically to the couchdb project. It would be a
lot simpler if we choose to rely on rebar only for the installation So i
would be happy if we can settle on that.


- benoît

>
>
> On 13 December 2012 08:25, Benoit Chesneau <bchesneau@gmail.com> wrote:
>
> > Hi all,
> >
> > As you know I am working on porting some part of rcouch to couchdb to
> make
> > it more otpish and such.
> >
> > Part of it requires to play with the build chain of couchdb and I'm
> trying
> > to list all the requirements and answers to the following questions. I
> also
> > would like to work on a one **reproducible* and **automated** way to
> build
> > couchdb on different platforms without requiring manual patching and
> such.
> >
> > - Why do we need autotools? Part of the story is related to this "VPATH"
> > thing, but I am still waiting for a clear explanation not convoluted in
> > details or related to a man. Ie. what are the usescases in human words.
> How
> > it is used today. by who. Other is related to the dependencies check but
> > seeing how many people are using rcouch and build-couchdb I have some
> > doubts about the usefulness of that. Also autotools are really only
> useful
> > on linux.
> >
> > - windows toolchain: so far i have these wikis and posts:
> >
> > http://wiki.apache.org/couchdb/Installing_on_Windows
> > http://wiki.apache.org/couchdb/Quirks_on_Windows
> > http://wiki.apache.org/couchdb/Building_from_source_on_Windows
> >
> >
> http://www.muse.net.nz/blog/2012/01/10/building-erlang-and-couchdb-on-windows-like-a-boss/
> >
> > also glazier doc : https://github.com/dch/glazier
> >
> > While I really appreciate all the efforts around it looks rather hackish
> > for now and I would like to help to make it more generic and easy to use
> > for others. So what are the requirements, quirks and problems found for
> > now? (I'm not speaking about couchdb issues). Can we have a clear list
> > about that ?
> >
> > - osx build wit the desktop addition: how it is sorted to be
> > build automatically  What arre the requirements & co?
> >
> > - others ? I know that some people worked in embedding couchdb on other
> > platforms. Having a list of the patches they use, why they aren't in the
> > project upstream and other details would be cool. For ex having the full
> > story of iErl4 and couchbae-android-framework of couchbase would
> > be interesting. But there are others around. I will also do my part on
> > that.
> >
> > Voilà. Improving the toolchain of couchdb would be a really nice goal for
> > 1.4 and I hope we can achieve this in time.
> >
> > - benoît
> >
>
>
>
> --
> NS
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message