Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Subject: Re: CouchDB 2.0 as Snap
To: dev@couchdb.apache.org
References: <35aa50dc-665c-3e7c-f406-b5b51b13d26e@gmail.com>
 <CABHX5y+Yw5z=sNjryPQtQj+eT=Y9g+U_PKu-1Zvk+G-dSt0HYw@mail.gmail.com>
 <e8376a95-d7e3-ceb3-1845-364572ff2745@gmail.com>
 <2B63806F-804A-4F4E-A6D0-2B8BBD790D68@apache.org>
 <cc606cae-165e-d08c-3e10-b2727e10149b@gmail.com>
 <A35800EA-7FCC-4CE8-BDE1-93BA33EAB5EE@apache.org>
 <573e3885-f334-a6ae-1086-90d67ad582c9@gmail.com>
 <E638C9E3-5424-41E0-8FDB-F57F7E3AF5B8@apache.org>
 <eb161cc7-5b69-8c75-0001-e48a1bb7d9be@gmail.com>
 <84D70CFC-6FC5-4DC1-8AC7-465468912839@apache.org>
From: Michael Hall <mhall119@gmail.com>
Message-ID: <65aea1c0-5d27-9245-04a9-a0231a20467e@gmail.com>
Date: Fri, 7 Oct 2016 16:57:50 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Thunderbird/45.3.0
MIME-Version: 1.0
In-Reply-To: <84D70CFC-6FC5-4DC1-8AC7-465468912839@apache.org>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
archived-at: Fri, 07 Oct 2016 20:58:13 -0000

I've done a little more work on the snapcraft.yaml, and integrated it
with couchdb's build files. Pull request is
https://github.com/apache/couchdb/pull/442

You can now ./configure && make snap

Michael Hall
mhall119@gmail.com

On 10/06/2016 12:57 PM, Adam Kocoloski wrote:
> Nice work Michael!
> 
> I noticed your snap.ini overrides the bind address for both the cluster port and the node-local port to have them listen on all interfaces. I think it’s worth discussing whether we want that to be the default for the snap. There’s a reason CouchDB defaults to listening only on the loopback interface. Otherwise it looks good to me.
> 
> Adam
> 
>> On Oct 6, 2016, at 8:39 AM, Michael Hall <mhall119@gmail.com> wrote:
>>
>> Hey everyone,
>>
>> Sorry for the long delay, but I got some help from a coworker and
>> between the two of us we have fixed the issue with the systemd service.
>>
>> If you put the attached files into a directory with the couchdb
>> directory from ./rel/ you get after building, then run "snapcraft snap"
>> you will get a ~40MB couchdb_2.0_amd64.snap (or whatever arch you're on)
>> that, when installed with "snap install couchdb_2.0_amd64.snap
>> --force-dangerous" will get you a running couchdb instance on
>> http://localhost:5984 (see attached screenshot). The --force-dangerous
>> is only needed because it's a local (untrusted) file, once it's being
>> published into the snap store that won't be needed and user can install
>> it with a simple "snap install couchdb".
>>
>> It's configured to put local.ini and couchdb.log into SNAP_DATA, which
>> will be /var/snap/couchdb/<version>/ and the actual database files in
>> SNAP_COMMON which will be /var/snap/couchdb/common/. The first will be
>> forward-copied every time you install a new version, the second is
>> unversioned so you won't be duplicating large database files on upgrades.
>>
>> I'd like to get this into upstream now that it produces a working snap,
>> and from there it can be improved as needed based on feedback from users.
>>
>> Michael Hall
>> mhall119@gmail.com
>>
>> On 09/19/2016 07:36 PM, Robert Newson wrote:
>>> Make a separate systemd service for epmd and have the couch one depend on it. There is a parameter you can add to couch's vm.args file to prevent it even trying to start epmd. 
>>>
>>> Sent from my iPhone
>>>
>>>> On 19 Sep 2016, at 22:47, Michael Hall <mhall119@gmail.com> wrote:
>>>>
>>>> Thanks to help from Jan and Wohali on IRC, I was able to manually build
>>>> couchdb from the 2.0.x branch, and then snap-package the resulting
>>>> binary. I have attached the snapcraft.yaml used for this. Put this file
>>>> in a directory with the couchdb directory built in ./rel/, then run
>>>> "snapcraft snap" to build couchdb_2.0_amd64.snap
>>>>
>>>> The snap package will create a systemd service file for running couchdb
>>>> as a daemon, but due to the way it launches a background epmd process
>>>> this isn't working right (systemd thinks it failed to start and keeps
>>>> trying to restart it until it givesup). Because of that, I've also
>>>> included a /snap/bin/couchdb.run which will manually kick it off, but
>>>> this should only be temporary until the daemon process can be fixed.
>>>>
>>>> One last caveat, you'll need to copy /snap/couchdb/current/etc/*.ini
>>>> into /var/snap/couchdb/current/ and mkdir /var/snap/couchdb/current/data
>>>> before running it. This could be done at runtime either by couchdb
>>>> itself, or with a custom wrapper script for the snap command.
>>>>
>>>> Michael Hall
>>>> mhall119@gmail.com
>>>>
>>>>> On 09/19/2016 01:19 PM, Jan Lehnardt wrote:
>>>>>
>>>>>> On 19 Sep 2016, at 19:13, Michael Hall <mhall119@gmail.com> wrote:
>>>>>>
>>>>>> Maybe I'm using the wrong branch, because the Makefile has an "install"
>>>>>> target but not a "release" target. I'm using developer-preview-2.0, if
>>>>>> that's not the correct one, which should I use?
>>>>>
>>>>> Please use the `2.0.x` branch.
>>>>>
>>>>> Best
>>>>> Jan
>>>>> --
>>>>>
>>>>>>
>>>>>> Michael Hall
>>>>>> mhall119@gmail.com
>>>>>>
>>>>>>> On 09/19/2016 12:10 PM, Jan Lehnardt wrote:
>>>>>>> Heya, nice effort here :)
>>>>>>>
>>>>>>> CouchDB 2.0 doesn’t use autotools. It mimics them minimally, but only
>>>>>>> insofar as it is useful for CouchDB and not for tools that expect
>>>>>>> autotools-like behaviour.
>>>>>>>
>>>>>>> Over time, we want to make it so that the CouchDB install procedure
>>>>>>> fits right into normal tooling, but we are not there yet.
>>>>>>>
>>>>>>> Especially, `make install` is not available in 2.0. Instead, we
>>>>>>> have `make release` which produces a location independent directory
>>>>>>> `./rel/couchdb` that you can move into your system where you need it.
>>>>>>>
>>>>>>> There is no way to externalise log files or so from a setup perspective
>>>>>>> (although it can be configured in local.ini).
>>>>>>>
>>>>>>> HTH
>>>>>>>
>>>>>>> Best
>>>>>>> Jan
>>>>>>> --
>>>>>>>
>>>>>>>> On 19 Sep 2016, at 17:48, Michael Hall <mhall119@gmail.com> wrote:
>>>>>>>>
>>>>>>>> I have attached the snapcraft.yaml file I've started. This is used by
>>>>>>>> the snapcraft tool to build and package a .snap file (just run
>>>>>>>> `snapcraft snap` in the same directory as this file).
>>>>>>>>
>>>>>>>> You can see that most of it is dedicated to grabbing the source,
>>>>>>>> specifying build dependencies (build-packages) and runtime dependencies
>>>>>>>> (stage-packages). The 'autotools' plugin will run the standard
>>>>>>>> "./configure; make; make install" steps on the source, and while the
>>>>>>>> output of those claims to be successful, make returns with a non-zero
>>>>>>>> status code ($?=2) which causes snapcraft to abort after building.
>>>>>>>>
>>>>>>>> As mentioned previously, this could be significantly simplified if it
>>>>>>>> could use the build processes already in place. In that case the
>>>>>>>> snapcraft.yaml would only need to be pointed to the local directory
>>>>>>>> containing the binary files needed to include in the .snap package. If
>>>>>>>> somebody wants to give that a try, I can put together a new
>>>>>>>> snapcraft.yaml that will do that.
>>>>>>>>
>>>>>>>>
>>>>>>>> Michael Hall
>>>>>>>> mhall119@gmail.com
>>>>>>>>
>>>>>>>>> On 09/19/2016 02:56 AM, Constantin Teodorescu wrote:
>>>>>>>>> It would be nice to have two snap packages:
>>>>>>>>> - CouchDB 2.0 UN-CLUSTERED
>>>>>>>>> - CouchDB 2.0 CLUSTERED VERSION
>>>>>>>>>
>>>>>>>>> That will encourage a lot of "standalone" CouchDB users to upgrade to a 2.0
>>>>>>>>> version without the clustering overload stuff, and thus make a big pool of
>>>>>>>>> 2.0 testers and bug-reporters!
>>>>>>>>> Teo
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Mon, Sep 19, 2016 at 4:47 AM, Michael Hall <mhall119@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>> First off, congratulations on the upcoming 2.0 release!
>>>>>>>>>>
>>>>>>>>>> I would love to see this new version available as a Snap package for
>>>>>>>>>> users of Ubuntu 16.04 LTS, since the archive version will be frozen on
>>>>>>>>>> 1.6.0 for the next 5 years of it's lifecycle.
>>>>>>>>>>
>>>>>>>>>> Snaps are self-contained packages that include all of the dependencies
>>>>>>>>>> they need, which lets them run as you (the upstream) intended across new
>>>>>>>>>> releases of Ubuntu, Debian, Arch, and many other distros. They run in a
>>>>>>>>>> sandbox that protects them from changes made to the user's system, but
>>>>>>>>>> with a number of optional interfaces if you need deeper interaction or
>>>>>>>>>> to share data with other apps.
>>>>>>>>>>
>>>>>>>>>> Every snap includes its own file tree, and is run on top of the same
>>>>>>>>>> base image regardless of distro or form factor. This keeps the
>>>>>>>>>> application's own files isolated from other apps and the host system, in
>>>>>>>>>> a read-only filesystem, which makes updating them safe and simple while
>>>>>>>>>> keeping you in control of the whole stack that your application runs on.
>>>>>>>>>> The snappy runtime then provides writable areas for storing both
>>>>>>>>>> versioned and unversioned data, as well as system-wide or per-user data.
>>>>>>>>>>
>>>>>>>>>> We also provide a Snap Store, which combines the speed of
>>>>>>>>>> self-publishing with the discoverability of a central archive. It is
>>>>>>>>>> used by default across all Ubuntu 16.04 flavors and derivatives, and any
>>>>>>>>>> distro where snaps have been enabled. Thanks to Snap's confinement,
>>>>>>>>>> applications can be published immediately after uploading. This means
>>>>>>>>>> that your application and updates are available to tens of millions of
>>>>>>>>>> users as soon as you press the button.
>>>>>>>>>>
>>>>>>>>>> I started the work on producing a Snap package for Couchdb 2.0, but as I
>>>>>>>>>> couldn't find a binary release I had to try building it from source and
>>>>>>>>>> unfortunately I was not successful on that step. I am happy to share my
>>>>>>>>>> packaging configuration with anybody here who knows the build process
>>>>>>>>>> better than me, but it would be even simpler to create the snap package
>>>>>>>>>> at the end of whatever process you already have to build binary
>>>>>>>>>> releases. I am happy to help with either or both approaches, and you can
>>>>>>>>>> also learn more about the snap format and tools here: http://snapcraft.io/
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Michael Hall
>>>>>>>>>> mhall119@gmail.com
>>>>>>>>>>
>>>>>>>>>
>>>>>>>> <snapcraft.yaml>
>>>>>>>
>>>>>
>>>> <snapcraft.yaml>
>>>
>> <snap.ini><snapcraft.yaml><snap_run.txt>
>