Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of bchesneau@gmail.com
 designates 209.85.223.180 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAHdjipK3UOg3FbnA_sGUpcP8YXOa0PYfnn3Zx0JmTodPjJdu+A@mail.gmail.com>
References: <40BCC84F-C695-492E-B56E-08C60955A3D8@apache.org>
	<CAHdjipK3UOg3FbnA_sGUpcP8YXOa0PYfnn3Zx0JmTodPjJdu+A@mail.gmail.com>
Date: Thu, 1 Nov 2012 07:49:26 +0100
Message-ID: 
 <CAJNb-9pQmqPYCT3v_QXH-W_ZtGQwXmMrcXPCwHTimvc7x12_QA@mail.gmail.com>
Subject: Re: CouchDB Plugins First Draft
From: Benoit Chesneau <bchesneau@gmail.com>
To: "dev@couchdb.apache.org" <dev@couchdb.apache.org>
Content-Type: multipart/alternative; boundary=14dae934055f871eba04cd696b79

--14dae934055f871eba04cd696b79
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

On Wed, Oct 31, 2012 at 11:28 PM, Alexander Shorin <kxepal@gmail.com> wrote=
:

> Hi Jan.
>
> More detailed and explained question from IRC meeting.
>
> Why there is need to reinvent own package manager when most modern
> systems (even Windows) already has package manager?
>
>
> What problem tried to be solved?
> ------------------------------------------------------------
>
> Easy find, install and activate any CouchDB plugin in one-shot action,
> right? Following example from gist:
>
> $ cpm # couch plugin manager
> $ cpm search geocouch
> $ cpm install geocouch
>
> While it tries to simplify extending of CouchDB, it brings a lot of
> problems into system
>
>
> What problems cpm brings?
> -----------------------------------------------------
>
> 1. Dependencies.
>
> While any system package already able to resolve dependencies,
> conflicts, provide slots for different versions, cpm will need to
> reimplement this features. Geocouch isn't good example, next one is
> much better:
>
> $ cpm install couchdb-lucene # what about dependencies?
>
> I don't think (and hope it will not) that this command will install
> JRE and other java things into my system while I'm only expects this
> plugin. System package managers already know what need and which
> versions of to properly install lucene. Should this `cpm install`
> command only check for suitable environment before landing? Or may be
> cpm will know how to cooperate with system package managers to propose
> install required dependencies?
>
> 2. Namespace.
>
> Since CouchDB installs system-wide (most cases) should plugins be
> installed in same way? Or each user may have his own set of active
> plugins? But it looks like that only CouchDB admin may operate with
> plugins from Futon, so plugins have to live at least by path
> accessiable for couchdb user. So there couldn't be personalised
> plugins and npm system wouldn't work. So, cpm utility is only useful
> for root user which is already has access to system package manager.
>
> 3. BigCouch
>
> How plugins system proposed to be work after BigCouch merge? Would
> plugins be installed for all shards or only just for current one? Will
> cluster works right if some shards would have geocouch-1.0, some of
> them has geocouch-1.3 with breaking changes and for others it just
> missed?
>
> Why system package manager?
> ---------------------------------------------------------
>
> $ flaggie dev-db/couchdb +lucene # or echo "dev-db/couchdb lucene" >>
> /etc/portage/package.use
> $ emerge couchdb
>
> And we'd got CouchDB with couchdb-lucene plugin and handled all
> required dependencies. Now system takes care about packages
> consistency and we're always sure and accidental iceadtea uninstall
> would be passed silently since it have to break our couchdb-lucene
> extension.
>
> In same way other projects are already have implemented their
> "plugins" system: nginx, php. Yes, they have not plugins, but modules,
> but since they acts on system level there is no easy and secure way to
> let everyone enable/disable/install any thirdparty component and this
> way is the right one.
>
> Following RabbitMQ project, they ships with all main plugins on board
> - you need just to enable with through cli utility. If you need to
> install some third party plugin, just copy his directory to rabbits
> plugins one. But to be honest, RabbitMQ plugin manager tool doesn't
> tries to act as package manager: it only shows, enables and disables
> plugins, nothing more - that's why their approach works fine.
>
>
> So I've smoothly went from original question to set of other ones, but
> I suppose they are also interested to be answered since they are
> defined show cpm will work and handle plugins.
>
> P.S. Sorry for poorly mind dump style, 2:40 am there. I could try to
> explain something in other way if it wasn't clear enough...
>
> --
> ,,,^..^,,,
>
>
> On Mon, Oct 29, 2012 at 8:41 PM, Jan Lehnardt <jan@apache.org> wrote:
> > Hey all,
> >
> > during one of the past IRC Meetings I promised to write up
> > what I have in mind for a CouchDB Plugin System.
> >
> > Here is my first draft:
> >
> >   https://gist.github.com/3974676
> >
> > This is very early stage, but I wanted to get the idea out
> > of the door. If anyone feels compelled to go into further
> > detail, or even implement some of this, more power to you!
> >
> > Note: This is especially interesting for the non-Erlang
> > folks here that need an excuse to contribute :)
> >
> > For the frontend, we=92d need some mockups on how the Futon
> > interaction would work (that is regardless of the state
> > of Futon.Current and Futon.Next).
> >
> > On the backend, you could start figuring out what the
> > registry could look like.
> >
> > I=92m looking forward to your feedback!
> >
> > Cheers!
> > Jan
> > --
> >
>
I second that , we need to find a good way to handle plugin.  I think we
have 4 cases to solve reading the gist:

1. upload from the HTTP api an Erlang plugin.
2. install a couchapp via HTPP
3. install an erlang plugin  from the fs
4. install a couchapp from the fs


Both 2 & 4 can be easily handled today. Though it raises
some interesting problems:

- how to allow a user to push a couchapp
- how to make sure that the couchapp can have access to a specific db on
the node and the cluster.

1 & 3 have some other questions:

- how is done the build? Is the plugin sent as a binary or is build on the
target machine?
- how to fix dependencies ? In that case , the requirements are generally
started when the plugin start and simply fail if the requirement isn't
found. This is what do elasticsearch or rabbitmq for ex.
- What if a plugin is installed while the node is running ? Should we
handle hot upgrade?
- What happen when the plugin is unloaded or unregistered. Especially if it
depends on some dependencies.
- sandboxing. How to make sure a "native" plugin won't do bad things on my
system
- how to deploy the plugin on a cluster?

Imo we should have a look on how it's handled in the mobiles right now:

- get an app from a location
- check it's security properties
- check requirements
- ask to the authorized users if the app can have access to a db/resource
...

But, hat if we distinct backend plusgins in erlang from the other. In my
understanding we need both to extend internals of couchdb. But maybe we can
find to another system allowing us to extend couchdb. Such plugin could be
then written in javascript, lua, elixir ... And could be sandboxed.

This is actually something I am working on and I will release preliminary
work at the end of the week or smth like it but in short the idea I have is
to extend the couchapp system for internal couchapps which are executed in
couchdb instead of the browser and have access to the full api. Networks
access & disk are limited  except if specifically allowed. Using such
features allows :

- installation and upgrade via HTTP
- plugin replication: you don't have to install cheff, puppet... plugins
are contained in a doc and can be replicated like others.
- the security system of the frontend couchapps could be reused

Anyway I quite like the description of plugins described by jan  and I am
happy to help on that . Btw why not using the wiki instead of github to
handle the spec writing?

- benoit

--14dae934055f871eba04cd696b79--